de.tudarmstadt.ukp.dkpro.core.mallet.wordembeddings

Class MalletEmbeddingsTrainer

java.lang.Object

org.apache.uima.analysis_component.AnalysisComponent_ImplBase

org.apache.uima.analysis_component.Annotator_ImplBase

org.apache.uima.analysis_component.JCasAnnotator_ImplBase

org.apache.uima.fit.component.JCasConsumer_ImplBase

de.tudarmstadt.ukp.dkpro.core.api.io.JCasFileWriter_ImplBase

de.tudarmstadt.ukp.dkpro.core.mallet.MalletModelTrainer

de.tudarmstadt.ukp.dkpro.core.mallet.wordembeddings.MalletEmbeddingsTrainer

All Implemented Interfaces:

org.apache.uima.analysis_component.AnalysisComponent

public class MalletEmbeddingsTrainer
extends MalletModelTrainer

Compute word embeddings from the given collection using skip-grams.

Set MalletModelTrainer.PARAM_TOKEN_FEATURE_PATH to define what is considered as a token (Tokens, Lemmas, etc.).

Set MalletModelTrainer.PARAM_COVERING_ANNOTATION_TYPE to define what is considered a document (sentences, paragraphs, etc.).

Since:

1.9.0

Nested Class Summary

Nested classes/interfaces inherited from class de.tudarmstadt.ukp.dkpro.core.api.io.JCasFileWriter_ImplBase

JCasFileWriter_ImplBase.NamedOutputStream

Field Summary

Fields

Modifier and Type	Field and Description
static String	PARAM_DIMENSIONS The dimensionality of the output word embeddings (default: 50).
static String	PARAM_EXAMPLE_WORD An example word that is output with its nearest neighbours once in a while (default: null, i.e.
static String	PARAM_MIN_DOCUMENT_LENGTH Ignore documents with fewer tokens than this value (default: 10).
static String	PARAM_NUM_NEGATIVE_SAMPLES The number of negative samples to be generated for each token (default: 5).
static String	PARAM_WINDOW_SIZE The context size when generating embeddings (default: 5).

Fields inherited from class de.tudarmstadt.ukp.dkpro.core.mallet.MalletModelTrainer

PARAM_COVERING_ANNOTATION_TYPE, PARAM_FILTER_REGEX, PARAM_FILTER_REGEX_REPLACEMENT, PARAM_LOWERCASE, PARAM_MIN_TOKEN_LENGTH, PARAM_NUM_THREADS, PARAM_STOPWORDS_FILE, PARAM_STOPWORDS_REPLACEMENT, PARAM_TOKEN_FEATURE_PATH, PARAM_USE_CHARACTERS

Fields inherited from class de.tudarmstadt.ukp.dkpro.core.api.io.JCasFileWriter_ImplBase

JAR_PREFIX, PARAM_COMPRESSION, PARAM_ESCAPE_DOCUMENT_ID, PARAM_OVERWRITE, PARAM_SINGULAR_TARGET, PARAM_STRIP_EXTENSION, PARAM_TARGET_LOCATION, PARAM_USE_DOCUMENT_ID

Constructor Summary

Constructors

Constructor and Description
MalletEmbeddingsTrainer()

Method Summary

Modifier and Type	Method and Description
void	collectionProcessComplete()

Methods inherited from class de.tudarmstadt.ukp.dkpro.core.mallet.MalletModelTrainer

getInstanceList, getNumThreads, initialize, process

Methods inherited from class de.tudarmstadt.ukp.dkpro.core.api.io.JCasFileWriter_ImplBase

getCompressionMethod, getOutputStream, getOutputStream, getRelativePath, getTargetLocation, isStripExtension, isUseDocumentId

Methods inherited from class org.apache.uima.fit.component.JCasConsumer_ImplBase

getLogger

Methods inherited from class org.apache.uima.analysis_component.JCasAnnotator_ImplBase

getRequiredCasInterface, process

Methods inherited from class org.apache.uima.analysis_component.Annotator_ImplBase

getCasInstancesRequired, hasNext, next

Methods inherited from class org.apache.uima.analysis_component.AnalysisComponent_ImplBase

batchProcessComplete, destroy, getContext, getResultSpecification, reconfigure, setResultSpecification

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PARAM_NUM_NEGATIVE_SAMPLES

public static final String PARAM_NUM_NEGATIVE_SAMPLES

The number of negative samples to be generated for each token (default: 5).

See Also:

Constant Field Values

PARAM_DIMENSIONS

public static final String PARAM_DIMENSIONS

The dimensionality of the output word embeddings (default: 50).

See Also:

Constant Field Values

PARAM_WINDOW_SIZE

public static final String PARAM_WINDOW_SIZE

The context size when generating embeddings (default: 5).

See Also:

Constant Field Values

PARAM_EXAMPLE_WORD

public static final String PARAM_EXAMPLE_WORD

An example word that is output with its nearest neighbours once in a while (default: null, i.e. none).

See Also:

Constant Field Values

PARAM_MIN_DOCUMENT_LENGTH

public static final String PARAM_MIN_DOCUMENT_LENGTH

Ignore documents with fewer tokens than this value (default: 10).

See Also:

Constant Field Values

Constructor Detail

MalletEmbeddingsTrainer

public MalletEmbeddingsTrainer()

Method Detail

collectionProcessComplete

public void collectionProcessComplete()
                               throws org.apache.uima.analysis_engine.AnalysisEngineProcessException

Specified by:

collectionProcessComplete in interface org.apache.uima.analysis_component.AnalysisComponent

Overrides:

collectionProcessComplete in class JCasFileWriter_ImplBase

Throws:

org.apache.uima.analysis_engine.AnalysisEngineProcessException