de.tudarmstadt.ukp.dkpro.core.mallet.lda

Class MalletLdaTopicModelTrainer

java.lang.Object

org.apache.uima.analysis_component.AnalysisComponent_ImplBase

org.apache.uima.analysis_component.Annotator_ImplBase

org.apache.uima.analysis_component.JCasAnnotator_ImplBase

org.apache.uima.fit.component.JCasConsumer_ImplBase

de.tudarmstadt.ukp.dkpro.core.api.io.JCasFileWriter_ImplBase

de.tudarmstadt.ukp.dkpro.core.mallet.MalletModelTrainer

de.tudarmstadt.ukp.dkpro.core.mallet.lda.MalletLdaTopicModelTrainer

All Implemented Interfaces:

org.apache.uima.analysis_component.AnalysisComponent

public class MalletLdaTopicModelTrainer
extends MalletModelTrainer

Estimate an LDA topic model using Mallet and write it to a file. It stores all incoming CAS' to Mallet Instances before estimating the model, using a ParallelTopicModel.

Set MalletModelTrainer.PARAM_TOKEN_FEATURE_PATH to define what is considered as a token (Tokens, Lemmas, etc.).

Set MalletModelTrainer.PARAM_COVERING_ANNOTATION_TYPE to define what is considered a document (sentences, paragraphs, etc.).

Nested Class Summary

Nested classes/interfaces inherited from class de.tudarmstadt.ukp.dkpro.core.api.io.JCasFileWriter_ImplBase

JCasFileWriter_ImplBase.NamedOutputStream

Field Summary

Fields

Modifier and Type	Field and Description
static String	PARAM_ALPHA_SUM The sum of alphas over all topics.
static String	PARAM_BETA Beta for a single dimension of the Dirichlet prior.
static String	PARAM_BURNIN_PERIOD The number of iterations before hyper-parameter optimization begins.
static String	PARAM_DISPLAY_INTERVAL The interval in which to display the estimated topics.
static String	PARAM_DISPLAY_N_TOPIC_WORDS The number of top words to display during estimation.
static String	PARAM_N_ITERATIONS The number of iterations during model estimation.
static String	PARAM_N_TOPICS The number of topics to estimate.
static String	PARAM_OPTIMIZE_INTERVAL Interval for optimizing Dirichlet hyper-parameters.
static String	PARAM_RANDOM_SEED Set random seed.
static String	PARAM_SAVE_INTERVAL Define how frequently a serialized model is saved to disk during estimation.
static String	PARAM_USE_SYMMETRIC_ALPHA Use a symmetric alpha value during model estimation? Default: false.

Fields inherited from class de.tudarmstadt.ukp.dkpro.core.mallet.MalletModelTrainer

PARAM_COVERING_ANNOTATION_TYPE, PARAM_FILTER_REGEX, PARAM_FILTER_REGEX_REPLACEMENT, PARAM_LOWERCASE, PARAM_MIN_TOKEN_LENGTH, PARAM_NUM_THREADS, PARAM_STOPWORDS_FILE, PARAM_STOPWORDS_REPLACEMENT, PARAM_TOKEN_FEATURE_PATH, PARAM_USE_CHARACTERS

Fields inherited from class de.tudarmstadt.ukp.dkpro.core.api.io.JCasFileWriter_ImplBase

JAR_PREFIX, PARAM_COMPRESSION, PARAM_ESCAPE_DOCUMENT_ID, PARAM_OVERWRITE, PARAM_SINGULAR_TARGET, PARAM_STRIP_EXTENSION, PARAM_TARGET_LOCATION, PARAM_USE_DOCUMENT_ID

Constructor Summary

Constructors

Constructor and Description
MalletLdaTopicModelTrainer()

Method Summary

Modifier and Type	Method and Description
void	collectionProcessComplete()

Methods inherited from class de.tudarmstadt.ukp.dkpro.core.mallet.MalletModelTrainer

getInstanceList, getNumThreads, initialize, process

Methods inherited from class de.tudarmstadt.ukp.dkpro.core.api.io.JCasFileWriter_ImplBase

getCompressionMethod, getOutputStream, getOutputStream, getRelativePath, getTargetLocation, isStripExtension, isUseDocumentId

Methods inherited from class org.apache.uima.fit.component.JCasConsumer_ImplBase

getLogger

Methods inherited from class org.apache.uima.analysis_component.JCasAnnotator_ImplBase

getRequiredCasInterface, process

Methods inherited from class org.apache.uima.analysis_component.Annotator_ImplBase

getCasInstancesRequired, hasNext, next

Methods inherited from class org.apache.uima.analysis_component.AnalysisComponent_ImplBase

batchProcessComplete, destroy, getContext, getResultSpecification, reconfigure, setResultSpecification

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PARAM_N_TOPICS

public static final String PARAM_N_TOPICS

The number of topics to estimate.

See Also:

Constant Field Values

PARAM_N_ITERATIONS

public static final String PARAM_N_ITERATIONS

The number of iterations during model estimation. Default: 1000.

See Also:

Constant Field Values

PARAM_BURNIN_PERIOD

public static final String PARAM_BURNIN_PERIOD

The number of iterations before hyper-parameter optimization begins. Default: 100

See Also:

Constant Field Values

PARAM_OPTIMIZE_INTERVAL

public static final String PARAM_OPTIMIZE_INTERVAL

Interval for optimizing Dirichlet hyper-parameters. Default: 50

See Also:

Constant Field Values

PARAM_RANDOM_SEED

public static final String PARAM_RANDOM_SEED

Set random seed. If set to -1 (default), uses random generator.

See Also:

Constant Field Values

PARAM_SAVE_INTERVAL

public static final String PARAM_SAVE_INTERVAL

Define how frequently a serialized model is saved to disk during estimation. Default: 0 (only save when estimation is done).

See Also:

Constant Field Values

PARAM_USE_SYMMETRIC_ALPHA

public static final String PARAM_USE_SYMMETRIC_ALPHA

Use a symmetric alpha value during model estimation? Default: false.

See Also:

Constant Field Values

PARAM_DISPLAY_INTERVAL

public static final String PARAM_DISPLAY_INTERVAL

The interval in which to display the estimated topics. Default: 50.

See Also:

Constant Field Values

PARAM_DISPLAY_N_TOPIC_WORDS

public static final String PARAM_DISPLAY_N_TOPIC_WORDS

The number of top words to display during estimation. Default: 7.

See Also:

Constant Field Values

PARAM_ALPHA_SUM

public static final String PARAM_ALPHA_SUM

The sum of alphas over all topics. Default: 1.0.

Another recommended value is 50 / T (number of topics).

See Also:

Constant Field Values

PARAM_BETA

public static final String PARAM_BETA

Beta for a single dimension of the Dirichlet prior. Default: 0.01.

See Also:

Constant Field Values

Constructor Detail

MalletLdaTopicModelTrainer

public MalletLdaTopicModelTrainer()

Method Detail

collectionProcessComplete

public void collectionProcessComplete()
                               throws org.apache.uima.analysis_engine.AnalysisEngineProcessException

Specified by:

collectionProcessComplete in interface org.apache.uima.analysis_component.AnalysisComponent

Overrides:

collectionProcessComplete in class JCasFileWriter_ImplBase

Throws:

org.apache.uima.analysis_engine.AnalysisEngineProcessException