org.dkpro.core.mallet

Class MalletModelTrainer

java.lang.Object

org.apache.uima.analysis_component.AnalysisComponent_ImplBase

org.apache.uima.analysis_component.Annotator_ImplBase

org.apache.uima.analysis_component.JCasAnnotator_ImplBase

org.apache.uima.fit.component.JCasConsumer_ImplBase

org.dkpro.core.api.io.JCasFileWriter_ImplBase

org.dkpro.core.mallet.MalletModelTrainer

All Implemented Interfaces:

org.apache.uima.analysis_component.AnalysisComponent

Direct Known Subclasses:

MalletEmbeddingsTrainer, MalletLdaTopicModelTrainer

public abstract class MalletModelTrainer
extends JCasFileWriter_ImplBase

This abstract class defines parameters and methods that are common for Mallet model estimators.

It creates a Mallet InstanceList from the input documents so that inheriting estimators can create a model, typically implemented by overriding the JCasFileWriter_ImplBase.collectionProcessComplete() method.

Since:

1.9.0

See Also:

MalletEmbeddingsTrainer, MalletLdaTopicModelTrainer

Nested Class Summary

Nested classes/interfaces inherited from class org.dkpro.core.api.io.JCasFileWriter_ImplBase

JCasFileWriter_ImplBase.NamedOutputStream

Field Summary

Fields

Modifier and Type	Field and Description
static String	PARAM_COVERING_ANNOTATION_TYPE If specified, the text contained in the given segmentation type annotations are fed as separate units ("documents") to the topic model estimator e.g.
static String	PARAM_FILTER_REGEX Regular expression of tokens to be filtered.
static String	PARAM_FILTER_REGEX_REPLACEMENT Value with which tokens matching the regular expression are replaced.
static String	PARAM_LOWERCASE If set to true (default: false), all tokens are lowercased.
static String	PARAM_MIN_TOKEN_LENGTH Ignore tokens (or any other annotation type, as specified by PARAM_TOKEN_FEATURE_PATH) that are shorter than the given value.
static String	PARAM_NUM_THREADS The number of threads to use during model estimation.
static String	PARAM_STOPWORDS_FILE The location of the stopwords file.
static String	PARAM_STOPWORDS_REPLACEMENT If set, stopwords found in the PARAM_STOPWORDS_FILE location are not removed, but replaced by the given string (e.g.
static String	PARAM_TOKEN_FEATURE_PATH The annotation type to use as input tokens for the model estimation.
static String	PARAM_USE_CHARACTERS If true (default: false), estimate character embeddings.

Fields inherited from class org.dkpro.core.api.io.JCasFileWriter_ImplBase

JAR_PREFIX, PARAM_COMPRESSION, PARAM_ESCAPE_FILENAME, PARAM_OVERWRITE, PARAM_SINGULAR_TARGET, PARAM_STRIP_EXTENSION, PARAM_TARGET_LOCATION, PARAM_USE_DOCUMENT_ID

Constructor Summary

Constructors

Constructor and Description
MalletModelTrainer()

Method Summary

Modifier and Type	Method and Description
cc.mallet.types.InstanceList	getInstanceList()
protected int	getNumThreads()
void	initialize(org.apache.uima.UimaContext context)
void	process(org.apache.uima.jcas.JCas aJCas)

Methods inherited from class org.dkpro.core.api.io.JCasFileWriter_ImplBase

collectionProcessComplete, getCompressionMethod, getOutputStream, getOutputStream, getRelativePath, getTargetLocation, isStripExtension, isUseDocumentId

Methods inherited from class org.apache.uima.analysis_component.JCasAnnotator_ImplBase

getRequiredCasInterface, process

Methods inherited from class org.apache.uima.analysis_component.Annotator_ImplBase

getCasInstancesRequired, hasNext, next

Methods inherited from class org.apache.uima.analysis_component.AnalysisComponent_ImplBase

batchProcessComplete, destroy, getContext, getLogger, getResultSpecification, reconfigure, setResultSpecification

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PARAM_TOKEN_FEATURE_PATH

public static final String PARAM_TOKEN_FEATURE_PATH

The annotation type to use as input tokens for the model estimation. For lemmas, use de.tudarmstadt.ukp.dkpro.core.api.segmentation.type.Token/lemma/value

See Also:

Constant Field Values

PARAM_NUM_THREADS

public static final String PARAM_NUM_THREADS

The number of threads to use during model estimation. If not set, the number of threads is automatically set by ComponentParameters.computeNumThreads(int).

Warning: do not set this to more than 1 when using very small (test) data sets on MalletEmbeddingsTrainer! This might prevent the process from terminating.

See Also:

Constant Field Values

PARAM_MIN_TOKEN_LENGTH

public static final String PARAM_MIN_TOKEN_LENGTH

Ignore tokens (or any other annotation type, as specified by PARAM_TOKEN_FEATURE_PATH) that are shorter than the given value.

See Also:

Constant Field Values

PARAM_COVERING_ANNOTATION_TYPE

public static final String PARAM_COVERING_ANNOTATION_TYPE

If specified, the text contained in the given segmentation type annotations are fed as separate units ("documents") to the topic model estimator e.g. de.tudarmstadt.ukp.dkpro.core.api.segmentation.type.sentence. Text that is not within such annotations is ignored.

By default, the full text is used as a document.

See Also:

Constant Field Values

PARAM_USE_CHARACTERS

public static final String PARAM_USE_CHARACTERS

If true (default: false), estimate character embeddings. PARAM_TOKEN_FEATURE_PATH is ignored.

See Also:

Constant Field Values

PARAM_LOWERCASE

public static final String PARAM_LOWERCASE

If set to true (default: false), all tokens are lowercased.

See Also:

Constant Field Values

PARAM_STOPWORDS_FILE

public static final String PARAM_STOPWORDS_FILE

The location of the stopwords file.

See Also:

Constant Field Values

PARAM_STOPWORDS_REPLACEMENT

public static final String PARAM_STOPWORDS_REPLACEMENT

If set, stopwords found in the PARAM_STOPWORDS_FILE location are not removed, but replaced by the given string (e.g. STOP).

See Also:

Constant Field Values

PARAM_FILTER_REGEX

public static final String PARAM_FILTER_REGEX

Regular expression of tokens to be filtered.

See Also:

Constant Field Values

PARAM_FILTER_REGEX_REPLACEMENT

public static final String PARAM_FILTER_REGEX_REPLACEMENT

Value with which tokens matching the regular expression are replaced.

See Also:

Constant Field Values

Constructor Detail

MalletModelTrainer

public MalletModelTrainer()

Method Detail

initialize

public void initialize(org.apache.uima.UimaContext context)
                throws org.apache.uima.resource.ResourceInitializationException

Specified by:

initialize in interface org.apache.uima.analysis_component.AnalysisComponent

Overrides:

initialize in class org.apache.uima.fit.component.JCasConsumer_ImplBase

Throws:

org.apache.uima.resource.ResourceInitializationException

process

public void process(org.apache.uima.jcas.JCas aJCas)
             throws org.apache.uima.analysis_engine.AnalysisEngineProcessException

Specified by:

process in class org.apache.uima.analysis_component.JCasAnnotator_ImplBase

Throws:

org.apache.uima.analysis_engine.AnalysisEngineProcessException

getNumThreads

protected int getNumThreads()

getInstanceList

public cc.mallet.types.InstanceList getInstanceList()