de.tudarmstadt.ukp.dkpro.core.tokit

Class TokenMerger

java.lang.Object

org.apache.uima.analysis_component.AnalysisComponent_ImplBase

org.apache.uima.analysis_component.Annotator_ImplBase

org.apache.uima.analysis_component.JCasAnnotator_ImplBase

org.apache.uima.fit.component.JCasAnnotator_ImplBase

de.tudarmstadt.ukp.dkpro.core.tokit.TokenMerger

All Implemented Interfaces:

org.apache.uima.analysis_component.AnalysisComponent

public class TokenMerger
extends org.apache.uima.fit.component.JCasAnnotator_ImplBase

Merges any Tokens that are covered by a given annotation type. E.g. this component can be used to create a single tokens from all tokens that constitute a multi-token named entity.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TokenMerger.LemmaMode

Field Summary

Fields

Modifier and Type	Field and Description
protected String	language
static String	PARAM_ANNOTATION_TYPE Annotation type for which tokens should be merged.
static String	PARAM_CONSTRAINT A constraint on the annotations that should be considered in form of a JXPath statement.
static String	PARAM_CPOS_VALUE Set a new coarse POS value for the new merged token.
static String	PARAM_LANGUAGE Use this language instead of the document language to resolve the model and tag set mapping.
static String	PARAM_LEMMA_MODE Configure what should happen to the lemma of the merged tokens.
static String	PARAM_POS_MAPPING_LOCATION Override the tagset mapping.
static String	PARAM_POS_TYPE Set a new POS tag for the new merged token.
static String	PARAM_POS_VALUE Set a new POS value for the new merged token.
protected String	posMappingLocation

Constructor Summary

Constructors

Constructor and Description
TokenMerger()

Method Summary

Modifier and Type	Method and Description
void	initialize(org.apache.uima.UimaContext aContext)
void	process(org.apache.uima.jcas.JCas aJCas)

Methods inherited from class org.apache.uima.fit.component.JCasAnnotator_ImplBase

getLogger

Methods inherited from class org.apache.uima.analysis_component.JCasAnnotator_ImplBase

getRequiredCasInterface, process

Methods inherited from class org.apache.uima.analysis_component.Annotator_ImplBase

getCasInstancesRequired, hasNext, next

Methods inherited from class org.apache.uima.analysis_component.AnalysisComponent_ImplBase

batchProcessComplete, collectionProcessComplete, destroy, getContext, getResultSpecification, reconfigure, setResultSpecification

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PARAM_ANNOTATION_TYPE

public static final String PARAM_ANNOTATION_TYPE

Annotation type for which tokens should be merged.

See Also:

Constant Field Values

PARAM_CONSTRAINT

public static final String PARAM_CONSTRAINT

A constraint on the annotations that should be considered in form of a JXPath statement. Example: set PARAM_ANNOTATION_TYPE to a NamedEntity type and set the PARAM_CONSTRAINT to ".[value = 'LOCATION']" to merge only tokens that are part of a location named entity.

See Also:

Constant Field Values

PARAM_LEMMA_MODE

public static final String PARAM_LEMMA_MODE

Configure what should happen to the lemma of the merged tokens. It is possible to JOIN the lemmata to a single lemma (space separated), to REMOVE the lemma or LEAVE the lemma of the first token as-is.

See Also:

Constant Field Values

PARAM_POS_VALUE

public static final String PARAM_POS_VALUE

Set a new POS value for the new merged token. This is the actual tag set value and is subject to tagset mapping. For example when merging tokens for named entities, the new POS value may be set to "NNP" (English/Penn Treebank Tagset).

See Also:

Constant Field Values

PARAM_CPOS_VALUE

public static final String PARAM_CPOS_VALUE

Set a new coarse POS value for the new merged token. This is the actual tag set value and is subject to tagset mapping. For example when merging tokens for named entities, the new POS value may be set to "NNP" (English/Penn Treebank Tagset).

See Also:

Constant Field Values

PARAM_POS_TYPE

public static final String PARAM_POS_TYPE

Set a new POS tag for the new merged token. This is the mapped type. If this is specified, tag set mapping will not be performed. This parameter has no effect unless PARAM_POS_VALUE is also set.

See Also:

Constant Field Values

PARAM_LANGUAGE

public static final String PARAM_LANGUAGE

Use this language instead of the document language to resolve the model and tag set mapping.

See Also:

Constant Field Values

language

protected String language

PARAM_POS_MAPPING_LOCATION

public static final String PARAM_POS_MAPPING_LOCATION

Override the tagset mapping.

See Also:

Constant Field Values

posMappingLocation

protected String posMappingLocation

Constructor Detail

TokenMerger

public TokenMerger()

Method Detail

initialize

public void initialize(org.apache.uima.UimaContext aContext)
                throws org.apache.uima.resource.ResourceInitializationException

Specified by:

initialize in interface org.apache.uima.analysis_component.AnalysisComponent

Overrides:

initialize in class org.apache.uima.fit.component.JCasAnnotator_ImplBase

Throws:

org.apache.uima.resource.ResourceInitializationException

process

public void process(org.apache.uima.jcas.JCas aJCas)
             throws org.apache.uima.analysis_engine.AnalysisEngineProcessException

Specified by:

process in class org.apache.uima.analysis_component.JCasAnnotator_ImplBase

Throws:

org.apache.uima.analysis_engine.AnalysisEngineProcessException