de.tudarmstadt.ukp.dkpro.core.textnormalizer.annotations

Class TrailingCharacterRemover

java.lang.Object

org.apache.uima.analysis_component.AnalysisComponent_ImplBase

org.apache.uima.analysis_component.Annotator_ImplBase

org.apache.uima.analysis_component.JCasAnnotator_ImplBase

org.apache.uima.fit.component.JCasAnnotator_ImplBase

de.tudarmstadt.ukp.dkpro.core.textnormalizer.annotations.TrailingCharacterRemover

All Implemented Interfaces:

org.apache.uima.analysis_component.AnalysisComponent

public class TrailingCharacterRemover
extends org.apache.uima.fit.component.JCasAnnotator_ImplBase

Removing trailing character (sequences) from tokens, e.g. punctuation.

Field Summary

Fields

Modifier and Type	Field and Description
static String	PARAM_MIN_TOKEN_LENGTH All tokens that are shorter than the minimum token length after removing trailing chars are completely removed.
static String	PARAM_PATTERN A regex to be trimmed from the end of tokens.

Constructor Summary

Constructors

Constructor and Description
TrailingCharacterRemover()

Method Summary

Modifier and Type	Method and Description
void	initialize(org.apache.uima.UimaContext context)
void	process(org.apache.uima.jcas.JCas aJCas)

Methods inherited from class org.apache.uima.fit.component.JCasAnnotator_ImplBase

getLogger

Methods inherited from class org.apache.uima.analysis_component.JCasAnnotator_ImplBase

getRequiredCasInterface, process

Methods inherited from class org.apache.uima.analysis_component.Annotator_ImplBase

getCasInstancesRequired, hasNext, next

Methods inherited from class org.apache.uima.analysis_component.AnalysisComponent_ImplBase

batchProcessComplete, collectionProcessComplete, destroy, getContext, getResultSpecification, reconfigure, setResultSpecification

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PARAM_PATTERN

public static final String PARAM_PATTERN

A regex to be trimmed from the end of tokens.

Default: "[\\Q,-“^»*’()&/\"'©§'—«·=\\E0-9A-Z]+" (remove punctuations, special characters and capital letters).

See Also:

Constant Field Values

PARAM_MIN_TOKEN_LENGTH

public static final String PARAM_MIN_TOKEN_LENGTH

All tokens that are shorter than the minimum token length after removing trailing chars are completely removed. By default (1), empty tokens are removed. Set to 0 or a negative value if no tokens should be removed.

Shorter tokens that do not have trailing chars removed are always retained, regardless of their length.

See Also:

Constant Field Values

Constructor Detail

TrailingCharacterRemover

public TrailingCharacterRemover()

Method Detail

initialize

public void initialize(org.apache.uima.UimaContext context)
                throws org.apache.uima.resource.ResourceInitializationException

Specified by:

initialize in interface org.apache.uima.analysis_component.AnalysisComponent

Overrides:

initialize in class org.apache.uima.fit.component.JCasAnnotator_ImplBase

Throws:

org.apache.uima.resource.ResourceInitializationException

process

public void process(org.apache.uima.jcas.JCas aJCas)
             throws org.apache.uima.analysis_engine.AnalysisEngineProcessException

Specified by:

process in class org.apache.uima.analysis_component.JCasAnnotator_ImplBase

Throws:

org.apache.uima.analysis_engine.AnalysisEngineProcessException