de.tudarmstadt.ukp.dkpro.core.tokit

Class BreakIteratorSegmenter

java.lang.Object

org.apache.uima.analysis_component.AnalysisComponent_ImplBase

org.apache.uima.analysis_component.Annotator_ImplBase

org.apache.uima.analysis_component.JCasAnnotator_ImplBase

org.apache.uima.fit.component.JCasAnnotator_ImplBase

de.tudarmstadt.ukp.dkpro.core.api.segmentation.SegmenterBase

de.tudarmstadt.ukp.dkpro.core.tokit.BreakIteratorSegmenter

All Implemented Interfaces:

org.apache.uima.analysis_component.AnalysisComponent

public class BreakIteratorSegmenter
extends SegmenterBase

BreakIterator segmenter.

Field Summary

Fields

Modifier and Type	Field and Description
static String	PARAM_SPLIT_AT_APOSTROPHE Per default the Java BreakIterator does not split off contractions like John's into two tokens.

Fields inherited from class de.tudarmstadt.ukp.dkpro.core.api.segmentation.SegmenterBase

PARAM_LANGUAGE, PARAM_STRICT_ZONING, PARAM_WRITE_FORM, PARAM_WRITE_SENTENCE, PARAM_WRITE_TOKEN, PARAM_ZONE_TYPES

Constructor Summary

Constructors

Constructor and Description
BreakIteratorSegmenter()

Method Summary

Modifier and Type	Method and Description
protected void	process(org.apache.uima.jcas.JCas aJCas, String text, int zoneBegin)

Methods inherited from class de.tudarmstadt.ukp.dkpro.core.api.segmentation.SegmenterBase

createSentence, createToken, createToken, createToken, getLanguage, getLocale, getZoneTypes, isEmpty, isStrictZoning, isWriteSentence, isWriteToken, limit, process, trim, trimChar

Methods inherited from class org.apache.uima.fit.component.JCasAnnotator_ImplBase

getLogger, initialize

Methods inherited from class org.apache.uima.analysis_component.JCasAnnotator_ImplBase

getRequiredCasInterface, process

Methods inherited from class org.apache.uima.analysis_component.Annotator_ImplBase

getCasInstancesRequired, hasNext, next

Methods inherited from class org.apache.uima.analysis_component.AnalysisComponent_ImplBase

batchProcessComplete, collectionProcessComplete, destroy, getContext, getResultSpecification, reconfigure, setResultSpecification

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PARAM_SPLIT_AT_APOSTROPHE

public static final String PARAM_SPLIT_AT_APOSTROPHE

Per default the Java BreakIterator does not split off contractions like John's into two tokens. When this parameter is enabled, a non-default token split is generated when an apostrophe (') is encountered.

See Also:

Constant Field Values

Constructor Detail

BreakIteratorSegmenter

public BreakIteratorSegmenter()

Method Detail

process

protected void process(org.apache.uima.jcas.JCas aJCas,
                       String text,
                       int zoneBegin)
                throws org.apache.uima.analysis_engine.AnalysisEngineProcessException

Specified by:

process in class SegmenterBase

Throws:

org.apache.uima.analysis_engine.AnalysisEngineProcessException