PatternBasedTokenSegmenter (DKPro Core - Master Aggregator 1.7.0 API)

java.lang.Object
- org.apache.uima.analysis_component.AnalysisComponent_ImplBase
- - org.apache.uima.analysis_component.Annotator_ImplBase
  - - org.apache.uima.analysis_component.JCasAnnotator_ImplBase
    - - org.apache.uima.fit.component.JCasAnnotator_ImplBase
      - de.tudarmstadt.ukp.dkpro.core.tokit.PatternBasedTokenSegmenter

All Implemented Interfaces:

org.apache.uima.analysis_component.AnalysisComponent
```
public class PatternBasedTokenSegmenter
extends org.apache.uima.fit.component.JCasAnnotator_ImplBase
```
Split up existing tokens again at particular split-chars. The prefix states whether the split chars should be added as separate Tokens. If the INCLUDE_PREFIX precedes the split pattern, the pattern is included. Consequently, patterns following the EXCLUDE_PREFIX, will not be added as a Token.

Author:

Richard Eckart de Castilho

Field Summary

Fields
Modifier and Type	Field and Description
`static String`	`EXCLUDE_PREFIX`
`static String`	`INCLUDE_PREFIX`
`static String`	`PARAM_DELETE_COVER` Wether to remove the original token.
`static String`	`PARAM_PATTERNS` A list of regular expressions, prefixed with `INCLUDE_PREFIX` or `EXCLUDE_PREFIX`.

Constructor Summary

Constructors
Constructor and Description

PatternBasedTokenSegmenter()

Constructors
Constructor and Description
`PatternBasedTokenSegmenter()`

Method Summary

Methods
Modifier and Type Method and Description

void initialize(org.apache.uima.UimaContext aContext)

void process(org.apache.uima.jcas.JCas aJCas)
- Methods inherited from class org.apache.uima.fit.component.JCasAnnotator_ImplBase
  getLogger
- Methods inherited from class org.apache.uima.analysis_component.JCasAnnotator_ImplBase
  getRequiredCasInterface, process
- Methods inherited from class org.apache.uima.analysis_component.Annotator_ImplBase
  getCasInstancesRequired, hasNext, next
- Methods inherited from class org.apache.uima.analysis_component.AnalysisComponent_ImplBase
  batchProcessComplete, collectionProcessComplete, destroy, getContext, getResultSpecification, reconfigure, setResultSpecification
- Methods inherited from class java.lang.Object
  clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods
Modifier and Type	Method and Description
`void`	`initialize(org.apache.uima.UimaContext aContext)`
`void`	`process(org.apache.uima.jcas.JCas aJCas)`

- Field Detail
  - INCLUDE_PREFIX
```
public static final String INCLUDE_PREFIX
```
    See Also:
    Constant Field Values
  - EXCLUDE_PREFIX
```
public static final String EXCLUDE_PREFIX
```
    See Also:
    Constant Field Values
  - PARAM_DELETE_COVER
```
public static final String PARAM_DELETE_COVER
```
    Wether to remove the original token. Default: true
    
    See Also:
    Constant Field Values
  - PARAM_PATTERNS
```
public static final String PARAM_PATTERNS
```
    A list of regular expressions, prefixed with INCLUDE_PREFIX or EXCLUDE_PREFIX. If neither of the prefixes is used, EXCLUDE_PREFIX is assumed.
    
    See Also:
    Constant Field Values
- Constructor Detail
  - PatternBasedTokenSegmenter
```
public PatternBasedTokenSegmenter()
```
- Method Detail
  - initialize
```
public void initialize(org.apache.uima.UimaContext aContext)
                throws org.apache.uima.resource.ResourceInitializationException
```
    Specified by:
    
    initialize in interface org.apache.uima.analysis_component.AnalysisComponent
    
    Overrides:
    
    initialize in class org.apache.uima.fit.component.JCasAnnotator_ImplBase
    
    Throws:
    
    org.apache.uima.resource.ResourceInitializationException
  - process
```
public void process(org.apache.uima.jcas.JCas aJCas)
             throws org.apache.uima.analysis_engine.AnalysisEngineProcessException
```
    Specified by:
    
    process in class org.apache.uima.analysis_component.JCasAnnotator_ImplBase
    
    Throws:
    
    org.apache.uima.analysis_engine.AnalysisEngineProcessException

Class PatternBasedTokenSegmenter

Field Summary

Constructor Summary

Method Summary

Methods inherited from class org.apache.uima.fit.component.JCasAnnotator_ImplBase

Methods inherited from class org.apache.uima.analysis_component.JCasAnnotator_ImplBase

Methods inherited from class org.apache.uima.analysis_component.Annotator_ImplBase

Methods inherited from class org.apache.uima.analysis_component.AnalysisComponent_ImplBase

Methods inherited from class java.lang.Object

Field Detail

INCLUDE_PREFIX

EXCLUDE_PREFIX

PARAM_DELETE_COVER

PARAM_PATTERNS

Constructor Detail

PatternBasedTokenSegmenter

Method Detail

initialize

process