de.tudarmstadt.ukp.dkpro.core.api.segmentation.type

Class Token

java.lang.Object

org.apache.uima.cas.impl.FeatureStructureImpl

org.apache.uima.jcas.cas.TOP

org.apache.uima.jcas.cas.AnnotationBase

org.apache.uima.jcas.tcas.Annotation

de.tudarmstadt.ukp.dkpro.core.api.segmentation.type.Token

All Implemented Interfaces:

Cloneable, org.apache.uima.cas.AnnotationBaseFS, org.apache.uima.cas.FeatureStructure, org.apache.uima.cas.text.AnnotationFS

Direct Known Subclasses:

JapaneseToken

public class Token
extends org.apache.uima.jcas.tcas.Annotation

Token is one of the two types commonly produced by a segmenter (the other being Sentence). A Token usually represents a word, although it may be used to represent multiple tightly connected words (e.g. "New York") or parts of a word (e.g. the possessive "'s"). One may choose to split compound words into multiple tokens, e.g. ("CamelCase" -> "Camel", "Case"; "Zauberstab" -> "Zauber", "stab"). Most processing components operate on Tokens, usually within the limits of the surrounding Sentence. E.g. a part-of-speech tagger analyses each Token in a Sentence and assigns a part-of-speech to each Token.

Updated by JCasGen Tue Mar 07 16:08:28 CET 2017 XML source: /Users/bluefire/git/dkpro-core/dkpro-core-api-segmentation-asl/src/main/resources/desc/type/LexicalUnits_customized.xml

Field Summary

Fields

Modifier and Type	Field and Description
static int	type
static int	typeIndexID

Fields inherited from class org.apache.uima.jcas.cas.TOP

addr, jcasType

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	Token() Never called.
	Token(int addr, org.apache.uima.jcas.cas.TOP_Type type) Internal - constructor used by generator
	Token(org.apache.uima.jcas.JCas jcas)
	Token(org.apache.uima.jcas.JCas jcas, int begin, int end)

Method Summary

Modifier and Type	Method and Description
TokenForm	getForm() getter for form - gets Potentially normalized form of the token text that should be used instead of the covered text if set.
String	getFormValue()
String	getId() getter for id - gets If this unit had an ID in the source format from which it was imported, it may be stored here.
Lemma	getLemma() getter for lemma - gets
String	getLemmaValue()
MorphologicalFeatures	getMorph() getter for morph - gets The morphological feature associated with this token.
org.apache.uima.jcas.tcas.Annotation	getParent() getter for parent - gets the parent of this token.
POS	getPos() getter for pos - gets
String	getPosValue()
Stem	getStem() getter for stem - gets
String	getStemValue()
String	getSyntacticFunction() getter for syntacticFunction - gets
String	getText()
int	getTypeIndexID()
void	setForm(TokenForm v) setter for form - sets Potentially normalized form of the token text that should be used instead of the covered text if set.
void	setId(String v) setter for id - sets If this unit had an ID in the source format from which it was imported, it may be stored here.
void	setLemma(Lemma v) setter for lemma - sets
void	setMorph(MorphologicalFeatures v) setter for morph - sets The morphological feature associated with this token.
void	setParent(org.apache.uima.jcas.tcas.Annotation v) setter for parent - sets the parent of this token.
void	setPos(POS v) setter for pos - sets
void	setStem(Stem v) setter for stem - sets
void	setSyntacticFunction(String v) setter for syntacticFunction - sets
void	setText(String aText) Set the token text.

Methods inherited from class org.apache.uima.jcas.tcas.Annotation

getBegin, getCoveredText, getEnd, getStart, setBegin, setEnd

Methods inherited from class org.apache.uima.jcas.cas.AnnotationBase

getSofa, getView

Methods inherited from class org.apache.uima.jcas.cas.TOP

addToIndexes, addToIndexes, equals, getAddress, getCAS, getCASImpl, getLowLevelCas, hashCode, removeFromIndexes, removeFromIndexes

Methods inherited from class org.apache.uima.cas.impl.FeatureStructureImpl

clone, getavoidcollisionTypeCode, getBooleanValue, getByteValue, getDoubleValue, getFeatureValue, getFeatureValueAsString, getFloatValue, getIntValue, getLongValue, getShortValue, getStringValue, getType, prettyPrint, prettyPrint, prettyPrint, setBooleanValue, setByteValue, setDoubleValue, setFeatureValue, setFeatureValueFromString, setFloatValue, setIntValue, setLongValue, setShortValue, setStringValue, toString, toString

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.uima.cas.AnnotationBaseFS

getView

Methods inherited from interface org.apache.uima.cas.FeatureStructure

clone, equals, getBooleanValue, getByteValue, getCAS, getDoubleValue, getFeatureValue, getFeatureValueAsString, getFloatValue, getIntValue, getLongValue, getShortValue, getStringValue, getType, hashCode, setBooleanValue, setByteValue, setDoubleValue, setFeatureValue, setFeatureValueFromString, setFloatValue, setIntValue, setLongValue, setShortValue, setStringValue

Field Detail

typeIndexID

public static final int typeIndexID

type

public static final int type

Constructor Detail

Token

protected Token()

Never called. Disable default constructor

Token

public Token(int addr,
             org.apache.uima.jcas.cas.TOP_Type type)

Internal - constructor used by generator

Parameters:

addr - low level Feature Structure reference

type - the type of this Feature Structure

Token

public Token(org.apache.uima.jcas.JCas jcas)

Parameters:

jcas - JCas to which this Feature Structure belongs

Token

public Token(org.apache.uima.jcas.JCas jcas,
             int begin,
             int end)

Parameters:

jcas - JCas to which this Feature Structure belongs

begin - offset to the begin spot in the SofA

end - offset to the end spot in the SofA

Method Detail

getTypeIndexID

public int getTypeIndexID()

Overrides:

getTypeIndexID in class org.apache.uima.jcas.tcas.Annotation

Returns:

index of the type

getParent

public org.apache.uima.jcas.tcas.Annotation getParent()

getter for parent - gets the parent of this token. This feature is meant to be used in when the token participates in a constituency parse and then refers to a constituent containing this token. The type of this feature is Annotation to avoid adding a dependency on the syntax API module.

Returns:

value of the feature

setParent

public void setParent(org.apache.uima.jcas.tcas.Annotation v)

setter for parent - sets the parent of this token. This feature is meant to be used in when the token participates in a constituency parse and then refers to a constituent containing this token. The type of this feature is Annotation to avoid adding a dependency on the syntax API module.

Parameters:

v - value to set into the feature

getLemma

public Lemma getLemma()

getter for lemma - gets

Returns:

value of the feature

setLemma

public void setLemma(Lemma v)

setter for lemma - sets

Parameters:

v - value to set into the feature

getStem

public Stem getStem()

getter for stem - gets

Returns:

value of the feature

setStem

public void setStem(Stem v)

setter for stem - sets

Parameters:

v - value to set into the feature

getPos

public POS getPos()

getter for pos - gets

Returns:

value of the feature

setPos

public void setPos(POS v)

setter for pos - sets

Parameters:

v - value to set into the feature

getMorph

public MorphologicalFeatures getMorph()

getter for morph - gets The morphological feature associated with this token.

Returns:

value of the feature

setMorph

public void setMorph(MorphologicalFeatures v)

setter for morph - sets The morphological feature associated with this token.

Parameters:

v - value to set into the feature

getId

public String getId()

getter for id - gets If this unit had an ID in the source format from which it was imported, it may be stored here. IDs are typically not assigned by DKPro Core components. If an ID is present, it should be respected by writers.

Returns:

value of the feature

setId

public void setId(String v)

setter for id - sets If this unit had an ID in the source format from which it was imported, it may be stored here. IDs are typically not assigned by DKPro Core components. If an ID is present, it should be respected by writers.

Parameters:

v - value to set into the feature

getForm

public TokenForm getForm()

getter for form - gets Potentially normalized form of the token text that should be used instead of the covered text if set.

Returns:

value of the feature

setForm

public void setForm(TokenForm v)

setter for form - sets Potentially normalized form of the token text that should be used instead of the covered text if set.

Parameters:

v - value to set into the feature

getSyntacticFunction

public String getSyntacticFunction()

getter for syntacticFunction - gets

Returns:

value of the feature

setSyntacticFunction

public void setSyntacticFunction(String v)

setter for syntacticFunction - sets

Parameters:

v - value to set into the feature

getPosValue

public String getPosValue()

Returns:

the POS value if there is a POS annotation linked to this token.

getStemValue

public String getStemValue()

Returns:

the stem value if there is a Stem annotation linked to this token.

getLemmaValue

public String getLemmaValue()

Returns:

the lemma value if there is a Lemma annotation linked to this token.

getFormValue

public String getFormValue()

Returns:

the form value if there is a TokenForm annotation linked to this token.

getText

public String getText()

Returns:

the token text taking into account a TokenForm annotation that might exist.

setText

public void setText(String aText)

Set the token text. Depending on whether the text is different to the covered text a TokenForm annotation is created or not. This method can only be used if the document text has already been set. If a document text is constructed incrementally, setForm(TokenForm) has to be called manually and whether or not a form is necessary needs to be determined based on the current state of the document text being built.

Parameters:

aText - the token text.