de.tudarmstadt.ukp.dkpro.core.tokit

Classes

• AnnotationByLengthFilter
• BreakIteratorSegmenter
• CamelCaseTokenSegmenter
• GermanSeparatedParticleAnnotator
• LineBasedSentenceSegmenter
• ParagraphSplitter
• PatternBasedTokenSegmenter
• RegexTokenizer
• TokenMerger
• TokenTrimmer
• WhitespaceTokenizer

Enums

• TokenMerger.LemmaMode