Field Detail

• PARAM_INPUT_TYPES

  public static final String PARAM_INPUT_TYPES

  Types to generate n-grams from. Example: Token.class.getName() + "/pos/PosValue" for part-of-speech n-grams

  See Also:

  Constant Field Values

• PARAM_TARGET_LOCATION

  public static final String PARAM_TARGET_LOCATION

  Location to which the output is written.

  See Also:

  Constant Field Values

• PARAM_TARGET_ENCODING

  public static final String PARAM_TARGET_ENCODING

  Character encoding of the output data.

  See Also:

  Constant Field Values

• PARAM_MIN_NGRAM_LENGTH

  public static final String PARAM_MIN_NGRAM_LENGTH

  Minimum n-gram length. Default: 1

  See Also:

  Constant Field Values

• PARAM_MAX_NGRAM_LENGTH

  public static final String PARAM_MAX_NGRAM_LENGTH

  Maximum n-gram length. Default: 3

  See Also:

  Constant Field Values

• PARAM_LOWERCASE

  public static final String PARAM_LOWERCASE

  Create a lower case index.

  See Also:

  Constant Field Values

• PARAM_CREATE_INDEXES

  public static final String PARAM_CREATE_INDEXES

  Create the indexes that jWeb1T needs to operate. (default: true)

  See Also:

  Constant Field Values

• PARAM_MIN_FREQUENCY

  public static final String PARAM_MIN_FREQUENCY

  Specifies the minimum frequency a NGram must have to be written to the final index. The specified value is interpreted as inclusive value, the default is 1. Thus, all NGrams with a frequency of at least 1 or higher will be written.

  See Also:

  Constant Field Values

• PARAM_SPLIT_TRESHOLD

  public static final String PARAM_SPLIT_TRESHOLD

  The input file(s) is/are split into smaller files for quick access. An own file is created if the first two starting letters (or the starting letter if the word has a length of 1 character) account for at least x% of all starting letters in the input file(s). The default value for splitting a file is 1.0%. Every word that has starting characters which does not suffice the threshold is written with other words that also did not meet the threshold into an own file for miscellaneous words. A high threshold will lead to only a few, but large files and a most likely very large misc. file. A low threshold results in many small files. Use a zero or a negative value to write everything to one file.

  See Also:

  Constant Field Values

• PARAM_CONTEXT_TYPE

  public static final String PARAM_CONTEXT_TYPE

  The type being used for segments

  See Also:

  Constant Field Values

• contextType

  protected String contextType

Constructor Detail

• Web1TFormatWriter

  public Web1TFormatWriter()

Method Detail

• initialize

  public void initialize(org.apache.uima.UimaContext context)
                  throws org.apache.uima.resource.ResourceInitializationException

  Specified by:

  initialize in interface org.apache.uima.analysis_component.AnalysisComponent

  Overrides:

  initialize in class org.apache.uima.fit.component.JCasAnnotator_ImplBase

  Throws:

  org.apache.uima.resource.ResourceInitializationException

• process

  public void process(org.apache.uima.jcas.JCas jcas)
               throws org.apache.uima.analysis_engine.AnalysisEngineProcessException

  Specified by:

  process in class org.apache.uima.analysis_component.JCasAnnotator_ImplBase

  Throws:

  org.apache.uima.analysis_engine.AnalysisEngineProcessException

• collectionProcessComplete

  public void collectionProcessComplete()
                                 throws org.apache.uima.analysis_engine.AnalysisEngineProcessException

  The input files for each ngram level is read, splitted according to the frequency of the words starting letter in the files and the split files are individually sorted and consolidated.

  Specified by:

  collectionProcessComplete in interface org.apache.uima.analysis_component.AnalysisComponent

  Overrides:

  collectionProcessComplete in class org.apache.uima.analysis_component.AnalysisComponent_ImplBase

  Throws:

  org.apache.uima.analysis_engine.AnalysisEngineProcessException