de.tudarmstadt.ukp.dkpro.core.frequency

Class Web1TProviderBase

java.lang.Object

de.tudarmstadt.ukp.dkpro.core.api.frequency.provider.FrequencyCountProviderBase

de.tudarmstadt.ukp.dkpro.core.frequency.Web1TProviderBase

All Implemented Interfaces:

FrequencyCountProvider

Direct Known Subclasses:

Web1TFileAccessProvider, Web1TInMemoryProvider

public abstract class Web1TProviderBase
extends FrequencyCountProviderBase

Field Summary

Fields

Modifier and Type	Field and Description
protected String	basePath
static String	BOS
static String	EOS
protected String	language
protected com.googlecode.jweb1t.Searcher	searcher

Constructor Summary

Constructors

Constructor and Description
Web1TProviderBase()

Method Summary

Modifier and Type	Method and Description
protected long	getFrequencyFromProvider(String phrase)
String	getLanguage() Returns the language for which the frequencies are provided.
Iterator<String>	getNgramIterator(int n) Returns an iterator over all ngrams of a given size in the corpus.
long	getNrOfDistinctNgrams(int n) Returns the number of distinct ngrams of a given size in the corpus.
long	getNrOfNgrams(int n) Returns the number of ngrams of a given size in the corpus.
long	getNrOfTokens()

Methods inherited from class de.tudarmstadt.ukp.dkpro.core.api.frequency.provider.FrequencyCountProviderBase

getFrequency, getID, getLogLikelihood, getLogProbability, getProbability, getScaleDownFactor, setScaleDownFactor

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

BOS

public static final String BOS

See Also:

Constant Field Values

EOS

public static final String EOS

See Also:

Constant Field Values

searcher

protected com.googlecode.jweb1t.Searcher searcher

basePath

protected String basePath

language

protected String language

Constructor Detail

Web1TProviderBase

public Web1TProviderBase()

Method Detail

getNrOfNgrams

public long getNrOfNgrams(int n)

Description copied from interface: FrequencyCountProvider

Returns the number of ngrams of a given size in the corpus.

Parameters:

n - the ngram size.

Returns:

the number of ngrams.

getNrOfTokens

public long getNrOfTokens()

Returns:

The number of tokens in the corpus.

getNrOfDistinctNgrams

public long getNrOfDistinctNgrams(int n)

Description copied from interface: FrequencyCountProvider

Returns the number of distinct ngrams of a given size in the corpus.

Parameters:

n - the ngram size.

Returns:

the number of ngrams.

getFrequencyFromProvider

protected long getFrequencyFromProvider(String phrase)
                                 throws IOException

Specified by:

getFrequencyFromProvider in class FrequencyCountProviderBase

Throws:

IOException

getNgramIterator

public Iterator<String> getNgramIterator(int n)
                                  throws IOException

Description copied from interface: FrequencyCountProvider

Returns an iterator over all ngrams of a given size in the corpus.

Parameters:

n - the ngram size.

Returns:

an iterator over the ngrams.

Throws:

IOException - if the information cannot be retrieved.

getLanguage

public String getLanguage()

Description copied from interface: FrequencyCountProvider

Returns the language for which the frequencies are provided.

Returns:

the language.