de.tudarmstadt.ukp.dkpro.core.api.frequency

Class FrequencyCountResourceBase

java.lang.Object

org.apache.uima.resource.Resource_ImplBase

org.apache.uima.fit.component.Resource_ImplBase

de.tudarmstadt.ukp.dkpro.core.api.frequency.FrequencyCountResourceBase

All Implemented Interfaces:

FrequencyCountProvider, org.apache.uima.fit.component.ExternalResourceAware, org.apache.uima.resource.Resource

Direct Known Subclasses:

BerkeleyLmFrequencyCountProvider, TestFrequencyCountResource, Web1TFrequencyCountResource, Web1TInMemoryFrequencyCountResource

public abstract class FrequencyCountResourceBase
extends org.apache.uima.fit.component.Resource_ImplBase
implements FrequencyCountProvider

Field Summary

Fields

Modifier and Type	Field and Description
static String	PARAM_SCALE_DOWN_FACTOR Scales down the frequencies by the given factor.
protected FrequencyCountProvider	provider
protected String	scaleDownFactor

Fields inherited from interface org.apache.uima.resource.Resource

PARAM_AGGREGATE_SOFA_MAPPINGS, PARAM_CONFIG_MANAGER, PARAM_CONFIG_PARAM_SETTINGS, PARAM_EXTERNAL_OVERRIDE_SETTINGS, PARAM_PERFORMANCE_TUNING_SETTINGS, PARAM_RESOURCE_MANAGER, PARAM_UIMA_CONTEXT

Constructor Summary

Constructors

Constructor and Description
FrequencyCountResourceBase()

Method Summary

Modifier and Type	Method and Description
long	getFrequency(String phrase) Get frequency for a phrase
String	getID() Returns an ID that identifies the frequency provider
String	getLanguage() Returns the language for which the frequencies are provided.
double	getLogProbability(String phrase) Get the log probability for a phrase
Iterator<String>	getNgramIterator(int n) Returns an iterator over all ngrams of a given size in the corpus.
long	getNrOfDistinctNgrams(int n) Returns the number of distinct ngrams of a given size in the corpus.
long	getNrOfNgrams(int n) Returns the number of ngrams of a given size in the corpus.
long	getNrOfTokens()
double	getProbability(String phrase) Get the probability (=normalized frequency) for a phrase. The frequency count is normalized by the number of phrases in the corpus.
protected abstract void	initializeProvider()

Methods inherited from class org.apache.uima.fit.component.Resource_ImplBase

afterResourcesInitialized, getLogger, getResourceName, initialize

Methods inherited from class org.apache.uima.resource.Resource_ImplBase

destroy, getCasManager, getMetaData, getRelativePathResolver, getResourceManager, getUimaContext, getUimaContextAdmin, setLogger, setMetaData

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PARAM_SCALE_DOWN_FACTOR

public static final String PARAM_SCALE_DOWN_FACTOR

Scales down the frequencies by the given factor.

See Also:

Constant Field Values

scaleDownFactor

protected String scaleDownFactor

provider

protected FrequencyCountProvider provider

Constructor Detail

FrequencyCountResourceBase

public FrequencyCountResourceBase()

Method Detail

getFrequency

public long getFrequency(String phrase)
                  throws IOException

Description copied from interface: FrequencyCountProvider

Get frequency for a phrase

Specified by:

getFrequency in interface FrequencyCountProvider

Parameters:

phrase - phrase to search

Returns:

frequency of the phrase. Returns 0 if the phrase does not exist in the corpus.

Throws:

IOException - if the information cannot be retrieved.

getProbability

public double getProbability(String phrase)
                      throws IOException

Description copied from interface: FrequencyCountProvider

Get the probability (=normalized frequency) for a phrase.
The frequency count is normalized by the number of phrases in the corpus.

Specified by:

getProbability in interface FrequencyCountProvider

Parameters:

phrase - phrase to search

Returns:

probability (=normalized frequency) of the phrase. Returns null if the phrase does not exist in the corpus.

Throws:

IOException - if the information cannot be retrieved.

getLogProbability

public double getLogProbability(String phrase)
                         throws IOException

Description copied from interface: FrequencyCountProvider

Get the log probability for a phrase

Specified by:

getLogProbability in interface FrequencyCountProvider

Parameters:

phrase - phrase to search

Returns:

The log probability of the. Returns 0 if the phrase does not exist in the corpus.

Throws:

IOException - if the information cannot be retrieved.

getNrOfTokens

public long getNrOfTokens()
                   throws IOException

Specified by:

getNrOfTokens in interface FrequencyCountProvider

Returns:

The number of tokens in the corpus.

Throws:

IOException - if the information cannot be retrieved.

getNrOfNgrams

public long getNrOfNgrams(int n)
                   throws IOException

Description copied from interface: FrequencyCountProvider

Returns the number of ngrams of a given size in the corpus.

Specified by:

getNrOfNgrams in interface FrequencyCountProvider

Parameters:

n - the ngram size.

Returns:

the number of ngrams.

Throws:

IOException - if the information cannot be retrieved.

getNrOfDistinctNgrams

public long getNrOfDistinctNgrams(int n)
                           throws IOException

Description copied from interface: FrequencyCountProvider

Returns the number of distinct ngrams of a given size in the corpus.

Specified by:

getNrOfDistinctNgrams in interface FrequencyCountProvider

Parameters:

n - the ngram size.

Returns:

the number of ngrams.

Throws:

IOException - if the information cannot be retrieved.

getNgramIterator

public Iterator<String> getNgramIterator(int n)
                                  throws IOException

Description copied from interface: FrequencyCountProvider

Returns an iterator over all ngrams of a given size in the corpus.

Specified by:

getNgramIterator in interface FrequencyCountProvider

Parameters:

n - the ngram size.

Returns:

an iterator over the ngrams.

Throws:

IOException - if the information cannot be retrieved.

getLanguage

public String getLanguage()
                   throws IOException

Description copied from interface: FrequencyCountProvider

Returns the language for which the frequencies are provided.

Specified by:

getLanguage in interface FrequencyCountProvider

Returns:

the language.

Throws:

IOException - if the information cannot be retrieved.

getID

public String getID()

Description copied from interface: FrequencyCountProvider

Returns an ID that identifies the frequency provider

Specified by:

getID in interface FrequencyCountProvider

Returns:

the ID.

initializeProvider

protected abstract void initializeProvider()
                                    throws Exception

Throws:

Exception