org.dkpro.core.api.embeddings

Interface Vectorizer

All Known Implementing Classes:

BinaryVectorizer, TextFormatVectorizer

public interface Vectorizer

An interface for vectorizers mapping tokens to embedding vectors.

See Also:

BinaryVectorizer, TextFormatVectorizer

Method Summary

Modifier and Type	Method and Description
boolean	contains(String token) True if the token is known by the vectorizer.
int	dimensions() The dimensionality of the embeddings
boolean	isCaseless()
int	size() The total number of known tokens.
float[]	unknownVector() The vector for unknown tokens.
float[]	vectorize(String token) Get the vector for a token.

Method Detail

vectorize

float[] vectorize(String token)
           throws IOException

Get the vector for a token. If the token is unknown, implementing classes should return the unknownVector().

Parameters:

token - a token String

Returns:

a float array

Throws:

IOException - if there was an error accessing the vector file.

contains

boolean contains(String token)

True if the token is known by the vectorizer.

Parameters:

token - a token String

Returns:

true if the token is known

unknownVector

float[] unknownVector()

The vector for unknown tokens.

Returns:

a float array

dimensions

int dimensions()

The dimensionality of the embeddings

Returns:

an int

size

int size()

The total number of known tokens.

Returns:

an int

isCaseless

boolean isCaseless()