org.dkpro.core.api.embeddings.text

Class TextFormatVectorizer

java.lang.Object

org.dkpro.core.api.embeddings.text.TextFormatVectorizer

All Implemented Interfaces:

Vectorizer

public class TextFormatVectorizer
extends Object
implements Vectorizer

A Vectorizer backed by a Map<String, float[].

Use load(File) to initialize.

Method Summary

Modifier and Type	Method and Description
boolean	contains(String token) True if the token is known by the vectorizer.
int	dimensions() The dimensionality of the embeddings
boolean	isCaseless()
static Vectorizer	load(File f) Load a text-format embeddings file (assuming no header line).
static Vectorizer	load(File embeddingsFile, boolean hasHeaderLine) Load a text-format embeddings file.
int	size() The total number of known tokens.
float[]	unknownVector() The vector for unknown tokens.
float[]	vectorize(String token) Get the vector for a token.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

load

public static Vectorizer load(File f)
                       throws IOException

Load a text-format embeddings file (assuming no header line).

Parameters:

f - the File containing the embeddings in text format

Returns:

a new TextFormatVectorizer

Throws:

IOException - if an I/O error occurs

load

public static Vectorizer load(File embeddingsFile,
                              boolean hasHeaderLine)
                       throws IOException

Load a text-format embeddings file.

Parameters:

embeddingsFile - the File containing the embeddings in text format

hasHeaderLine - if true, the first line in the file is expected to be a header line

Returns:

a new TextFormatVectorizer

Throws:

IOException - if an I/O error occurs

vectorize

public float[] vectorize(String token)

Description copied from interface: Vectorizer

Get the vector for a token. If the token is unknown, implementing classes should return the Vectorizer.unknownVector().

Specified by:

vectorize in interface Vectorizer

Parameters:

token - a token String

Returns:

a float array

contains

public boolean contains(String token)

Description copied from interface: Vectorizer

True if the token is known by the vectorizer.

Specified by:

contains in interface Vectorizer

Parameters:

token - a token String

Returns:

true if the token is known

unknownVector

public float[] unknownVector()

Description copied from interface: Vectorizer

The vector for unknown tokens.

Specified by:

unknownVector in interface Vectorizer

Returns:

a float array

dimensions

public int dimensions()

Description copied from interface: Vectorizer

The dimensionality of the embeddings

Specified by:

dimensions in interface Vectorizer

Returns:

an int

size

public int size()

Description copied from interface: Vectorizer

The total number of known tokens.

Specified by:

size in interface Vectorizer

Returns:

an int

isCaseless

public boolean isCaseless()

Specified by:

isCaseless in interface Vectorizer