opennlp.tools.ngram
Class Token

java.lang.Object
  opennlp.tools.ngram.Token

public class Token
extends java.lang.Object

Representation of a token which is shared by all instances of that token.

Method Summary

static Token	create(java.lang.String token) Creates a new Token instance.
static TokenList	create(java.lang.String[] tokenStrings) Creates a token array from the provdided string array.
boolean	equals(java.lang.Object obj)
java.lang.String	getToken() Retrives the token string.
int	hashCode()
java.lang.String	toString() Represents the token as human-readable string.

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Method Detail

getToken

public java.lang.String getToken()

Retrives the token string. Note: Do not use toString() for token text retrival.

Returns:

the token text

hashCode

public int hashCode()

Overrides:

hashCode in class java.lang.Object

equals

public boolean equals(java.lang.Object obj)

Overrides:

equals in class java.lang.Object

toString

public java.lang.String toString()

Represents the token as human-readable string.

Overrides:

toString in class java.lang.Object

create

public static Token create(java.lang.String token)

Creates a new Token instance.

Parameters:

token -

Returns:

the new token

create

public static TokenList create(java.lang.String[] tokenStrings)

Creates a token array from the provdided string array.

Parameters:

tokenStrings -

Returns:

the strings as tokens