The WordTokenization type exposes the following members.
Constructors
| Name | Description | |
|---|---|---|
 | WordTokenization |
Construct a new tokenization.
|
Methods
| Name | Description | |
|---|---|---|
| AddChangeListener | Overloaded. |
| Equals | (Inherited from Object.) |
 | Finalize |
Allows an Object to attempt to free resources and perform other cleanup operations before the Object is reclaimed by garbage collection.
(Inherited from Object.) |
| GetHashCode |
Serves as a hash function for a particular type.
(Inherited from Object.) |
| GetType |
Gets the Type of the current instance.
(Inherited from Object.) |
| IsUnchanging |
A particular ChangeType can never be raised by this Changeable.
(Inherited from Unchangeable.) |
| MemberwiseClone |
Creates a shallow copy of the current Object.
(Inherited from Object.) |
| ParseString |
Parse a string.
|
| ParseToken |
Returns the symbol for a single token.
|
| RemoveChangeListener | Overloaded. |
| SplitString |
Split up a string.
|
| TokenizeSymbol |
Return a token representing a single symbol.
|
| TokenizeSymbolList |
Return a string representation of a list of symbols.
|
| ToString | (Inherited from Object.) |
Properties
| Name | Description | |
|---|---|---|
 | Alphabet |
The alphabet to which this tokenization applies.
|
| Annotation |
Should return the associated annotation object.
|
| TokenType |
Determine the style of tokenization represented by this object.
|
