Module: tf.keras.preprocessing.text

Utilities for text input preprocessing.

Deprecated: tf.keras.preprocessing.text APIs are not recommended for new code. Prefer tf.keras.utils.text_dataset_from_directory and tf.keras.layers.TextVectorization which provide a more efficient approach for preprocessing text input. For an introduction to these APIs, see the text loading tutorial and preprocessing layer guide.

Classes

class Tokenizer: Text tokenization utility class.

Functions

hashing_trick(...): Converts a text to a sequence of indexes in a fixed-size hashing space.

one_hot(...): One-hot encodes a text into a list of word indexes of size n.

text_to_word_sequence(...): Converts a text to a sequence of words (or tokens).

tokenizer_from_json(...): Parses a JSON tokenizer configuration and returns a tokenizer instance.