tflite_support.metadata_writers.metadata_info.SentencePieceTokenizerMd

A container for the sentence piece tokenizer 1 metadata information.

View aliases

Main aliases

tflite_support.metadata_writers.audio_classifier.metadata_info.SentencePieceTokenizerMd, tflite_support.metadata_writers.audio_classifier.metadata_writer.metadata_info.SentencePieceTokenizerMd, tflite_support.metadata_writers.bert_nl_classifier.metadata_info.SentencePieceTokenizerMd, tflite_support.metadata_writers.bert_nl_classifier.metadata_writer.metadata_info.SentencePieceTokenizerMd, tflite_support.metadata_writers.image_classifier.metadata_info.SentencePieceTokenizerMd, tflite_support.metadata_writers.image_classifier.metadata_writer.metadata_info.SentencePieceTokenizerMd, tflite_support.metadata_writers.image_segmenter.metadata_info.SentencePieceTokenizerMd, tflite_support.metadata_writers.image_segmenter.metadata_writer.metadata_info.SentencePieceTokenizerMd, tflite_support.metadata_writers.nl_classifier.metadata_info.SentencePieceTokenizerMd, tflite_support.metadata_writers.nl_classifier.metadata_writer.metadata_info.SentencePieceTokenizerMd, tflite_support.metadata_writers.object_detector.metadata_info.SentencePieceTokenizerMd, tflite_support.metadata_writers.object_detector.metadata_writer.metadata_info.SentencePieceTokenizerMd

tflite_support.metadata_writers.metadata_info.SentencePieceTokenizerMd(
    sentence_piece_model_path: str, vocab_file_path: Optional[str] = None
)

Args
sentence_piece_model_path	path to the sentence piece model file.
vocab_file_path	path to the vocabulary file.

Methods

create_metadata

View source

create_metadata() -> tflite_support.metadata_schema_py_generated.ProcessUnitT

Creates the sentence piece tokenizer metadata based on the information.

Returns
A Flatbuffers Python object of the sentence piece tokenizer metadata.