View source on GitHub |
DataLoader class for Text Searcher.
Inherits From: DataLoader
tflite_model_maker.searcher.TextDataLoader(
embedder: text_embedder.TextEmbedder
) -> None
Used in the notebooks
Used in the tutorials |
---|
Args | |
---|---|
embedder
|
Embedder to generate embedding from raw input image. |
Methods
append
append(
data_loader: 'DataLoader'
) -> None
Appends the dataset.
Don't check if embedders from the two data loader are the same in this function. Users are responsible to keep the embedder identical.
Args | |
---|---|
data_loader
|
The data loader in which the data will be appended. |
create
@classmethod
create( text_embedder_path: str, l2_normalize: bool = False ) -> 'DataLoader'
Creates DataLoader for the Text Searcher task.
Args | |
---|---|
text_embedder_path
|
Path to the ".tflite" text embedder model. case and L2 norm is thus achieved through TF Lite inference. |
l2_normalize
|
Whether to normalize the returned feature vector with L2 norm. Use this option only if the model does not already contain a native L2_NORMALIZATION TF Lite Op. In most cases, this is already the case and L2 norm is thus achieved through TF Lite inference. |
Returns | |
---|---|
DataLoader object created for the Text Searcher task. |
load_from_csv
load_from_csv(
path: str,
text_column: str,
metadata_column: str,
delimiter: str = ',',
quotechar: str = '\'"
) -> None
Loads text data from csv file that includes a "header" line with titles.
Users can load text from different csv files one by one. For instance,
# Creates data_loader instance.
data_loader = text_searcher_dataloader.DataLoader.create(tflite_path)
# Loads text, first from `text_path1` and secondly from `text_path2`.
data_loader.load_from_csv(
text_path1, text_column='text', metadata_column='metadata')
data_loader.load_from_csv(
text_path2, text_column='text', metadata_column='metadata')
Args | |
---|---|
path
|
Text csv file path to be loaded. |
text_column
|
Column name for input text. |
metadata_column
|
Column name for user metadata associated with each input text. |
delimiter
|
Character used to separate fields. |
quotechar
|
Character used to quote fields containing special characters. |
__len__
__len__()