View source on GitHub |
DataLoader for object detector.
tflite_model_maker.object_detector.DataLoader(
tfrecord_file_patten, size, label_map, annotations_json_file=None
)
Used in the notebooks
Used in the tutorials |
---|
Methods
from_cache
@classmethod
from_cache( cache_prefix )
Loads the data from cache.
Args | |
---|---|
cache_prefix
|
The cache prefix including the cache directory and the cache prefix filename, e.g: '/tmp/cache/train'. |
Returns | |
---|---|
ObjectDetectorDataLoader object. |
from_csv
@classmethod
from_csv( filename: str, images_dir: Optional[str] = None, delimiter: str = ',', quotechar: str = '\'", num_shards: int = 10, max_num_images: Optional[int] = None, cache_dir: Optional[str] = None, cache_prefix_filename: Optional[str] = None ) -> List[Optional[DetectorDataLoader]]
Loads the data from the csv file.
The csv format is shown in https://cloud.google.com/vision/automl/object-detection/docs/csv-format We supports bounding box with 2 vertices for now. We support the files in the local machine as well.
Args | |
---|---|
filename
|
Name of the csv file. |
images_dir
|
Path to directory that store raw images. If None, the image path in the csv file is the path to Google Cloud Storage or the absolute path in the local machine. |
delimiter
|
Character used to separate fields. |
quotechar
|
Character used to quote fields containing special characters. |
num_shards
|
Number of shards for output file. |
max_num_images
|
Max number of imags to process. |
cache_dir
|
The cache directory to save TFRecord, metadata and json file. When cache_dir is None, a temporary folder will be created and will not be removed automatically after training which makes it can be used later. |
cache_prefix_filename
|
The cache prefix filename. If None, will
automatically generate it based on filename .
|
Returns | |
---|---|
train_data, validation_data, test_data which are ObjectDetectorDataLoader objects. Can be None if without such data. |
from_pascal_voc
@classmethod
from_pascal_voc( images_dir: str, annotations_dir: str, label_map: Union[List[str], Dict[int, str], str], annotation_filenames: Optional[Collection[str]] = None, ignore_difficult_instances: bool = False, num_shards: int = 100, max_num_images: Optional[int] = None, cache_dir: Optional[str] = None, cache_prefix_filename: Optional[str] = None ) -> DetectorDataLoader
Loads from dataset with PASCAL VOC format.
Refer to https://towardsdatascience.com/coco-data-format-for-object-detection-a4c5eaf518c5 for the description of PASCAL VOC data format.
LabelImg Tool (https://github.com/tzutalin/labelImg) can annotate the image and save annotations as XML files in PASCAL VOC data format.
Annotations are in the folder: annotations_dir
.
Raw images are in the foloder: images_dir
.
Args | |
---|---|
images_dir
|
Path to directory that store raw images. |
annotations_dir
|
Path to the annotations directory. |
label_map
|
Variable shows mapping label integers ids to string label
names. 0 is the reserved key for background . Label names can't be
duplicated. Supported format: 1. Dict, map label integers ids to string
label names, e.g.
{1: 'person', 2: 'notperson'}. 2. List, a list of label names. e.g.
['person', 'notperson'] which is
the same as setting label_map={1: 'person', 2: 'notperson'}.
|
annotation_filenames
|
Collection of annotation filenames (strings) to be
loaded. For instance, if there're 3 annotation files [0.xml, 1.xml,
2.xml] in annotations_dir , setting annotation_filenames=['0', '1']
makes this method only load [0.xml, 1.xml].
|
ignore_difficult_instances
|
Whether to ignore difficult instances.
difficult can be set inside object item in the annotation xml file.
|
num_shards
|
Number of shards for output file. |
max_num_images
|
Max number of imags to process. |
cache_dir
|
The cache directory to save TFRecord, metadata and json file. When cache_dir is not set, a temporary folder will be created and will not be removed automatically after training which makes it can be used later. |
cache_prefix_filename
|
The cache prefix filename. If not set, will
automatically generate it based on image_dir , annotations_dir and
annotation_filenames .
|
Returns | |
---|---|
ObjectDetectorDataLoader object. |
gen_dataset
gen_dataset(
model_spec, batch_size=None, is_training=False, use_fake_data=False
)
Generate a batched tf.data.Dataset for training/evaluation.
Args | |
---|---|
model_spec
|
Specification for the model. |
batch_size
|
A integer, the returned dataset will be batched by this size. |
is_training
|
A boolean, when True, the returned dataset will be optionally shuffled and repeated as an endless dataset. |
use_fake_data
|
Use fake input. |
Returns | |
---|---|
A TF dataset ready to be consumed by Keras model. |
split
split(
fraction
)
This function isn't implemented for the object detection task.
__len__
__len__()