Performs object detection on images.
The API expects a TFLite model with TFLite Model Metadata..
The API supports models with one image input tensor and four output tensors. To be more specific, here are the requirements.
- Input image tensor (
kTfLiteUInt8
/kTfLiteFloat32
)- image input of size
[batch x height x width x channels]
. - batch inference is not supported (
batch
is required to be 1). - only RGB inputs are supported (
channels
is required to be 3). - if type is
kTfLiteFloat32
, NormalizationOptions are required to be attached to the metadata for input normalization.
- image input of size
DetectionPostProcess
op, i.e:
- Location tensor (
kTfLiteFloat32
):- tensor of size
[1 x num_results x 4]
, the inner array representing bounding boxes in the form [top, left, right, bottom]. BoundingBoxProperties
are required to be attached to the metadata and must specifytype=BOUNDARIES
andcoordinate_type=RATIO
.
- tensor of size
kTfLiteFloat32
):
- tensor of size
[1 x num_results]
, each value representing the integer index of a class. - if label maps are attached to the metadata as
TENSOR_VALUE_LABELS
associated files, they are used to convert the tensor values into labels.
kTfLiteFloat32
):
- tensor of size
[1 x num_results]
, each value representing the score of the detected object.
kTfLiteFloat32
):
- integer num_results as a tensor of size
[1]
.
An example of such model can be found on TensorFlow Hub..
Nested Classes
class | ObjectDetector.ObjectDetectorOptions | Options for setting up an ObjectDetector. |
Public Methods
Inherited Methods
Public Methods
public static ObjectDetector createFromBuffer (ByteBuffer modelBuffer)
Creates an ObjectDetector
instance with a model buffer and the default ObjectDetector.ObjectDetectorOptions
.
Parameters
modelBuffer | a direct ByteBuffer or a MappedByteBuffer of the detection
model |
---|
Throws
IllegalArgumentException | if the model buffer is not a direct ByteBuffer or a
MappedByteBuffer * @throws IllegalStateException if there is an internal error |
---|---|
RuntimeException | if there is an otherwise unspecified error |
public static ObjectDetector createFromBufferAndOptions (ByteBuffer modelBuffer, ObjectDetector.ObjectDetectorOptions options)
Creates an ObjectDetector
instance with a model buffer and ObjectDetector.ObjectDetectorOptions
.
Parameters
modelBuffer | a direct ByteBuffer or a MappedByteBuffer of the detection
model |
---|---|
options |
Throws
IllegalArgumentException | if the model buffer is not a direct ByteBuffer or a
MappedByteBuffer |
---|---|
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
public static ObjectDetector createFromFile (Context context, String modelPath)
Creates an ObjectDetector
instance from the default ObjectDetector.ObjectDetectorOptions
.
Parameters
context | |
---|---|
modelPath | path to the detection model with metadata in the assets |
Throws
IOException | if an I/O error occurs when loading the tflite model |
---|---|
IllegalArgumentException | if an argument is invalid |
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
public static ObjectDetector createFromFile (File modelFile)
Creates an ObjectDetector
instance from the default ObjectDetector.ObjectDetectorOptions
.
Parameters
modelFile | the detection model File instance |
---|
Throws
IOException | if an I/O error occurs when loading the tflite model |
---|---|
IllegalArgumentException | if an argument is invalid |
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
public static ObjectDetector createFromFileAndOptions (Context context, String modelPath, ObjectDetector.ObjectDetectorOptions options)
Creates an ObjectDetector
instance from ObjectDetector.ObjectDetectorOptions
.
Parameters
context | |
---|---|
modelPath | path to the detection model with metadata in the assets |
options |
Throws
IOException | if an I/O error occurs when loading the tflite model |
---|---|
IllegalArgumentException | if an argument is invalid |
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
public static ObjectDetector createFromFileAndOptions (File modelFile, ObjectDetector.ObjectDetectorOptions options)
Creates an ObjectDetector
instance from ObjectDetector.ObjectDetectorOptions
.
Parameters
modelFile | the detection model File instance |
---|---|
options |
Throws
IOException | if an I/O error occurs when loading the tflite model |
---|---|
IllegalArgumentException | if an argument is invalid |
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
public List<Detection> detect (MlImage image)
Performs actual detection on the provided MlImage
.
Parameters
image | an MlImage object that represents an image |
---|
Throws
IllegalStateException | if there is an internal error |
---|---|
RuntimeException | if there is an otherwise unspecified error |
IllegalArgumentException | if the storage type or format of the image is unsupported |
public List<Detection> detect (TensorImage image, ImageProcessingOptions options)
Performs actual detection on the provided image.
ObjectDetector
supports the following TensorImage
color space types:
ObjectDetector
supports the following options:
- image rotation (through
ImageProcessingOptions.Builder.setOrientation(ImageProcessingOptions.Orientation)
). It defaults toImageProcessingOptions.Orientation.TOP_LEFT
.
Parameters
image | a UINT8 TensorImage object that represents an RGB or YUV image |
---|---|
options | the options to configure how to preprocess the image |
Throws
IllegalStateException | if there is an internal error |
---|---|
RuntimeException | if there is an otherwise unspecified error |
IllegalArgumentException | if the color space type of image is unsupported |
public List<Detection> detect (MlImage image, ImageProcessingOptions options)
Performs actual detection on the provided MlImage
with ImageProcessingOptions
.
ObjectDetector
supports the following options:
- image rotation (through
ImageProcessingOptions.Builder.setOrientation(ImageProcessingOptions.Orientation)
). It defaults toImageProcessingOptions.Orientation.TOP_LEFT
.MlImage.getRotation()
is not effective.
Parameters
image | an MlImage object that represents an image |
---|---|
options | the options to configure how to preprocess the image |
Throws
IllegalStateException | if there is an internal error |
---|---|
RuntimeException | if there is an otherwise unspecified error |
IllegalArgumentException | if the storage type or format of the image is unsupported |
public List<Detection> detect (TensorImage image)
Performs actual detection on the provided image.
ObjectDetector
supports the following TensorImage
color space types:
Parameters
image | a UINT8 TensorImage object that represents an RGB or YUV image |
---|
Throws
IllegalStateException | if there is an internal error |
---|---|
RuntimeException | if there is an otherwise unspecified error |
IllegalArgumentException | if the color space type of image is unsupported |