Performs similarity search on images.
The API expects a TFLite model with optional, but strongly recommended, TFLite Model Metadata..
- Input image tensor (
kTfLiteUInt8
/kTfLiteFloat32
)- image input of size
[batch x height x width x channels]
. - batch inference is not supported (
batch
is required to be 1). - only RGB inputs are supported (
channels
is required to be 3). - if type is
kTfLiteFloat32
, NormalizationOptions are required to be attached to the metadata for input normalization.
- image input of size
kTfLiteUInt8
/kTfLiteFloat32
)
N
components corresponding to theN
dimensions of the returned feature vector for this output layer.- Either 2 or 4 dimensions, i.e.
[1 x N]
or[1 x 1 x 1 x N]
.
TODO(b/180502532): add pointer to example model.
TODO(b/222671076): add factory create methods without options, such as `createFromFile`, once the single file format (index file packed in the model) is supported.
Nested Classes
class | ImageSearcher.ImageSearcherOptions | Options for setting up an ImageSearcher. |
Public Methods
Inherited Methods
Public Methods
public static ImageSearcher createFromBufferAndOptions (ByteBuffer modelBuffer, ImageSearcher.ImageSearcherOptions options)
Creates an ImageSearcher
instance with a model buffer and ImageSearcher.ImageSearcherOptions
.
Parameters
modelBuffer | a direct ByteBuffer or a MappedByteBuffer of the search
model |
---|---|
options |
Throws
IllegalArgumentException | if the model buffer is not a direct ByteBuffer or a
MappedByteBuffer |
---|---|
IOException | if an I/O error occurs when loading the index file |
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
public static ImageSearcher createFromBufferAndOptionsImpl (ByteBuffer modelBuffer, ImageSearcher.ImageSearcherOptions options, int indexFd)
Parameters
modelBuffer | |
---|---|
options | |
indexFd |
public static ImageSearcher createFromFileAndOptions (Context context, String modelPath, ImageSearcher.ImageSearcherOptions options)
Creates an ImageSearcher
instance from ImageSearcher.ImageSearcherOptions
.
Parameters
context | |
---|---|
modelPath | path of the search model with metadata in the assets |
options |
Throws
IOException | if an I/O error occurs when loading the tflite model or the index file |
---|---|
IllegalArgumentException | if an argument is invalid |
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
public static ImageSearcher createFromFileAndOptions (File modelFile, ImageSearcher.ImageSearcherOptions options)
Creates an ImageSearcher
instance.
Parameters
modelFile | the search model File instance |
---|---|
options |
Throws
IOException | if an I/O error occurs when loading the tflite model or the index file |
---|---|
IllegalArgumentException | if an argument is invalid |
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
public List<NearestNeighbor> search (TensorImage image, ImageProcessingOptions options)
Performs embedding extraction on the provided TensorImage
with ImageProcessingOptions
, followed by nearest-neighbor search in the index.
ImageSearcher
supports the following options:
- Region of interest (ROI) (through
ImageProcessingOptions.Builder.setRoi(Rect)
). It defaults to the entire image. - image rotation (through
ImageProcessingOptions.Builder.setOrientation(ImageProcessingOptions.Orientation)
). It defaults toImageProcessingOptions.Orientation.TOP_LEFT
.
ImageSearcher
supports the following TensorImage
color space types:
Parameters
image | a UINT8 TensorImage object that represents an RGB or YUV image |
---|---|
options |
Throws
IllegalArgumentException | if the color space type of image is unsupported |
---|
public List<NearestNeighbor> search (MlImage image)
Performs embedding extraction on the provided MlImage
, followed by nearest-neighbor
search in the index.
Parameters
image | an MlImage object that represents an image |
---|
Throws
IllegalArgumentException | if the storage type or format of the image is unsupported |
---|
public List<NearestNeighbor> search (MlImage image, ImageProcessingOptions options)
Performs embedding extraction on the provided MlImage
with ImageProcessingOptions
, followed by nearest-neighbor search in the index.
ImageSearcher
supports the following options:
- Region of interest (ROI) (through
ImageProcessingOptions.Builder.setRoi(Rect)
). It defaults to the entire image. - image rotation (through
ImageProcessingOptions.Builder.setOrientation(ImageProcessingOptions.Orientation)
). It defaults toImageProcessingOptions.Orientation.TOP_LEFT
.MlImage.getRotation()
is not effective.
Parameters
image | a MlImage object that represents an image |
---|---|
options | configures options including ROI and rotation |
Throws
IllegalArgumentException | if the storage type or format of the image is unsupported |
---|
public List<NearestNeighbor> search (TensorImage image)
Performs embedding extraction on the provided TensorImage
, followed by nearest-neighbor
search in the index.
ImageSearcher
supports the following TensorImage
color space types:
Parameters
image | a UINT8 TensorImage object that represents an RGB or YUV image |
---|
Throws
IllegalArgumentException | if the color space type of image is unsupported |
---|