GpuDelegateFactory.Options

public static class GpuDelegateFactory.Options
Known Direct Subclasses

Delegate options.

Constants

int INFERENCE_PREFERENCE_FAST_SINGLE_ANSWER Delegate will be used only once, therefore, bootstrap/init time should be taken into account.
int INFERENCE_PREFERENCE_SUSTAINED_SPEED Prefer maximizing the throughput.

Public Constructors

Public Methods

boolean
int
String
String
boolean
GpuDelegateFactory.Options
setInferencePreference(int preference)
Sets the inference preference for precision/compilation/runtime tradeoffs.
GpuDelegateFactory.Options
setPrecisionLossAllowed(boolean precisionLossAllowed)
Sets whether precision loss is allowed.
GpuDelegateFactory.Options
setQuantizedModelsAllowed(boolean quantizedModelsAllowed)
Enables running quantized models with the delegate.
GpuDelegateFactory.Options
setSerializationParams(String serializationDir, String modelToken)
Enables serialization on the delegate.

Inherited Methods

Constants

public static final int INFERENCE_PREFERENCE_FAST_SINGLE_ANSWER

Delegate will be used only once, therefore, bootstrap/init time should be taken into account.

Constant Value: 0

public static final int INFERENCE_PREFERENCE_SUSTAINED_SPEED

Prefer maximizing the throughput. Same delegate will be used repeatedly on multiple inputs.

Constant Value: 1

Public Constructors

public Options ()

Public Methods

public boolean areQuantizedModelsAllowed ()

public int getInferencePreference ()

public String getModelToken ()

public String getSerializationDir ()

public boolean isPrecisionLossAllowed ()

public GpuDelegateFactory.Options setInferencePreference (int preference)

Sets the inference preference for precision/compilation/runtime tradeoffs.

Parameters
preference One of `INFERENCE_PREFERENCE_FAST_SINGLE_ANSWER` (default), `INFERENCE_PREFERENCE_SUSTAINED_SPEED`.

public GpuDelegateFactory.Options setPrecisionLossAllowed (boolean precisionLossAllowed)

Sets whether precision loss is allowed.

Parameters
precisionLossAllowed When `true` (default), the GPU may quantify tensors, downcast values, process in FP16. When `false`, computations are carried out in 32-bit floating point.

public GpuDelegateFactory.Options setQuantizedModelsAllowed (boolean quantizedModelsAllowed)

Enables running quantized models with the delegate.

WARNING: This is an experimental API and subject to change.

Parameters
quantizedModelsAllowed When true (default), the GPU may run quantized models.

public GpuDelegateFactory.Options setSerializationParams (String serializationDir, String modelToken)

Enables serialization on the delegate. Note non-null serializationDir and modelToken are required for serialization.

WARNING: This is an experimental API and subject to change.

Parameters
serializationDir The directory to use for storing data. Caller is responsible to ensure the model is not stored in a public directory. It's recommended to use Context.getCodeCacheDir() to provide a private location for the application on Android.
modelToken The token to be used to identify the model. Caller is responsible to ensure the token is unique to the model graph and data.