 Visualizza su TensorFlow.org |  Esegui in Google Colab |  Visualizza la fonte su GitHub |  Scarica taccuino |
Tensorflow Lite Authoring API fornisce un modo per mantenere i vostri tf.function modelli Compatibile con tensorflow Lite.
Impostare
import tensorflow as tf
Problema di compatibilità da TensorFlow a TensorFlow Lite
Se desideri utilizzare il tuo modello TF sui dispositivi, devi convertirlo in un modello TFLite per utilizzarlo dall'interprete TFLite. Durante la conversione, potresti riscontrare un errore di compatibilità a causa di operazioni TensorFlow non supportate dal set di operazioni integrato TFLite.
Questo è un tipo di problema fastidioso. Come puoi rilevarlo prima come il tempo di creazione del modello?
Si noti che il seguente codice avrà esito negativo sul converter.convert() chiamata.
@tf.function(input_signature=[
tf.TensorSpec(shape=[None], dtype=tf.float32)
])
def f(x):
return tf.cosh(x)
# Evaluate the tf.function
result = f(tf.constant([0.0]))
print (f"result = {result}")
result = [1.]
# Convert the tf.function
converter = tf.lite.TFLiteConverter.from_concrete_functions(
[f.get_concrete_function()], f)
try:
fb_model = converter.convert()
except Exception as e:
print(f"Got an exception: {e}")
Got an exception: /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:746:0: error: 'tf.Cosh' op is neither a custom op nor a flex op
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py:2538:0: note: called from
/tmp/ipykernel_11095/885400331.py:5:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1125:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py:672:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1143:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3402:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3557:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3157:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py:781:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:746:0: note: Error code: ERROR_NEEDS_FLEX_OPS
<unknown>:0: error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Cosh
Details:
tf.Cosh(tensor<?xf32>) -> (tensor<?xf32>) : {device = ""}
2021-11-15 12:14:34.777684: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:363] Ignored output_format.
2021-11-15 12:14:34.777724: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:366] Ignored drop_control_dependency.
loc(callsite("Cosh"("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py":746:0) at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py":2538:0 at callsite("/tmp/ipykernel_11095/885400331.py":5:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1125:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":672:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1143:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3402:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3557:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3157:0 at "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":781:0)))))))))): error: 'tf.Cosh' op is neither a custom op nor a flex op
error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Cosh
Details:
tf.Cosh(tensor<?xf32>) -> (tensor<?xf32>) : {device = ""}
Utilizzo semplice dell'authoring consapevole del target
Abbiamo introdotto l'API di creazione per rilevare il problema di compatibilità di TensorFlow Lite durante il periodo di creazione del modello.
Hai solo bisogno di aggiungere @tf.lite.experimental.authoring.compatible decoratore per avvolgere il vostro tf.function modello per verificare la compatibilità TFLite.
Dopodiché, la compatibilità verrà verificata automaticamente quando valuterai il tuo modello.
@tf.lite.experimental.authoring.compatible
@tf.function(input_signature=[
tf.TensorSpec(shape=[None], dtype=tf.float32)
])
def f(x):
return tf.cosh(x)
# Evaluate the tf.function
result = f(tf.constant([0.0]))
print (f"result = {result}")
COMPATIBILITY WARNING: op 'tf.Cosh' require(s) "Select TF Ops" for model conversion for TensorFlow Lite. https://www.tensorflow.org/lite/guide/ops_select
Op: tf.Cosh
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:746
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py:2538
- /tmp/ipykernel_11095/885400331.py:5
result = [1.]
2021-11-15 12:14:34.816991: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:363] Ignored output_format.
2021-11-15 12:14:34.817023: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:366] Ignored drop_control_dependency.
loc(callsite("Cosh"("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py":746:0) at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py":2538:0 at callsite("/tmp/ipykernel_11095/885400331.py":5:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1125:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":672:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1143:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3402:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3557:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3157:0 at "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":781:0)))))))))): error: 'tf.Cosh' op is neither a custom op nor a flex op
error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Cosh
Details:
tf.Cosh(tensor<?xf32>) -> (tensor<?xf32>) : {device = ""}
Se qualche problema di compatibilità tensorflow Lite viene trovato, mostrerà COMPATIBILITY WARNING o COMPATIBILITY ERROR con la posizione esatta del op problematico. In questo esempio, mostra la posizione dei tf.Cosh op nel modello tf.function.
È anche possibile controllare il registro di compatibilità con il <function_name>.get_compatibility_log() metodo.
compatibility_log = '\n'.join(f.get_compatibility_log())
print (f"compatibility_log = {compatibility_log}")
compatibility_log = COMPATIBILITY WARNING: op 'tf.Cosh' require(s) "Select TF Ops" for model conversion for TensorFlow Lite. https://www.tensorflow.org/lite/guide/ops_select Op: tf.Cosh - /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:746 - /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py:2538 - /tmp/ipykernel_11095/885400331.py:5
Sollevare un'eccezione per un'incompatibilità
È possibile fornire un'opzione per l' @tf.lite.experimental.authoring.compatible decoratore. L' raise_exception opzione offre un'eccezione quando si sta cercando di valutare il modello decorato.
@tf.lite.experimental.authoring.compatible(raise_exception=True)
@tf.function(input_signature=[
tf.TensorSpec(shape=[None], dtype=tf.float32)
])
def f(x):
return tf.cosh(x)
# Evaluate the tf.function
try:
result = f(tf.constant([0.0]))
print (f"result = {result}")
except Exception as e:
print(f"Got an exception: {e}")
COMPATIBILITY WARNING: op 'tf.Cosh' require(s) "Select TF Ops" for model conversion for TensorFlow Lite. https://www.tensorflow.org/lite/guide/ops_select
Op: tf.Cosh
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:746
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py:2538
- /tmp/ipykernel_11095/885400331.py:5
Got an exception: CompatibilityException at <tensorflow.python.eager.def_function.Function object at 0x7ff2e016fc50>
2021-11-15 12:14:34.863629: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:363] Ignored output_format.
2021-11-15 12:14:34.863657: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:366] Ignored drop_control_dependency.
loc(callsite("Cosh"("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py":746:0) at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py":2538:0 at callsite("/tmp/ipykernel_11095/885400331.py":5:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1125:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":672:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1143:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3402:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3557:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3157:0 at "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":781:0)))))))))): error: 'tf.Cosh' op is neither a custom op nor a flex op
error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Cosh
Details:
tf.Cosh(tensor<?xf32>) -> (tensor<?xf32>) : {device = ""}
Specificare l'utilizzo di "Seleziona operazioni TF"
Se siete già a conoscenza di Select TF ops utilizzo, si può dire questo al API Authoring impostando converter_target_spec . E 'lo stesso tf.lite.TargetSpec oggetto che verrà utilizzato per tf.lite.TFLiteConverter API.
target_spec = tf.lite.TargetSpec()
target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS,
tf.lite.OpsSet.SELECT_TF_OPS,
]
@tf.lite.experimental.authoring.compatible(converter_target_spec=target_spec, raise_exception=True)
@tf.function(input_signature=[
tf.TensorSpec(shape=[None], dtype=tf.float32)
])
def f(x):
return tf.cosh(x)
# Evaluate the tf.function
result = f(tf.constant([0.0]))
print (f"result = {result}")
2021-11-15 12:14:34.921628: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:363] Ignored output_format.
WARNING:absl:Buffer deduplication procedure will be skipped when flatbuffer library is not properly loaded
result = [1.]
2021-11-15 12:14:34.921661: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:366] Ignored drop_control_dependency.
2021-11-15 12:14:34.934915: W tensorflow/compiler/mlir/lite/flatbuffer_export.cc:1891] TFLite interpreter needs to link Flex delegate in order to run the model since it contains the following Select TFop(s):
Flex ops: FlexCosh
Details:
tf.Cosh(tensor<?xf32>) -> (tensor<?xf32>) : {device = ""}
See instructions: https://www.tensorflow.org/lite/guide/ops_select
Verifica della compatibilità della GPU
Se si vuole garantire il vostro modello è compatibile con la GPU delegato di tensorflow Lite, è possibile impostare experimental_supported_backends di tf.lite.TargetSpec .
L'esempio seguente mostra come garantire la compatibilità del delegato GPU del modello. Si noti che questo modello presenta problemi di compatibilità poiché utilizza un tensore 2D con l'operatore tf.slice e l'operatore tf.cosh non supportato. Vedrete due COMPATIBILITY WARNING con le informazioni sulla posizione.
target_spec = tf.lite.TargetSpec()
target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS,
tf.lite.OpsSet.SELECT_TF_OPS,
]
target_spec.experimental_supported_backends = ["GPU"]
@tf.lite.experimental.authoring.compatible(converter_target_spec=target_spec)
@tf.function(input_signature=[
tf.TensorSpec(shape=[4, 4], dtype=tf.float32)
])
def func(x):
y = tf.cosh(x)
return y + tf.slice(x, [1, 1], [1, 1])
result = func(tf.ones(shape=(4,4), dtype=tf.float32))
'tfl.slice' op is not GPU compatible: SLICE supports for 3 or 4 dimensional tensors only, but node has 2 dimensional tensors.
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py:1096
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/util/traceback_utils.py:150
- /tmp/ipykernel_11095/3833138856.py:13
'tf.Cosh' op is not GPU compatible: Not supported custom op FlexCosh
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:746
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py:2538
- /tmp/ipykernel_11095/3833138856.py:12
COMPATIBILITY WARNING: op 'tf.Cosh, tfl.slice' aren't compatible with TensorFlow Lite GPU delegate. https://www.tensorflow.org/lite/performance/gpu
2021-11-15 12:14:34.998044: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:363] Ignored output_format.
2021-11-15 12:14:34.998077: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:366] Ignored drop_control_dependency.
2021-11-15 12:14:35.012457: W tensorflow/compiler/mlir/lite/flatbuffer_export.cc:1891] TFLite interpreter needs to link Flex delegate in order to run the model since it contains the following Select TFop(s):
Flex ops: FlexCosh
Details:
tf.Cosh(tensor<4x4xf32>) -> (tensor<4x4xf32>) : {device = ""}
See instructions: https://www.tensorflow.org/lite/guide/ops_select
loc(callsite("Cosh"("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py":746:0) at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py":2538:0 at callsite("/tmp/ipykernel_11095/3833138856.py":12:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1125:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":672:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1143:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3402:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3557:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3157:0 at "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":781:0)))))))))): error: 'tf.Cosh' op is not GPU compatible: Not supported custom op FlexCosh
loc(callsite("Slice"("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py":1096:0) at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/util/traceback_utils.py":150:0 at callsite("/tmp/ipykernel_11095/3833138856.py":13:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1125:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":672:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1143:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3402:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3557:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3157:0 at "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":781:0)))))))))): error: 'tfl.slice' op is not GPU compatible: SLICE supports for 3 or 4 dimensional tensors only, but node has 2 dimensional tensors.
Per saperne di più
Per ulteriori informazioni, fare riferimento a: