 Xem trên TensorFlow.org |  Chạy trong Google Colab |  Xem nguồn trên GitHub |  Tải xuống sổ ghi chép |
TensorFlow Lite Authoring API cung cấp một cách để duy trì bạn tf.function mô hình compatibile với TensorFlow Lite.
Thành lập
import tensorflow as tf
Vấn đề tương thích giữa TensorFlow với TensorFlow Lite
Nếu bạn muốn sử dụng mô hình TF của mình trên các thiết bị, bạn cần chuyển đổi mô hình đó sang mô hình TFLite để sử dụng nó từ trình thông dịch TFLite. Trong quá trình chuyển đổi, bạn có thể gặp phải lỗi tương thích do các hoạt động TensorFlow không được hỗ trợ bởi bộ op nội trang TFLite.
Đây là một loại vấn đề khó chịu. Làm thế nào bạn có thể phát hiện nó sớm hơn như thời gian tác giả mô hình?
Lưu ý rằng đoạn mã sau sẽ thất bại trên converter.convert() gọi.
@tf.function(input_signature=[
tf.TensorSpec(shape=[None], dtype=tf.float32)
])
def f(x):
return tf.cosh(x)
# Evaluate the tf.function
result = f(tf.constant([0.0]))
print (f"result = {result}")
result = [1.]
# Convert the tf.function
converter = tf.lite.TFLiteConverter.from_concrete_functions(
[f.get_concrete_function()], f)
try:
fb_model = converter.convert()
except Exception as e:
print(f"Got an exception: {e}")
Got an exception: /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:746:0: error: 'tf.Cosh' op is neither a custom op nor a flex op
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py:2538:0: note: called from
/tmp/ipykernel_11095/885400331.py:5:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1125:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py:672:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1143:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3402:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3557:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3157:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py:781:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:746:0: note: Error code: ERROR_NEEDS_FLEX_OPS
<unknown>:0: error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Cosh
Details:
tf.Cosh(tensor<?xf32>) -> (tensor<?xf32>) : {device = ""}
2021-11-15 12:14:34.777684: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:363] Ignored output_format.
2021-11-15 12:14:34.777724: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:366] Ignored drop_control_dependency.
loc(callsite("Cosh"("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py":746:0) at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py":2538:0 at callsite("/tmp/ipykernel_11095/885400331.py":5:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1125:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":672:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1143:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3402:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3557:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3157:0 at "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":781:0)))))))))): error: 'tf.Cosh' op is neither a custom op nor a flex op
error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Cosh
Details:
tf.Cosh(tensor<?xf32>) -> (tensor<?xf32>) : {device = ""}
Cách sử dụng Tác giả Nhận biết Mục tiêu Đơn giản
Chúng tôi đã giới thiệu API tác giả để phát hiện sự cố tương thích với TensorFlow Lite trong thời gian tạo mô hình.
Bạn chỉ cần thêm @tf.lite.experimental.authoring.compatible trang trí quấn bạn tf.function mô hình để kiểm tra khả năng tương thích TFLite.
Sau đó, tính tương thích sẽ được tự động kiểm tra khi bạn đánh giá mô hình của mình.
@tf.lite.experimental.authoring.compatible
@tf.function(input_signature=[
tf.TensorSpec(shape=[None], dtype=tf.float32)
])
def f(x):
return tf.cosh(x)
# Evaluate the tf.function
result = f(tf.constant([0.0]))
print (f"result = {result}")
COMPATIBILITY WARNING: op 'tf.Cosh' require(s) "Select TF Ops" for model conversion for TensorFlow Lite. https://www.tensorflow.org/lite/guide/ops_select
Op: tf.Cosh
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:746
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py:2538
- /tmp/ipykernel_11095/885400331.py:5
result = [1.]
2021-11-15 12:14:34.816991: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:363] Ignored output_format.
2021-11-15 12:14:34.817023: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:366] Ignored drop_control_dependency.
loc(callsite("Cosh"("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py":746:0) at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py":2538:0 at callsite("/tmp/ipykernel_11095/885400331.py":5:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1125:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":672:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1143:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3402:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3557:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3157:0 at "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":781:0)))))))))): error: 'tf.Cosh' op is neither a custom op nor a flex op
error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Cosh
Details:
tf.Cosh(tensor<?xf32>) -> (tensor<?xf32>) : {device = ""}
Nếu bất kỳ vấn đề tương thích TensorFlow Lite được tìm thấy, nó sẽ hiển thị COMPATIBILITY WARNING hoặc COMPATIBILITY ERROR với vị trí chính xác của các op có vấn đề. Trong ví dụ này, nó cho thấy vị trí của tf.Cosh op trong mô hình tf.function của bạn.
Bạn cũng có thể kiểm tra các bản ghi tương thích với <function_name>.get_compatibility_log() phương pháp.
compatibility_log = '\n'.join(f.get_compatibility_log())
print (f"compatibility_log = {compatibility_log}")
compatibility_log = COMPATIBILITY WARNING: op 'tf.Cosh' require(s) "Select TF Ops" for model conversion for TensorFlow Lite. https://www.tensorflow.org/lite/guide/ops_select Op: tf.Cosh - /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:746 - /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py:2538 - /tmp/ipykernel_11095/885400331.py:5
Nâng cao một ngoại lệ cho một sự không tương thích
Bạn có thể cung cấp một tùy chọn để các @tf.lite.experimental.authoring.compatible trang trí. Các raise_exception tùy chọn cung cấp cho bạn một ngoại lệ khi bạn đang cố gắng để đánh giá mô hình trang trí.
@tf.lite.experimental.authoring.compatible(raise_exception=True)
@tf.function(input_signature=[
tf.TensorSpec(shape=[None], dtype=tf.float32)
])
def f(x):
return tf.cosh(x)
# Evaluate the tf.function
try:
result = f(tf.constant([0.0]))
print (f"result = {result}")
except Exception as e:
print(f"Got an exception: {e}")
COMPATIBILITY WARNING: op 'tf.Cosh' require(s) "Select TF Ops" for model conversion for TensorFlow Lite. https://www.tensorflow.org/lite/guide/ops_select
Op: tf.Cosh
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:746
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py:2538
- /tmp/ipykernel_11095/885400331.py:5
Got an exception: CompatibilityException at <tensorflow.python.eager.def_function.Function object at 0x7ff2e016fc50>
2021-11-15 12:14:34.863629: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:363] Ignored output_format.
2021-11-15 12:14:34.863657: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:366] Ignored drop_control_dependency.
loc(callsite("Cosh"("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py":746:0) at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py":2538:0 at callsite("/tmp/ipykernel_11095/885400331.py":5:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1125:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":672:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1143:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3402:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3557:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3157:0 at "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":781:0)))))))))): error: 'tf.Cosh' op is neither a custom op nor a flex op
error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Cosh
Details:
tf.Cosh(tensor<?xf32>) -> (tensor<?xf32>) : {device = ""}
Chỉ định mức sử dụng "Chọn hoạt động TF"
Nếu bạn đã nhận thức Chọn TF ops sử dụng, bạn có thể nói điều này với các API Authoring bằng cách thiết lập converter_target_spec . Đó là cùng tf.lite.TargetSpec đối tượng bạn sẽ sử dụng nó cho tf.lite.TFLiteConverter API.
target_spec = tf.lite.TargetSpec()
target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS,
tf.lite.OpsSet.SELECT_TF_OPS,
]
@tf.lite.experimental.authoring.compatible(converter_target_spec=target_spec, raise_exception=True)
@tf.function(input_signature=[
tf.TensorSpec(shape=[None], dtype=tf.float32)
])
def f(x):
return tf.cosh(x)
# Evaluate the tf.function
result = f(tf.constant([0.0]))
print (f"result = {result}")
2021-11-15 12:14:34.921628: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:363] Ignored output_format.
WARNING:absl:Buffer deduplication procedure will be skipped when flatbuffer library is not properly loaded
result = [1.]
2021-11-15 12:14:34.921661: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:366] Ignored drop_control_dependency.
2021-11-15 12:14:34.934915: W tensorflow/compiler/mlir/lite/flatbuffer_export.cc:1891] TFLite interpreter needs to link Flex delegate in order to run the model since it contains the following Select TFop(s):
Flex ops: FlexCosh
Details:
tf.Cosh(tensor<?xf32>) -> (tensor<?xf32>) : {device = ""}
See instructions: https://www.tensorflow.org/lite/guide/ops_select
Kiểm tra khả năng tương thích của GPU
Nếu bạn muốn đảm bảo mô hình của bạn là compatibile với GPU đại biểu của TensorFlow Lite, bạn có thể đặt experimental_supported_backends của tf.lite.TargetSpec .
Ví dụ sau đây cho thấy cách đảm bảo khả năng tương thích của đại biểu GPU đối với mô hình của bạn. Lưu ý rằng mô hình này có vấn đề về khả năng tương thích vì nó sử dụng tensor 2D với toán tử tf.slice và toán tử tf.cosh không được hỗ trợ. Bạn sẽ thấy hai COMPATIBILITY WARNING với các thông tin địa điểm.
target_spec = tf.lite.TargetSpec()
target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS,
tf.lite.OpsSet.SELECT_TF_OPS,
]
target_spec.experimental_supported_backends = ["GPU"]
@tf.lite.experimental.authoring.compatible(converter_target_spec=target_spec)
@tf.function(input_signature=[
tf.TensorSpec(shape=[4, 4], dtype=tf.float32)
])
def func(x):
y = tf.cosh(x)
return y + tf.slice(x, [1, 1], [1, 1])
result = func(tf.ones(shape=(4,4), dtype=tf.float32))
'tfl.slice' op is not GPU compatible: SLICE supports for 3 or 4 dimensional tensors only, but node has 2 dimensional tensors.
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py:1096
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/util/traceback_utils.py:150
- /tmp/ipykernel_11095/3833138856.py:13
'tf.Cosh' op is not GPU compatible: Not supported custom op FlexCosh
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:746
- /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py:2538
- /tmp/ipykernel_11095/3833138856.py:12
COMPATIBILITY WARNING: op 'tf.Cosh, tfl.slice' aren't compatible with TensorFlow Lite GPU delegate. https://www.tensorflow.org/lite/performance/gpu
2021-11-15 12:14:34.998044: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:363] Ignored output_format.
2021-11-15 12:14:34.998077: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:366] Ignored drop_control_dependency.
2021-11-15 12:14:35.012457: W tensorflow/compiler/mlir/lite/flatbuffer_export.cc:1891] TFLite interpreter needs to link Flex delegate in order to run the model since it contains the following Select TFop(s):
Flex ops: FlexCosh
Details:
tf.Cosh(tensor<4x4xf32>) -> (tensor<4x4xf32>) : {device = ""}
See instructions: https://www.tensorflow.org/lite/guide/ops_select
loc(callsite("Cosh"("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py":746:0) at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py":2538:0 at callsite("/tmp/ipykernel_11095/3833138856.py":12:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1125:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":672:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1143:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3402:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3557:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3157:0 at "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":781:0)))))))))): error: 'tf.Cosh' op is not GPU compatible: Not supported custom op FlexCosh
loc(callsite("Slice"("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py":1096:0) at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/util/traceback_utils.py":150:0 at callsite("/tmp/ipykernel_11095/3833138856.py":13:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1125:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":672:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py":1143:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3402:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3557:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py":3157:0 at "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py":781:0)))))))))): error: 'tfl.slice' op is not GPU compatible: SLICE supports for 3 or 4 dimensional tensors only, but node has 2 dimensional tensors.
Đọc thêm
Để biết thêm thông tin, vui lòng tham khảo: