 TensorFlow.org에서 보기
|
 Google Colab에서 실행하기
|
 GitHub에서 소그 보기 |
 노트북 다운로드하기 |
TensorFlow Lite Authoring API는 TensorFlow Lite와 tf.function 모델의 호환성을 유지하는 방법을 제공합니다.
설정
import tensorflow as tf
2022-12-15 01:08:27.118741: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory 2022-12-15 01:08:27.118852: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory 2022-12-15 01:08:27.118862: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
TensorFlow와 TensorFlow Lite 간의 호환성 문제
장치에서 TF 모델을 사용하려면 TFLite 인터프리터에서 사용할 수 있게 이를 TFLite 모델로 변환해야 합니다. 변환하는 동안 TFLite 기본 제공 op 세트에서 지원하지 않는 TensorFlow op로 인해 호환성 오류가 발생할 수 있습니다.
이것은 다소 성가신 문제입니다. 모델 저작 단계에서 이를 조기에 감지할 수 있는 방법은 무엇일까요?
다음 코드는 converter.convert() 호출에서 실패합니다.
@tf.function(input_signature=[
tf.TensorSpec(shape=[None], dtype=tf.float32)
])
def f(x):
return tf.cosh(x)
# Evaluate the tf.function
result = f(tf.constant([0.0]))
print (f"result = {result}")
result = [1.]
# Convert the tf.function
converter = tf.lite.TFLiteConverter.from_concrete_functions(
[f.get_concrete_function()], f)
try:
fb_model = converter.convert()
except Exception as e:
print(f"Got an exception: {e}")
Got an exception: /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/op_def_library.py:795:0: error: 'tf.Cosh' op is neither a custom op nor a flex op
/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py:2537:0: note: called from
/tmpfs/tmp/ipykernel_68101/885400331.py:5:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py:1258:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py:645:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py:1283:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py:284:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py:360:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py:157:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py:162:0: note: called from
/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/op_def_library.py:795:0: note: Error code: ERROR_NEEDS_FLEX_OPS
<unknown>:0: error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Cosh
Details:
tf.Cosh(tensor<?xf32>) -> (tensor<?xf32>) : {device = ""}
2022-12-15 01:08:31.774628: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:362] Ignored output_format.
2022-12-15 01:08:31.774671: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:365] Ignored drop_control_dependency.
loc(fused["Cosh:", callsite("Cosh"("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/op_def_library.py":795:0) at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py":2537:0 at callsite("/tmpfs/tmp/ipykernel_68101/885400331.py":5:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py":1258:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py":645:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py":1283:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":284:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":360:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":157:0 at "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":162:0)))))))))]): error: 'tf.Cosh' op is neither a custom op nor a flex op
error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Cosh
Details:
tf.Cosh(tensor<?xf32>) -> (tensor<?xf32>) : {device = ""}
단순 타겟 인식 저작 방법
모델 저작 단계에서 TensorFlow Lite 호환성 문제를 감지하기 위해 Authoring API를 도입했습니다.
TFLite 호환성을 확인하기 위해 tf.function 모델을 래핑하려면 @tf.lite.experimental.authoring.compatible 데코레이터를 추가하기만 하면 됩니다.
그러면 모델을 평가할 때 호환성이 자동으로 확인됩니다.
@tf.lite.experimental.authoring.compatible
@tf.function(input_signature=[
tf.TensorSpec(shape=[None], dtype=tf.float32)
])
def f(x):
return tf.cosh(x)
# Evaluate the tf.function
result = f(tf.constant([0.0]))
print (f"result = {result}")
COMPATIBILITY WARNING: op 'tf.Cosh' require(s) "Select TF Ops" for model conversion for TensorFlow Lite. https://www.tensorflow.org/lite/guide/ops_select
Op: tf.Cosh
- /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/op_def_library.py:795
- /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py:2537
- /tmpfs/tmp/ipykernel_68101/885400331.py:5
result = [1.]
2022-12-15 01:08:31.833978: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:362] Ignored output_format.
2022-12-15 01:08:31.834017: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:365] Ignored drop_control_dependency.
loc(fused["Cosh:", callsite("Cosh"("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/op_def_library.py":795:0) at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py":2537:0 at callsite("/tmpfs/tmp/ipykernel_68101/885400331.py":5:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py":1258:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py":645:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py":1283:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":284:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":360:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":157:0 at "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":162:0)))))))))]): error: 'tf.Cosh' op is neither a custom op nor a flex op
error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Cosh
Details:
tf.Cosh(tensor<?xf32>) -> (tensor<?xf32>) : {device = ""}
TensorFlow Lite 호환성 문제가 발견되면 문제가 있는 op의 정확한 위치와 함께 COMPATIBILITY WARNING 또는 COMPATIBILITY ERROR가 표시됩니다. 이 예에서는 tf.function 모델에서 tf.Cosh op의 위치를 보여줍니다.
<function_name>.get_compatibility_log() 메서드로 호환성 로그를 확인할 수도 있습니다.
compatibility_log = '\n'.join(f.get_compatibility_log())
print (f"compatibility_log = {compatibility_log}")
compatibility_log = COMPATIBILITY WARNING: op 'tf.Cosh' require(s) "Select TF Ops" for model conversion for TensorFlow Lite. https://www.tensorflow.org/lite/guide/ops_select Op: tf.Cosh - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/op_def_library.py:795 - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py:2537 - /tmpfs/tmp/ipykernel_68101/885400331.py:5
비호환성에 대한 예외 발생
@tf.lite.experimental.authoring.compatible 데코레이터에 옵션을 제공할 수 있습니다. raise_exception 옵션은 데코레이팅된 모델을 평가하려고 할 때 예외를 제공합니다.
@tf.lite.experimental.authoring.compatible(raise_exception=True)
@tf.function(input_signature=[
tf.TensorSpec(shape=[None], dtype=tf.float32)
])
def f(x):
return tf.cosh(x)
# Evaluate the tf.function
try:
result = f(tf.constant([0.0]))
print (f"result = {result}")
except Exception as e:
print(f"Got an exception: {e}")
COMPATIBILITY WARNING: op 'tf.Cosh' require(s) "Select TF Ops" for model conversion for TensorFlow Lite. https://www.tensorflow.org/lite/guide/ops_select
Op: tf.Cosh
- /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/op_def_library.py:795
- /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py:2537
- /tmpfs/tmp/ipykernel_68101/885400331.py:5
Got an exception: CompatibilityException at <tensorflow.python.eager.polymorphic_function.polymorphic_function.Function object at 0x7f252aea1be0>
2022-12-15 01:08:31.898689: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:362] Ignored output_format.
2022-12-15 01:08:31.898738: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:365] Ignored drop_control_dependency.
loc(fused["Cosh:", callsite("Cosh"("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/op_def_library.py":795:0) at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py":2537:0 at callsite("/tmpfs/tmp/ipykernel_68101/885400331.py":5:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py":1258:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py":645:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py":1283:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":284:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":360:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":157:0 at "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":162:0)))))))))]): error: 'tf.Cosh' op is neither a custom op nor a flex op
error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Cosh
Details:
tf.Cosh(tensor<?xf32>) -> (tensor<?xf32>) : {device = ""}
"TF op 선택" 지정 방법
Select TF ops 방법을 이미 알고 있다면 converter_target_spec을 설정하여 Authoring API에 이를 알릴 수 있습니다. tf.lite.TFLiteConverter API에 사용할 동일한 tf.lite.TargetSpec 객체입니다.
target_spec = tf.lite.TargetSpec()
target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS,
tf.lite.OpsSet.SELECT_TF_OPS,
]
@tf.lite.experimental.authoring.compatible(converter_target_spec=target_spec, raise_exception=True)
@tf.function(input_signature=[
tf.TensorSpec(shape=[None], dtype=tf.float32)
])
def f(x):
return tf.cosh(x)
# Evaluate the tf.function
result = f(tf.constant([0.0]))
print (f"result = {result}")
2022-12-15 01:08:31.976118: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:362] Ignored output_format.
2022-12-15 01:08:31.976157: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:365] Ignored drop_control_dependency.
2022-12-15 01:08:31.989689: W tensorflow/compiler/mlir/lite/flatbuffer_export.cc:2046] TFLite interpreter needs to link Flex delegate in order to run the model since it contains the following Select TFop(s):
Flex ops: FlexCosh
Details:
tf.Cosh(tensor<?xf32>) -> (tensor<?xf32>) : {device = ""}
See instructions: https://www.tensorflow.org/lite/guide/ops_select
result = [1.]
GPU 호환성 확인
모델이 TensorFlow Lite의 GPU 대리자와 호환되도록 하려면 tf.lite.TargetSpec의 experimental_supported_backends를 설정할 수 있습니다.
다음 예제는 모델의 GPU 대리자 호환성을 보장하는 방법을 보여줍니다. 이 모델은 tf.slice 연산자 및 지원되지 않는 tf.cosh 연산자와 함께 2D 텐서를 사용하기 때문에 호환성 문제가 있습니다. 위치 정보와 함께 두 개의 COMPATIBILITY WARNING가 표시됩니다.
target_spec = tf.lite.TargetSpec()
target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS,
tf.lite.OpsSet.SELECT_TF_OPS,
]
target_spec.experimental_supported_backends = ["GPU"]
@tf.lite.experimental.authoring.compatible(converter_target_spec=target_spec)
@tf.function(input_signature=[
tf.TensorSpec(shape=[4, 4], dtype=tf.float32)
])
def func(x):
y = tf.cosh(x)
return y + tf.slice(x, [1, 1], [1, 1])
result = func(tf.ones(shape=(4,4), dtype=tf.float32))
'tfl.slice' op is not GPU compatible: SLICE supports for 3 or 4 dimensional tensors only, but node has 2 dimensional tensors.
- /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/util/dispatch.py:1176
- /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/util/traceback_utils.py:150
- /tmpfs/tmp/ipykernel_68101/3833138856.py:13
'tf.Cosh' op is not GPU compatible: Not supported custom op FlexCosh
- /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/op_def_library.py:795
- /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py:2537
- /tmpfs/tmp/ipykernel_68101/3833138856.py:12
COMPATIBILITY WARNING: op 'tf.Cosh, tfl.slice' aren't compatible with TensorFlow Lite GPU delegate. https://www.tensorflow.org/lite/performance/gpu
2022-12-15 01:08:32.075694: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:362] Ignored output_format.
2022-12-15 01:08:32.075738: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:365] Ignored drop_control_dependency.
2022-12-15 01:08:32.089652: W tensorflow/compiler/mlir/lite/flatbuffer_export.cc:2046] TFLite interpreter needs to link Flex delegate in order to run the model since it contains the following Select TFop(s):
Flex ops: FlexCosh
Details:
tf.Cosh(tensor<4x4xf32>) -> (tensor<4x4xf32>) : {device = ""}
See instructions: https://www.tensorflow.org/lite/guide/ops_select
loc(fused["Cosh:", callsite("Cosh"("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/op_def_library.py":795:0) at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py":2537:0 at callsite("/tmpfs/tmp/ipykernel_68101/3833138856.py":12:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py":1258:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py":645:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py":1283:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":284:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":360:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":157:0 at "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":162:0)))))))))]): error: 'tf.Cosh' op is not GPU compatible: Not supported custom op FlexCosh
loc(fused["Slice:", callsite("Slice"("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/util/dispatch.py":1176:0) at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/util/traceback_utils.py":150:0 at callsite("/tmpfs/tmp/ipykernel_68101/3833138856.py":13:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py":1258:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py":645:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py":1283:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":284:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":360:0 at callsite("/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":157:0 at "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py":162:0)))))))))]): error: 'tfl.slice' op is not GPU compatible: SLICE supports for 3 or 4 dimensional tensors only, but node has 2 dimensional tensors.
더 읽어보기
자세한 내용은 다음을 참조하세요.