Przykłady migracji: szablony estymatorów

Zobacz na TensorFlow.org Uruchom w Google Colab Wyświetl źródło na GitHub Pobierz notatnik

Gotowe (lub gotowe) estymatory były tradycyjnie używane w TensorFlow 1 jako szybkie i łatwe sposoby uczenia modeli dla różnych typowych przypadków użycia. TensorFlow 2 zapewnia proste przybliżone zamienniki wielu z nich za pomocą modeli Keras. W przypadku tych gotowych estymatorów, które nie mają wbudowanych zamienników TensorFlow 2, nadal możesz dość łatwo zbudować własny zamiennik.

Ten przewodnik omawia kilka przykładów bezpośrednich odpowiedników i niestandardowych podstawień, aby zademonstrować, jak modele wywodzące się z tf.estimator z TensorFlow 1 można przenieść do TF2 za pomocą Keras.

Mianowicie, ten przewodnik zawiera przykłady migracji:

Typowym prekursorem uczenia modelu jest wstępne przetwarzanie funkcji, które jest wykonywane dla modeli TensorFlow 1 Estimator za pomocą tf.feature_column . Aby uzyskać więcej informacji na temat wstępnego przetwarzania funkcji w TensorFlow 2, zobacz ten przewodnik dotyczący migracji kolumn funkcji .

Ustawiać

Zacznij od kilku niezbędnych importów TensorFlow,

pip install tensorflow_decision_forests

import keras
import pandas as pd
import tensorflow as tf
import tensorflow.compat.v1 as tf1
import tensorflow_decision_forests as tfdf

WARNING:root:TF Parameter Server distributed training not available (this is expected for the pre-build release).

przygotować kilka prostych danych do demonstracji ze standardowego zbioru danych Titanica,

x_train = pd.read_csv('https://storage.googleapis.com/tf-datasets/titanic/train.csv')
x_eval = pd.read_csv('https://storage.googleapis.com/tf-datasets/titanic/eval.csv')
x_train['sex'].replace(('male', 'female'), (0, 1), inplace=True)
x_eval['sex'].replace(('male', 'female'), (0, 1), inplace=True)

x_train['alone'].replace(('n', 'y'), (0, 1), inplace=True)
x_eval['alone'].replace(('n', 'y'), (0, 1), inplace=True)

x_train['class'].replace(('First', 'Second', 'Third'), (1, 2, 3), inplace=True)
x_eval['class'].replace(('First', 'Second', 'Third'), (1, 2, 3), inplace=True)

x_train.drop(['embark_town', 'deck'], axis=1, inplace=True)
x_eval.drop(['embark_town', 'deck'], axis=1, inplace=True)

y_train = x_train.pop('survived')
y_eval = x_eval.pop('survived')

# Data setup for TensorFlow 1 with `tf.estimator`
def _input_fn():
  return tf1.data.Dataset.from_tensor_slices((dict(x_train), y_train)).batch(32)


def _eval_input_fn():
  return tf1.data.Dataset.from_tensor_slices((dict(x_eval), y_eval)).batch(32)


FEATURE_NAMES = [
    'age', 'fare', 'sex', 'n_siblings_spouses', 'parch', 'class', 'alone'
]

feature_columns = []
for fn in FEATURE_NAMES:
  feat_col = tf1.feature_column.numeric_column(fn, dtype=tf.float32)
  feature_columns.append(feat_col)

i stworzyć metodę tworzenia instancji uproszczonego optymalizatora próbek do użycia z naszymi różnymi modelami TensorFlow 1 Estimator i TensorFlow 2 Keras.

def create_sample_optimizer(tf_version):
  if tf_version == 'tf1':
    optimizer = lambda: tf.keras.optimizers.Ftrl(
        l1_regularization_strength=0.001,
        learning_rate=tf1.train.exponential_decay(
            learning_rate=0.1,
            global_step=tf1.train.get_global_step(),
            decay_steps=10000,
            decay_rate=0.9))
  elif tf_version == 'tf2':
    optimizer = tf.keras.optimizers.Ftrl(
        l1_regularization_strength=0.001,
        learning_rate=tf.keras.optimizers.schedules.ExponentialDecay(
            initial_learning_rate=0.1, decay_steps=10000, decay_rate=0.9))
  return optimizer

Przykład 1: Migracja z LinearEstimator

TF1: Korzystanie z LinearEstimator

W TensorFlow 1 można użyć tf.estimator.LinearEstimator do utworzenia podstawowego modelu liniowego dla problemów z regresją i klasyfikacją.

linear_estimator = tf.estimator.LinearEstimator(
    head=tf.estimator.BinaryClassHead(),
    feature_columns=feature_columns,
    optimizer=create_sample_optimizer('tf1'))

INFO:tensorflow:Using default config.
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpvoycvffz
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpvoycvffz
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpvoycvffz', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpvoycvffz', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}

linear_estimator.train(input_fn=_input_fn, steps=100)
linear_estimator.evaluate(input_fn=_eval_input_fn, steps=10)

WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/training_util.py:401: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/training_util.py:401: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/canned/linear.py:1478: UserWarning: `layer.add_variable` is deprecated and will be removed in a future version. Please use `layer.add_weight` method instead.
  getter=tf.compat.v1.get_variable)
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/keras/optimizer_v2/ftrl.py:149: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/keras/optimizer_v2/ftrl.py:149: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpvoycvffz/model.ckpt.
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpvoycvffz/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:loss = 0.6931472, step = 0
INFO:tensorflow:loss = 0.6931472, step = 0
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 20...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 20...
INFO:tensorflow:Saving checkpoints for 20 into /tmp/tmpvoycvffz/model.ckpt.
INFO:tensorflow:Saving checkpoints for 20 into /tmp/tmpvoycvffz/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 20...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 20...
INFO:tensorflow:Loss for final step: 0.55268794.
INFO:tensorflow:Loss for final step: 0.55268794.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2022-01-29T02:21:45
INFO:tensorflow:Starting evaluation at 2022-01-29T02:21:45
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpvoycvffz/model.ckpt-20
INFO:tensorflow:Restoring parameters from /tmp/tmpvoycvffz/model.ckpt-20
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Evaluation [1/10]
INFO:tensorflow:Evaluation [1/10]
INFO:tensorflow:Evaluation [2/10]
INFO:tensorflow:Evaluation [2/10]
INFO:tensorflow:Evaluation [3/10]
INFO:tensorflow:Evaluation [3/10]
INFO:tensorflow:Evaluation [4/10]
INFO:tensorflow:Evaluation [4/10]
INFO:tensorflow:Evaluation [5/10]
INFO:tensorflow:Evaluation [5/10]
INFO:tensorflow:Evaluation [6/10]
INFO:tensorflow:Evaluation [6/10]
INFO:tensorflow:Evaluation [7/10]
INFO:tensorflow:Evaluation [7/10]
INFO:tensorflow:Evaluation [8/10]
INFO:tensorflow:Evaluation [8/10]
INFO:tensorflow:Evaluation [9/10]
INFO:tensorflow:Evaluation [9/10]
INFO:tensorflow:Inference Time : 0.50224s
INFO:tensorflow:Inference Time : 0.50224s
INFO:tensorflow:Finished evaluation at 2022-01-29-02:21:45
INFO:tensorflow:Finished evaluation at 2022-01-29-02:21:45
INFO:tensorflow:Saving dict for global step 20: accuracy = 0.70075756, accuracy_baseline = 0.625, auc = 0.75472915, auc_precision_recall = 0.65362054, average_loss = 0.5759378, global_step = 20, label/mean = 0.375, loss = 0.5704812, precision = 0.6388889, prediction/mean = 0.41331062, recall = 0.46464646
INFO:tensorflow:Saving dict for global step 20: accuracy = 0.70075756, accuracy_baseline = 0.625, auc = 0.75472915, auc_precision_recall = 0.65362054, average_loss = 0.5759378, global_step = 20, label/mean = 0.375, loss = 0.5704812, precision = 0.6388889, prediction/mean = 0.41331062, recall = 0.46464646
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 20: /tmp/tmpvoycvffz/model.ckpt-20
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 20: /tmp/tmpvoycvffz/model.ckpt-20
{'accuracy': 0.70075756,
 'accuracy_baseline': 0.625,
 'auc': 0.75472915,
 'auc_precision_recall': 0.65362054,
 'average_loss': 0.5759378,
 'label/mean': 0.375,
 'loss': 0.5704812,
 'precision': 0.6388889,
 'prediction/mean': 0.41331062,
 'recall': 0.46464646,
 'global_step': 20}

TF2: Korzystanie z Keras LinearModel

W TensorFlow 2 możesz utworzyć instancję Keras tf.compat.v1.keras.models.LinearModel , która jest substytutem tf.estimator.LinearEstimator . Ścieżka tf.compat.v1.keras służy do oznaczenia, że ​​gotowy model istnieje w celu zapewnienia zgodności.

linear_model = tf.compat.v1.keras.experimental.LinearModel()
linear_model.compile(loss='mse', optimizer=create_sample_optimizer('tf2'), metrics=['accuracy'])
linear_model.fit(x_train, y_train, epochs=10)
linear_model.evaluate(x_eval, y_eval, return_dict=True)

Epoch 1/10
20/20 [==============================] - 0s 2ms/step - loss: 2.8157 - accuracy: 0.6300
Epoch 2/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2758 - accuracy: 0.6427
Epoch 3/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2470 - accuracy: 0.6699
Epoch 4/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1954 - accuracy: 0.7177
Epoch 5/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1931 - accuracy: 0.7145
Epoch 6/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1816 - accuracy: 0.7496
Epoch 7/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1766 - accuracy: 0.7751
Epoch 8/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2198 - accuracy: 0.7560
Epoch 9/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1657 - accuracy: 0.7959
Epoch 10/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1738 - accuracy: 0.7959
9/9 [==============================] - 0s 2ms/step - loss: 0.2278 - accuracy: 0.6780
{'loss': 0.22778697311878204, 'accuracy': 0.6780303120613098}

Przykład 2: Migracja z DNNEstimator

TF1: Korzystanie z DNNEstimator

W TensorFlow 1 można użyć tf.estimator.DNNEstimator do utworzenia bazowego modelu DNN dla problemów z regresją i klasyfikacją.

dnn_estimator = tf.estimator.DNNEstimator(
    head=tf.estimator.BinaryClassHead(),
    feature_columns=feature_columns,
    hidden_units=[128],
    activation_fn=tf.nn.relu,
    optimizer=create_sample_optimizer('tf1'))

INFO:tensorflow:Using default config.
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmphckb8f81
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmphckb8f81
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmphckb8f81', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmphckb8f81', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}

dnn_estimator.train(input_fn=_input_fn, steps=100)
dnn_estimator.evaluate(input_fn=_eval_input_fn, steps=10)

INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmphckb8f81/model.ckpt.
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmphckb8f81/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:loss = 2.1811047, step = 0
INFO:tensorflow:loss = 2.1811047, step = 0
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 20...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 20...
INFO:tensorflow:Saving checkpoints for 20 into /tmp/tmphckb8f81/model.ckpt.
INFO:tensorflow:Saving checkpoints for 20 into /tmp/tmphckb8f81/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 20...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 20...
INFO:tensorflow:Loss for final step: 0.5881681.
INFO:tensorflow:Loss for final step: 0.5881681.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2022-01-29T02:21:48
INFO:tensorflow:Starting evaluation at 2022-01-29T02:21:48
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmphckb8f81/model.ckpt-20
INFO:tensorflow:Restoring parameters from /tmp/tmphckb8f81/model.ckpt-20
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Evaluation [1/10]
INFO:tensorflow:Evaluation [1/10]
INFO:tensorflow:Evaluation [2/10]
INFO:tensorflow:Evaluation [2/10]
INFO:tensorflow:Evaluation [3/10]
INFO:tensorflow:Evaluation [3/10]
INFO:tensorflow:Evaluation [4/10]
INFO:tensorflow:Evaluation [4/10]
INFO:tensorflow:Evaluation [5/10]
INFO:tensorflow:Evaluation [5/10]
INFO:tensorflow:Evaluation [6/10]
INFO:tensorflow:Evaluation [6/10]
INFO:tensorflow:Evaluation [7/10]
INFO:tensorflow:Evaluation [7/10]
INFO:tensorflow:Evaluation [8/10]
INFO:tensorflow:Evaluation [8/10]
INFO:tensorflow:Evaluation [9/10]
INFO:tensorflow:Evaluation [9/10]
INFO:tensorflow:Inference Time : 0.47075s
INFO:tensorflow:Inference Time : 0.47075s
INFO:tensorflow:Finished evaluation at 2022-01-29-02:21:49
INFO:tensorflow:Finished evaluation at 2022-01-29-02:21:49
INFO:tensorflow:Saving dict for global step 20: accuracy = 0.7083333, accuracy_baseline = 0.625, auc = 0.70716256, auc_precision_recall = 0.6146256, average_loss = 0.60399944, global_step = 20, label/mean = 0.375, loss = 0.5986442, precision = 0.6486486, prediction/mean = 0.41256863, recall = 0.4848485
INFO:tensorflow:Saving dict for global step 20: accuracy = 0.7083333, accuracy_baseline = 0.625, auc = 0.70716256, auc_precision_recall = 0.6146256, average_loss = 0.60399944, global_step = 20, label/mean = 0.375, loss = 0.5986442, precision = 0.6486486, prediction/mean = 0.41256863, recall = 0.4848485
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 20: /tmp/tmphckb8f81/model.ckpt-20
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 20: /tmp/tmphckb8f81/model.ckpt-20
{'accuracy': 0.7083333,
 'accuracy_baseline': 0.625,
 'auc': 0.70716256,
 'auc_precision_recall': 0.6146256,
 'average_loss': 0.60399944,
 'label/mean': 0.375,
 'loss': 0.5986442,
 'precision': 0.6486486,
 'prediction/mean': 0.41256863,
 'recall': 0.4848485,
 'global_step': 20}

TF2: Używanie Keras do tworzenia niestandardowego modelu DNN

W TensorFlow 2 możesz utworzyć niestandardowy model DNN, aby zastąpić model wygenerowany przez tf.estimator.DNNEstimator , z podobnymi poziomami dostosowania określonego przez użytkownika (na przykład, jak w poprzednim przykładzie, możliwość dostosowania wybranego optymalizatora modelu) .

Podobny przepływ pracy może być użyty do zastąpienia tf.estimator.experimental.RNNEstimator Keras RNN. Keras zapewnia szereg wbudowanych, konfigurowalnych opcji za pośrednictwem tf.keras.layers.RNN , tf.keras.layers.LSTM i tf.keras.layers.GRU — więcej szczegółów znajdziesz tutaj .

dnn_model = tf.keras.models.Sequential(
    [tf.keras.layers.Dense(128, activation='relu'),
     tf.keras.layers.Dense(1)])

dnn_model.compile(loss='mse', optimizer=create_sample_optimizer('tf2'), metrics=['accuracy'])

dnn_model.fit(x_train, y_train, epochs=10)
dnn_model.evaluate(x_eval, y_eval, return_dict=True)

Epoch 1/10
20/20 [==============================] - 0s 2ms/step - loss: 551.2993 - accuracy: 0.5997
Epoch 2/10
20/20 [==============================] - 0s 2ms/step - loss: 16.8562 - accuracy: 0.6427
Epoch 3/10
20/20 [==============================] - 0s 2ms/step - loss: 0.3048 - accuracy: 0.7161
Epoch 4/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2475 - accuracy: 0.7416
Epoch 5/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2334 - accuracy: 0.7512
Epoch 6/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2200 - accuracy: 0.7416
Epoch 7/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2012 - accuracy: 0.7656
Epoch 8/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2025 - accuracy: 0.7624
Epoch 9/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2185 - accuracy: 0.7703
Epoch 10/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2046 - accuracy: 0.7687
9/9 [==============================] - 0s 2ms/step - loss: 0.2227 - accuracy: 0.6856
{'loss': 0.2227054387331009, 'accuracy': 0.685606062412262}

Przykład 3: Migracja z DNNLinearCombinedEstimator

TF1: Korzystanie z DNNLinearCombinedEstimator

W TensorFlow 1 można użyć tf.estimator.DNNLinearCombinedEstimator do stworzenia bazowego połączonego modelu problemów z regresją i klasyfikacją z możliwością dostosowywania zarówno komponentów liniowych, jak i DNN.

optimizer = create_sample_optimizer('tf1')

combined_estimator = tf.estimator.DNNLinearCombinedEstimator(
    head=tf.estimator.BinaryClassHead(),
    # Wide settings
    linear_feature_columns=feature_columns,
    linear_optimizer=optimizer,
    # Deep settings
    dnn_feature_columns=feature_columns,
    dnn_hidden_units=[128],
    dnn_optimizer=optimizer)

INFO:tensorflow:Using default config.
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpwl5e5eaq
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpwl5e5eaq
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpwl5e5eaq', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpwl5e5eaq', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}

combined_estimator.train(input_fn=_input_fn, steps=100)
combined_estimator.evaluate(input_fn=_eval_input_fn, steps=10)

INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/canned/linear.py:1478: UserWarning: `layer.add_variable` is deprecated and will be removed in a future version. Please use `layer.add_weight` method instead.
  getter=tf.compat.v1.get_variable)
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpwl5e5eaq/model.ckpt.
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpwl5e5eaq/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:loss = 2.5475807, step = 0
INFO:tensorflow:loss = 2.5475807, step = 0
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 20...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 20...
INFO:tensorflow:Saving checkpoints for 20 into /tmp/tmpwl5e5eaq/model.ckpt.
INFO:tensorflow:Saving checkpoints for 20 into /tmp/tmpwl5e5eaq/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 20...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 20...
INFO:tensorflow:Loss for final step: 0.58060575.
INFO:tensorflow:Loss for final step: 0.58060575.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2022-01-29T02:21:53
INFO:tensorflow:Starting evaluation at 2022-01-29T02:21:53
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpwl5e5eaq/model.ckpt-20
INFO:tensorflow:Restoring parameters from /tmp/tmpwl5e5eaq/model.ckpt-20
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Evaluation [1/10]
INFO:tensorflow:Evaluation [1/10]
INFO:tensorflow:Evaluation [2/10]
INFO:tensorflow:Evaluation [2/10]
INFO:tensorflow:Evaluation [3/10]
INFO:tensorflow:Evaluation [3/10]
INFO:tensorflow:Evaluation [4/10]
INFO:tensorflow:Evaluation [4/10]
INFO:tensorflow:Evaluation [5/10]
INFO:tensorflow:Evaluation [5/10]
INFO:tensorflow:Evaluation [6/10]
INFO:tensorflow:Evaluation [6/10]
INFO:tensorflow:Evaluation [7/10]
INFO:tensorflow:Evaluation [7/10]
INFO:tensorflow:Evaluation [8/10]
INFO:tensorflow:Evaluation [8/10]
INFO:tensorflow:Evaluation [9/10]
INFO:tensorflow:Evaluation [9/10]
INFO:tensorflow:Inference Time : 0.54029s
INFO:tensorflow:Inference Time : 0.54029s
INFO:tensorflow:Finished evaluation at 2022-01-29-02:21:53
INFO:tensorflow:Finished evaluation at 2022-01-29-02:21:53
INFO:tensorflow:Saving dict for global step 20: accuracy = 0.6931818, accuracy_baseline = 0.625, auc = 0.73532283, auc_precision_recall = 0.630229, average_loss = 0.65179086, global_step = 20, label/mean = 0.375, loss = 0.63768697, precision = 0.60714287, prediction/mean = 0.4162652, recall = 0.5151515
INFO:tensorflow:Saving dict for global step 20: accuracy = 0.6931818, accuracy_baseline = 0.625, auc = 0.73532283, auc_precision_recall = 0.630229, average_loss = 0.65179086, global_step = 20, label/mean = 0.375, loss = 0.63768697, precision = 0.60714287, prediction/mean = 0.4162652, recall = 0.5151515
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 20: /tmp/tmpwl5e5eaq/model.ckpt-20
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 20: /tmp/tmpwl5e5eaq/model.ckpt-20
{'accuracy': 0.6931818,
 'accuracy_baseline': 0.625,
 'auc': 0.73532283,
 'auc_precision_recall': 0.630229,
 'average_loss': 0.65179086,
 'label/mean': 0.375,
 'loss': 0.63768697,
 'precision': 0.60714287,
 'prediction/mean': 0.4162652,
 'recall': 0.5151515,
 'global_step': 20}

TF2: Używanie Keras WideDeepModel

W TensorFlow 2 można utworzyć instancję Keras tf.compat.v1.keras.models.WideDeepModel , aby zastąpić instancję wygenerowaną przez tf.estimator.DNNLinearCombinedEstimator , z podobnymi poziomami dostosowania określonymi przez użytkownika (na przykład, jak w poprzedni przykład, możliwość dostosowania wybranego optymalizatora modelu).

Ten WideDeepModel jest skonstruowany na podstawie składowego LinearModel i niestandardowego modelu DNN, z których oba zostały omówione w poprzednich dwóch przykładach. W razie potrzeby można również użyć niestandardowego modelu liniowego zamiast wbudowanego Keras LinearModel .

Jeśli chcesz zbudować własny model zamiast gotowego estymatora, sprawdź, jak zbudować model keras.Sequential . Więcej informacji na temat niestandardowych szkoleń i optymalizatorów można znaleźć w tym przewodniku .

# Create LinearModel and DNN Model as in Examples 1 and 2
optimizer = create_sample_optimizer('tf2')

linear_model = tf.compat.v1.keras.experimental.LinearModel()
linear_model.compile(loss='mse', optimizer=optimizer, metrics=['accuracy'])
linear_model.fit(x_train, y_train, epochs=10, verbose=0)

dnn_model = tf.keras.models.Sequential(
    [tf.keras.layers.Dense(128, activation='relu'),
     tf.keras.layers.Dense(1)])
dnn_model.compile(loss='mse', optimizer=optimizer, metrics=['accuracy'])

combined_model = tf.compat.v1.keras.experimental.WideDeepModel(linear_model,
                                                               dnn_model)
combined_model.compile(
    optimizer=[optimizer, optimizer], loss='mse', metrics=['accuracy'])
combined_model.fit([x_train, x_train], y_train, epochs=10)
combined_model.evaluate(x_eval, y_eval, return_dict=True)

Epoch 1/10
20/20 [==============================] - 0s 2ms/step - loss: 1118.0448 - accuracy: 0.6715
Epoch 2/10
20/20 [==============================] - 0s 2ms/step - loss: 0.5682 - accuracy: 0.7305
Epoch 3/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2719 - accuracy: 0.7671
Epoch 4/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2032 - accuracy: 0.7831
Epoch 5/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1911 - accuracy: 0.7783
Epoch 6/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1895 - accuracy: 0.7863
Epoch 7/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1882 - accuracy: 0.7863
Epoch 8/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1717 - accuracy: 0.7974
Epoch 9/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1701 - accuracy: 0.7927
Epoch 10/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1684 - accuracy: 0.7990
9/9 [==============================] - 0s 2ms/step - loss: 0.1930 - accuracy: 0.7424
{'loss': 0.19299836456775665, 'accuracy': 0.7424242496490479}

Przykład 4: Migracja z BoostedTreesEstimator

TF1: Korzystanie z narzędzia BoostedTreesEstimator

W TensorFlow 1 można użyć tf.estimator.BoostedTreesEstimator do stworzenia linii bazowej w celu stworzenia bazowego modelu Gradient Boosting przy użyciu zestawu drzew decyzyjnych dla problemów regresji i klasyfikacji. Ta funkcja nie jest już dostępna w TensorFlow 2.

bt_estimator = tf1.estimator.BoostedTreesEstimator(
    head=tf.estimator.BinaryClassHead(),
    n_batches_per_layer=1,
    max_depth=10,
    n_trees=1000,
    feature_columns=feature_columns)

bt_estimator.train(input_fn=_input_fn, steps=1000)
bt_estimator.evaluate(input_fn=_eval_input_fn, steps=100)

TF2: Korzystanie z lasów decyzyjnych TensorFlow

W TensorFlow 2 najbliższym, wstępnie zapakowanym substytutem modelu wygenerowanego przez tf.estimator.BoostedTreesEstimator jest model utworzony za pomocą tfdf.keras.GradientBoostedTreesModel , który tworzy sekwencyjnie uczoną sekwencję płytkich drzew decyzyjnych, z których każde ma na celu „uczenie się” na podstawie błędów wykonane przez jego poprzedników w sekwencji.

GradientBoostedTreesModel zapewnia więcej opcji dostosowywania, pozwalając na określenie wszystkiego, od podstawowych ograniczeń głębokości po warunki wczesnego zatrzymania. Zobacz więcej szczegółów atrybutu GradientBoostedTreesModel .

gbt_model = tfdf.keras.GradientBoostedTreesModel(
    task=tfdf.keras.Task.CLASSIFICATION)
gbt_model.compile(metrics=['mse', 'accuracy'])

Use /tmp/tmpbr1acn2_ as temporary training directory

train_df, eval_df = x_train.copy(), x_eval.copy()
train_df['survived'], eval_df['survived'] = y_train, y_eval

train_dataset = tfdf.keras.pd_dataframe_to_tf_dataset(train_df, label='survived')
eval_dataset = tfdf.keras.pd_dataframe_to_tf_dataset(eval_df, label='survived')

gbt_model.fit(train_dataset)
gbt_model.evaluate(eval_dataset, return_dict=True)

Starting reading the dataset
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_decision_forests/keras/core.py:2036: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only
  features_dataframe = dataframe.drop(label, 1)
1/1 [==============================] - ETA: 0s
Dataset read in 0:00:03.161776
Training model
Model trained in 0:00:00.102649
Compiling model
1/1 [==============================] - 3s 3s/step
[INFO kernel.cc:1153] Loading model from path
[INFO abstract_model.cc:1063] Engine "GradientBoostedTreesQuickScorerExtended" built
[INFO kernel.cc:1001] Use fast generic engine
WARNING:tensorflow:AutoGraph could not transform <function simple_ml_inference_op_with_handle at 0x7f95e9db4e60> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING:tensorflow:AutoGraph could not transform <function simple_ml_inference_op_with_handle at 0x7f95e9db4e60> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING: AutoGraph could not transform <function simple_ml_inference_op_with_handle at 0x7f95e9db4e60> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
1/1 [==============================] - 0s 388ms/step - loss: 0.0000e+00 - mse: 0.1308 - accuracy: 0.8144
{'loss': 0.0, 'mse': 0.13076548278331757, 'accuracy': 0.814393937587738}

W TensorFlow 2 dostępny jest również inny substytut TFDF dla modelu generowanego przez tf.estimator.BoostedTreesEstimator - tfdf.keras.RandomForestModel . RandomForestModel tworzy solidnego, odpornego na nadmierne dopasowanie ucznia składającego się z głosującej populacji głębokich drzew decyzyjnych, z których każdy jest szkolony na losowych podzbiorach wejściowego zestawu danych szkoleniowych.

RandomForestModel i GradientBoostedTreesModel zapewniają podobnie rozbudowane poziomy dostosowywania. Wybór między nimi jest specyficzny dla problemu i zależny od zadania lub aplikacji.

Zapoznaj się z dokumentacją interfejsu API, aby uzyskać więcej informacji na temat RandomForestModel i GradientBoostedTreesModel .

rf_model = tfdf.keras.RandomForestModel(
    task=tfdf.keras.Task.CLASSIFICATION)
rf_model.compile(metrics=['mse', 'accuracy'])

Use /tmp/tmpluh2ebcj as temporary training directory

rf_model.fit(train_dataset)
rf_model.evaluate(eval_dataset, return_dict=True)

Starting reading the dataset
1/1 [==============================] - ETA: 0s
Dataset read in 0:00:00.094262
Training model
Model trained in 0:00:00.083656
Compiling model
1/1 [==============================] - 0s 260ms/step
[INFO kernel.cc:1153] Loading model from path
[INFO kernel.cc:1001] Use fast generic engine
1/1 [==============================] - 0s 123ms/step - loss: 0.0000e+00 - mse: 0.1270 - accuracy: 0.8636
{'loss': 0.0, 'mse': 0.12698587775230408, 'accuracy': 0.8636363744735718}