 View on TensorFlow.org |
 Google Colab で実行 |
 GitHub でソースを表示{ |
概要
グラフ正則化は、Neural Graph Learning（Bui et al.、2018）の広範なパラダイムに基づく固有の手法です。その中核となる考え方は、グラフ正則化された対象を持つニューラルネットワークモデルを、ラベル付けされたデータとラベル付けされていないデータの両方を使用してトレーニングすることです。
このチュートリアルでは、自然な（有機的な）グラフを形成する文書を分類するためにグラフ正則化を使用することについて見ていきます。
Neural Structured Learning（NSL）フレームワークを使用してグラフ正則化モデルを作成する、一般的な方策は以下の通りです。
- 入力グラフとサンプル特徴からトレーニングデータを生成します。グラフのノードはサンプルに対応し、グラフのエッジはサンプルのペア間の類似性に対応します。結果として得られるトレーニングデータには、元のノード特徴に加え、近傍特徴が含まれます。
KerasSequential API、Functional API、または Subclass API を使用して、基本モデルとしてニューラルネットワークを作成します。- NSL フレームワークが提供する
GraphRegularizationラッパークラスで基本モデルをラップし、新しいグラフKerasモデルを作成します。この新しいモデルは、トレーニング目的の正則化項にグラフ正則化損失を含みます。 - グラフ
Kerasモデルをトレーニングして評価します。
セットアップ
Neural Structured Learning パッケージをインストールします。
pip install --quiet neural-structured-learning依存関係とインポート
import neural_structured_learning as nsl
import tensorflow as tf
# Resets notebook state
tf.keras.backend.clear_session()
print("Version: ", tf.__version__)
print("Eager mode: ", tf.executing_eagerly())
print(
"GPU is",
"available" if tf.config.list_physical_devices("GPU") else "NOT AVAILABLE")
Version: 2.4.1 Eager mode: True GPU is available
Cora データセット
Cora データセットはノードが機械学習論文を表現し、エッジが論文のペア間の引用を表現する引用グラフです。このタスクには、各論文を 7 つのカテゴリのいずれかに分類することを目的とする文書分類が含まれています。言い換えると、これは 7 つのクラスを持つマルチクラス分類問題です。
グラフ
元のグラフは有向グラフです。しかし、この例の目的のためには無向グラフを考慮します。つまり、論文 A が論文 B を引用している場合、論文 B も論文 A を引用していると考えます。これは必ずしも正しいわけではありませんが、この例においては、引用を類似性のプロキシとみなしており、通常は可換性を持つとみなされます。
特徴
入力された各論文には、次の 2 つの特徴が効果的に含まれています。
単語：論文中のテキストを密でマルチホットな Bag of Words（BoW）表現にしたもの。Cora データセットの語彙には 1433 個のユニークな単語が含まれています。つまり、この特徴の長さは 1433 で、 'i' の位置の値は語彙中の単語 'i' が論文中に存在するかどうかを示す 0 か 1 です。
ラベル: 論文のクラス ID（カテゴリ）を表現する単一の整数。
Cora データセットをダウンロードする
wget --quiet -P /tmp https://linqs-data.soe.ucsc.edu/public/lbc/cora.tgztar -C /tmp -xvzf /tmp/cora.tgz
cora/ cora/README cora/cora.cites cora/cora.content
Cora データセットを NSL 形式に変換する
Cora データセットを前処理し、Neural Structured Learning に必要な形式に変換するために、NSL の GitHub リポジトリに含まれている 'preprocess_cora_dataset.py' スクリプトを実行します。このスクリプトは以下を行います。
- 元のノード特徴とグラフを使用して近傍特徴を生成します。
tf.train.Exampleインスタンスを含むトレーニングデータとテストデータの分割を生成します。- 結果として得られたトレーニングデータとテストデータを
TFRecord形式で永続化します。
!wget https://raw.githubusercontent.com/tensorflow/neural-structured-learning/master/neural_structured_learning/examples/preprocess/cora/preprocess_cora_dataset.py
!python preprocess_cora_dataset.py \
--input_cora_content=/tmp/cora/cora.content \
--input_cora_graph=/tmp/cora/cora.cites \
--max_nbrs=5 \
--output_train_data=/tmp/cora/train_merged_examples.tfr \
--output_test_data=/tmp/cora/test_examples.tfr
--2021-02-12 22:29:54-- https://raw.githubusercontent.com/tensorflow/neural-structured-learning/master/neural_structured_learning/examples/preprocess/cora/preprocess_cora_dataset.py
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.110.133, 185.199.108.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response...
200 OK
Length: 11640 (11K) [text/plain]
Saving to: ‘preprocess_cora_dataset.py’
preproces 0%[ ] 0 --.-KB/s
preprocess_cora_dat 100%[===================>] 11.37K --.-KB/s in 0.001s
2021-02-12 22:29:54 (19.3 MB/s) - ‘preprocess_cora_dataset.py’ saved [11640/11640]
2021-02-12 22:29:55.197371: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
Reading graph file: /tmp/cora/cora.cites...
Done reading 5429 edges from: /tmp/cora/cora.cites (0.01 seconds).
Making all edges bi-directional...
Done (0.01 seconds). Total graph nodes: 2708
Joining seed and neighbor tf.train.Examples with graph edges...
Done creating and writing 2155 merged tf.train.Examples (1.36 seconds).
Out-degree histogram: [(1, 386), (2, 468), (3, 452), (4, 309), (5, 540)]
Output training data written to TFRecord file: /tmp/cora/train_merged_examples.tfr.
Output test data written to TFRecord file: /tmp/cora/test_examples.tfr.
Total running time: 0.04 minutes.
グローバル変数
トレーニングデータとテストデータへのファイルパスは、上記 'preprocess_cora_dataset.py' スクリプトの呼び出しに使用したコマンドラインフラグの値に基づきます。
### Experiment dataset
TRAIN_DATA_PATH = '/tmp/cora/train_merged_examples.tfr'
TEST_DATA_PATH = '/tmp/cora/test_examples.tfr'
### Constants used to identify neighbor features in the input.
NBR_FEATURE_PREFIX = 'NL_nbr_'
NBR_WEIGHT_SUFFIX = '_weight'
ハイパーパラメータ
HParams のインスタンスを使用して、トレーニングと評価に使用する様々なハイパーパラメータと定数をインクルードします。それぞれについての簡単な説明を以下に示します。
num_classes：7 つの異なるクラスがあります。
max_seq_length：これは語彙のサイズであり、入力内のすべてのインスタンスは、密でマルチホットな Bag of Words 表現を持ちます。言い換えれば、ある単語の値が 1 ならば、その単語が入力内に存在することを示し、値が 0 ならば、その単語が入力内に存在しないことを示します。
distance_type：これはサンプルをその近傍と正則化する際に使用する距離メトリックです。
graph_regularization_multiplier：これは全体の損失関数においてグラフ正則化項の相対的な重みを制御します。
num_neighbors：グラフ正則化に使用する近傍の数を指定します。この値は上で
preprocess_cora_dataset.pyの実行時に使用したコマンドライン引数max_nbrs以下でなければなりません。num_fc_units：ニューラルネットワーク内の完全に接続されたレイヤー数。
train_epochs：トレーニングのエポック数。
batch_size：トレーニングや評価に使用するバッチサイズ。
dropout_rate：完全に接続された各レイヤーの後のドロップアウト率を制御します。
eval_steps：評価が完了したと判断するまでに処理を行うバッチ数。
None設定にすると、テストセット内の全てのインスタンスを評価します。
class HParams(object):
"""Hyperparameters used for training."""
def __init__(self):
### dataset parameters
self.num_classes = 7
self.max_seq_length = 1433
### neural graph learning parameters
self.distance_type = nsl.configs.DistanceType.L2
self.graph_regularization_multiplier = 0.1
self.num_neighbors = 1
### model architecture
self.num_fc_units = [50, 50]
### training parameters
self.train_epochs = 100
self.batch_size = 128
self.dropout_rate = 0.5
### eval parameters
self.eval_steps = None # All instances in the test set are evaluated.
HPARAMS = HParams()
トレーニングデータとテストデータを読み込む
先にこのノートブックで説明したように、入力のトレーニングデータとテストデータは 'preprocess_cora_dataset.py' によって作成されています。これらのデータを 2 つの tf.data.Dataset オブジェクトに読み込みます。1 つはトレーニング用、もう1 つはテスト用です。
モデルの入力レイヤー内では、各サンプルから「単語」と「ラベル」の特徴だけでなく、hparams.num_neighbors の値に基づき対応する近傍特徴も抽出します。近傍が hparams.num_neighbors よりも少ないインスタンスでは、存在しない近傍特徴にダミー値を割り当てます。
def make_dataset(file_path, training=False):
"""Creates a `tf.data.TFRecordDataset`.
Args:
file_path: Name of the file in the `.tfrecord` format containing
`tf.train.Example` objects.
training: Boolean indicating if we are in training mode.
Returns:
An instance of `tf.data.TFRecordDataset` containing the `tf.train.Example`
objects.
"""
def parse_example(example_proto):
"""Extracts relevant fields from the `example_proto`.
Args:
example_proto: An instance of `tf.train.Example`.
Returns:
A pair whose first value is a dictionary containing relevant features
and whose second value contains the ground truth label.
"""
# The 'words' feature is a multi-hot, bag-of-words representation of the
# original raw text. A default value is required for examples that don't
# have the feature.
feature_spec = {
'words':
tf.io.FixedLenFeature([HPARAMS.max_seq_length],
tf.int64,
default_value=tf.constant(
0,
dtype=tf.int64,
shape=[HPARAMS.max_seq_length])),
'label':
tf.io.FixedLenFeature((), tf.int64, default_value=-1),
}
# We also extract corresponding neighbor features in a similar manner to
# the features above during training.
if training:
for i in range(HPARAMS.num_neighbors):
nbr_feature_key = '{}{}_{}'.format(NBR_FEATURE_PREFIX, i, 'words')
nbr_weight_key = '{}{}{}'.format(NBR_FEATURE_PREFIX, i,
NBR_WEIGHT_SUFFIX)
feature_spec[nbr_feature_key] = tf.io.FixedLenFeature(
[HPARAMS.max_seq_length],
tf.int64,
default_value=tf.constant(
0, dtype=tf.int64, shape=[HPARAMS.max_seq_length]))
# We assign a default value of 0.0 for the neighbor weight so that
# graph regularization is done on samples based on their exact number
# of neighbors. In other words, non-existent neighbors are discounted.
feature_spec[nbr_weight_key] = tf.io.FixedLenFeature(
[1], tf.float32, default_value=tf.constant([0.0]))
features = tf.io.parse_single_example(example_proto, feature_spec)
label = features.pop('label')
return features, label
dataset = tf.data.TFRecordDataset([file_path])
if training:
dataset = dataset.shuffle(10000)
dataset = dataset.map(parse_example)
dataset = dataset.batch(HPARAMS.batch_size)
return dataset
train_dataset = make_dataset(TRAIN_DATA_PATH, training=True)
test_dataset = make_dataset(TEST_DATA_PATH)
トレーニングデータセットの中身を覗いてみましょう。
for feature_batch, label_batch in train_dataset.take(1):
print('Feature list:', list(feature_batch.keys()))
print('Batch of inputs:', feature_batch['words'])
nbr_feature_key = '{}{}_{}'.format(NBR_FEATURE_PREFIX, 0, 'words')
nbr_weight_key = '{}{}{}'.format(NBR_FEATURE_PREFIX, 0, NBR_WEIGHT_SUFFIX)
print('Batch of neighbor inputs:', feature_batch[nbr_feature_key])
print('Batch of neighbor weights:',
tf.reshape(feature_batch[nbr_weight_key], [-1]))
print('Batch of labels:', label_batch)
Feature list: ['NL_nbr_0_weight', 'NL_nbr_0_words', 'words'] Batch of inputs: tf.Tensor( [[0 0 0 ... 0 0 0] [0 0 0 ... 0 0 0] [0 0 0 ... 0 0 0] ... [0 0 0 ... 0 0 0] [0 0 0 ... 0 0 0] [0 0 0 ... 0 0 0]], shape=(128, 1433), dtype=int64) Batch of neighbor inputs: tf.Tensor( [[0 0 0 ... 0 0 0] [0 0 0 ... 0 0 0] [0 0 0 ... 0 0 0] ... [0 0 0 ... 0 0 0] [0 0 0 ... 0 0 0] [0 0 0 ... 0 0 0]], shape=(128, 1433), dtype=int64) Batch of neighbor weights: tf.Tensor( [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.], shape=(128,), dtype=float32) Batch of labels: tf.Tensor( [1 3 3 6 3 3 2 0 3 2 6 3 2 1 2 3 2 3 1 1 2 0 5 2 2 2 1 2 2 2 0 2 0 1 1 6 6 5 6 5 6 5 1 0 1 4 1 5 1 3 3 0 6 1 1 2 6 5 0 3 6 4 2 6 2 3 2 3 2 0 3 1 2 2 0 2 3 4 1 2 0 4 6 2 4 3 3 4 0 1 3 3 3 6 2 6 1 1 2 0 3 3 2 5 4 4 1 3 1 3 5 3 3 5 2 6 2 3 5 3 0 3 1 6 1 1 3 3], shape=(128,), dtype=int64)
テストデータセットの中身を覗いてみましょう。
for feature_batch, label_batch in test_dataset.take(1):
print('Feature list:', list(feature_batch.keys()))
print('Batch of inputs:', feature_batch['words'])
print('Batch of labels:', label_batch)
Feature list: ['words'] Batch of inputs: tf.Tensor( [[0 0 0 ... 0 0 0] [0 0 0 ... 0 0 0] [0 0 0 ... 0 0 0] ... [0 0 0 ... 0 0 0] [0 0 0 ... 0 0 0] [0 0 0 ... 0 0 0]], shape=(128, 1433), dtype=int64) Batch of labels: tf.Tensor( [5 2 2 2 1 2 6 3 2 3 6 1 3 6 4 4 2 3 3 0 2 0 5 2 1 0 6 3 6 4 2 2 3 0 4 2 2 2 2 3 2 2 2 0 2 2 2 2 4 2 3 4 0 2 6 2 1 4 2 0 0 1 4 2 6 0 5 2 2 3 2 5 2 5 2 3 2 2 2 2 2 6 6 3 2 4 2 6 3 2 2 6 2 4 2 2 1 3 4 6 0 0 2 4 2 1 3 6 6 2 6 6 6 1 4 6 4 3 6 6 0 0 2 6 2 4 0 0], shape=(128,), dtype=int64)
モデルの定義
グラフ正則化の使い方を実証するために、まずこの問題の基本モデルを構築します。2 つの隠れレイヤーとその間にドロップアウトを持つ単純なフィードフォワード ニューラルネットワークを使用します。ここでは tf.Keras フレームワークでサポートされているすべてのモデルタイプ（Sequential モデル、Functional モデル、Subclass モデル）を使用して基本モデルを作成します。
Sequential 基本モデル
def make_mlp_sequential_model(hparams):
"""Creates a sequential multi-layer perceptron model."""
model = tf.keras.Sequential()
model.add(
tf.keras.layers.InputLayer(
input_shape=(hparams.max_seq_length,), name='words'))
# Input is already one-hot encoded in the integer format. We cast it to
# floating point format here.
model.add(
tf.keras.layers.Lambda(lambda x: tf.keras.backend.cast(x, tf.float32)))
for num_units in hparams.num_fc_units:
model.add(tf.keras.layers.Dense(num_units, activation='relu'))
# For sequential models, by default, Keras ensures that the 'dropout' layer
# is invoked only during training.
model.add(tf.keras.layers.Dropout(hparams.dropout_rate))
model.add(tf.keras.layers.Dense(hparams.num_classes, activation='softmax'))
return model
Functional 基本モデル
def make_mlp_functional_model(hparams):
"""Creates a functional API-based multi-layer perceptron model."""
inputs = tf.keras.Input(
shape=(hparams.max_seq_length,), dtype='int64', name='words')
# Input is already one-hot encoded in the integer format. We cast it to
# floating point format here.
cur_layer = tf.keras.layers.Lambda(
lambda x: tf.keras.backend.cast(x, tf.float32))(
inputs)
for num_units in hparams.num_fc_units:
cur_layer = tf.keras.layers.Dense(num_units, activation='relu')(cur_layer)
# For functional models, by default, Keras ensures that the 'dropout' layer
# is invoked only during training.
cur_layer = tf.keras.layers.Dropout(hparams.dropout_rate)(cur_layer)
outputs = tf.keras.layers.Dense(
hparams.num_classes, activation='softmax')(
cur_layer)
model = tf.keras.Model(inputs, outputs=outputs)
return model
Subclass 基本モデル
def make_mlp_subclass_model(hparams):
"""Creates a multi-layer perceptron subclass model in Keras."""
class MLP(tf.keras.Model):
"""Subclass model defining a multi-layer perceptron."""
def __init__(self):
super(MLP, self).__init__()
# Input is already one-hot encoded in the integer format. We create a
# layer to cast it to floating point format here.
self.cast_to_float_layer = tf.keras.layers.Lambda(
lambda x: tf.keras.backend.cast(x, tf.float32))
self.dense_layers = [
tf.keras.layers.Dense(num_units, activation='relu')
for num_units in hparams.num_fc_units
]
self.dropout_layer = tf.keras.layers.Dropout(hparams.dropout_rate)
self.output_layer = tf.keras.layers.Dense(
hparams.num_classes, activation='softmax')
def call(self, inputs, training=False):
cur_layer = self.cast_to_float_layer(inputs['words'])
for dense_layer in self.dense_layers:
cur_layer = dense_layer(cur_layer)
cur_layer = self.dropout_layer(cur_layer, training=training)
outputs = self.output_layer(cur_layer)
return outputs
return MLP()
基本モデルを作成する
# Create a base MLP model using the functional API.
# Alternatively, you can also create a sequential or subclass base model using
# the make_mlp_sequential_model() or make_mlp_subclass_model() functions
# respectively, defined above. Note that if a subclass model is used, its
# summary cannot be generated until it is built.
base_model_tag, base_model = 'FUNCTIONAL', make_mlp_functional_model(HPARAMS)
base_model.summary()
Model: "model" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= words (InputLayer) [(None, 1433)] 0 _________________________________________________________________ lambda (Lambda) (None, 1433) 0 _________________________________________________________________ dense (Dense) (None, 50) 71700 _________________________________________________________________ dropout (Dropout) (None, 50) 0 _________________________________________________________________ dense_1 (Dense) (None, 50) 2550 _________________________________________________________________ dropout_1 (Dropout) (None, 50) 0 _________________________________________________________________ dense_2 (Dense) (None, 7) 357 ================================================================= Total params: 74,607 Trainable params: 74,607 Non-trainable params: 0 _________________________________________________________________
基本 MLP モデルをトレーニングする
# Compile and train the base MLP model
base_model.compile(
optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
base_model.fit(train_dataset, epochs=HPARAMS.train_epochs, verbose=1)
Epoch 1/100 /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['NL_nbr_0_weight', 'NL_nbr_0_words'] which did not match any model input. They will be ignored by the model. [n for n in tensors.keys() if n not in ref_input_names]) 1/Unknown - 1s 765ms/step - loss: 1.9805 - accuracy: 0.1250 7/Unknown - 1s 10ms/step - loss: 1.9773 - accuracy: 0.1224 12/Unknown - 1s 10ms/step - loss: 1.9693 - accuracy: 0.1349 17/17 [==============================] - 1s 11ms/step - loss: 1.9608 - accuracy: 0.1465 Epoch 2/100 1/17 [>.............................] - ETA: 0s - loss: 1.8903 - accuracy: 0.2578 6/17 [=========>....................] - ETA: 0s - loss: 1.8705 - accuracy: 0.2707 11/17 [==================>...........] - ETA: 0s - loss: 1.8632 - accuracy: 0.2760 16/17 [===========================>..] - ETA: 0s - loss: 1.8594 - accuracy: 0.2781 17/17 [==============================] - 0s 11ms/step - loss: 1.8578 - accuracy: 0.2793 Epoch 3/100 1/17 [>.............................] - ETA: 0s - loss: 1.8223 - accuracy: 0.3125 6/17 [=========>....................] - ETA: 0s - loss: 1.7910 - accuracy: 0.3464 11/17 [==================>...........] - ETA: 0s - loss: 1.7842 - accuracy: 0.3477 17/17 [==============================] - ETA: 0s - loss: 1.7756 - accuracy: 0.3469 17/17 [==============================] - 0s 11ms/step - loss: 1.7744 - accuracy: 0.3468 Epoch 4/100 1/17 [>.............................] - ETA: 0s - loss: 1.7688 - accuracy: 0.2969 6/17 [=========>....................] - ETA: 0s - loss: 1.7310 - accuracy: 0.3281 11/17 [==================>...........] - ETA: 0s - loss: 1.7062 - accuracy: 0.3423 16/17 [===========================>..] - ETA: 0s - loss: 1.6904 - accuracy: 0.3510 17/17 [==============================] - 0s 10ms/step - loss: 1.6850 - accuracy: 0.3542 Epoch 5/100 1/17 [>.............................] - ETA: 0s - loss: 1.6577 - accuracy: 0.3984 7/17 [===========>..................] - ETA: 0s - loss: 1.5866 - accuracy: 0.3907 12/17 [====================>.........] - ETA: 0s - loss: 1.5691 - accuracy: 0.3983 17/17 [==============================] - ETA: 0s - loss: 1.5537 - accuracy: 0.4052 17/17 [==============================] - 0s 11ms/step - loss: 1.5511 - accuracy: 0.4065 Epoch 6/100 1/17 [>.............................] - ETA: 0s - loss: 1.3940 - accuracy: 0.4844 6/17 [=========>....................] - ETA: 0s - loss: 1.4218 - accuracy: 0.4883 11/17 [==================>...........] - ETA: 0s - loss: 1.4066 - accuracy: 0.5032 16/17 [===========================>..] - ETA: 0s - loss: 1.3881 - accuracy: 0.5136 17/17 [==============================] - 0s 11ms/step - loss: 1.3826 - accuracy: 0.5161 Epoch 7/100 1/17 [>.............................] - ETA: 0s - loss: 1.1945 - accuracy: 0.5547 6/17 [=========>....................] - ETA: 0s - loss: 1.2136 - accuracy: 0.5672 11/17 [==================>...........] - ETA: 0s - loss: 1.2078 - accuracy: 0.5811 16/17 [===========================>..] - ETA: 0s - loss: 1.2059 - accuracy: 0.5866 17/17 [==============================] - 0s 11ms/step - loss: 1.2052 - accuracy: 0.5874 Epoch 8/100 1/17 [>.............................] - ETA: 0s - loss: 1.1082 - accuracy: 0.6406 6/17 [=========>....................] - ETA: 0s - loss: 1.1025 - accuracy: 0.6471 11/17 [==================>...........] - ETA: 0s - loss: 1.0974 - accuracy: 0.6428 16/17 [===========================>..] - ETA: 0s - loss: 1.0903 - accuracy: 0.6432 17/17 [==============================] - 0s 11ms/step - loss: 1.0876 - accuracy: 0.6437 Epoch 9/100 1/17 [>.............................] - ETA: 0s - loss: 0.9366 - accuracy: 0.7031 7/17 [===========>..................] - ETA: 0s - loss: 0.9649 - accuracy: 0.6846 12/17 [====================>.........] - ETA: 0s - loss: 0.9659 - accuracy: 0.6845 17/17 [==============================] - ETA: 0s - loss: 0.9627 - accuracy: 0.6862 17/17 [==============================] - 0s 10ms/step - loss: 0.9621 - accuracy: 0.6866 Epoch 10/100 1/17 [>.............................] - ETA: 0s - loss: 0.9635 - accuracy: 0.6484 7/17 [===========>..................] - ETA: 0s - loss: 0.9047 - accuracy: 0.6962 12/17 [====================>.........] - ETA: 0s - loss: 0.8992 - accuracy: 0.6985 17/17 [==============================] - ETA: 0s - loss: 0.8902 - accuracy: 0.7032 17/17 [==============================] - 0s 11ms/step - loss: 0.8881 - accuracy: 0.7042 Epoch 11/100 1/17 [>.............................] - ETA: 0s - loss: 0.8266 - accuracy: 0.7422 6/17 [=========>....................] - ETA: 0s - loss: 0.8203 - accuracy: 0.7285 11/17 [==================>...........] - ETA: 0s - loss: 0.8131 - accuracy: 0.7298 16/17 [===========================>..] - ETA: 0s - loss: 0.8062 - accuracy: 0.7350 17/17 [==============================] - 0s 11ms/step - loss: 0.8042 - accuracy: 0.7365 Epoch 12/100 1/17 [>.............................] - ETA: 0s - loss: 0.5993 - accuracy: 0.7891 6/17 [=========>....................] - ETA: 0s - loss: 0.7032 - accuracy: 0.7673 11/17 [==================>...........] - ETA: 0s - loss: 0.7157 - accuracy: 0.7668 16/17 [===========================>..] - ETA: 0s - loss: 0.7171 - accuracy: 0.7675 17/17 [==============================] - 0s 11ms/step - loss: 0.7164 - accuracy: 0.7680 Epoch 13/100 1/17 [>.............................] - ETA: 0s - loss: 0.6514 - accuracy: 0.7969 6/17 [=========>....................] - ETA: 0s - loss: 0.6292 - accuracy: 0.8140 11/17 [==================>...........] - ETA: 0s - loss: 0.6383 - accuracy: 0.8087 16/17 [===========================>..] - ETA: 0s - loss: 0.6378 - accuracy: 0.8079 17/17 [==============================] - 0s 11ms/step - loss: 0.6374 - accuracy: 0.8080 Epoch 14/100 1/17 [>.............................] - ETA: 0s - loss: 0.6004 - accuracy: 0.8047 6/17 [=========>....................] - ETA: 0s - loss: 0.5620 - accuracy: 0.8300 11/17 [==================>...........] - ETA: 0s - loss: 0.5745 - accuracy: 0.8225 16/17 [===========================>..] - ETA: 0s - loss: 0.5809 - accuracy: 0.8178 17/17 [==============================] - 0s 10ms/step - loss: 0.5826 - accuracy: 0.8164 Epoch 15/100 1/17 [>.............................] - ETA: 0s - loss: 0.3991 - accuracy: 0.8984 6/17 [=========>....................] - ETA: 0s - loss: 0.4764 - accuracy: 0.8629 11/17 [==================>...........] - ETA: 0s - loss: 0.5018 - accuracy: 0.8507 16/17 [===========================>..] - ETA: 0s - loss: 0.5140 - accuracy: 0.8440 17/17 [==============================] - 0s 12ms/step - loss: 0.5169 - accuracy: 0.8426 Epoch 16/100 1/17 [>.............................] - ETA: 0s - loss: 0.5253 - accuracy: 0.8828 6/17 [=========>....................] - ETA: 0s - loss: 0.5670 - accuracy: 0.8338 11/17 [==================>...........] - ETA: 0s - loss: 0.5622 - accuracy: 0.8326 16/17 [===========================>..] - ETA: 0s - loss: 0.5520 - accuracy: 0.8343 17/17 [==============================] - 0s 11ms/step - loss: 0.5486 - accuracy: 0.8348 Epoch 17/100 1/17 [>.............................] - ETA: 0s - loss: 0.5084 - accuracy: 0.8281 6/17 [=========>....................] - ETA: 0s - loss: 0.4796 - accuracy: 0.8442 11/17 [==================>...........] - ETA: 0s - loss: 0.4740 - accuracy: 0.8520 16/17 [===========================>..] - ETA: 0s - loss: 0.4700 - accuracy: 0.8557 17/17 [==============================] - 0s 11ms/step - loss: 0.4695 - accuracy: 0.8565 Epoch 18/100 1/17 [>.............................] - ETA: 0s - loss: 0.3931 - accuracy: 0.8828 7/17 [===========>..................] - ETA: 0s - loss: 0.4412 - accuracy: 0.8667 13/17 [=====================>........] - ETA: 0s - loss: 0.4353 - accuracy: 0.8689 17/17 [==============================] - 0s 10ms/step - loss: 0.4333 - accuracy: 0.8688 Epoch 19/100 1/17 [>.............................] - ETA: 0s - loss: 0.3538 - accuracy: 0.9062 6/17 [=========>....................] - ETA: 0s - loss: 0.4203 - accuracy: 0.8745 11/17 [==================>...........] - ETA: 0s - loss: 0.4199 - accuracy: 0.8721 16/17 [===========================>..] - ETA: 0s - loss: 0.4162 - accuracy: 0.8734 17/17 [==============================] - 0s 10ms/step - loss: 0.4156 - accuracy: 0.8735 Epoch 20/100 1/17 [>.............................] - ETA: 0s - loss: 0.4317 - accuracy: 0.8438 7/17 [===========>..................] - ETA: 0s - loss: 0.3869 - accuracy: 0.8834 12/17 [====================>.........] - ETA: 0s - loss: 0.3850 - accuracy: 0.8863 17/17 [==============================] - ETA: 0s - loss: 0.3805 - accuracy: 0.8877 17/17 [==============================] - 0s 11ms/step - loss: 0.3798 - accuracy: 0.8881 Epoch 21/100 1/17 [>.............................] - ETA: 0s - loss: 0.3472 - accuracy: 0.9219 6/17 [=========>....................] - ETA: 0s - loss: 0.3531 - accuracy: 0.9081 11/17 [==================>...........] - ETA: 0s - loss: 0.3504 - accuracy: 0.9031 16/17 [===========================>..] - ETA: 0s - loss: 0.3475 - accuracy: 0.9024 17/17 [==============================] - 0s 11ms/step - loss: 0.3469 - accuracy: 0.9021 Epoch 22/100 1/17 [>.............................] - ETA: 0s - loss: 0.2655 - accuracy: 0.9141 6/17 [=========>....................] - ETA: 0s - loss: 0.3037 - accuracy: 0.9068 11/17 [==================>...........] - ETA: 0s - loss: 0.3084 - accuracy: 0.9080 16/17 [===========================>..] - ETA: 0s - loss: 0.3096 - accuracy: 0.9089 17/17 [==============================] - 0s 11ms/step - loss: 0.3103 - accuracy: 0.9090 Epoch 23/100 1/17 [>.............................] - ETA: 0s - loss: 0.3321 - accuracy: 0.8750 6/17 [=========>....................] - ETA: 0s - loss: 0.3262 - accuracy: 0.8813 11/17 [==================>...........] - ETA: 0s - loss: 0.3285 - accuracy: 0.8853 16/17 [===========================>..] - ETA: 0s - loss: 0.3291 - accuracy: 0.8880 17/17 [==============================] - 0s 11ms/step - loss: 0.3284 - accuracy: 0.8891 Epoch 24/100 1/17 [>.............................] - ETA: 0s - loss: 0.2106 - accuracy: 0.9609 6/17 [=========>....................] - ETA: 0s - loss: 0.2597 - accuracy: 0.9283 11/17 [==================>...........] - ETA: 0s - loss: 0.2690 - accuracy: 0.9233 16/17 [===========================>..] - ETA: 0s - loss: 0.2747 - accuracy: 0.9203 17/17 [==============================] - 0s 11ms/step - loss: 0.2758 - accuracy: 0.9196 Epoch 25/100 1/17 [>.............................] - ETA: 0s - loss: 0.2664 - accuracy: 0.9062 6/17 [=========>....................] - ETA: 0s - loss: 0.2880 - accuracy: 0.9078 11/17 [==================>...........] - ETA: 0s - loss: 0.2800 - accuracy: 0.9118 16/17 [===========================>..] - ETA: 0s - loss: 0.2784 - accuracy: 0.9122 17/17 [==============================] - 0s 10ms/step - loss: 0.2780 - accuracy: 0.9124 Epoch 26/100 1/17 [>.............................] - ETA: 0s - loss: 0.2145 - accuracy: 0.9531 6/17 [=========>....................] - ETA: 0s - loss: 0.2105 - accuracy: 0.9521 11/17 [==================>...........] - ETA: 0s - loss: 0.2144 - accuracy: 0.9482 16/17 [===========================>..] - ETA: 0s - loss: 0.2218 - accuracy: 0.9440 17/17 [==============================] - 0s 11ms/step - loss: 0.2244 - accuracy: 0.9427 Epoch 27/100 1/17 [>.............................] - ETA: 0s - loss: 0.2343 - accuracy: 0.9219 6/17 [=========>....................] - ETA: 0s - loss: 0.2557 - accuracy: 0.9163 11/17 [==================>...........] - ETA: 0s - loss: 0.2574 - accuracy: 0.9190 16/17 [===========================>..] - ETA: 0s - loss: 0.2563 - accuracy: 0.9208 17/17 [==============================] - 0s 11ms/step - loss: 0.2555 - accuracy: 0.9215 Epoch 28/100 1/17 [>.............................] - ETA: 0s - loss: 0.2723 - accuracy: 0.9453 7/17 [===========>..................] - ETA: 0s - loss: 0.2319 - accuracy: 0.9428 12/17 [====================>.........] - ETA: 0s - loss: 0.2280 - accuracy: 0.9412 17/17 [==============================] - ETA: 0s - loss: 0.2260 - accuracy: 0.9411 17/17 [==============================] - 0s 11ms/step - loss: 0.2261 - accuracy: 0.9410 Epoch 29/100 1/17 [>.............................] - ETA: 0s - loss: 0.2911 - accuracy: 0.9141 6/17 [=========>....................] - ETA: 0s - loss: 0.2640 - accuracy: 0.9197 12/17 [====================>.........] - ETA: 0s - loss: 0.2563 - accuracy: 0.9224 17/17 [==============================] - ETA: 0s - loss: 0.2549 - accuracy: 0.9227 17/17 [==============================] - 0s 10ms/step - loss: 0.2545 - accuracy: 0.9228 Epoch 30/100 1/17 [>.............................] - ETA: 0s - loss: 0.2085 - accuracy: 0.9297 6/17 [=========>....................] - ETA: 0s - loss: 0.2244 - accuracy: 0.9281 11/17 [==================>...........] - ETA: 0s - loss: 0.2209 - accuracy: 0.9312 16/17 [===========================>..] - ETA: 0s - loss: 0.2170 - accuracy: 0.9345 17/17 [==============================] - 0s 11ms/step - loss: 0.2161 - accuracy: 0.9354 Epoch 31/100 1/17 [>.............................] - ETA: 0s - loss: 0.2069 - accuracy: 0.9453 6/17 [=========>....................] - ETA: 0s - loss: 0.2075 - accuracy: 0.9448 11/17 [==================>...........] - ETA: 0s - loss: 0.2072 - accuracy: 0.9456 16/17 [===========================>..] - ETA: 0s - loss: 0.2066 - accuracy: 0.9450 17/17 [==============================] - 0s 11ms/step - loss: 0.2065 - accuracy: 0.9445 Epoch 32/100 1/17 [>.............................] - ETA: 0s - loss: 0.2472 - accuracy: 0.9141 6/17 [=========>....................] - ETA: 0s - loss: 0.2327 - accuracy: 0.9230 11/17 [==================>...........] - ETA: 0s - loss: 0.2264 - accuracy: 0.9284 16/17 [===========================>..] - ETA: 0s - loss: 0.2196 - accuracy: 0.9325 17/17 [==============================] - 0s 11ms/step - loss: 0.2176 - accuracy: 0.9336 Epoch 33/100 1/17 [>.............................] - ETA: 0s - loss: 0.1912 - accuracy: 0.9531 6/17 [=========>....................] - ETA: 0s - loss: 0.2087 - accuracy: 0.9398 11/17 [==================>...........] - ETA: 0s - loss: 0.2051 - accuracy: 0.9403 16/17 [===========================>..] - ETA: 0s - loss: 0.2019 - accuracy: 0.9416 17/17 [==============================] - 0s 11ms/step - loss: 0.2013 - accuracy: 0.9421 Epoch 34/100 1/17 [>.............................] - ETA: 0s - loss: 0.1356 - accuracy: 0.9766 6/17 [=========>....................] - ETA: 0s - loss: 0.1527 - accuracy: 0.9607 12/17 [====================>.........] - ETA: 0s - loss: 0.1599 - accuracy: 0.9543 17/17 [==============================] - ETA: 0s - loss: 0.1644 - accuracy: 0.9516 17/17 [==============================] - 0s 10ms/step - loss: 0.1651 - accuracy: 0.9513 Epoch 35/100 1/17 [>.............................] - ETA: 0s - loss: 0.2358 - accuracy: 0.9375 6/17 [=========>....................] - ETA: 0s - loss: 0.2021 - accuracy: 0.9496 11/17 [==================>...........] - ETA: 0s - loss: 0.1919 - accuracy: 0.9510 16/17 [===========================>..] - ETA: 0s - loss: 0.1862 - accuracy: 0.9515 17/17 [==============================] - 0s 10ms/step - loss: 0.1848 - accuracy: 0.9514 Epoch 36/100 1/17 [>.............................] - ETA: 0s - loss: 0.1592 - accuracy: 0.9453 6/17 [=========>....................] - ETA: 0s - loss: 0.1512 - accuracy: 0.9608 11/17 [==================>...........] - ETA: 0s - loss: 0.1594 - accuracy: 0.9586 16/17 [===========================>..] - ETA: 0s - loss: 0.1628 - accuracy: 0.9563 17/17 [==============================] - 0s 11ms/step - loss: 0.1634 - accuracy: 0.9558 Epoch 37/100 1/17 [>.............................] - ETA: 0s - loss: 0.1561 - accuracy: 0.9766 6/17 [=========>....................] - ETA: 0s - loss: 0.1514 - accuracy: 0.9689 12/17 [====================>.........] - ETA: 0s - loss: 0.1593 - accuracy: 0.9623 17/17 [==============================] - ETA: 0s - loss: 0.1604 - accuracy: 0.9601 17/17 [==============================] - 0s 10ms/step - loss: 0.1605 - accuracy: 0.9598 Epoch 38/100 1/17 [>.............................] - ETA: 0s - loss: 0.1276 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.1480 - accuracy: 0.9638 11/17 [==================>...........] - ETA: 0s - loss: 0.1449 - accuracy: 0.9646 16/17 [===========================>..] - ETA: 0s - loss: 0.1439 - accuracy: 0.9652 17/17 [==============================] - 0s 10ms/step - loss: 0.1438 - accuracy: 0.9651 Epoch 39/100 1/17 [>.............................] - ETA: 0s - loss: 0.1554 - accuracy: 0.9688 6/17 [=========>....................] - ETA: 0s - loss: 0.1585 - accuracy: 0.9626 11/17 [==================>...........] - ETA: 0s - loss: 0.1602 - accuracy: 0.9585 16/17 [===========================>..] - ETA: 0s - loss: 0.1606 - accuracy: 0.9572 17/17 [==============================] - 0s 11ms/step - loss: 0.1602 - accuracy: 0.9569 Epoch 40/100 1/17 [>.............................] - ETA: 0s - loss: 0.1876 - accuracy: 0.9297 6/17 [=========>....................] - ETA: 0s - loss: 0.1625 - accuracy: 0.9493 11/17 [==================>...........] - ETA: 0s - loss: 0.1536 - accuracy: 0.9550 16/17 [===========================>..] - ETA: 0s - loss: 0.1498 - accuracy: 0.9572 17/17 [==============================] - 0s 10ms/step - loss: 0.1487 - accuracy: 0.9576 Epoch 41/100 1/17 [>.............................] - ETA: 0s - loss: 0.1340 - accuracy: 0.9922 6/17 [=========>....................] - ETA: 0s - loss: 0.1369 - accuracy: 0.9817 11/17 [==================>...........] - ETA: 0s - loss: 0.1313 - accuracy: 0.9787 16/17 [===========================>..] - ETA: 0s - loss: 0.1318 - accuracy: 0.9753 17/17 [==============================] - 0s 10ms/step - loss: 0.1324 - accuracy: 0.9742 Epoch 42/100 1/17 [>.............................] - ETA: 0s - loss: 0.0767 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.0991 - accuracy: 0.9769 11/17 [==================>...........] - ETA: 0s - loss: 0.1106 - accuracy: 0.9721 16/17 [===========================>..] - ETA: 0s - loss: 0.1157 - accuracy: 0.9703 17/17 [==============================] - 0s 10ms/step - loss: 0.1173 - accuracy: 0.9698 Epoch 43/100 1/17 [>.............................] - ETA: 0s - loss: 0.1004 - accuracy: 0.9688 6/17 [=========>....................] - ETA: 0s - loss: 0.1136 - accuracy: 0.9664 10/17 [================>.............] - ETA: 0s - loss: 0.1129 - accuracy: 0.9685 15/17 [=========================>....] - ETA: 0s - loss: 0.1149 - accuracy: 0.9686 17/17 [==============================] - 0s 12ms/step - loss: 0.1148 - accuracy: 0.9690 Epoch 44/100 1/17 [>.............................] - ETA: 0s - loss: 0.1246 - accuracy: 0.9531 6/17 [=========>....................] - ETA: 0s - loss: 0.1272 - accuracy: 0.9622 11/17 [==================>...........] - ETA: 0s - loss: 0.1238 - accuracy: 0.9657 16/17 [===========================>..] - ETA: 0s - loss: 0.1219 - accuracy: 0.9670 17/17 [==============================] - 0s 11ms/step - loss: 0.1214 - accuracy: 0.9672 Epoch 45/100 1/17 [>.............................] - ETA: 0s - loss: 0.1064 - accuracy: 0.9766 6/17 [=========>....................] - ETA: 0s - loss: 0.1191 - accuracy: 0.9660 11/17 [==================>...........] - ETA: 0s - loss: 0.1231 - accuracy: 0.9663 16/17 [===========================>..] - ETA: 0s - loss: 0.1280 - accuracy: 0.9648 17/17 [==============================] - 0s 11ms/step - loss: 0.1289 - accuracy: 0.9645 Epoch 46/100 1/17 [>.............................] - ETA: 0s - loss: 0.0947 - accuracy: 0.9922 6/17 [=========>....................] - ETA: 0s - loss: 0.1262 - accuracy: 0.9655 11/17 [==================>...........] - ETA: 0s - loss: 0.1266 - accuracy: 0.9630 16/17 [===========================>..] - ETA: 0s - loss: 0.1257 - accuracy: 0.9628 17/17 [==============================] - 0s 11ms/step - loss: 0.1255 - accuracy: 0.9628 Epoch 47/100 1/17 [>.............................] - ETA: 0s - loss: 0.1034 - accuracy: 0.9766 6/17 [=========>....................] - ETA: 0s - loss: 0.1137 - accuracy: 0.9703 11/17 [==================>...........] - ETA: 0s - loss: 0.1173 - accuracy: 0.9690 16/17 [===========================>..] - ETA: 0s - loss: 0.1157 - accuracy: 0.9696 17/17 [==============================] - 0s 10ms/step - loss: 0.1151 - accuracy: 0.9697 Epoch 48/100 1/17 [>.............................] - ETA: 0s - loss: 0.1170 - accuracy: 0.9688 6/17 [=========>....................] - ETA: 0s - loss: 0.1157 - accuracy: 0.9656 11/17 [==================>...........] - ETA: 0s - loss: 0.1151 - accuracy: 0.9669 16/17 [===========================>..] - ETA: 0s - loss: 0.1154 - accuracy: 0.9671 17/17 [==============================] - 0s 11ms/step - loss: 0.1153 - accuracy: 0.9672 Epoch 49/100 1/17 [>.............................] - ETA: 0s - loss: 0.1051 - accuracy: 0.9688 6/17 [=========>....................] - ETA: 0s - loss: 0.1027 - accuracy: 0.9682 11/17 [==================>...........] - ETA: 0s - loss: 0.1054 - accuracy: 0.9677 16/17 [===========================>..] - ETA: 0s - loss: 0.1071 - accuracy: 0.9679 17/17 [==============================] - 0s 11ms/step - loss: 0.1074 - accuracy: 0.9681 Epoch 50/100 1/17 [>.............................] - ETA: 0s - loss: 0.1135 - accuracy: 0.9531 6/17 [=========>....................] - ETA: 0s - loss: 0.1183 - accuracy: 0.9592 11/17 [==================>...........] - ETA: 0s - loss: 0.1224 - accuracy: 0.9596 16/17 [===========================>..] - ETA: 0s - loss: 0.1207 - accuracy: 0.9610 17/17 [==============================] - 0s 11ms/step - loss: 0.1201 - accuracy: 0.9616 Epoch 51/100 1/17 [>.............................] - ETA: 0s - loss: 0.1087 - accuracy: 0.9922 6/17 [=========>....................] - ETA: 0s - loss: 0.1014 - accuracy: 0.9852 11/17 [==================>...........] - ETA: 0s - loss: 0.1032 - accuracy: 0.9815 16/17 [===========================>..] - ETA: 0s - loss: 0.1035 - accuracy: 0.9789 17/17 [==============================] - 0s 11ms/step - loss: 0.1033 - accuracy: 0.9784 Epoch 52/100 1/17 [>.............................] - ETA: 0s - loss: 0.1232 - accuracy: 0.9531 6/17 [=========>....................] - ETA: 0s - loss: 0.0997 - accuracy: 0.9666 11/17 [==================>...........] - ETA: 0s - loss: 0.0986 - accuracy: 0.9686 16/17 [===========================>..] - ETA: 0s - loss: 0.0977 - accuracy: 0.9698 17/17 [==============================] - 0s 11ms/step - loss: 0.0972 - accuracy: 0.9701 Epoch 53/100 1/17 [>.............................] - ETA: 0s - loss: 0.1443 - accuracy: 0.9531 7/17 [===========>..................] - ETA: 0s - loss: 0.1108 - accuracy: 0.9718 13/17 [=====================>........] - ETA: 0s - loss: 0.1055 - accuracy: 0.9741 17/17 [==============================] - 0s 10ms/step - loss: 0.1056 - accuracy: 0.9733 Epoch 54/100 1/17 [>.............................] - ETA: 0s - loss: 0.1016 - accuracy: 0.9766 6/17 [=========>....................] - ETA: 0s - loss: 0.1139 - accuracy: 0.9688 11/17 [==================>...........] - ETA: 0s - loss: 0.1116 - accuracy: 0.9693 16/17 [===========================>..] - ETA: 0s - loss: 0.1083 - accuracy: 0.9703 17/17 [==============================] - 0s 11ms/step - loss: 0.1073 - accuracy: 0.9707 Epoch 55/100 1/17 [>.............................] - ETA: 0s - loss: 0.1136 - accuracy: 0.9688 6/17 [=========>....................] - ETA: 0s - loss: 0.1000 - accuracy: 0.9701 11/17 [==================>...........] - ETA: 0s - loss: 0.0999 - accuracy: 0.9705 16/17 [===========================>..] - ETA: 0s - loss: 0.0992 - accuracy: 0.9705 17/17 [==============================] - 0s 11ms/step - loss: 0.0989 - accuracy: 0.9705 Epoch 56/100 1/17 [>.............................] - ETA: 0s - loss: 0.0739 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.0848 - accuracy: 0.9804 11/17 [==================>...........] - ETA: 0s - loss: 0.0857 - accuracy: 0.9794 16/17 [===========================>..] - ETA: 0s - loss: 0.0862 - accuracy: 0.9790 17/17 [==============================] - 0s 11ms/step - loss: 0.0868 - accuracy: 0.9787 Epoch 57/100 1/17 [>.............................] - ETA: 0s - loss: 0.0793 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.0959 - accuracy: 0.9746 11/17 [==================>...........] - ETA: 0s - loss: 0.0971 - accuracy: 0.9736 16/17 [===========================>..] - ETA: 0s - loss: 0.0962 - accuracy: 0.9743 17/17 [==============================] - 0s 11ms/step - loss: 0.0956 - accuracy: 0.9745 Epoch 58/100 1/17 [>.............................] - ETA: 0s - loss: 0.0851 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.0804 - accuracy: 0.9830 11/17 [==================>...........] - ETA: 0s - loss: 0.0813 - accuracy: 0.9818 16/17 [===========================>..] - ETA: 0s - loss: 0.0829 - accuracy: 0.9807 17/17 [==============================] - 0s 11ms/step - loss: 0.0833 - accuracy: 0.9805 Epoch 59/100 1/17 [>.............................] - ETA: 0s - loss: 0.0581 - accuracy: 0.9922 6/17 [=========>....................] - ETA: 0s - loss: 0.0888 - accuracy: 0.9816 11/17 [==================>...........] - ETA: 0s - loss: 0.0870 - accuracy: 0.9812 16/17 [===========================>..] - ETA: 0s - loss: 0.0852 - accuracy: 0.9807 17/17 [==============================] - 0s 11ms/step - loss: 0.0848 - accuracy: 0.9805 Epoch 60/100 1/17 [>.............................] - ETA: 0s - loss: 0.0648 - accuracy: 0.9766 6/17 [=========>....................] - ETA: 0s - loss: 0.1052 - accuracy: 0.9717 11/17 [==================>...........] - ETA: 0s - loss: 0.1054 - accuracy: 0.9727 16/17 [===========================>..] - ETA: 0s - loss: 0.1029 - accuracy: 0.9737 17/17 [==============================] - 0s 11ms/step - loss: 0.1015 - accuracy: 0.9743 Epoch 61/100 1/17 [>.............................] - ETA: 0s - loss: 0.1203 - accuracy: 0.9688 6/17 [=========>....................] - ETA: 0s - loss: 0.1010 - accuracy: 0.9699 11/17 [==================>...........] - ETA: 0s - loss: 0.0940 - accuracy: 0.9720 16/17 [===========================>..] - ETA: 0s - loss: 0.0906 - accuracy: 0.9731 17/17 [==============================] - 0s 11ms/step - loss: 0.0894 - accuracy: 0.9735 Epoch 62/100 1/17 [>.............................] - ETA: 0s - loss: 0.0756 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.0685 - accuracy: 0.9834 11/17 [==================>...........] - ETA: 0s - loss: 0.0732 - accuracy: 0.9797 16/17 [===========================>..] - ETA: 0s - loss: 0.0752 - accuracy: 0.9782 17/17 [==============================] - 0s 11ms/step - loss: 0.0755 - accuracy: 0.9780 Epoch 63/100 1/17 [>.............................] - ETA: 0s - loss: 0.1212 - accuracy: 0.9609 6/17 [=========>....................] - ETA: 0s - loss: 0.0842 - accuracy: 0.9770 11/17 [==================>...........] - ETA: 0s - loss: 0.0772 - accuracy: 0.9782 16/17 [===========================>..] - ETA: 0s - loss: 0.0739 - accuracy: 0.9791 17/17 [==============================] - 0s 12ms/step - loss: 0.0736 - accuracy: 0.9793 Epoch 64/100 1/17 [>.............................] - ETA: 0s - loss: 0.0800 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.0885 - accuracy: 0.9771 11/17 [==================>...........] - ETA: 0s - loss: 0.0874 - accuracy: 0.9758 16/17 [===========================>..] - ETA: 0s - loss: 0.0868 - accuracy: 0.9754 17/17 [==============================] - 0s 11ms/step - loss: 0.0867 - accuracy: 0.9751 Epoch 65/100 1/17 [>.............................] - ETA: 0s - loss: 0.0619 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.0703 - accuracy: 0.9772 11/17 [==================>...........] - ETA: 0s - loss: 0.0695 - accuracy: 0.9780 16/17 [===========================>..] - ETA: 0s - loss: 0.0704 - accuracy: 0.9782 17/17 [==============================] - 0s 11ms/step - loss: 0.0708 - accuracy: 0.9783 Epoch 66/100 1/17 [>.............................] - ETA: 0s - loss: 0.0723 - accuracy: 0.9688 6/17 [=========>....................] - ETA: 0s - loss: 0.0671 - accuracy: 0.9760 11/17 [==================>...........] - ETA: 0s - loss: 0.0690 - accuracy: 0.9776 16/17 [===========================>..] - ETA: 0s - loss: 0.0708 - accuracy: 0.9784 17/17 [==============================] - 0s 11ms/step - loss: 0.0712 - accuracy: 0.9784 Epoch 67/100 1/17 [>.............................] - ETA: 0s - loss: 0.0589 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.0625 - accuracy: 0.9827 11/17 [==================>...........] - ETA: 0s - loss: 0.0665 - accuracy: 0.9817 16/17 [===========================>..] - ETA: 0s - loss: 0.0678 - accuracy: 0.9815 17/17 [==============================] - 0s 11ms/step - loss: 0.0678 - accuracy: 0.9816 Epoch 68/100 1/17 [>.............................] - ETA: 0s - loss: 0.0615 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.0586 - accuracy: 0.9817 11/17 [==================>...........] - ETA: 0s - loss: 0.0639 - accuracy: 0.9790 16/17 [===========================>..] - ETA: 0s - loss: 0.0686 - accuracy: 0.9773 17/17 [==============================] - 0s 12ms/step - loss: 0.0697 - accuracy: 0.9771 Epoch 69/100 1/17 [>.............................] - ETA: 0s - loss: 0.1682 - accuracy: 0.9688 6/17 [=========>....................] - ETA: 0s - loss: 0.1094 - accuracy: 0.9780 11/17 [==================>...........] - ETA: 0s - loss: 0.1009 - accuracy: 0.9775 16/17 [===========================>..] - ETA: 0s - loss: 0.0969 - accuracy: 0.9767 17/17 [==============================] - 0s 11ms/step - loss: 0.0960 - accuracy: 0.9764 Epoch 70/100 1/17 [>.............................] - ETA: 0s - loss: 0.0557 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.0714 - accuracy: 0.9802 12/17 [====================>.........] - ETA: 0s - loss: 0.0689 - accuracy: 0.9810 17/17 [==============================] - 0s 10ms/step - loss: 0.0707 - accuracy: 0.9809 Epoch 71/100 1/17 [>.............................] - ETA: 0s - loss: 0.0534 - accuracy: 0.9922 6/17 [=========>....................] - ETA: 0s - loss: 0.0806 - accuracy: 0.9793 11/17 [==================>...........] - ETA: 0s - loss: 0.0832 - accuracy: 0.9772 16/17 [===========================>..] - ETA: 0s - loss: 0.0820 - accuracy: 0.9772 17/17 [==============================] - 0s 11ms/step - loss: 0.0818 - accuracy: 0.9771 Epoch 72/100 1/17 [>.............................] - ETA: 0s - loss: 0.0838 - accuracy: 0.9688 7/17 [===========>..................] - ETA: 0s - loss: 0.0722 - accuracy: 0.9765 12/17 [====================>.........] - ETA: 0s - loss: 0.0732 - accuracy: 0.9772 17/17 [==============================] - ETA: 0s - loss: 0.0737 - accuracy: 0.9776 17/17 [==============================] - 0s 10ms/step - loss: 0.0739 - accuracy: 0.9775 Epoch 73/100 1/17 [>.............................] - ETA: 0s - loss: 0.0667 - accuracy: 0.9766 6/17 [=========>....................] - ETA: 0s - loss: 0.0759 - accuracy: 0.9761 10/17 [================>.............] - ETA: 0s - loss: 0.0737 - accuracy: 0.9777 15/17 [=========================>....] - ETA: 0s - loss: 0.0716 - accuracy: 0.9790 17/17 [==============================] - 0s 12ms/step - loss: 0.0710 - accuracy: 0.9796 Epoch 74/100 1/17 [>.............................] - ETA: 0s - loss: 0.0426 - accuracy: 0.9922 6/17 [=========>....................] - ETA: 0s - loss: 0.0548 - accuracy: 0.9877 11/17 [==================>...........] - ETA: 0s - loss: 0.0579 - accuracy: 0.9855 17/17 [==============================] - ETA: 0s - loss: 0.0617 - accuracy: 0.9827 17/17 [==============================] - 0s 10ms/step - loss: 0.0621 - accuracy: 0.9824 Epoch 75/100 1/17 [>.............................] - ETA: 0s - loss: 0.0482 - accuracy: 0.9922 7/17 [===========>..................] - ETA: 0s - loss: 0.0520 - accuracy: 0.9889 12/17 [====================>.........] - ETA: 0s - loss: 0.0547 - accuracy: 0.9884 17/17 [==============================] - ETA: 0s - loss: 0.0564 - accuracy: 0.9881 17/17 [==============================] - 0s 10ms/step - loss: 0.0567 - accuracy: 0.9881 Epoch 76/100 1/17 [>.............................] - ETA: 0s - loss: 0.0360 - accuracy: 1.0000 7/17 [===========>..................] - ETA: 0s - loss: 0.0459 - accuracy: 0.9901 12/17 [====================>.........] - ETA: 0s - loss: 0.0482 - accuracy: 0.9894 17/17 [==============================] - ETA: 0s - loss: 0.0494 - accuracy: 0.9891 17/17 [==============================] - 0s 11ms/step - loss: 0.0496 - accuracy: 0.9890 Epoch 77/100 1/17 [>.............................] - ETA: 0s - loss: 0.0366 - accuracy: 1.0000 6/17 [=========>....................] - ETA: 0s - loss: 0.0513 - accuracy: 0.9904 11/17 [==================>...........] - ETA: 0s - loss: 0.0568 - accuracy: 0.9868 16/17 [===========================>..] - ETA: 0s - loss: 0.0599 - accuracy: 0.9844 17/17 [==============================] - 0s 11ms/step - loss: 0.0609 - accuracy: 0.9837 Epoch 78/100 1/17 [>.............................] - ETA: 0s - loss: 0.0744 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.0807 - accuracy: 0.9820 11/17 [==================>...........] - ETA: 0s - loss: 0.0809 - accuracy: 0.9808 16/17 [===========================>..] - ETA: 0s - loss: 0.0788 - accuracy: 0.9811 17/17 [==============================] - 0s 11ms/step - loss: 0.0779 - accuracy: 0.9812 Epoch 79/100 1/17 [>.............................] - ETA: 0s - loss: 0.0734 - accuracy: 0.9766 5/17 [=======>......................] - ETA: 0s - loss: 0.0609 - accuracy: 0.9799 10/17 [================>.............] - ETA: 0s - loss: 0.0596 - accuracy: 0.9826 15/17 [=========================>....] - ETA: 0s - loss: 0.0594 - accuracy: 0.9834 17/17 [==============================] - 0s 12ms/step - loss: 0.0591 - accuracy: 0.9837 Epoch 80/100 1/17 [>.............................] - ETA: 0s - loss: 0.0563 - accuracy: 0.9922 6/17 [=========>....................] - ETA: 0s - loss: 0.0679 - accuracy: 0.9814 11/17 [==================>...........] - ETA: 0s - loss: 0.0687 - accuracy: 0.9789 16/17 [===========================>..] - ETA: 0s - loss: 0.0679 - accuracy: 0.9790 17/17 [==============================] - 0s 11ms/step - loss: 0.0673 - accuracy: 0.9791 Epoch 81/100 1/17 [>.............................] - ETA: 0s - loss: 0.0364 - accuracy: 0.9922 6/17 [=========>....................] - ETA: 0s - loss: 0.0496 - accuracy: 0.9858 11/17 [==================>...........] - ETA: 0s - loss: 0.0551 - accuracy: 0.9844 16/17 [===========================>..] - ETA: 0s - loss: 0.0565 - accuracy: 0.9840 17/17 [==============================] - 0s 10ms/step - loss: 0.0568 - accuracy: 0.9839 Epoch 82/100 1/17 [>.............................] - ETA: 0s - loss: 0.0612 - accuracy: 0.9766 6/17 [=========>....................] - ETA: 0s - loss: 0.0583 - accuracy: 0.9813 11/17 [==================>...........] - ETA: 0s - loss: 0.0575 - accuracy: 0.9825 16/17 [===========================>..] - ETA: 0s - loss: 0.0568 - accuracy: 0.9830 17/17 [==============================] - 0s 11ms/step - loss: 0.0568 - accuracy: 0.9830 Epoch 83/100 1/17 [>.............................] - ETA: 0s - loss: 0.0646 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.0695 - accuracy: 0.9812 11/17 [==================>...........] - ETA: 0s - loss: 0.0648 - accuracy: 0.9827 16/17 [===========================>..] - ETA: 0s - loss: 0.0637 - accuracy: 0.9830 17/17 [==============================] - 0s 11ms/step - loss: 0.0635 - accuracy: 0.9830 Epoch 84/100 1/17 [>.............................] - ETA: 0s - loss: 0.0510 - accuracy: 0.9922 6/17 [=========>....................] - ETA: 0s - loss: 0.0506 - accuracy: 0.9884 11/17 [==================>...........] - ETA: 0s - loss: 0.0524 - accuracy: 0.9876 16/17 [===========================>..] - ETA: 0s - loss: 0.0559 - accuracy: 0.9854 17/17 [==============================] - 0s 11ms/step - loss: 0.0570 - accuracy: 0.9846 Epoch 85/100 1/17 [>.............................] - ETA: 0s - loss: 0.0554 - accuracy: 0.9922 6/17 [=========>....................] - ETA: 0s - loss: 0.0652 - accuracy: 0.9870 11/17 [==================>...........] - ETA: 0s - loss: 0.0632 - accuracy: 0.9860 16/17 [===========================>..] - ETA: 0s - loss: 0.0621 - accuracy: 0.9854 17/17 [==============================] - 0s 11ms/step - loss: 0.0617 - accuracy: 0.9854 Epoch 86/100 1/17 [>.............................] - ETA: 0s - loss: 0.0902 - accuracy: 0.9766 6/17 [=========>....................] - ETA: 0s - loss: 0.0701 - accuracy: 0.9804 11/17 [==================>...........] - ETA: 0s - loss: 0.0655 - accuracy: 0.9823 16/17 [===========================>..] - ETA: 0s - loss: 0.0630 - accuracy: 0.9830 17/17 [==============================] - 0s 10ms/step - loss: 0.0624 - accuracy: 0.9831 Epoch 87/100 1/17 [>.............................] - ETA: 0s - loss: 0.0294 - accuracy: 1.0000 6/17 [=========>....................] - ETA: 0s - loss: 0.0379 - accuracy: 0.9923 11/17 [==================>...........] - ETA: 0s - loss: 0.0427 - accuracy: 0.9898 16/17 [===========================>..] - ETA: 0s - loss: 0.0453 - accuracy: 0.9887 17/17 [==============================] - 0s 11ms/step - loss: 0.0462 - accuracy: 0.9884 Epoch 88/100 1/17 [>.............................] - ETA: 0s - loss: 0.0468 - accuracy: 0.9922 6/17 [=========>....................] - ETA: 0s - loss: 0.0512 - accuracy: 0.9910 11/17 [==================>...........] - ETA: 0s - loss: 0.0516 - accuracy: 0.9894 16/17 [===========================>..] - ETA: 0s - loss: 0.0521 - accuracy: 0.9885 17/17 [==============================] - 0s 11ms/step - loss: 0.0520 - accuracy: 0.9884 Epoch 89/100 1/17 [>.............................] - ETA: 0s - loss: 0.0607 - accuracy: 0.9688 6/17 [=========>....................] - ETA: 0s - loss: 0.0469 - accuracy: 0.9851 11/17 [==================>...........] - ETA: 0s - loss: 0.0457 - accuracy: 0.9871 16/17 [===========================>..] - ETA: 0s - loss: 0.0465 - accuracy: 0.9875 17/17 [==============================] - 0s 11ms/step - loss: 0.0468 - accuracy: 0.9875 Epoch 90/100 1/17 [>.............................] - ETA: 0s - loss: 0.0455 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.0543 - accuracy: 0.9811 11/17 [==================>...........] - ETA: 0s - loss: 0.0576 - accuracy: 0.9805 16/17 [===========================>..] - ETA: 0s - loss: 0.0592 - accuracy: 0.9806 17/17 [==============================] - 0s 11ms/step - loss: 0.0600 - accuracy: 0.9806 Epoch 91/100 1/17 [>.............................] - ETA: 0s - loss: 0.0658 - accuracy: 0.9688 6/17 [=========>....................] - ETA: 0s - loss: 0.0546 - accuracy: 0.9780 11/17 [==================>...........] - ETA: 0s - loss: 0.0522 - accuracy: 0.9801 16/17 [===========================>..] - ETA: 0s - loss: 0.0511 - accuracy: 0.9818 17/17 [==============================] - 0s 10ms/step - loss: 0.0507 - accuracy: 0.9823 Epoch 92/100 1/17 [>.............................] - ETA: 0s - loss: 0.0803 - accuracy: 0.9688 6/17 [=========>....................] - ETA: 0s - loss: 0.0557 - accuracy: 0.9809 11/17 [==================>...........] - ETA: 0s - loss: 0.0533 - accuracy: 0.9826 16/17 [===========================>..] - ETA: 0s - loss: 0.0529 - accuracy: 0.9838 17/17 [==============================] - 0s 11ms/step - loss: 0.0528 - accuracy: 0.9841 Epoch 93/100 1/17 [>.............................] - ETA: 0s - loss: 0.0573 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.0508 - accuracy: 0.9860 11/17 [==================>...........] - ETA: 0s - loss: 0.0490 - accuracy: 0.9867 16/17 [===========================>..] - ETA: 0s - loss: 0.0484 - accuracy: 0.9865 17/17 [==============================] - 0s 11ms/step - loss: 0.0483 - accuracy: 0.9865 Epoch 94/100 1/17 [>.............................] - ETA: 0s - loss: 0.0552 - accuracy: 0.9922 6/17 [=========>....................] - ETA: 0s - loss: 0.0650 - accuracy: 0.9833 11/17 [==================>...........] - ETA: 0s - loss: 0.0631 - accuracy: 0.9834 16/17 [===========================>..] - ETA: 0s - loss: 0.0617 - accuracy: 0.9832 17/17 [==============================] - 0s 11ms/step - loss: 0.0615 - accuracy: 0.9832 Epoch 95/100 1/17 [>.............................] - ETA: 0s - loss: 0.0376 - accuracy: 0.9766 6/17 [=========>....................] - ETA: 0s - loss: 0.0477 - accuracy: 0.9808 11/17 [==================>...........] - ETA: 0s - loss: 0.0471 - accuracy: 0.9840 16/17 [===========================>..] - ETA: 0s - loss: 0.0470 - accuracy: 0.9854 17/17 [==============================] - 0s 11ms/step - loss: 0.0470 - accuracy: 0.9856 Epoch 96/100 1/17 [>.............................] - ETA: 0s - loss: 0.0329 - accuracy: 1.0000 6/17 [=========>....................] - ETA: 0s - loss: 0.0462 - accuracy: 0.9893 11/17 [==================>...........] - ETA: 0s - loss: 0.0427 - accuracy: 0.9899 16/17 [===========================>..] - ETA: 0s - loss: 0.0421 - accuracy: 0.9899 17/17 [==============================] - 0s 11ms/step - loss: 0.0419 - accuracy: 0.9900 Epoch 97/100 1/17 [>.............................] - ETA: 0s - loss: 0.0524 - accuracy: 0.9844 6/17 [=========>....................] - ETA: 0s - loss: 0.0433 - accuracy: 0.9936 11/17 [==================>...........] - ETA: 0s - loss: 0.0408 - accuracy: 0.9950 16/17 [===========================>..] - ETA: 0s - loss: 0.0405 - accuracy: 0.9946 17/17 [==============================] - 0s 11ms/step - loss: 0.0407 - accuracy: 0.9942 Epoch 98/100 1/17 [>.............................] - ETA: 0s - loss: 0.0802 - accuracy: 0.9766 7/17 [===========>..................] - ETA: 0s - loss: 0.0575 - accuracy: 0.9833 13/17 [=====================>........] - ETA: 0s - loss: 0.0581 - accuracy: 0.9824 17/17 [==============================] - 0s 10ms/step - loss: 0.0573 - accuracy: 0.9826 Epoch 99/100 1/17 [>.............................] - ETA: 0s - loss: 0.0500 - accuracy: 0.9922 6/17 [=========>....................] - ETA: 0s - loss: 0.0473 - accuracy: 0.9882 11/17 [==================>...........] - ETA: 0s - loss: 0.0458 - accuracy: 0.9885 16/17 [===========================>..] - ETA: 0s - loss: 0.0464 - accuracy: 0.9878 17/17 [==============================] - 0s 10ms/step - loss: 0.0465 - accuracy: 0.9877 Epoch 100/100 1/17 [>.............................] - ETA: 0s - loss: 0.0157 - accuracy: 1.0000 7/17 [===========>..................] - ETA: 0s - loss: 0.0418 - accuracy: 0.9888 12/17 [====================>.........] - ETA: 0s - loss: 0.0442 - accuracy: 0.9877 17/17 [==============================] - ETA: 0s - loss: 0.0443 - accuracy: 0.9879 17/17 [==============================] - 0s 10ms/step - loss: 0.0442 - accuracy: 0.9880 <tensorflow.python.keras.callbacks.History at 0x7f7860e3d048>
基本 MLP モデルを評価する
# Helper function to print evaluation metrics.
def print_metrics(model_desc, eval_metrics):
"""Prints evaluation metrics.
Args:
model_desc: A description of the model.
eval_metrics: A dictionary mapping metric names to corresponding values. It
must contain the loss and accuracy metrics.
"""
print('\n')
print('Eval accuracy for ', model_desc, ': ', eval_metrics['accuracy'])
print('Eval loss for ', model_desc, ': ', eval_metrics['loss'])
if 'graph_loss' in eval_metrics:
print('Eval graph loss for ', model_desc, ': ', eval_metrics['graph_loss'])
eval_results = dict(
zip(base_model.metrics_names,
base_model.evaluate(test_dataset, steps=HPARAMS.eval_steps)))
print_metrics('Base MLP model', eval_results)
1/Unknown - 0s 145ms/step - loss: 1.1800 - accuracy: 0.7812 5/5 [==============================] - 0s 9ms/step - loss: 1.2943 - accuracy: 0.7939 Eval accuracy for Base MLP model : 0.7938517332077026 Eval loss for Base MLP model : 1.2943289279937744
MLP モデルをグラフ正則化でトレーニングする
既存の tf.Keras.Model の損失項にグラフ正則化を組み込む場合に必要なのは、数行のコードのみです。基本モデルをラップして、損失にグラフ正則化を含んだ新しい tf.Keras サブクラスモデルを作成します。
グラフ正則化の増分効果を評価するために、基本モデルの新しいインスタンスを作成します。これは、base_model は既に数回のイテレーションでトレーニングされているため、このトレーニング済みモデルを再利用してグラフ正則化モデルを作成しても base_model の公平な比較をすることができないからです。
# Build a new base MLP model.
base_reg_model_tag, base_reg_model = 'FUNCTIONAL', make_mlp_functional_model(
HPARAMS)
# Wrap the base MLP model with graph regularization.
graph_reg_config = nsl.configs.make_graph_reg_config(
max_neighbors=HPARAMS.num_neighbors,
multiplier=HPARAMS.graph_regularization_multiplier,
distance_type=HPARAMS.distance_type,
sum_over_axis=-1)
graph_reg_model = nsl.keras.GraphRegularization(base_reg_model,
graph_reg_config)
graph_reg_model.compile(
optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
graph_reg_model.fit(train_dataset, epochs=HPARAMS.train_epochs, verbose=1)
Epoch 1/100
/tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/framework/indexed_slices.py:437: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/GraphRegularization/graph_loss/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/GraphRegularization/graph_loss/Reshape:0", shape=(None, 7), dtype=float32), dense_shape=Tensor("gradient_tape/GraphRegularization/graph_loss/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.
"shape. This may consume a large amount of memory." % value)
1/Unknown - 2s 2s/step - loss: 1.9655 - accuracy: 0.1094 - scaled_graph_loss: 7.7787e-04
6/Unknown - 2s 11ms/step - loss: 1.9661 - accuracy: 0.1291 - scaled_graph_loss: 7.4272e-04
11/Unknown - 2s 11ms/step - loss: 1.9609 - accuracy: 0.1417 - scaled_graph_loss: 7.4203e-04
16/Unknown - 2s 11ms/step - loss: 1.9559 - accuracy: 0.1495 - scaled_graph_loss: 7.4046e-04
17/17 [==============================] - 2s 11ms/step - loss: 1.9542 - accuracy: 0.1519 - scaled_graph_loss: 7.4008e-04
Epoch 2/100
1/17 [>.............................] - ETA: 0s - loss: 1.9255 - accuracy: 0.1953 - scaled_graph_loss: 8.0138e-04
6/17 [=========>....................] - ETA: 0s - loss: 1.8947 - accuracy: 0.2208 - scaled_graph_loss: 8.1104e-04
11/17 [==================>...........] - ETA: 0s - loss: 1.8853 - accuracy: 0.2338 - scaled_graph_loss: 8.2701e-04
16/17 [===========================>..] - ETA: 0s - loss: 1.8801 - accuracy: 0.2426 - scaled_graph_loss: 8.5188e-04
17/17 [==============================] - 0s 11ms/step - loss: 1.8780 - accuracy: 0.2452 - scaled_graph_loss: 8.6243e-04
Epoch 3/100
1/17 [>.............................] - ETA: 0s - loss: 1.8250 - accuracy: 0.2578 - scaled_graph_loss: 0.0013
6/17 [=========>....................] - ETA: 0s - loss: 1.8139 - accuracy: 0.3002 - scaled_graph_loss: 0.0014
11/17 [==================>...........] - ETA: 0s - loss: 1.8050 - accuracy: 0.3123 - scaled_graph_loss: 0.0015
16/17 [===========================>..] - ETA: 0s - loss: 1.7985 - accuracy: 0.3182 - scaled_graph_loss: 0.0016
17/17 [==============================] - 0s 11ms/step - loss: 1.7961 - accuracy: 0.3197 - scaled_graph_loss: 0.0016
Epoch 4/100
1/17 [>.............................] - ETA: 0s - loss: 1.6700 - accuracy: 0.4062 - scaled_graph_loss: 0.0027
7/17 [===========>..................] - ETA: 0s - loss: 1.7008 - accuracy: 0.3662 - scaled_graph_loss: 0.0029
12/17 [====================>.........] - ETA: 0s - loss: 1.6944 - accuracy: 0.3728 - scaled_graph_loss: 0.0030
17/17 [==============================] - ETA: 0s - loss: 1.6876 - accuracy: 0.3767 - scaled_graph_loss: 0.0031
17/17 [==============================] - 0s 10ms/step - loss: 1.6863 - accuracy: 0.3774 - scaled_graph_loss: 0.0032
Epoch 5/100
1/17 [>.............................] - ETA: 0s - loss: 1.5436 - accuracy: 0.4375 - scaled_graph_loss: 0.0051
6/17 [=========>....................] - ETA: 0s - loss: 1.5912 - accuracy: 0.3805 - scaled_graph_loss: 0.0051
11/17 [==================>...........] - ETA: 0s - loss: 1.5869 - accuracy: 0.3844 - scaled_graph_loss: 0.0052
16/17 [===========================>..] - ETA: 0s - loss: 1.5754 - accuracy: 0.3945 - scaled_graph_loss: 0.0054
17/17 [==============================] - 0s 11ms/step - loss: 1.5712 - accuracy: 0.3973 - scaled_graph_loss: 0.0054
Epoch 6/100
1/17 [>.............................] - ETA: 0s - loss: 1.3969 - accuracy: 0.5000 - scaled_graph_loss: 0.0076
6/17 [=========>....................] - ETA: 0s - loss: 1.4542 - accuracy: 0.4696 - scaled_graph_loss: 0.0083
11/17 [==================>...........] - ETA: 0s - loss: 1.4422 - accuracy: 0.4719 - scaled_graph_loss: 0.0084
16/17 [===========================>..] - ETA: 0s - loss: 1.4287 - accuracy: 0.4771 - scaled_graph_loss: 0.0086
17/17 [==============================] - 0s 11ms/step - loss: 1.4242 - accuracy: 0.4789 - scaled_graph_loss: 0.0087
Epoch 7/100
1/17 [>.............................] - ETA: 0s - loss: 1.4118 - accuracy: 0.5000 - scaled_graph_loss: 0.0112
6/17 [=========>....................] - ETA: 0s - loss: 1.3519 - accuracy: 0.5288 - scaled_graph_loss: 0.0115
11/17 [==================>...........] - ETA: 0s - loss: 1.3323 - accuracy: 0.5347 - scaled_graph_loss: 0.0120
16/17 [===========================>..] - ETA: 0s - loss: 1.3151 - accuracy: 0.5426 - scaled_graph_loss: 0.0124
17/17 [==============================] - 0s 11ms/step - loss: 1.3093 - accuracy: 0.5452 - scaled_graph_loss: 0.0125
Epoch 8/100
1/17 [>.............................] - ETA: 0s - loss: 1.2320 - accuracy: 0.5469 - scaled_graph_loss: 0.0158
6/17 [=========>....................] - ETA: 0s - loss: 1.1801 - accuracy: 0.5830 - scaled_graph_loss: 0.0161
11/17 [==================>...........] - ETA: 0s - loss: 1.1534 - accuracy: 0.6027 - scaled_graph_loss: 0.0165
16/17 [===========================>..] - ETA: 0s - loss: 1.1448 - accuracy: 0.6071 - scaled_graph_loss: 0.0168
17/17 [==============================] - 0s 11ms/step - loss: 1.1419 - accuracy: 0.6088 - scaled_graph_loss: 0.0169
Epoch 9/100
1/17 [>.............................] - ETA: 0s - loss: 1.1959 - accuracy: 0.5547 - scaled_graph_loss: 0.0210
6/17 [=========>....................] - ETA: 0s - loss: 1.0976 - accuracy: 0.6181 - scaled_graph_loss: 0.0208
11/17 [==================>...........] - ETA: 0s - loss: 1.0531 - accuracy: 0.6441 - scaled_graph_loss: 0.0205
16/17 [===========================>..] - ETA: 0s - loss: 1.0337 - accuracy: 0.6555 - scaled_graph_loss: 0.0206
17/17 [==============================] - 0s 11ms/step - loss: 1.0283 - accuracy: 0.6588 - scaled_graph_loss: 0.0207
Epoch 10/100
1/17 [>.............................] - ETA: 0s - loss: 0.9404 - accuracy: 0.6719 - scaled_graph_loss: 0.0248
6/17 [=========>....................] - ETA: 0s - loss: 0.9541 - accuracy: 0.6840 - scaled_graph_loss: 0.0244
11/17 [==================>...........] - ETA: 0s - loss: 0.9400 - accuracy: 0.6956 - scaled_graph_loss: 0.0242
16/17 [===========================>..] - ETA: 0s - loss: 0.9244 - accuracy: 0.7053 - scaled_graph_loss: 0.0242
17/17 [==============================] - 0s 11ms/step - loss: 0.9211 - accuracy: 0.7076 - scaled_graph_loss: 0.0243
Epoch 11/100
1/17 [>.............................] - ETA: 0s - loss: 0.7688 - accuracy: 0.8125 - scaled_graph_loss: 0.0269
6/17 [=========>....................] - ETA: 0s - loss: 0.7919 - accuracy: 0.7792 - scaled_graph_loss: 0.0260
11/17 [==================>...........] - ETA: 0s - loss: 0.7958 - accuracy: 0.7734 - scaled_graph_loss: 0.0262
16/17 [===========================>..] - ETA: 0s - loss: 0.8007 - accuracy: 0.7708 - scaled_graph_loss: 0.0262
17/17 [==============================] - 0s 11ms/step - loss: 0.8022 - accuracy: 0.7699 - scaled_graph_loss: 0.0262
Epoch 12/100
1/17 [>.............................] - ETA: 0s - loss: 0.7896 - accuracy: 0.7656 - scaled_graph_loss: 0.0340
6/17 [=========>....................] - ETA: 0s - loss: 0.7932 - accuracy: 0.7580 - scaled_graph_loss: 0.0295
11/17 [==================>...........] - ETA: 0s - loss: 0.7880 - accuracy: 0.7611 - scaled_graph_loss: 0.0286
16/17 [===========================>..] - ETA: 0s - loss: 0.7820 - accuracy: 0.7619 - scaled_graph_loss: 0.0284
17/17 [==============================] - 0s 11ms/step - loss: 0.7787 - accuracy: 0.7628 - scaled_graph_loss: 0.0284
Epoch 13/100
1/17 [>.............................] - ETA: 0s - loss: 0.7713 - accuracy: 0.7422 - scaled_graph_loss: 0.0354
6/17 [=========>....................] - ETA: 0s - loss: 0.7083 - accuracy: 0.7836 - scaled_graph_loss: 0.0312
11/17 [==================>...........] - ETA: 0s - loss: 0.7057 - accuracy: 0.7893 - scaled_graph_loss: 0.0302
16/17 [===========================>..] - ETA: 0s - loss: 0.7006 - accuracy: 0.7940 - scaled_graph_loss: 0.0299
17/17 [==============================] - 0s 10ms/step - loss: 0.6991 - accuracy: 0.7949 - scaled_graph_loss: 0.0298
Epoch 14/100
1/17 [>.............................] - ETA: 0s - loss: 0.6625 - accuracy: 0.8672 - scaled_graph_loss: 0.0329
6/17 [=========>....................] - ETA: 0s - loss: 0.6673 - accuracy: 0.8380 - scaled_graph_loss: 0.0303
11/17 [==================>...........] - ETA: 0s - loss: 0.6503 - accuracy: 0.8367 - scaled_graph_loss: 0.0301
16/17 [===========================>..] - ETA: 0s - loss: 0.6399 - accuracy: 0.8355 - scaled_graph_loss: 0.0299
17/17 [==============================] - 0s 11ms/step - loss: 0.6366 - accuracy: 0.8353 - scaled_graph_loss: 0.0298
Epoch 15/100
1/17 [>.............................] - ETA: 0s - loss: 0.4995 - accuracy: 0.8281 - scaled_graph_loss: 0.0302
6/17 [=========>....................] - ETA: 0s - loss: 0.5362 - accuracy: 0.8253 - scaled_graph_loss: 0.0316
11/17 [==================>...........] - ETA: 0s - loss: 0.5371 - accuracy: 0.8285 - scaled_graph_loss: 0.0316
16/17 [===========================>..] - ETA: 0s - loss: 0.5426 - accuracy: 0.8305 - scaled_graph_loss: 0.0316
17/17 [==============================] - 0s 11ms/step - loss: 0.5447 - accuracy: 0.8312 - scaled_graph_loss: 0.0316
Epoch 16/100
1/17 [>.............................] - ETA: 0s - loss: 0.4324 - accuracy: 0.8906 - scaled_graph_loss: 0.0254
6/17 [=========>....................] - ETA: 0s - loss: 0.5007 - accuracy: 0.8671 - scaled_graph_loss: 0.0281
11/17 [==================>...........] - ETA: 0s - loss: 0.5130 - accuracy: 0.8619 - scaled_graph_loss: 0.0290
16/17 [===========================>..] - ETA: 0s - loss: 0.5159 - accuracy: 0.8604 - scaled_graph_loss: 0.0294
17/17 [==============================] - 0s 11ms/step - loss: 0.5165 - accuracy: 0.8604 - scaled_graph_loss: 0.0295
Epoch 17/100
1/17 [>.............................] - ETA: 0s - loss: 0.5017 - accuracy: 0.8516 - scaled_graph_loss: 0.0291
6/17 [=========>....................] - ETA: 0s - loss: 0.4680 - accuracy: 0.8779 - scaled_graph_loss: 0.0297
11/17 [==================>...........] - ETA: 0s - loss: 0.4736 - accuracy: 0.8759 - scaled_graph_loss: 0.0304
16/17 [===========================>..] - ETA: 0s - loss: 0.4771 - accuracy: 0.8725 - scaled_graph_loss: 0.0307
17/17 [==============================] - 0s 11ms/step - loss: 0.4780 - accuracy: 0.8717 - scaled_graph_loss: 0.0307
Epoch 18/100
1/17 [>.............................] - ETA: 0s - loss: 0.5563 - accuracy: 0.8750 - scaled_graph_loss: 0.0355
6/17 [=========>....................] - ETA: 0s - loss: 0.5085 - accuracy: 0.8781 - scaled_graph_loss: 0.0310
11/17 [==================>...........] - ETA: 0s - loss: 0.4938 - accuracy: 0.8768 - scaled_graph_loss: 0.0303
16/17 [===========================>..] - ETA: 0s - loss: 0.4823 - accuracy: 0.8761 - scaled_graph_loss: 0.0303
17/17 [==============================] - 0s 11ms/step - loss: 0.4786 - accuracy: 0.8763 - scaled_graph_loss: 0.0304
Epoch 19/100
1/17 [>.............................] - ETA: 0s - loss: 0.5442 - accuracy: 0.8516 - scaled_graph_loss: 0.0397
6/17 [=========>....................] - ETA: 0s - loss: 0.4863 - accuracy: 0.8582 - scaled_graph_loss: 0.0347
11/17 [==================>...........] - ETA: 0s - loss: 0.4575 - accuracy: 0.8706 - scaled_graph_loss: 0.0333
16/17 [===========================>..] - ETA: 0s - loss: 0.4469 - accuracy: 0.8752 - scaled_graph_loss: 0.0329
17/17 [==============================] - 0s 10ms/step - loss: 0.4446 - accuracy: 0.8762 - scaled_graph_loss: 0.0328
Epoch 20/100
1/17 [>.............................] - ETA: 0s - loss: 0.4771 - accuracy: 0.8516 - scaled_graph_loss: 0.0301
6/17 [=========>....................] - ETA: 0s - loss: 0.4145 - accuracy: 0.8848 - scaled_graph_loss: 0.0318
11/17 [==================>...........] - ETA: 0s - loss: 0.4048 - accuracy: 0.8903 - scaled_graph_loss: 0.0321
16/17 [===========================>..] - ETA: 0s - loss: 0.3979 - accuracy: 0.8940 - scaled_graph_loss: 0.0322
17/17 [==============================] - 0s 11ms/step - loss: 0.3954 - accuracy: 0.8953 - scaled_graph_loss: 0.0322
Epoch 21/100
1/17 [>.............................] - ETA: 0s - loss: 0.3749 - accuracy: 0.8984 - scaled_graph_loss: 0.0316
7/17 [===========>..................] - ETA: 0s - loss: 0.3690 - accuracy: 0.8988 - scaled_graph_loss: 0.0319
12/17 [====================>.........] - ETA: 0s - loss: 0.3743 - accuracy: 0.8961 - scaled_graph_loss: 0.0320
17/17 [==============================] - ETA: 0s - loss: 0.3742 - accuracy: 0.8965 - scaled_graph_loss: 0.0320
17/17 [==============================] - 0s 11ms/step - loss: 0.3739 - accuracy: 0.8967 - scaled_graph_loss: 0.0320
Epoch 22/100
1/17 [>.............................] - ETA: 0s - loss: 0.4791 - accuracy: 0.8750 - scaled_graph_loss: 0.0333
6/17 [=========>....................] - ETA: 0s - loss: 0.4148 - accuracy: 0.8914 - scaled_graph_loss: 0.0336
11/17 [==================>...........] - ETA: 0s - loss: 0.3925 - accuracy: 0.8990 - scaled_graph_loss: 0.0332
16/17 [===========================>..] - ETA: 0s - loss: 0.3856 - accuracy: 0.9005 - scaled_graph_loss: 0.0329
17/17 [==============================] - 0s 11ms/step - loss: 0.3835 - accuracy: 0.9009 - scaled_graph_loss: 0.0329
Epoch 23/100
1/17 [>.............................] - ETA: 0s - loss: 0.3412 - accuracy: 0.9062 - scaled_graph_loss: 0.0338
6/17 [=========>....................] - ETA: 0s - loss: 0.3279 - accuracy: 0.9171 - scaled_graph_loss: 0.0332
11/17 [==================>...........] - ETA: 0s - loss: 0.3246 - accuracy: 0.9203 - scaled_graph_loss: 0.0329
16/17 [===========================>..] - ETA: 0s - loss: 0.3242 - accuracy: 0.9201 - scaled_graph_loss: 0.0330
17/17 [==============================] - 0s 10ms/step - loss: 0.3242 - accuracy: 0.9201 - scaled_graph_loss: 0.0330
Epoch 24/100
1/17 [>.............................] - ETA: 0s - loss: 0.2642 - accuracy: 0.9219 - scaled_graph_loss: 0.0240
6/17 [=========>....................] - ETA: 0s - loss: 0.3000 - accuracy: 0.9226 - scaled_graph_loss: 0.0289
11/17 [==================>...........] - ETA: 0s - loss: 0.3047 - accuracy: 0.9209 - scaled_graph_loss: 0.0301
16/17 [===========================>..] - ETA: 0s - loss: 0.3034 - accuracy: 0.9214 - scaled_graph_loss: 0.0307
17/17 [==============================] - 0s 11ms/step - loss: 0.3034 - accuracy: 0.9214 - scaled_graph_loss: 0.0310
Epoch 25/100
1/17 [>.............................] - ETA: 0s - loss: 0.3625 - accuracy: 0.9062 - scaled_graph_loss: 0.0402
6/17 [=========>....................] - ETA: 0s - loss: 0.3102 - accuracy: 0.9202 - scaled_graph_loss: 0.0357
11/17 [==================>...........] - ETA: 0s - loss: 0.2963 - accuracy: 0.9251 - scaled_graph_loss: 0.0350
16/17 [===========================>..] - ETA: 0s - loss: 0.2919 - accuracy: 0.9275 - scaled_graph_loss: 0.0346
17/17 [==============================] - 0s 11ms/step - loss: 0.2909 - accuracy: 0.9281 - scaled_graph_loss: 0.0345
Epoch 26/100
1/17 [>.............................] - ETA: 0s - loss: 0.2874 - accuracy: 0.9375 - scaled_graph_loss: 0.0355
6/17 [=========>....................] - ETA: 0s - loss: 0.3039 - accuracy: 0.9232 - scaled_graph_loss: 0.0366
11/17 [==================>...........] - ETA: 0s - loss: 0.2961 - accuracy: 0.9246 - scaled_graph_loss: 0.0355
16/17 [===========================>..] - ETA: 0s - loss: 0.2931 - accuracy: 0.9247 - scaled_graph_loss: 0.0348
17/17 [==============================] - 0s 11ms/step - loss: 0.2921 - accuracy: 0.9249 - scaled_graph_loss: 0.0347
Epoch 27/100
1/17 [>.............................] - ETA: 0s - loss: 0.2245 - accuracy: 0.9609 - scaled_graph_loss: 0.0333
6/17 [=========>....................] - ETA: 0s - loss: 0.2328 - accuracy: 0.9533 - scaled_graph_loss: 0.0339
11/17 [==================>...........] - ETA: 0s - loss: 0.2391 - accuracy: 0.9502 - scaled_graph_loss: 0.0339
16/17 [===========================>..] - ETA: 0s - loss: 0.2428 - accuracy: 0.9487 - scaled_graph_loss: 0.0336
17/17 [==============================] - 0s 12ms/step - loss: 0.2439 - accuracy: 0.9483 - scaled_graph_loss: 0.0335
Epoch 28/100
1/17 [>.............................] - ETA: 0s - loss: 0.2914 - accuracy: 0.9453 - scaled_graph_loss: 0.0294
6/17 [=========>....................] - ETA: 0s - loss: 0.2592 - accuracy: 0.9478 - scaled_graph_loss: 0.0316
11/17 [==================>...........] - ETA: 0s - loss: 0.2549 - accuracy: 0.9473 - scaled_graph_loss: 0.0326
16/17 [===========================>..] - ETA: 0s - loss: 0.2526 - accuracy: 0.9454 - scaled_graph_loss: 0.0329
17/17 [==============================] - 0s 11ms/step - loss: 0.2524 - accuracy: 0.9445 - scaled_graph_loss: 0.0330
Epoch 29/100
1/17 [>.............................] - ETA: 0s - loss: 0.1702 - accuracy: 0.9531 - scaled_graph_loss: 0.0262
6/17 [=========>....................] - ETA: 0s - loss: 0.2242 - accuracy: 0.9436 - scaled_graph_loss: 0.0307
11/17 [==================>...........] - ETA: 0s - loss: 0.2307 - accuracy: 0.9418 - scaled_graph_loss: 0.0317
16/17 [===========================>..] - ETA: 0s - loss: 0.2312 - accuracy: 0.9420 - scaled_graph_loss: 0.0319
17/17 [==============================] - 0s 11ms/step - loss: 0.2310 - accuracy: 0.9424 - scaled_graph_loss: 0.0319
Epoch 30/100
1/17 [>.............................] - ETA: 0s - loss: 0.2223 - accuracy: 0.9609 - scaled_graph_loss: 0.0307
6/17 [=========>....................] - ETA: 0s - loss: 0.2332 - accuracy: 0.9484 - scaled_graph_loss: 0.0332
11/17 [==================>...........] - ETA: 0s - loss: 0.2376 - accuracy: 0.9420 - scaled_graph_loss: 0.0332
16/17 [===========================>..] - ETA: 0s - loss: 0.2388 - accuracy: 0.9394 - scaled_graph_loss: 0.0333
17/17 [==============================] - 0s 11ms/step - loss: 0.2389 - accuracy: 0.9388 - scaled_graph_loss: 0.0334
Epoch 31/100
1/17 [>.............................] - ETA: 0s - loss: 0.1854 - accuracy: 0.9531 - scaled_graph_loss: 0.0376
6/17 [=========>....................] - ETA: 0s - loss: 0.2312 - accuracy: 0.9458 - scaled_graph_loss: 0.0375
11/17 [==================>...........] - ETA: 0s - loss: 0.2252 - accuracy: 0.9507 - scaled_graph_loss: 0.0366
16/17 [===========================>..] - ETA: 0s - loss: 0.2215 - accuracy: 0.9521 - scaled_graph_loss: 0.0357
17/17 [==============================] - 0s 11ms/step - loss: 0.2204 - accuracy: 0.9523 - scaled_graph_loss: 0.0355
Epoch 32/100
1/17 [>.............................] - ETA: 0s - loss: 0.2194 - accuracy: 0.9609 - scaled_graph_loss: 0.0331
6/17 [=========>....................] - ETA: 0s - loss: 0.2197 - accuracy: 0.9525 - scaled_graph_loss: 0.0338
11/17 [==================>...........] - ETA: 0s - loss: 0.2183 - accuracy: 0.9516 - scaled_graph_loss: 0.0337
16/17 [===========================>..] - ETA: 0s - loss: 0.2164 - accuracy: 0.9523 - scaled_graph_loss: 0.0335
17/17 [==============================] - 0s 11ms/step - loss: 0.2159 - accuracy: 0.9525 - scaled_graph_loss: 0.0334
Epoch 33/100
1/17 [>.............................] - ETA: 0s - loss: 0.2320 - accuracy: 0.9453 - scaled_graph_loss: 0.0294
6/17 [=========>....................] - ETA: 0s - loss: 0.2048 - accuracy: 0.9528 - scaled_graph_loss: 0.0345
11/17 [==================>...........] - ETA: 0s - loss: 0.2026 - accuracy: 0.9552 - scaled_graph_loss: 0.0344
16/17 [===========================>..] - ETA: 0s - loss: 0.2023 - accuracy: 0.9560 - scaled_graph_loss: 0.0345
17/17 [==============================] - 0s 11ms/step - loss: 0.2022 - accuracy: 0.9561 - scaled_graph_loss: 0.0345
Epoch 34/100
1/17 [>.............................] - ETA: 0s - loss: 0.1820 - accuracy: 0.9766 - scaled_graph_loss: 0.0323
6/17 [=========>....................] - ETA: 0s - loss: 0.1916 - accuracy: 0.9632 - scaled_graph_loss: 0.0348
11/17 [==================>...........] - ETA: 0s - loss: 0.1918 - accuracy: 0.9616 - scaled_graph_loss: 0.0346
16/17 [===========================>..] - ETA: 0s - loss: 0.1925 - accuracy: 0.9605 - scaled_graph_loss: 0.0345
17/17 [==============================] - 0s 11ms/step - loss: 0.1926 - accuracy: 0.9601 - scaled_graph_loss: 0.0345
Epoch 35/100
1/17 [>.............................] - ETA: 0s - loss: 0.1713 - accuracy: 0.9688 - scaled_graph_loss: 0.0346
7/17 [===========>..................] - ETA: 0s - loss: 0.2006 - accuracy: 0.9541 - scaled_graph_loss: 0.0362
12/17 [====================>.........] - ETA: 0s - loss: 0.2039 - accuracy: 0.9502 - scaled_graph_loss: 0.0346
17/17 [==============================] - ETA: 0s - loss: 0.2048 - accuracy: 0.9493 - scaled_graph_loss: 0.0343
17/17 [==============================] - 0s 11ms/step - loss: 0.2049 - accuracy: 0.9493 - scaled_graph_loss: 0.0343
Epoch 36/100
1/17 [>.............................] - ETA: 0s - loss: 0.1823 - accuracy: 0.9531 - scaled_graph_loss: 0.0381
6/17 [=========>....................] - ETA: 0s - loss: 0.1728 - accuracy: 0.9604 - scaled_graph_loss: 0.0351
11/17 [==================>...........] - ETA: 0s - loss: 0.1726 - accuracy: 0.9613 - scaled_graph_loss: 0.0342
16/17 [===========================>..] - ETA: 0s - loss: 0.1729 - accuracy: 0.9624 - scaled_graph_loss: 0.0336
17/17 [==============================] - 0s 11ms/step - loss: 0.1732 - accuracy: 0.9627 - scaled_graph_loss: 0.0335
Epoch 37/100
1/17 [>.............................] - ETA: 0s - loss: 0.2129 - accuracy: 0.9531 - scaled_graph_loss: 0.0285
7/17 [===========>..................] - ETA: 0s - loss: 0.1986 - accuracy: 0.9555 - scaled_graph_loss: 0.0311
12/17 [====================>.........] - ETA: 0s - loss: 0.1957 - accuracy: 0.9557 - scaled_graph_loss: 0.0321
17/17 [==============================] - ETA: 0s - loss: 0.1919 - accuracy: 0.9571 - scaled_graph_loss: 0.0326
17/17 [==============================] - 0s 11ms/step - loss: 0.1914 - accuracy: 0.9573 - scaled_graph_loss: 0.0327
Epoch 38/100
1/17 [>.............................] - ETA: 0s - loss: 0.1837 - accuracy: 0.9531 - scaled_graph_loss: 0.0336
6/17 [=========>....................] - ETA: 0s - loss: 0.1780 - accuracy: 0.9589 - scaled_graph_loss: 0.0328
11/17 [==================>...........] - ETA: 0s - loss: 0.1767 - accuracy: 0.9581 - scaled_graph_loss: 0.0328
16/17 [===========================>..] - ETA: 0s - loss: 0.1775 - accuracy: 0.9579 - scaled_graph_loss: 0.0330
17/17 [==============================] - 0s 11ms/step - loss: 0.1781 - accuracy: 0.9578 - scaled_graph_loss: 0.0332
Epoch 39/100
1/17 [>.............................] - ETA: 0s - loss: 0.1690 - accuracy: 0.9766 - scaled_graph_loss: 0.0270
6/17 [=========>....................] - ETA: 0s - loss: 0.1600 - accuracy: 0.9791 - scaled_graph_loss: 0.0302
11/17 [==================>...........] - ETA: 0s - loss: 0.1643 - accuracy: 0.9755 - scaled_graph_loss: 0.0315
16/17 [===========================>..] - ETA: 0s - loss: 0.1648 - accuracy: 0.9735 - scaled_graph_loss: 0.0322
17/17 [==============================] - 0s 11ms/step - loss: 0.1650 - accuracy: 0.9730 - scaled_graph_loss: 0.0324
Epoch 40/100
1/17 [>.............................] - ETA: 0s - loss: 0.1678 - accuracy: 0.9688 - scaled_graph_loss: 0.0356
6/17 [=========>....................] - ETA: 0s - loss: 0.1722 - accuracy: 0.9557 - scaled_graph_loss: 0.0337
11/17 [==================>...........] - ETA: 0s - loss: 0.1676 - accuracy: 0.9592 - scaled_graph_loss: 0.0331
16/17 [===========================>..] - ETA: 0s - loss: 0.1649 - accuracy: 0.9616 - scaled_graph_loss: 0.0328
17/17 [==============================] - 0s 11ms/step - loss: 0.1650 - accuracy: 0.9621 - scaled_graph_loss: 0.0328
Epoch 41/100
1/17 [>.............................] - ETA: 0s - loss: 0.1968 - accuracy: 0.9609 - scaled_graph_loss: 0.0379
6/17 [=========>....................] - ETA: 0s - loss: 0.1756 - accuracy: 0.9625 - scaled_graph_loss: 0.0333
11/17 [==================>...........] - ETA: 0s - loss: 0.1739 - accuracy: 0.9634 - scaled_graph_loss: 0.0337
16/17 [===========================>..] - ETA: 0s - loss: 0.1723 - accuracy: 0.9643 - scaled_graph_loss: 0.0338
17/17 [==============================] - 0s 11ms/step - loss: 0.1721 - accuracy: 0.9644 - scaled_graph_loss: 0.0339
Epoch 42/100
1/17 [>.............................] - ETA: 0s - loss: 0.2247 - accuracy: 0.9453 - scaled_graph_loss: 0.0389
6/17 [=========>....................] - ETA: 0s - loss: 0.1814 - accuracy: 0.9654 - scaled_graph_loss: 0.0377
11/17 [==================>...........] - ETA: 0s - loss: 0.1713 - accuracy: 0.9683 - scaled_graph_loss: 0.0365
16/17 [===========================>..] - ETA: 0s - loss: 0.1677 - accuracy: 0.9687 - scaled_graph_loss: 0.0358
17/17 [==============================] - 0s 11ms/step - loss: 0.1672 - accuracy: 0.9687 - scaled_graph_loss: 0.0356
Epoch 43/100
1/17 [>.............................] - ETA: 0s - loss: 0.1990 - accuracy: 0.9375 - scaled_graph_loss: 0.0334
6/17 [=========>....................] - ETA: 0s - loss: 0.1761 - accuracy: 0.9501 - scaled_graph_loss: 0.0338
11/17 [==================>...........] - ETA: 0s - loss: 0.1686 - accuracy: 0.9560 - scaled_graph_loss: 0.0340
16/17 [===========================>..] - ETA: 0s - loss: 0.1654 - accuracy: 0.9591 - scaled_graph_loss: 0.0343
17/17 [==============================] - 0s 11ms/step - loss: 0.1642 - accuracy: 0.9600 - scaled_graph_loss: 0.0343
Epoch 44/100
1/17 [>.............................] - ETA: 0s - loss: 0.1262 - accuracy: 0.9766 - scaled_graph_loss: 0.0354
6/17 [=========>....................] - ETA: 0s - loss: 0.1432 - accuracy: 0.9736 - scaled_graph_loss: 0.0334
11/17 [==================>...........] - ETA: 0s - loss: 0.1458 - accuracy: 0.9737 - scaled_graph_loss: 0.0336
16/17 [===========================>..] - ETA: 0s - loss: 0.1470 - accuracy: 0.9735 - scaled_graph_loss: 0.0335
17/17 [==============================] - 0s 11ms/step - loss: 0.1469 - accuracy: 0.9735 - scaled_graph_loss: 0.0334
Epoch 45/100
1/17 [>.............................] - ETA: 0s - loss: 0.1375 - accuracy: 0.9766 - scaled_graph_loss: 0.0334
6/17 [=========>....................] - ETA: 0s - loss: 0.1482 - accuracy: 0.9730 - scaled_graph_loss: 0.0338
11/17 [==================>...........] - ETA: 0s - loss: 0.1495 - accuracy: 0.9711 - scaled_graph_loss: 0.0340
16/17 [===========================>..] - ETA: 0s - loss: 0.1483 - accuracy: 0.9707 - scaled_graph_loss: 0.0340
17/17 [==============================] - 0s 10ms/step - loss: 0.1478 - accuracy: 0.9708 - scaled_graph_loss: 0.0340
Epoch 46/100
1/17 [>.............................] - ETA: 0s - loss: 0.1644 - accuracy: 0.9609 - scaled_graph_loss: 0.0370
6/17 [=========>....................] - ETA: 0s - loss: 0.1605 - accuracy: 0.9594 - scaled_graph_loss: 0.0388
11/17 [==================>...........] - ETA: 0s - loss: 0.1551 - accuracy: 0.9626 - scaled_graph_loss: 0.0377
16/17 [===========================>..] - ETA: 0s - loss: 0.1539 - accuracy: 0.9638 - scaled_graph_loss: 0.0370
17/17 [==============================] - 0s 11ms/step - loss: 0.1537 - accuracy: 0.9640 - scaled_graph_loss: 0.0367
Epoch 47/100
1/17 [>.............................] - ETA: 0s - loss: 0.1467 - accuracy: 0.9688 - scaled_graph_loss: 0.0320
6/17 [=========>....................] - ETA: 0s - loss: 0.1633 - accuracy: 0.9640 - scaled_graph_loss: 0.0357
11/17 [==================>...........] - ETA: 0s - loss: 0.1553 - accuracy: 0.9677 - scaled_graph_loss: 0.0356
16/17 [===========================>..] - ETA: 0s - loss: 0.1521 - accuracy: 0.9689 - scaled_graph_loss: 0.0356
17/17 [==============================] - 0s 11ms/step - loss: 0.1513 - accuracy: 0.9691 - scaled_graph_loss: 0.0355
Epoch 48/100
1/17 [>.............................] - ETA: 0s - loss: 0.0923 - accuracy: 1.0000 - scaled_graph_loss: 0.0345
6/17 [=========>....................] - ETA: 0s - loss: 0.1217 - accuracy: 0.9798 - scaled_graph_loss: 0.0338
11/17 [==================>...........] - ETA: 0s - loss: 0.1230 - accuracy: 0.9780 - scaled_graph_loss: 0.0326
16/17 [===========================>..] - ETA: 0s - loss: 0.1248 - accuracy: 0.9769 - scaled_graph_loss: 0.0326
17/17 [==============================] - 0s 11ms/step - loss: 0.1252 - accuracy: 0.9768 - scaled_graph_loss: 0.0327
Epoch 49/100
1/17 [>.............................] - ETA: 0s - loss: 0.1742 - accuracy: 0.9688 - scaled_graph_loss: 0.0443
6/17 [=========>....................] - ETA: 0s - loss: 0.1537 - accuracy: 0.9699 - scaled_graph_loss: 0.0369
11/17 [==================>...........] - ETA: 0s - loss: 0.1492 - accuracy: 0.9710 - scaled_graph_loss: 0.0356
16/17 [===========================>..] - ETA: 0s - loss: 0.1449 - accuracy: 0.9721 - scaled_graph_loss: 0.0353
17/17 [==============================] - 0s 11ms/step - loss: 0.1443 - accuracy: 0.9722 - scaled_graph_loss: 0.0352
Epoch 50/100
1/17 [>.............................] - ETA: 0s - loss: 0.1326 - accuracy: 0.9766 - scaled_graph_loss: 0.0317
6/17 [=========>....................] - ETA: 0s - loss: 0.1353 - accuracy: 0.9724 - scaled_graph_loss: 0.0326
11/17 [==================>...........] - ETA: 0s - loss: 0.1343 - accuracy: 0.9726 - scaled_graph_loss: 0.0331
16/17 [===========================>..] - ETA: 0s - loss: 0.1342 - accuracy: 0.9729 - scaled_graph_loss: 0.0333
17/17 [==============================] - 0s 11ms/step - loss: 0.1339 - accuracy: 0.9731 - scaled_graph_loss: 0.0333
Epoch 51/100
1/17 [>.............................] - ETA: 0s - loss: 0.1408 - accuracy: 0.9688 - scaled_graph_loss: 0.0395
6/17 [=========>....................] - ETA: 0s - loss: 0.1495 - accuracy: 0.9707 - scaled_graph_loss: 0.0396
11/17 [==================>...........] - ETA: 0s - loss: 0.1436 - accuracy: 0.9728 - scaled_graph_loss: 0.0377
16/17 [===========================>..] - ETA: 0s - loss: 0.1393 - accuracy: 0.9739 - scaled_graph_loss: 0.0365
17/17 [==============================] - 0s 11ms/step - loss: 0.1385 - accuracy: 0.9741 - scaled_graph_loss: 0.0362
Epoch 52/100
1/17 [>.............................] - ETA: 0s - loss: 0.1405 - accuracy: 0.9844 - scaled_graph_loss: 0.0311
6/17 [=========>....................] - ETA: 0s - loss: 0.1388 - accuracy: 0.9733 - scaled_graph_loss: 0.0322
11/17 [==================>...........] - ETA: 0s - loss: 0.1391 - accuracy: 0.9723 - scaled_graph_loss: 0.0331
16/17 [===========================>..] - ETA: 0s - loss: 0.1355 - accuracy: 0.9730 - scaled_graph_loss: 0.0332
17/17 [==============================] - 0s 11ms/step - loss: 0.1347 - accuracy: 0.9732 - scaled_graph_loss: 0.0333
Epoch 53/100
1/17 [>.............................] - ETA: 0s - loss: 0.0947 - accuracy: 0.9844 - scaled_graph_loss: 0.0354
6/17 [=========>....................] - ETA: 0s - loss: 0.1166 - accuracy: 0.9780 - scaled_graph_loss: 0.0358
11/17 [==================>...........] - ETA: 0s - loss: 0.1202 - accuracy: 0.9779 - scaled_graph_loss: 0.0355
16/17 [===========================>..] - ETA: 0s - loss: 0.1219 - accuracy: 0.9784 - scaled_graph_loss: 0.0354
17/17 [==============================] - 0s 11ms/step - loss: 0.1222 - accuracy: 0.9785 - scaled_graph_loss: 0.0353
Epoch 54/100
1/17 [>.............................] - ETA: 0s - loss: 0.1137 - accuracy: 0.9844 - scaled_graph_loss: 0.0369
6/17 [=========>....................] - ETA: 0s - loss: 0.1223 - accuracy: 0.9753 - scaled_graph_loss: 0.0349
11/17 [==================>...........] - ETA: 0s - loss: 0.1271 - accuracy: 0.9724 - scaled_graph_loss: 0.0356
16/17 [===========================>..] - ETA: 0s - loss: 0.1261 - accuracy: 0.9735 - scaled_graph_loss: 0.0354
17/17 [==============================] - 0s 11ms/step - loss: 0.1258 - accuracy: 0.9738 - scaled_graph_loss: 0.0354
Epoch 55/100
1/17 [>.............................] - ETA: 0s - loss: 0.1099 - accuracy: 0.9922 - scaled_graph_loss: 0.0441
6/17 [=========>....................] - ETA: 0s - loss: 0.1159 - accuracy: 0.9827 - scaled_graph_loss: 0.0375
11/17 [==================>...........] - ETA: 0s - loss: 0.1187 - accuracy: 0.9797 - scaled_graph_loss: 0.0359
16/17 [===========================>..] - ETA: 0s - loss: 0.1207 - accuracy: 0.9775 - scaled_graph_loss: 0.0354
17/17 [==============================] - 0s 11ms/step - loss: 0.1209 - accuracy: 0.9771 - scaled_graph_loss: 0.0352
Epoch 56/100
1/17 [>.............................] - ETA: 0s - loss: 0.1086 - accuracy: 0.9844 - scaled_graph_loss: 0.0367
6/17 [=========>....................] - ETA: 0s - loss: 0.1324 - accuracy: 0.9804 - scaled_graph_loss: 0.0372
11/17 [==================>...........] - ETA: 0s - loss: 0.1301 - accuracy: 0.9792 - scaled_graph_loss: 0.0361
16/17 [===========================>..] - ETA: 0s - loss: 0.1287 - accuracy: 0.9787 - scaled_graph_loss: 0.0354
17/17 [==============================] - 0s 11ms/step - loss: 0.1279 - accuracy: 0.9787 - scaled_graph_loss: 0.0352
Epoch 57/100
1/17 [>.............................] - ETA: 0s - loss: 0.1188 - accuracy: 0.9688 - scaled_graph_loss: 0.0193
6/17 [=========>....................] - ETA: 0s - loss: 0.1399 - accuracy: 0.9646 - scaled_graph_loss: 0.0267
11/17 [==================>...........] - ETA: 0s - loss: 0.1329 - accuracy: 0.9684 - scaled_graph_loss: 0.0295
16/17 [===========================>..] - ETA: 0s - loss: 0.1287 - accuracy: 0.9710 - scaled_graph_loss: 0.0308
17/17 [==============================] - 0s 11ms/step - loss: 0.1273 - accuracy: 0.9719 - scaled_graph_loss: 0.0312
Epoch 58/100
1/17 [>.............................] - ETA: 0s - loss: 0.1194 - accuracy: 0.9766 - scaled_graph_loss: 0.0212
6/17 [=========>....................] - ETA: 0s - loss: 0.1207 - accuracy: 0.9740 - scaled_graph_loss: 0.0320
11/17 [==================>...........] - ETA: 0s - loss: 0.1216 - accuracy: 0.9738 - scaled_graph_loss: 0.0330
16/17 [===========================>..] - ETA: 0s - loss: 0.1207 - accuracy: 0.9746 - scaled_graph_loss: 0.0331
17/17 [==============================] - 0s 11ms/step - loss: 0.1206 - accuracy: 0.9747 - scaled_graph_loss: 0.0332
Epoch 59/100
1/17 [>.............................] - ETA: 0s - loss: 0.1078 - accuracy: 0.9766 - scaled_graph_loss: 0.0364
6/17 [=========>....................] - ETA: 0s - loss: 0.1112 - accuracy: 0.9792 - scaled_graph_loss: 0.0360
11/17 [==================>...........] - ETA: 0s - loss: 0.1109 - accuracy: 0.9806 - scaled_graph_loss: 0.0346
16/17 [===========================>..] - ETA: 0s - loss: 0.1108 - accuracy: 0.9813 - scaled_graph_loss: 0.0343
17/17 [==============================] - 0s 11ms/step - loss: 0.1109 - accuracy: 0.9814 - scaled_graph_loss: 0.0342
Epoch 60/100
1/17 [>.............................] - ETA: 0s - loss: 0.0989 - accuracy: 0.9922 - scaled_graph_loss: 0.0382
6/17 [=========>....................] - ETA: 0s - loss: 0.1116 - accuracy: 0.9826 - scaled_graph_loss: 0.0343
11/17 [==================>...........] - ETA: 0s - loss: 0.1146 - accuracy: 0.9799 - scaled_graph_loss: 0.0340
16/17 [===========================>..] - ETA: 0s - loss: 0.1166 - accuracy: 0.9781 - scaled_graph_loss: 0.0339
17/17 [==============================] - 0s 11ms/step - loss: 0.1168 - accuracy: 0.9778 - scaled_graph_loss: 0.0338
Epoch 61/100
1/17 [>.............................] - ETA: 0s - loss: 0.1400 - accuracy: 0.9688 - scaled_graph_loss: 0.0345
6/17 [=========>....................] - ETA: 0s - loss: 0.1151 - accuracy: 0.9804 - scaled_graph_loss: 0.0342
11/17 [==================>...........] - ETA: 0s - loss: 0.1123 - accuracy: 0.9819 - scaled_graph_loss: 0.0342
16/17 [===========================>..] - ETA: 0s - loss: 0.1126 - accuracy: 0.9820 - scaled_graph_loss: 0.0341
17/17 [==============================] - 0s 11ms/step - loss: 0.1125 - accuracy: 0.9820 - scaled_graph_loss: 0.0341
Epoch 62/100
1/17 [>.............................] - ETA: 0s - loss: 0.1052 - accuracy: 0.9844 - scaled_graph_loss: 0.0389
6/17 [=========>....................] - ETA: 0s - loss: 0.1102 - accuracy: 0.9806 - scaled_graph_loss: 0.0380
11/17 [==================>...........] - ETA: 0s - loss: 0.1072 - accuracy: 0.9823 - scaled_graph_loss: 0.0368
16/17 [===========================>..] - ETA: 0s - loss: 0.1055 - accuracy: 0.9826 - scaled_graph_loss: 0.0360
17/17 [==============================] - 0s 11ms/step - loss: 0.1055 - accuracy: 0.9824 - scaled_graph_loss: 0.0359
Epoch 63/100
1/17 [>.............................] - ETA: 0s - loss: 0.1581 - accuracy: 0.9609 - scaled_graph_loss: 0.0396
6/17 [=========>....................] - ETA: 0s - loss: 0.1347 - accuracy: 0.9712 - scaled_graph_loss: 0.0384
11/17 [==================>...........] - ETA: 0s - loss: 0.1242 - accuracy: 0.9751 - scaled_graph_loss: 0.0370
16/17 [===========================>..] - ETA: 0s - loss: 0.1195 - accuracy: 0.9767 - scaled_graph_loss: 0.0363
17/17 [==============================] - 0s 11ms/step - loss: 0.1183 - accuracy: 0.9771 - scaled_graph_loss: 0.0361
Epoch 64/100
1/17 [>.............................] - ETA: 0s - loss: 0.1065 - accuracy: 1.0000 - scaled_graph_loss: 0.0421
6/17 [=========>....................] - ETA: 0s - loss: 0.0984 - accuracy: 0.9920 - scaled_graph_loss: 0.0347
11/17 [==================>...........] - ETA: 0s - loss: 0.1024 - accuracy: 0.9874 - scaled_graph_loss: 0.0344
16/17 [===========================>..] - ETA: 0s - loss: 0.1055 - accuracy: 0.9842 - scaled_graph_loss: 0.0343
17/17 [==============================] - 0s 11ms/step - loss: 0.1063 - accuracy: 0.9835 - scaled_graph_loss: 0.0343
Epoch 65/100
1/17 [>.............................] - ETA: 0s - loss: 0.0853 - accuracy: 0.9922 - scaled_graph_loss: 0.0221
6/17 [=========>....................] - ETA: 0s - loss: 0.1041 - accuracy: 0.9832 - scaled_graph_loss: 0.0267
11/17 [==================>...........] - ETA: 0s - loss: 0.1111 - accuracy: 0.9791 - scaled_graph_loss: 0.0289
16/17 [===========================>..] - ETA: 0s - loss: 0.1117 - accuracy: 0.9786 - scaled_graph_loss: 0.0302
17/17 [==============================] - 0s 10ms/step - loss: 0.1117 - accuracy: 0.9786 - scaled_graph_loss: 0.0306
Epoch 66/100
1/17 [>.............................] - ETA: 0s - loss: 0.0754 - accuracy: 0.9922 - scaled_graph_loss: 0.0311
6/17 [=========>....................] - ETA: 0s - loss: 0.0949 - accuracy: 0.9840 - scaled_graph_loss: 0.0338
11/17 [==================>...........] - ETA: 0s - loss: 0.1053 - accuracy: 0.9797 - scaled_graph_loss: 0.0342
16/17 [===========================>..] - ETA: 0s - loss: 0.1085 - accuracy: 0.9786 - scaled_graph_loss: 0.0343
17/17 [==============================] - 0s 11ms/step - loss: 0.1091 - accuracy: 0.9783 - scaled_graph_loss: 0.0343
Epoch 67/100
1/17 [>.............................] - ETA: 0s - loss: 0.0757 - accuracy: 0.9922 - scaled_graph_loss: 0.0379
6/17 [=========>....................] - ETA: 0s - loss: 0.0894 - accuracy: 0.9900 - scaled_graph_loss: 0.0349
11/17 [==================>...........] - ETA: 0s - loss: 0.0941 - accuracy: 0.9890 - scaled_graph_loss: 0.0346
16/17 [===========================>..] - ETA: 0s - loss: 0.0954 - accuracy: 0.9884 - scaled_graph_loss: 0.0340
17/17 [==============================] - 0s 10ms/step - loss: 0.0958 - accuracy: 0.9882 - scaled_graph_loss: 0.0340
Epoch 68/100
1/17 [>.............................] - ETA: 0s - loss: 0.1053 - accuracy: 0.9844 - scaled_graph_loss: 0.0414
6/17 [=========>....................] - ETA: 0s - loss: 0.1071 - accuracy: 0.9853 - scaled_graph_loss: 0.0389
11/17 [==================>...........] - ETA: 0s - loss: 0.1103 - accuracy: 0.9843 - scaled_graph_loss: 0.0380
16/17 [===========================>..] - ETA: 0s - loss: 0.1084 - accuracy: 0.9842 - scaled_graph_loss: 0.0369
17/17 [==============================] - 0s 11ms/step - loss: 0.1077 - accuracy: 0.9842 - scaled_graph_loss: 0.0366
Epoch 69/100
1/17 [>.............................] - ETA: 0s - loss: 0.1291 - accuracy: 0.9609 - scaled_graph_loss: 0.0404
6/17 [=========>....................] - ETA: 0s - loss: 0.1163 - accuracy: 0.9721 - scaled_graph_loss: 0.0358
11/17 [==================>...........] - ETA: 0s - loss: 0.1122 - accuracy: 0.9748 - scaled_graph_loss: 0.0354
16/17 [===========================>..] - ETA: 0s - loss: 0.1101 - accuracy: 0.9763 - scaled_graph_loss: 0.0353
17/17 [==============================] - 0s 11ms/step - loss: 0.1092 - accuracy: 0.9767 - scaled_graph_loss: 0.0353
Epoch 70/100
1/17 [>.............................] - ETA: 0s - loss: 0.1553 - accuracy: 0.9531 - scaled_graph_loss: 0.0320
7/17 [===========>..................] - ETA: 0s - loss: 0.1359 - accuracy: 0.9688 - scaled_graph_loss: 0.0330
12/17 [====================>.........] - ETA: 0s - loss: 0.1228 - accuracy: 0.9748 - scaled_graph_loss: 0.0335
17/17 [==============================] - ETA: 0s - loss: 0.1167 - accuracy: 0.9773 - scaled_graph_loss: 0.0338
17/17 [==============================] - 0s 10ms/step - loss: 0.1159 - accuracy: 0.9777 - scaled_graph_loss: 0.0338
Epoch 71/100
1/17 [>.............................] - ETA: 0s - loss: 0.0821 - accuracy: 0.9922 - scaled_graph_loss: 0.0351
6/17 [=========>....................] - ETA: 0s - loss: 0.0877 - accuracy: 0.9890 - scaled_graph_loss: 0.0325
11/17 [==================>...........] - ETA: 0s - loss: 0.0879 - accuracy: 0.9883 - scaled_graph_loss: 0.0321
16/17 [===========================>..] - ETA: 0s - loss: 0.0875 - accuracy: 0.9882 - scaled_graph_loss: 0.0323
17/17 [==============================] - 0s 11ms/step - loss: 0.0875 - accuracy: 0.9881 - scaled_graph_loss: 0.0325
Epoch 72/100
1/17 [>.............................] - ETA: 0s - loss: 0.0810 - accuracy: 0.9922 - scaled_graph_loss: 0.0303
6/17 [=========>....................] - ETA: 0s - loss: 0.0875 - accuracy: 0.9874 - scaled_graph_loss: 0.0321
11/17 [==================>...........] - ETA: 0s - loss: 0.0900 - accuracy: 0.9870 - scaled_graph_loss: 0.0334
16/17 [===========================>..] - ETA: 0s - loss: 0.0905 - accuracy: 0.9865 - scaled_graph_loss: 0.0337
17/17 [==============================] - 0s 11ms/step - loss: 0.0905 - accuracy: 0.9864 - scaled_graph_loss: 0.0337
Epoch 73/100
1/17 [>.............................] - ETA: 0s - loss: 0.1077 - accuracy: 0.9766 - scaled_graph_loss: 0.0291
6/17 [=========>....................] - ETA: 0s - loss: 0.1069 - accuracy: 0.9738 - scaled_graph_loss: 0.0308
11/17 [==================>...........] - ETA: 0s - loss: 0.1044 - accuracy: 0.9752 - scaled_graph_loss: 0.0314
16/17 [===========================>..] - ETA: 0s - loss: 0.1024 - accuracy: 0.9765 - scaled_graph_loss: 0.0318
17/17 [==============================] - 0s 11ms/step - loss: 0.1021 - accuracy: 0.9767 - scaled_graph_loss: 0.0321
Epoch 74/100
1/17 [>.............................] - ETA: 0s - loss: 0.1033 - accuracy: 0.9766 - scaled_graph_loss: 0.0313
6/17 [=========>....................] - ETA: 0s - loss: 0.1050 - accuracy: 0.9770 - scaled_graph_loss: 0.0325
11/17 [==================>...........] - ETA: 0s - loss: 0.1052 - accuracy: 0.9770 - scaled_graph_loss: 0.0323
16/17 [===========================>..] - ETA: 0s - loss: 0.1050 - accuracy: 0.9772 - scaled_graph_loss: 0.0328
17/17 [==============================] - 0s 11ms/step - loss: 0.1047 - accuracy: 0.9773 - scaled_graph_loss: 0.0328
Epoch 75/100
1/17 [>.............................] - ETA: 0s - loss: 0.0894 - accuracy: 0.9844 - scaled_graph_loss: 0.0390
6/17 [=========>....................] - ETA: 0s - loss: 0.0917 - accuracy: 0.9803 - scaled_graph_loss: 0.0342
11/17 [==================>...........] - ETA: 0s - loss: 0.0930 - accuracy: 0.9809 - scaled_graph_loss: 0.0339
16/17 [===========================>..] - ETA: 0s - loss: 0.0953 - accuracy: 0.9811 - scaled_graph_loss: 0.0338
17/17 [==============================] - 0s 11ms/step - loss: 0.0958 - accuracy: 0.9812 - scaled_graph_loss: 0.0338
Epoch 76/100
1/17 [>.............................] - ETA: 0s - loss: 0.1193 - accuracy: 0.9688 - scaled_graph_loss: 0.0307
6/17 [=========>....................] - ETA: 0s - loss: 0.1032 - accuracy: 0.9775 - scaled_graph_loss: 0.0319
11/17 [==================>...........] - ETA: 0s - loss: 0.1027 - accuracy: 0.9784 - scaled_graph_loss: 0.0329
16/17 [===========================>..] - ETA: 0s - loss: 0.1005 - accuracy: 0.9798 - scaled_graph_loss: 0.0334
17/17 [==============================] - 0s 11ms/step - loss: 0.0997 - accuracy: 0.9802 - scaled_graph_loss: 0.0335
Epoch 77/100
1/17 [>.............................] - ETA: 0s - loss: 0.0831 - accuracy: 0.9922 - scaled_graph_loss: 0.0265
6/17 [=========>....................] - ETA: 0s - loss: 0.0808 - accuracy: 0.9885 - scaled_graph_loss: 0.0286
11/17 [==================>...........] - ETA: 0s - loss: 0.0827 - accuracy: 0.9886 - scaled_graph_loss: 0.0300
16/17 [===========================>..] - ETA: 0s - loss: 0.0847 - accuracy: 0.9879 - scaled_graph_loss: 0.0311
17/17 [==============================] - 0s 11ms/step - loss: 0.0853 - accuracy: 0.9877 - scaled_graph_loss: 0.0314
Epoch 78/100
1/17 [>.............................] - ETA: 0s - loss: 0.1499 - accuracy: 0.9609 - scaled_graph_loss: 0.0412
6/17 [=========>....................] - ETA: 0s - loss: 0.1179 - accuracy: 0.9740 - scaled_graph_loss: 0.0368
11/17 [==================>...........] - ETA: 0s - loss: 0.1070 - accuracy: 0.9786 - scaled_graph_loss: 0.0348
16/17 [===========================>..] - ETA: 0s - loss: 0.1027 - accuracy: 0.9806 - scaled_graph_loss: 0.0346
17/17 [==============================] - 0s 11ms/step - loss: 0.1016 - accuracy: 0.9810 - scaled_graph_loss: 0.0346
Epoch 79/100
1/17 [>.............................] - ETA: 0s - loss: 0.0959 - accuracy: 0.9922 - scaled_graph_loss: 0.0267
6/17 [=========>....................] - ETA: 0s - loss: 0.0973 - accuracy: 0.9816 - scaled_graph_loss: 0.0301
11/17 [==================>...........] - ETA: 0s - loss: 0.0978 - accuracy: 0.9807 - scaled_graph_loss: 0.0308
16/17 [===========================>..] - ETA: 0s - loss: 0.0980 - accuracy: 0.9808 - scaled_graph_loss: 0.0315
17/17 [==============================] - 0s 10ms/step - loss: 0.0978 - accuracy: 0.9809 - scaled_graph_loss: 0.0317
Epoch 80/100
1/17 [>.............................] - ETA: 0s - loss: 0.0929 - accuracy: 1.0000 - scaled_graph_loss: 0.0313
6/17 [=========>....................] - ETA: 0s - loss: 0.0889 - accuracy: 0.9878 - scaled_graph_loss: 0.0320
11/17 [==================>...........] - ETA: 0s - loss: 0.0890 - accuracy: 0.9871 - scaled_graph_loss: 0.0322
16/17 [===========================>..] - ETA: 0s - loss: 0.0906 - accuracy: 0.9865 - scaled_graph_loss: 0.0328
17/17 [==============================] - 0s 11ms/step - loss: 0.0908 - accuracy: 0.9864 - scaled_graph_loss: 0.0329
Epoch 81/100
1/17 [>.............................] - ETA: 0s - loss: 0.0517 - accuracy: 0.9922 - scaled_graph_loss: 0.0233
6/17 [=========>....................] - ETA: 0s - loss: 0.0610 - accuracy: 0.9936 - scaled_graph_loss: 0.0318
11/17 [==================>...........] - ETA: 0s - loss: 0.0676 - accuracy: 0.9921 - scaled_graph_loss: 0.0329
16/17 [===========================>..] - ETA: 0s - loss: 0.0728 - accuracy: 0.9907 - scaled_graph_loss: 0.0332
17/17 [==============================] - 0s 11ms/step - loss: 0.0742 - accuracy: 0.9902 - scaled_graph_loss: 0.0332
Epoch 82/100
1/17 [>.............................] - ETA: 0s - loss: 0.0890 - accuracy: 0.9844 - scaled_graph_loss: 0.0339
6/17 [=========>....................] - ETA: 0s - loss: 0.0938 - accuracy: 0.9871 - scaled_graph_loss: 0.0348
11/17 [==================>...........] - ETA: 0s - loss: 0.0926 - accuracy: 0.9875 - scaled_graph_loss: 0.0346
16/17 [===========================>..] - ETA: 0s - loss: 0.0916 - accuracy: 0.9874 - scaled_graph_loss: 0.0346
17/17 [==============================] - 0s 11ms/step - loss: 0.0910 - accuracy: 0.9875 - scaled_graph_loss: 0.0345
Epoch 83/100
1/17 [>.............................] - ETA: 0s - loss: 0.0926 - accuracy: 0.9766 - scaled_graph_loss: 0.0342
6/17 [=========>....................] - ETA: 0s - loss: 0.0870 - accuracy: 0.9854 - scaled_graph_loss: 0.0347
11/17 [==================>...........] - ETA: 0s - loss: 0.0909 - accuracy: 0.9851 - scaled_graph_loss: 0.0350
16/17 [===========================>..] - ETA: 0s - loss: 0.0908 - accuracy: 0.9850 - scaled_graph_loss: 0.0345
17/17 [==============================] - 0s 10ms/step - loss: 0.0908 - accuracy: 0.9848 - scaled_graph_loss: 0.0345
Epoch 84/100
1/17 [>.............................] - ETA: 0s - loss: 0.0811 - accuracy: 0.9844 - scaled_graph_loss: 0.0244
6/17 [=========>....................] - ETA: 0s - loss: 0.0881 - accuracy: 0.9793 - scaled_graph_loss: 0.0328
11/17 [==================>...........] - ETA: 0s - loss: 0.0859 - accuracy: 0.9812 - scaled_graph_loss: 0.0324
16/17 [===========================>..] - ETA: 0s - loss: 0.0850 - accuracy: 0.9827 - scaled_graph_loss: 0.0327
17/17 [==============================] - 0s 10ms/step - loss: 0.0848 - accuracy: 0.9831 - scaled_graph_loss: 0.0328
Epoch 85/100
1/17 [>.............................] - ETA: 0s - loss: 0.1330 - accuracy: 0.9609 - scaled_graph_loss: 0.0381
6/17 [=========>....................] - ETA: 0s - loss: 0.1137 - accuracy: 0.9745 - scaled_graph_loss: 0.0358
11/17 [==================>...........] - ETA: 0s - loss: 0.1060 - accuracy: 0.9776 - scaled_graph_loss: 0.0350
16/17 [===========================>..] - ETA: 0s - loss: 0.1009 - accuracy: 0.9799 - scaled_graph_loss: 0.0346
17/17 [==============================] - 0s 11ms/step - loss: 0.0997 - accuracy: 0.9804 - scaled_graph_loss: 0.0345
Epoch 86/100
1/17 [>.............................] - ETA: 0s - loss: 0.1109 - accuracy: 0.9844 - scaled_graph_loss: 0.0366
6/17 [=========>....................] - ETA: 0s - loss: 0.0983 - accuracy: 0.9855 - scaled_graph_loss: 0.0323
11/17 [==================>...........] - ETA: 0s - loss: 0.0927 - accuracy: 0.9859 - scaled_graph_loss: 0.0322
16/17 [===========================>..] - ETA: 0s - loss: 0.0904 - accuracy: 0.9858 - scaled_graph_loss: 0.0324
17/17 [==============================] - 0s 10ms/step - loss: 0.0901 - accuracy: 0.9859 - scaled_graph_loss: 0.0326
Epoch 87/100
1/17 [>.............................] - ETA: 0s - loss: 0.0853 - accuracy: 0.9844 - scaled_graph_loss: 0.0349
6/17 [=========>....................] - ETA: 0s - loss: 0.0810 - accuracy: 0.9873 - scaled_graph_loss: 0.0344
11/17 [==================>...........] - ETA: 0s - loss: 0.0821 - accuracy: 0.9870 - scaled_graph_loss: 0.0335
16/17 [===========================>..] - ETA: 0s - loss: 0.0822 - accuracy: 0.9872 - scaled_graph_loss: 0.0334
17/17 [==============================] - 0s 11ms/step - loss: 0.0825 - accuracy: 0.9873 - scaled_graph_loss: 0.0334
Epoch 88/100
1/17 [>.............................] - ETA: 0s - loss: 0.0529 - accuracy: 1.0000 - scaled_graph_loss: 0.0248
6/17 [=========>....................] - ETA: 0s - loss: 0.0770 - accuracy: 0.9899 - scaled_graph_loss: 0.0315
11/17 [==================>...........] - ETA: 0s - loss: 0.0800 - accuracy: 0.9888 - scaled_graph_loss: 0.0327
16/17 [===========================>..] - ETA: 0s - loss: 0.0805 - accuracy: 0.9886 - scaled_graph_loss: 0.0331
17/17 [==============================] - 0s 11ms/step - loss: 0.0805 - accuracy: 0.9885 - scaled_graph_loss: 0.0332
Epoch 89/100
1/17 [>.............................] - ETA: 0s - loss: 0.0534 - accuracy: 1.0000 - scaled_graph_loss: 0.0308
7/17 [===========>..................] - ETA: 0s - loss: 0.0670 - accuracy: 0.9917 - scaled_graph_loss: 0.0321
12/17 [====================>.........] - ETA: 0s - loss: 0.0727 - accuracy: 0.9900 - scaled_graph_loss: 0.0326
17/17 [==============================] - ETA: 0s - loss: 0.0769 - accuracy: 0.9887 - scaled_graph_loss: 0.0329
17/17 [==============================] - 0s 11ms/step - loss: 0.0776 - accuracy: 0.9885 - scaled_graph_loss: 0.0330
Epoch 90/100
1/17 [>.............................] - ETA: 0s - loss: 0.1247 - accuracy: 0.9766 - scaled_graph_loss: 0.0351
6/17 [=========>....................] - ETA: 0s - loss: 0.1061 - accuracy: 0.9778 - scaled_graph_loss: 0.0339
11/17 [==================>...........] - ETA: 0s - loss: 0.0994 - accuracy: 0.9790 - scaled_graph_loss: 0.0339
16/17 [===========================>..] - ETA: 0s - loss: 0.0966 - accuracy: 0.9797 - scaled_graph_loss: 0.0340
17/17 [==============================] - 0s 11ms/step - loss: 0.0960 - accuracy: 0.9799 - scaled_graph_loss: 0.0341
Epoch 91/100
1/17 [>.............................] - ETA: 0s - loss: 0.0697 - accuracy: 1.0000 - scaled_graph_loss: 0.0358
6/17 [=========>....................] - ETA: 0s - loss: 0.0743 - accuracy: 0.9914 - scaled_graph_loss: 0.0330
11/17 [==================>...........] - ETA: 0s - loss: 0.0768 - accuracy: 0.9895 - scaled_graph_loss: 0.0335
16/17 [===========================>..] - ETA: 0s - loss: 0.0785 - accuracy: 0.9888 - scaled_graph_loss: 0.0337
17/17 [==============================] - 0s 11ms/step - loss: 0.0787 - accuracy: 0.9888 - scaled_graph_loss: 0.0337
Epoch 92/100
1/17 [>.............................] - ETA: 0s - loss: 0.0482 - accuracy: 1.0000 - scaled_graph_loss: 0.0318
6/17 [=========>....................] - ETA: 0s - loss: 0.0661 - accuracy: 0.9944 - scaled_graph_loss: 0.0362
11/17 [==================>...........] - ETA: 0s - loss: 0.0724 - accuracy: 0.9923 - scaled_graph_loss: 0.0357
16/17 [===========================>..] - ETA: 0s - loss: 0.0734 - accuracy: 0.9916 - scaled_graph_loss: 0.0350
17/17 [==============================] - 0s 11ms/step - loss: 0.0736 - accuracy: 0.9914 - scaled_graph_loss: 0.0348
Epoch 93/100
1/17 [>.............................] - ETA: 0s - loss: 0.0716 - accuracy: 0.9922 - scaled_graph_loss: 0.0371
6/17 [=========>....................] - ETA: 0s - loss: 0.0794 - accuracy: 0.9912 - scaled_graph_loss: 0.0364
11/17 [==================>...........] - ETA: 0s - loss: 0.0796 - accuracy: 0.9903 - scaled_graph_loss: 0.0354
16/17 [===========================>..] - ETA: 0s - loss: 0.0804 - accuracy: 0.9894 - scaled_graph_loss: 0.0349
17/17 [==============================] - 0s 10ms/step - loss: 0.0806 - accuracy: 0.9892 - scaled_graph_loss: 0.0347
Epoch 94/100
1/17 [>.............................] - ETA: 0s - loss: 0.0763 - accuracy: 0.9922 - scaled_graph_loss: 0.0247
6/17 [=========>....................] - ETA: 0s - loss: 0.0683 - accuracy: 0.9930 - scaled_graph_loss: 0.0284
11/17 [==================>...........] - ETA: 0s - loss: 0.0703 - accuracy: 0.9922 - scaled_graph_loss: 0.0301
16/17 [===========================>..] - ETA: 0s - loss: 0.0719 - accuracy: 0.9914 - scaled_graph_loss: 0.0312
17/17 [==============================] - 0s 11ms/step - loss: 0.0723 - accuracy: 0.9912 - scaled_graph_loss: 0.0314
Epoch 95/100
1/17 [>.............................] - ETA: 0s - loss: 0.0373 - accuracy: 0.9922 - scaled_graph_loss: 0.0175
6/17 [=========>....................] - ETA: 0s - loss: 0.0560 - accuracy: 0.9905 - scaled_graph_loss: 0.0264
11/17 [==================>...........] - ETA: 0s - loss: 0.0615 - accuracy: 0.9893 - scaled_graph_loss: 0.0278
16/17 [===========================>..] - ETA: 0s - loss: 0.0658 - accuracy: 0.9888 - scaled_graph_loss: 0.0290
17/17 [==============================] - 0s 11ms/step - loss: 0.0671 - accuracy: 0.9887 - scaled_graph_loss: 0.0295
Epoch 96/100
1/17 [>.............................] - ETA: 0s - loss: 0.0799 - accuracy: 0.9844 - scaled_graph_loss: 0.0301
6/17 [=========>....................] - ETA: 0s - loss: 0.0771 - accuracy: 0.9884 - scaled_graph_loss: 0.0321
11/17 [==================>...........] - ETA: 0s - loss: 0.0770 - accuracy: 0.9889 - scaled_graph_loss: 0.0323
16/17 [===========================>..] - ETA: 0s - loss: 0.0777 - accuracy: 0.9888 - scaled_graph_loss: 0.0326
17/17 [==============================] - 0s 11ms/step - loss: 0.0780 - accuracy: 0.9887 - scaled_graph_loss: 0.0327
Epoch 97/100
1/17 [>.............................] - ETA: 0s - loss: 0.0966 - accuracy: 0.9844 - scaled_graph_loss: 0.0319
6/17 [=========>....................] - ETA: 0s - loss: 0.0917 - accuracy: 0.9851 - scaled_graph_loss: 0.0322
11/17 [==================>...........] - ETA: 0s - loss: 0.0876 - accuracy: 0.9861 - scaled_graph_loss: 0.0325
16/17 [===========================>..] - ETA: 0s - loss: 0.0851 - accuracy: 0.9869 - scaled_graph_loss: 0.0330
17/17 [==============================] - 0s 11ms/step - loss: 0.0843 - accuracy: 0.9871 - scaled_graph_loss: 0.0331
Epoch 98/100
1/17 [>.............................] - ETA: 0s - loss: 0.0758 - accuracy: 1.0000 - scaled_graph_loss: 0.0291
6/17 [=========>....................] - ETA: 0s - loss: 0.0777 - accuracy: 0.9919 - scaled_graph_loss: 0.0321
11/17 [==================>...........] - ETA: 0s - loss: 0.0784 - accuracy: 0.9914 - scaled_graph_loss: 0.0328
16/17 [===========================>..] - ETA: 0s - loss: 0.0792 - accuracy: 0.9905 - scaled_graph_loss: 0.0331
17/17 [==============================] - 0s 11ms/step - loss: 0.0796 - accuracy: 0.9901 - scaled_graph_loss: 0.0333
Epoch 99/100
1/17 [>.............................] - ETA: 0s - loss: 0.1032 - accuracy: 0.9844 - scaled_graph_loss: 0.0366
6/17 [=========>....................] - ETA: 0s - loss: 0.0962 - accuracy: 0.9812 - scaled_graph_loss: 0.0325
11/17 [==================>...........] - ETA: 0s - loss: 0.0909 - accuracy: 0.9829 - scaled_graph_loss: 0.0324
16/17 [===========================>..] - ETA: 0s - loss: 0.0880 - accuracy: 0.9843 - scaled_graph_loss: 0.0327
17/17 [==============================] - 0s 11ms/step - loss: 0.0871 - accuracy: 0.9847 - scaled_graph_loss: 0.0329
Epoch 100/100
1/17 [>.............................] - ETA: 0s - loss: 0.0772 - accuracy: 0.9922 - scaled_graph_loss: 0.0361
6/17 [=========>....................] - ETA: 0s - loss: 0.0768 - accuracy: 0.9877 - scaled_graph_loss: 0.0339
11/17 [==================>...........] - ETA: 0s - loss: 0.0772 - accuracy: 0.9869 - scaled_graph_loss: 0.0337
16/17 [===========================>..] - ETA: 0s - loss: 0.0782 - accuracy: 0.9862 - scaled_graph_loss: 0.0334
17/17 [==============================] - 0s 11ms/step - loss: 0.0787 - accuracy: 0.9859 - scaled_graph_loss: 0.0335
<tensorflow.python.keras.callbacks.History at 0x7f786083d518>
MLP モデルをグラフ正則化で評価する
eval_results = dict(
zip(graph_reg_model.metrics_names,
graph_reg_model.evaluate(test_dataset, steps=HPARAMS.eval_steps)))
print_metrics('MLP + graph regularization', eval_results)
1/Unknown - 0s 175ms/step - loss: 1.1986 - accuracy: 0.7891 5/5 [==============================] - 0s 8ms/step - loss: 1.2281 - accuracy: 0.8103 Eval accuracy for MLP + graph regularization : 0.8155515193939209 Eval loss for MLP + graph regularization : 1.2275753021240234
グラフ正則化モデルの精度は、基本モデル（base_model）の精度に比べて 2-3% 程度高くなります。
結論
Neural Structured Learning（NSL）フレームワークを用いて、自然な引用グラフ上の文書（Cora）分類のためのグラフ正則化の使用について実証しました。上級者向けチュートリアルでは、グラフ正則化を使用してニューラルネットワークをトレーニングする前に、サンプル埋め込みに基づいたグラフを合成します。このアプローチは、入力に明示的なグラフが含まれない場合に有用です。
ユーザーの方々には、グラフ正則化のさまざまなニューラルアーキテクチャを試してみると共に、監視の量を加減してさらに深く実験を行うことを推奨しています。