Automated hyper-parameter tuning

View on TensorFlow.org Run in Google Colab View on GitHub Download notebook

Welcome to the Automated hyper-parameter tuning tutorial. In this colab, you will learn how to improve your models using automated hyper-parameter tuning with TensorFlow Decision Forests.

Introduction

A learning algorithm trains a machine learning model on a training dataset. The parameters of a learning algorithm–called "hyper-parameters"–control how the model is trained and impact its quality. Therefore, finding the best hyper-parameters is an important stage of modeling.

Some hyper-parameters are simple to configure. For example, increasing the number of trees (num_trees) in a random forest increases the quality of the model until a plateau. Therefore, setting the largest value compatible with the serving constraints (more trees means a larger model) is a valid rule of thumb. However, other hyper-parameters have a more complex interaction with the model and cannot be chosen with such a simple rule. For example, increasing the maximum tree depth (max_depth) of a gradient boosted tree model can both increase or decrease the quality of the model. Furthermore, hyper-parameters can interact between each others, and the optimal value of a hyper-parameter cannot be found in isolation.

There are three main approaches to select the hyper-parameter values:

  1. The default approach: Learning algorithms come with default values. While not ideal in all cases, those values produce reasonable results in most situations. This approach is recommended as the first approach to use in any modeling. This page lists the default value of TF Decision Forests.

  2. The template hyper-parameter approach: In addition to the default values, TF Decision Forests also exposes the hyper-parameter templates. Those are benchmark-tuned hyper-parameter values with excellent performance but high training cost (e.g. hyperparameter_template="benchmark_rank1").

  3. The manual tuning approach: You can manually test different hyper-parameter values and select the one that performs best. The advanced users guide give some advice.

  4. The automated tuning approach: A tuning algorithm can be used to find automatically the best hyper-parameter values. This approach gives often the best results and does not require expertise. The main downside of this approach is the time it takes for large datasets.

In this colab, we shows the default and automated tuning approaches with the TensorFlow Decision Forests library.

Hyper-parameter tuning algorithms

Automated tuning algorithms work by generating and evaluating a large number of hyper-parameter values. Each of those iterations is called a "trial". The evaluation of a trial is expensive as it requires to train a new model each time. At the end of the tuning, the hyper-parameter with the best evaluation is used.

Tuning algorithm are configured as follow:

The search space

The search space is the list of hyper-parameters to optimize and the values they can take. For example, the maximum depth of a tree could be optimized for values in between 1 and 32. Exploring more hyper-parameters and more possible values often leads to better models but also takes more time. The hyper-parameters listed in the user manual are the most impactful ones to tune. The other hyper-parameters are listed in the documentation.

When the possible value of one hyper-parameter depends on the value of another hyper-parameter, the search space is said to be conditional.

The number of trials

The number of trials defines how many models will be trained and evaluated. Larger number of trials generally leads to better models, but takes more time.

The optimizer

The optimizer selects the next hyper-parameter to evaluate the past trial evaluations. The simplest and often reasonable optimizer is the one that selects the hyper-parameter at random.

The objective / trial score

The objective is the metric optimized by the tuner. Often, this metric is a measure of quality (e.g. accuracy, log loss) of the model evaluated on a validation dataset.

Train-valid-test

The validation dataset should be different from the training datasets: If the training and validation datasets are the same, the selected hyper-parameters will be irrelevant. The validation dataset should also be different from the testing dataset (also called holdout dataset): Because hyper-parameter tuning is a form of training, if the testing and validation datasets are the same, you are effectively training on the test dataset. In this case, you might overfit on your test dataset without a way to measure it.

Cross-validation

In the case of a small dataset, for example a dataset with less than 100k examples, hyper-parameter tuning can be coupled with cross-validation: Instead of being evaluated from a single training-test round, the objective/trial score is evaluated as the average of the metric over multiple cross-validation rounds.

Similarly as to the train-valid-and-test datasets, the cross-validation used to evaluate the objective/score during hyper-parameter tuning should be different from the cross-validation used to evaluate the quality of the model.

Out-of-bag evaluation

Some models, like Random Forests, can be evaluated on the training datasets using the "out-of-bag evaluation" method. While not as accurate as cross-validation, the "out-of-bag evaluation" is much faster than cross-validation and does not require a separate validation datasets.

In tensorflow decision forests

In TF-DF, the model "self" evaluation is always a fair way to evaluate a model. For example, an out-of-bag evaluation is used for Random Forest models while a validation dataset is used for Gradient Boosted models.

Hyper-parameter tuning with TF Decision Forests

TF-DF supports automatic hyper-parameter tuning with minimal configuration. In the next example, we will train and compare two models: One trained with default hyper-parameters, and one trained with hyper-parameter tuning.

Setup

# Install TensorFlow Dececision Forests
pip install tensorflow_decision_forests -U -qq

Install Wurlitzer. Wurlitzer is required to show the detailed training logs in colabs (with verbose=2).

pip install wurlitzer -U -qq

Import the necessary libraries.

import tensorflow_decision_forests as tfdf
import matplotlib.pyplot as plt
import pandas as pd
import tensorflow as tf
import numpy as np

The hidden code cell limits the output height in colab.

Define "set_cell_height".

Training a model without Automated hyper-parameter tuning

We will train a model on the Multiple Features Data Set multi-class classification dataset available on the UCI. Let's download the dataset.

!wget -q https://archive.ics.uci.edu/ml/machine-learning-databases/mfeat/mfeat-fou -O /tmp/mfeat-fou.csv
all_df = pd.read_csv("/tmp/mfeat-fou.csv", header=None, names=[f"coef_{i}" for i in range(76) ], sep="  ")
# The label values are not directly recorded in the dataset. Instead, the first 200 examples have class 0,
# the next 200 examples have class 1, and so on.
label="label"
all_df[label] = [i // 200 for i in range(2000)]
print(all_df)
coef_0    coef_1    coef_2    coef_3    coef_4    coef_5    coef_6  \
0     0.065882  0.197312  0.103826  0.270362  0.616078  0.035856  0.424572   
1     0.049142  0.175971  0.105515  0.227095  0.599280  0.041217  0.431078   
2     0.034172  0.227649  0.108766  0.127697  0.612494  0.056554  0.470639   
3     0.062336  0.217979  0.080243  0.289592  0.546316  0.045779  0.425545   
4     0.061970  0.198358  0.111239  0.253460  0.608455  0.023631  0.415246   
...        ...       ...       ...       ...       ...       ...       ...   
1995  0.171874  0.221525  0.426470  0.077220  0.264413  0.220274  0.452574   
1996  0.152025  0.347439  0.323534  0.087706  0.427077  0.313656  0.371549   
1997  0.335606  0.318426  0.257948  0.225381  0.417769  0.123067  0.333455   
1998  0.253910  0.171202  0.302622  0.181066  0.326383  0.167960  0.417409   
1999  0.271575  0.149037  0.232753  0.197720  0.236424  0.187395  0.467213   

        coef_7    coef_8    coef_9  ...   coef_67   coef_68   coef_69  \
0     0.089701  0.367773  0.037065  ...  0.113374  0.074343  0.281666   
1     0.096801  0.326739  0.059661  ...  0.069337  0.045386  0.254264   
2     0.041903  0.324267  0.044569  ...  0.218842  0.041087  0.360464   
3     0.022841  0.331454  0.119052  ...  0.085479  0.087658  0.151880   
4     0.091866  0.309310  0.049142  ...  0.106960  0.032283  0.248565   
...        ...       ...       ...  ...       ...       ...       ...   
1995  0.161113  0.356788  0.081889  ...  0.197440  0.197778  0.205866   
1996  0.262768  0.396422  0.124391  ...  0.161606  0.033621  0.173155   
1997  0.342044  0.398151  0.079550  ...  0.121659  0.316333  0.244846   
1998  0.399065  0.266153  0.097131  ...  0.168154  0.147167  0.268675   
1999  0.385117  0.191292  0.149162  ...  0.225634  0.061724  0.267944   

       coef_70   coef_71   coef_72   coef_73   coef_74   coef_75  label  
0     0.067795  0.344182  0.038963  0.394366  0.049971  0.344871      0  
1     0.045447  0.335659  0.021719  0.445277  0.083978  0.354092      0  
2     0.047154  0.377408  0.052099  0.445029  0.071234  0.261465      0  
3     0.075950  0.293462  0.022675  0.408291  0.063010  0.401376      0  
4     0.015674  0.386276  0.039481  0.434701  0.069218  0.405403      0  
...        ...       ...       ...       ...       ...       ...    ...  
1995  0.138887  0.160848  0.126190  0.247345  0.239117  0.136807      9  
1996  0.295227  0.259417  0.345691  0.068303  0.158352  0.282327      9  
1997  0.132893  0.042988  0.250740  0.141576  0.038776  0.114591      9  
1998  0.199125  0.128708  0.285220  0.230322  0.103664  0.130003      9  
1999  0.251182  0.170772  0.322067  0.275769  0.021074  0.085241      9  

[2000 rows x 77 columns]
/tmpfs/tmp/ipykernel_21727/163055519.py:2: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support regex separators (separators > 1 char and different from '\s+' are interpreted as regex); you can avoid this warning by specifying engine='python'.
  all_df = pd.read_csv("/tmp/mfeat-fou.csv", header=None, names=[f"coef_{i}" for i in range(76) ], sep="  ")

Split the dataset into a training and a testing dataset.

def split_dataset(dataset, test_ratio=0.30):
  """Splits a panda dataframe in two."""
  test_indices = np.random.rand(len(dataset)) < test_ratio
  return dataset[~test_indices], dataset[test_indices]

train_df, test_df = split_dataset(all_df)
print("{} examples in training, {} examples for testing.".format(len(train_df), len(test_df)))
label="label"
train_ds = tfdf.keras.pd_dataframe_to_tf_dataset(train_df, label=label)
test_ds = tfdf.keras.pd_dataframe_to_tf_dataset(test_df, label=label)
/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow_decision_forests/keras/core.py:2542: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only.
  features_dataframe = dataframe.drop(label, 1)
1420 examples in training, 580 examples for testing.
/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow_decision_forests/keras/core.py:2542: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only.
  features_dataframe = dataframe.drop(label, 1)

First, we train and evaluate the quality of a Gradient Boosted Trees model trained with the default hyper-parameters.

%%time
# Train a model with default hyper-parameters
model = tfdf.keras.GradientBoostedTreesModel()
model.fit(train_ds)
Warning: The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
WARNING:absl:The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
Use /tmpfs/tmp/tmpnfm0i97y as temporary training directory
Reading training dataset...
Training dataset read in 0:00:03.557221. Found 1420 examples.
Training model...
Model trained in 0:00:04.062735
Compiling model...
[INFO kernel.cc:1176] Loading model from path /tmpfs/tmp/tmpnfm0i97y/model/ with prefix bda9098251b04177
[INFO abstract_model.cc:1246] Engine "GradientBoostedTreesGeneric" built
[INFO kernel.cc:1022] Use fast generic engine
WARNING:tensorflow:AutoGraph could not transform <function simple_ml_inference_op_with_handle at 0x7fc75551b430> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING:tensorflow:AutoGraph could not transform <function simple_ml_inference_op_with_handle at 0x7fc75551b430> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING: AutoGraph could not transform <function simple_ml_inference_op_with_handle at 0x7fc75551b430> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
Model compiled.
CPU times: user 35.9 s, sys: 9.13 s, total: 45.1 s
Wall time: 9.17 s
<keras.callbacks.History at 0x7fc7539064f0>
# Evaluate the model
model.compile(["accuracy"])
test_accuracy = model.evaluate(test_ds, return_dict=True, verbose=0)["accuracy"]
print(f"Test accuracy without hyper-parameter tuning: {test_accuracy:.4f}")
Test accuracy without hyper-parameter tuning: 0.7966

The default hyper-parameters of the model are available with the learner_params function. The definition of those parameters is available in the documentation.

print("Default hyper-parameters of the model:\n", model.learner_params)
Default hyper-parameters of the model:
 {'adapt_subsample_for_maximum_training_duration': False, 'allow_na_conditions': False, 'apply_link_function': True, 'categorical_algorithm': 'CART', 'categorical_set_split_greedy_sampling': 0.1, 'categorical_set_split_max_num_items': -1, 'categorical_set_split_min_item_frequency': 1, 'compute_permutation_variable_importance': False, 'dart_dropout': 0.01, 'early_stopping': 'LOSS_INCREASE', 'early_stopping_num_trees_look_ahead': 30, 'focal_loss_alpha': 0.5, 'focal_loss_gamma': 2.0, 'forest_extraction': 'MART', 'goss_alpha': 0.2, 'goss_beta': 0.1, 'growing_strategy': 'LOCAL', 'honest': False, 'honest_fixed_separation': False, 'honest_ratio_leaf_examples': 0.5, 'in_split_min_examples_check': True, 'keep_non_leaf_label_distribution': True, 'l1_regularization': 0.0, 'l2_categorical_regularization': 1.0, 'l2_regularization': 0.0, 'lambda_loss': 1.0, 'loss': 'DEFAULT', 'max_depth': 6, 'max_num_nodes': None, 'maximum_model_size_in_memory_in_bytes': -1.0, 'maximum_training_duration_seconds': -1.0, 'min_examples': 5, 'missing_value_policy': 'GLOBAL_IMPUTATION', 'num_candidate_attributes': -1, 'num_candidate_attributes_ratio': -1.0, 'num_trees': 300, 'random_seed': 123456, 'sampling_method': 'NONE', 'selective_gradient_boosting_ratio': 0.01, 'shrinkage': 0.1, 'sorting_strategy': 'PRESORT', 'sparse_oblique_normalization': None, 'sparse_oblique_num_projections_exponent': None, 'sparse_oblique_projection_density_factor': None, 'sparse_oblique_weights': None, 'split_axis': 'AXIS_ALIGNED', 'subsample': 1.0, 'uplift_min_examples_in_treatment': 5, 'uplift_split_score': 'KULLBACK_LEIBLER', 'use_hessian_gain': False, 'validation_interval_in_trees': 1, 'validation_ratio': 0.1}

Training a model with automated hyper-parameter tuning

Hyper-paramter tuning is enabled by specifying the tuner constructor argument of the model. The tuner object contains all the configuration of the tuner (search space, optimizer, trial and objective).

# Configure the tuner.

# Create a Random Search tuner with 50 trials.
tuner = tfdf.tuner.RandomSearch(num_trials=50)

# Define the search space.
#
# Adding more parameters generaly improve the quality of the model, but make
# the tuning last longer.

tuner.choice("min_examples", [2, 5, 7, 10])
tuner.choice("categorical_algorithm", ["CART", "RANDOM"])

# Some hyper-parameters are only valid for specific values of other
# hyper-parameters. For example, the "max_depth" parameter is mostly useful when
# "growing_strategy=LOCAL" while "max_num_nodes" is better suited when
# "growing_strategy=BEST_FIRST_GLOBAL".

local_search_space = tuner.choice("growing_strategy", ["LOCAL"])
local_search_space.choice("max_depth", [3, 4, 5, 6, 8])

# merge=True indicates that the parameter (here "growing_strategy") is already
# defined, and that new values are added to it.
global_search_space = tuner.choice("growing_strategy", ["BEST_FIRST_GLOBAL"], merge=True)
global_search_space.choice("max_num_nodes", [16, 32, 64, 128, 256])

tuner.choice("use_hessian_gain", [True, False])
tuner.choice("shrinkage", [0.02, 0.05, 0.10, 0.15])
tuner.choice("num_candidate_attributes_ratio", [0.2, 0.5, 0.9, 1.0])

# Uncomment some (or all) of the following hyper-parameters to increase the
# quality of the search. The number of trial should be increased accordingly.

# tuner.choice("split_axis", ["AXIS_ALIGNED"])
# oblique_space = tuner.choice("split_axis", ["SPARSE_OBLIQUE"], merge=True)
# oblique_space.choice("sparse_oblique_normalization",
#                      ["NONE", "STANDARD_DEVIATION", "MIN_MAX"])
# oblique_space.choice("sparse_oblique_weights", ["BINARY", "CONTINUOUS"])
# oblique_space.choice("sparse_oblique_num_projections_exponent", [1.0, 1.5])
<tensorflow_decision_forests.component.tuner.tuner.SearchSpace at 0x7fc7a2995c70>
%%time
%set_cell_height 300

# Tune the model. Notice the `tuner=tuner`.
tuned_model = tfdf.keras.GradientBoostedTreesModel(tuner=tuner)
tuned_model.fit(train_ds, verbose=2)

# The `num_threads` model constructor argument (not specified in the example
# above) controls how many trials are run in parallel (one per thread). If
# `num_threads` is not specified (like in the example above), one thread is
# allocated for each available CPU core.
#
# If the training is interrupted (for example, by pressing on the "stop" button
# on the top-left of the colab cell), the best model so-far will be returned.

# In the training logs, you can see lines such as `[10/50] Score: -0.45 / -0.40
# HParams: ...`. This indicates that 10 of the 50 trials have been completed.
# And that the last trial returned a score of "-0.45" and that the best trial so
# far has a score of "-0.40". In this example, the model is optimized by
# logloss. Since scores are maximized and log loss should be minimized, the
# score is effectively minus the log loss.
<IPython.core.display.Javascript object>
Warning: The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
WARNING:absl:The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
Use /tmpfs/tmp/tmp89tz8qw2 as temporary training directory
Reading training dataset...
Training tensor examples:
Features: {'coef_0': <tf.Tensor 'data:0' shape=(None,) dtype=float64>, 'coef_1': <tf.Tensor 'data_1:0' shape=(None,) dtype=float64>, 'coef_2': <tf.Tensor 'data_12:0' shape=(None,) dtype=float64>, 'coef_3': <tf.Tensor 'data_23:0' shape=(None,) dtype=float64>, 'coef_4': <tf.Tensor 'data_34:0' shape=(None,) dtype=float64>, 'coef_5': <tf.Tensor 'data_45:0' shape=(None,) dtype=float64>, 'coef_6': <tf.Tensor 'data_56:0' shape=(None,) dtype=float64>, 'coef_7': <tf.Tensor 'data_67:0' shape=(None,) dtype=float64>, 'coef_8': <tf.Tensor 'data_74:0' shape=(None,) dtype=float64>, 'coef_9': <tf.Tensor 'data_75:0' shape=(None,) dtype=float64>, 'coef_10': <tf.Tensor 'data_2:0' shape=(None,) dtype=float64>, 'coef_11': <tf.Tensor 'data_3:0' shape=(None,) dtype=float64>, 'coef_12': <tf.Tensor 'data_4:0' shape=(None,) dtype=float64>, 'coef_13': <tf.Tensor 'data_5:0' shape=(None,) dtype=float64>, 'coef_14': <tf.Tensor 'data_6:0' shape=(None,) dtype=float64>, 'coef_15': <tf.Tensor 'data_7:0' shape=(None,) dtype=float64>, 'coef_16': <tf.Tensor 'data_8:0' shape=(None,) dtype=float64>, 'coef_17': <tf.Tensor 'data_9:0' shape=(None,) dtype=float64>, 'coef_18': <tf.Tensor 'data_10:0' shape=(None,) dtype=float64>, 'coef_19': <tf.Tensor 'data_11:0' shape=(None,) dtype=float64>, 'coef_20': <tf.Tensor 'data_13:0' shape=(None,) dtype=float64>, 'coef_21': <tf.Tensor 'data_14:0' shape=(None,) dtype=float64>, 'coef_22': <tf.Tensor 'data_15:0' shape=(None,) dtype=float64>, 'coef_23': <tf.Tensor 'data_16:0' shape=(None,) dtype=float64>, 'coef_24': <tf.Tensor 'data_17:0' shape=(None,) dtype=float64>, 'coef_25': <tf.Tensor 'data_18:0' shape=(None,) dtype=float64>, 'coef_26': <tf.Tensor 'data_19:0' shape=(None,) dtype=float64>, 'coef_27': <tf.Tensor 'data_20:0' shape=(None,) dtype=float64>, 'coef_28': <tf.Tensor 'data_21:0' shape=(None,) dtype=float64>, 'coef_29': <tf.Tensor 'data_22:0' shape=(None,) dtype=float64>, 'coef_30': <tf.Tensor 'data_24:0' shape=(None,) dtype=float64>, 'coef_31': <tf.Tensor 'data_25:0' shape=(None,) dtype=float64>, 'coef_32': <tf.Tensor 'data_26:0' shape=(None,) dtype=float64>, 'coef_33': <tf.Tensor 'data_27:0' shape=(None,) dtype=float64>, 'coef_34': <tf.Tensor 'data_28:0' shape=(None,) dtype=float64>, 'coef_35': <tf.Tensor 'data_29:0' shape=(None,) dtype=float64>, 'coef_36': <tf.Tensor 'data_30:0' shape=(None,) dtype=float64>, 'coef_37': <tf.Tensor 'data_31:0' shape=(None,) dtype=float64>, 'coef_38': <tf.Tensor 'data_32:0' shape=(None,) dtype=float64>, 'coef_39': <tf.Tensor 'data_33:0' shape=(None,) dtype=float64>, 'coef_40': <tf.Tensor 'data_35:0' shape=(None,) dtype=float64>, 'coef_41': <tf.Tensor 'data_36:0' shape=(None,) dtype=float64>, 'coef_42': <tf.Tensor 'data_37:0' shape=(None,) dtype=float64>, 'coef_43': <tf.Tensor 'data_38:0' shape=(None,) dtype=float64>, 'coef_44': <tf.Tensor 'data_39:0' shape=(None,) dtype=float64>, 'coef_45': <tf.Tensor 'data_40:0' shape=(None,) dtype=float64>, 'coef_46': <tf.Tensor 'data_41:0' shape=(None,) dtype=float64>, 'coef_47': <tf.Tensor 'data_42:0' shape=(None,) dtype=float64>, 'coef_48': <tf.Tensor 'data_43:0' shape=(None,) dtype=float64>, 'coef_49': <tf.Tensor 'data_44:0' shape=(None,) dtype=float64>, 'coef_50': <tf.Tensor 'data_46:0' shape=(None,) dtype=float64>, 'coef_51': <tf.Tensor 'data_47:0' shape=(None,) dtype=float64>, 'coef_52': <tf.Tensor 'data_48:0' shape=(None,) dtype=float64>, 'coef_53': <tf.Tensor 'data_49:0' shape=(None,) dtype=float64>, 'coef_54': <tf.Tensor 'data_50:0' shape=(None,) dtype=float64>, 'coef_55': <tf.Tensor 'data_51:0' shape=(None,) dtype=float64>, 'coef_56': <tf.Tensor 'data_52:0' shape=(None,) dtype=float64>, 'coef_57': <tf.Tensor 'data_53:0' shape=(None,) dtype=float64>, 'coef_58': <tf.Tensor 'data_54:0' shape=(None,) dtype=float64>, 'coef_59': <tf.Tensor 'data_55:0' shape=(None,) dtype=float64>, 'coef_60': <tf.Tensor 'data_57:0' shape=(None,) dtype=float64>, 'coef_61': <tf.Tensor 'data_58:0' shape=(None,) dtype=float64>, 'coef_62': <tf.Tensor 'data_59:0' shape=(None,) dtype=float64>, 'coef_63': <tf.Tensor 'data_60:0' shape=(None,) dtype=float64>, 'coef_64': <tf.Tensor 'data_61:0' shape=(None,) dtype=float64>, 'coef_65': <tf.Tensor 'data_62:0' shape=(None,) dtype=float64>, 'coef_66': <tf.Tensor 'data_63:0' shape=(None,) dtype=float64>, 'coef_67': <tf.Tensor 'data_64:0' shape=(None,) dtype=float64>, 'coef_68': <tf.Tensor 'data_65:0' shape=(None,) dtype=float64>, 'coef_69': <tf.Tensor 'data_66:0' shape=(None,) dtype=float64>, 'coef_70': <tf.Tensor 'data_68:0' shape=(None,) dtype=float64>, 'coef_71': <tf.Tensor 'data_69:0' shape=(None,) dtype=float64>, 'coef_72': <tf.Tensor 'data_70:0' shape=(None,) dtype=float64>, 'coef_73': <tf.Tensor 'data_71:0' shape=(None,) dtype=float64>, 'coef_74': <tf.Tensor 'data_72:0' shape=(None,) dtype=float64>, 'coef_75': <tf.Tensor 'data_73:0' shape=(None,) dtype=float64>}
Label: Tensor("data_76:0", shape=(None,), dtype=int64)
Weights: None
Normalized tensor features:
 {'coef_0': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast:0' shape=(None,) dtype=float32>), 'coef_1': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_1:0' shape=(None,) dtype=float32>), 'coef_2': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_2:0' shape=(None,) dtype=float32>), 'coef_3': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_3:0' shape=(None,) dtype=float32>), 'coef_4': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_4:0' shape=(None,) dtype=float32>), 'coef_5': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_5:0' shape=(None,) dtype=float32>), 'coef_6': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_6:0' shape=(None,) dtype=float32>), 'coef_7': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_7:0' shape=(None,) dtype=float32>), 'coef_8': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_8:0' shape=(None,) dtype=float32>), 'coef_9': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_9:0' shape=(None,) dtype=float32>), 'coef_10': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_10:0' shape=(None,) dtype=float32>), 'coef_11': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_11:0' shape=(None,) dtype=float32>), 'coef_12': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_12:0' shape=(None,) dtype=float32>), 'coef_13': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_13:0' shape=(None,) dtype=float32>), 'coef_14': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_14:0' shape=(None,) dtype=float32>), 'coef_15': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_15:0' shape=(None,) dtype=float32>), 'coef_16': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_16:0' shape=(None,) dtype=float32>), 'coef_17': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_17:0' shape=(None,) dtype=float32>), 'coef_18': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_18:0' shape=(None,) dtype=float32>), 'coef_19': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_19:0' shape=(None,) dtype=float32>), 'coef_20': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_20:0' shape=(None,) dtype=float32>), 'coef_21': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_21:0' shape=(None,) dtype=float32>), 'coef_22': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_22:0' shape=(None,) dtype=float32>), 'coef_23': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_23:0' shape=(None,) dtype=float32>), 'coef_24': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_24:0' shape=(None,) dtype=float32>), 'coef_25': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_25:0' shape=(None,) dtype=float32>), 'coef_26': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_26:0' shape=(None,) dtype=float32>), 'coef_27': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_27:0' shape=(None,) dtype=float32>), 'coef_28': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_28:0' shape=(None,) dtype=float32>), 'coef_29': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_29:0' shape=(None,) dtype=float32>), 'coef_30': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_30:0' shape=(None,) dtype=float32>), 'coef_31': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_31:0' shape=(None,) dtype=float32>), 'coef_32': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_32:0' shape=(None,) dtype=float32>), 'coef_33': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_33:0' shape=(None,) dtype=float32>), 'coef_34': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_34:0' shape=(None,) dtype=float32>), 'coef_35': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_35:0' shape=(None,) dtype=float32>), 'coef_36': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_36:0' shape=(None,) dtype=float32>), 'coef_37': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_37:0' shape=(None,) dtype=float32>), 'coef_38': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_38:0' shape=(None,) dtype=float32>), 'coef_39': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_39:0' shape=(None,) dtype=float32>), 'coef_40': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_40:0' shape=(None,) dtype=float32>), 'coef_41': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_41:0' shape=(None,) dtype=float32>), 'coef_42': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_42:0' shape=(None,) dtype=float32>), 'coef_43': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_43:0' shape=(None,) dtype=float32>), 'coef_44': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_44:0' shape=(None,) dtype=float32>), 'coef_45': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_45:0' shape=(None,) dtype=float32>), 'coef_46': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_46:0' shape=(None,) dtype=float32>), 'coef_47': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_47:0' shape=(None,) dtype=float32>), 'coef_48': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_48:0' shape=(None,) dtype=float32>), 'coef_49': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_49:0' shape=(None,) dtype=float32>), 'coef_50': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_50:0' shape=(None,) dtype=float32>), 'coef_51': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_51:0' shape=(None,) dtype=float32>), 'coef_52': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_52:0' shape=(None,) dtype=float32>), 'coef_53': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_53:0' shape=(None,) dtype=float32>), 'coef_54': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_54:0' shape=(None,) dtype=float32>), 'coef_55': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_55:0' shape=(None,) dtype=float32>), 'coef_56': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_56:0' shape=(None,) dtype=float32>), 'coef_57': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_57:0' shape=(None,) dtype=float32>), 'coef_58': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_58:0' shape=(None,) dtype=float32>), 'coef_59': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_59:0' shape=(None,) dtype=float32>), 'coef_60': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_60:0' shape=(None,) dtype=float32>), 'coef_61': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_61:0' shape=(None,) dtype=float32>), 'coef_62': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_62:0' shape=(None,) dtype=float32>), 'coef_63': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_63:0' shape=(None,) dtype=float32>), 'coef_64': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_64:0' shape=(None,) dtype=float32>), 'coef_65': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_65:0' shape=(None,) dtype=float32>), 'coef_66': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_66:0' shape=(None,) dtype=float32>), 'coef_67': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_67:0' shape=(None,) dtype=float32>), 'coef_68': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_68:0' shape=(None,) dtype=float32>), 'coef_69': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_69:0' shape=(None,) dtype=float32>), 'coef_70': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_70:0' shape=(None,) dtype=float32>), 'coef_71': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_71:0' shape=(None,) dtype=float32>), 'coef_72': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_72:0' shape=(None,) dtype=float32>), 'coef_73': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_73:0' shape=(None,) dtype=float32>), 'coef_74': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_74:0' shape=(None,) dtype=float32>), 'coef_75': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_75:0' shape=(None,) dtype=float32>)}
Training dataset read in 0:00:00.798424. Found 1420 examples.
Training model...
Standard output detected as not visible to the user e.g. running in a notebook. Creating a training log redirection. If training get stuck, try calling tfdf.keras.set_training_logs_redirection(False).
[INFO kernel.cc:809] Start Yggdrasil model training
[INFO kernel.cc:810] Collect training examples
[INFO kernel.cc:418] Number of batches: 2
[INFO kernel.cc:419] Number of examples: 1420
[INFO kernel.cc:832] Training dataset:
Number of records: 1420
Number of columns: 77

Number of columns by type:
    NUMERICAL: 76 (98.7013%)
    CATEGORICAL: 1 (1.2987%)

Columns:

NUMERICAL: 76 (98.7013%)
    0: "coef_0" NUMERICAL mean:0.185193 min:0.00492771 max:0.383341 sd:0.0928829
    1: "coef_1" NUMERICAL mean:0.374057 min:0.0090002 max:0.771281 sd:0.173715
    2: "coef_10" NUMERICAL mean:0.15629 min:0.00376415 max:0.40926 sd:0.0867228
    3: "coef_11" NUMERICAL mean:0.165407 min:0.00274976 max:0.506974 sd:0.0833784
    4: "coef_12" NUMERICAL mean:0.165901 min:0.00142364 max:0.471881 sd:0.0933784
    5: "coef_13" NUMERICAL mean:0.193121 min:0.00590215 max:0.635803 sd:0.109577
    6: "coef_14" NUMERICAL mean:0.143136 min:0.00647194 max:0.382988 sd:0.064732
    7: "coef_15" NUMERICAL mean:0.114066 min:0.00290088 max:0.366277 sd:0.0601661
    8: "coef_16" NUMERICAL mean:0.127982 min:0.00287311 max:0.462718 sd:0.0664961
    9: "coef_17" NUMERICAL mean:0.129778 min:0.00304677 max:0.396381 sd:0.0781749
    10: "coef_18" NUMERICAL mean:0.120858 min:0.00486002 max:0.407231 sd:0.0656658
    11: "coef_19" NUMERICAL mean:0.116757 min:0.00181387 max:0.345249 sd:0.0539648
    12: "coef_2" NUMERICAL mean:0.269064 min:0.0079055 max:0.665372 sd:0.137599
    13: "coef_20" NUMERICAL mean:0.108321 min:0.00243648 max:0.315663 sd:0.0530303
    14: "coef_21" NUMERICAL mean:0.108301 min:0.00261734 max:0.337783 sd:0.051511
    15: "coef_22" NUMERICAL mean:0.0986228 min:0.00263824 max:0.332622 sd:0.0585544
    16: "coef_23" NUMERICAL mean:0.115921 min:0.00418841 max:0.451662 sd:0.0652091
    17: "coef_24" NUMERICAL mean:0.106315 min:0.00182609 max:0.280131 sd:0.0473684
    18: "coef_25" NUMERICAL mean:0.0985301 min:0.00407151 max:0.312352 sd:0.0497109
    19: "coef_26" NUMERICAL mean:0.0955449 min:0.00480036 max:0.320455 sd:0.0474757
    20: "coef_27" NUMERICAL mean:0.090385 min:0.00293579 max:0.32458 sd:0.0500048
    21: "coef_28" NUMERICAL mean:0.0930036 min:0.00239058 max:0.312097 sd:0.0520835
    22: "coef_29" NUMERICAL mean:0.0943252 min:0.0044099 max:0.247468 sd:0.0445233
    23: "coef_3" NUMERICAL mean:0.263306 min:0.0101143 max:0.748012 sd:0.105796
    24: "coef_30" NUMERICAL mean:0.0931969 min:0.00282674 max:0.293574 sd:0.0457799
    25: "coef_31" NUMERICAL mean:0.088141 min:0.00174888 max:0.277819 sd:0.0430336
    26: "coef_32" NUMERICAL mean:0.0827424 min:0.00025893 max:0.292676 sd:0.0468864
    27: "coef_33" NUMERICAL mean:0.0829955 min:0.00256918 max:0.314669 sd:0.0438914
    28: "coef_34" NUMERICAL mean:0.0903238 min:0.00396516 max:0.278892 sd:0.0418604
    29: "coef_35" NUMERICAL mean:0.0864804 min:0.00158334 max:0.253353 sd:0.0426071
    30: "coef_36" NUMERICAL mean:0.0820904 min:0.00228121 max:0.257029 sd:0.041108
    31: "coef_37" NUMERICAL mean:0.0777956 min:0.00046895 max:0.291041 sd:0.0418326
    32: "coef_38" NUMERICAL mean:0.077968 min:0.00171066 max:0.251208 sd:0.040831
    33: "coef_39" NUMERICAL mean:0.0856294 min:0.00268666 max:0.247084 sd:0.0407453
    34: "coef_4" NUMERICAL mean:0.303393 min:0.00209035 max:0.721911 sd:0.15479
    35: "coef_40" NUMERICAL mean:0.0801986 min:0.00087734 max:0.243434 sd:0.0400049
    36: "coef_41" NUMERICAL mean:0.0773931 min:0.00206583 max:0.229024 sd:0.0380087
    37: "coef_42" NUMERICAL mean:0.0736305 min:0.00136188 max:0.23306 sd:0.0382668
    38: "coef_43" NUMERICAL mean:0.0712032 min:0.00113918 max:0.251665 sd:0.0383177
    39: "coef_44" NUMERICAL mean:0.083717 min:0.00308002 max:0.230327 sd:0.0409021
    40: "coef_45" NUMERICAL mean:0.0784619 min:0.00232565 max:0.225519 sd:0.0371967
    41: "coef_46" NUMERICAL mean:0.0745289 min:0.00223798 max:0.225919 sd:0.0394697
    42: "coef_47" NUMERICAL mean:0.0735969 min:0.00375832 max:0.234122 sd:0.0378476
    43: "coef_48" NUMERICAL mean:0.0858847 min:0.0029998 max:0.266345 sd:0.042262
    44: "coef_49" NUMERICAL mean:0.0834963 min:0.00115433 max:0.242921 sd:0.0411049
    45: "coef_5" NUMERICAL mean:0.158677 min:0.00267619 max:0.479913 sd:0.089747
    46: "coef_50" NUMERICAL mean:0.0806624 min:0.00280476 max:0.247757 sd:0.0425177
    47: "coef_51" NUMERICAL mean:0.0798156 min:0.00227356 max:0.235941 sd:0.0416608
    48: "coef_52" NUMERICAL mean:0.0948293 min:0.00345921 max:0.250377 sd:0.0453798
    49: "coef_53" NUMERICAL mean:0.0918011 min:0.00157472 max:0.257 sd:0.0439351
    50: "coef_54" NUMERICAL mean:0.0872454 min:0.00270626 max:0.256175 sd:0.0483413
    51: "coef_55" NUMERICAL mean:0.0850743 min:0.00156102 max:0.241507 sd:0.042891
    52: "coef_56" NUMERICAL mean:0.100832 min:0.00166723 max:0.277346 sd:0.0504132
    53: "coef_57" NUMERICAL mean:0.101564 min:0.00288512 max:0.284183 sd:0.0495377
    54: "coef_58" NUMERICAL mean:0.0906659 min:0.00173879 max:0.26817 sd:0.048872
    55: "coef_59" NUMERICAL mean:0.0947948 min:0.00237684 max:0.284695 sd:0.0531274
    56: "coef_6" NUMERICAL mean:0.367434 min:0.0119186 max:0.750938 sd:0.144447
    57: "coef_60" NUMERICAL mean:0.115679 min:0.00538156 max:0.300376 sd:0.0600483
    58: "coef_61" NUMERICAL mean:0.109685 min:0.00160526 max:0.319926 sd:0.0542184
    59: "coef_62" NUMERICAL mean:0.105632 min:0.00287046 max:0.31959 sd:0.0605518
    60: "coef_63" NUMERICAL mean:0.109943 min:0.00266793 max:0.351732 sd:0.0563731
    61: "coef_64" NUMERICAL mean:0.119207 min:0.00188382 max:0.427654 sd:0.0637886
    62: "coef_65" NUMERICAL mean:0.13092 min:0.00377491 max:0.371757 sd:0.0666428
    63: "coef_66" NUMERICAL mean:0.12369 min:0.00523224 max:0.394695 sd:0.0684848
    64: "coef_67" NUMERICAL mean:0.127944 min:0.00365267 max:0.418783 sd:0.0637265
    65: "coef_68" NUMERICAL mean:0.158167 min:0.00242352 max:0.47947 sd:0.0911508
    66: "coef_69" NUMERICAL mean:0.18051 min:0.0070905 max:0.490737 sd:0.0850876
    67: "coef_7" NUMERICAL mean:0.247664 min:0.00173686 max:0.52853 sd:0.123301
    68: "coef_70" NUMERICAL mean:0.148424 min:0.00377982 max:0.456282 sd:0.0866818
    69: "coef_71" NUMERICAL mean:0.165291 min:0.00778369 max:0.457965 sd:0.0910933
    70: "coef_72" NUMERICAL mean:0.231008 min:0.00397911 max:0.60137 sd:0.12293
    71: "coef_73" NUMERICAL mean:0.266279 min:0.0114335 max:0.620857 sd:0.13088
    72: "coef_74" NUMERICAL mean:0.144816 min:0.00113625 max:0.510432 sd:0.0856124
    73: "coef_75" NUMERICAL mean:0.216958 min:0.00457179 max:0.562397 sd:0.123788
    74: "coef_8" NUMERICAL mean:0.29034 min:0.010604 max:0.702383 sd:0.11784
    75: "coef_9" NUMERICAL mean:0.154272 min:0.00234154 max:0.584942 sd:0.0894681

CATEGORICAL: 1 (1.2987%)
    76: "__LABEL" CATEGORICAL integerized vocab-size:11 no-ood-item

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO kernel.cc:878] Configure learner
[WARNING gradient_boosted_trees.cc:1663] Subsample hyperparameter given but sampling method does not match.
[WARNING gradient_boosted_trees.cc:1676] GOSS alpha hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1685] GOSS beta hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1697] SelGB ratio hyperparameter given but SelGB is disabled.
[INFO kernel.cc:908] Training config:
learner: "HYPERPARAMETER_OPTIMIZER"
features: "coef_0"
features: "coef_1"
features: "coef_10"
features: "coef_11"
features: "coef_12"
features: "coef_13"
features: "coef_14"
features: "coef_15"
features: "coef_16"
features: "coef_17"
features: "coef_18"
features: "coef_19"
features: "coef_2"
features: "coef_20"
features: "coef_21"
features: "coef_22"
features: "coef_23"
features: "coef_24"
features: "coef_25"
features: "coef_26"
features: "coef_27"
features: "coef_28"
features: "coef_29"
features: "coef_3"
features: "coef_30"
features: "coef_31"
features: "coef_32"
features: "coef_33"
features: "coef_34"
features: "coef_35"
features: "coef_36"
features: "coef_37"
features: "coef_38"
features: "coef_39"
features: "coef_4"
features: "coef_40"
features: "coef_41"
features: "coef_42"
features: "coef_43"
features: "coef_44"
features: "coef_45"
features: "coef_46"
features: "coef_47"
features: "coef_48"
features: "coef_49"
features: "coef_5"
features: "coef_50"
features: "coef_51"
features: "coef_52"
features: "coef_53"
features: "coef_54"
features: "coef_55"
features: "coef_56"
features: "coef_57"
features: "coef_58"
features: "coef_59"
features: "coef_6"
features: "coef_60"
features: "coef_61"
features: "coef_62"
features: "coef_63"
features: "coef_64"
features: "coef_65"
features: "coef_66"
features: "coef_67"
features: "coef_68"
features: "coef_69"
features: "coef_7"
features: "coef_70"
features: "coef_71"
features: "coef_72"
features: "coef_73"
features: "coef_74"
features: "coef_75"
features: "coef_8"
features: "coef_9"
label: "__LABEL"
task: CLASSIFICATION
metadata {
  framework: "TF Keras"
}
[yggdrasil_decision_forests.model.hyperparameters_optimizer_v2.proto.hyperparameters_optimizer_config] {
  base_learner {
    learner: "GRADIENT_BOOSTED_TREES"
    features: "coef_0"
    features: "coef_1"
    features: "coef_10"
    features: "coef_11"
    features: "coef_12"
    features: "coef_13"
    features: "coef_14"
    features: "coef_15"
    features: "coef_16"
    features: "coef_17"
    features: "coef_18"
    features: "coef_19"
    features: "coef_2"
    features: "coef_20"
    features: "coef_21"
    features: "coef_22"
    features: "coef_23"
    features: "coef_24"
    features: "coef_25"
    features: "coef_26"
    features: "coef_27"
    features: "coef_28"
    features: "coef_29"
    features: "coef_3"
    features: "coef_30"
    features: "coef_31"
    features: "coef_32"
    features: "coef_33"
    features: "coef_34"
    features: "coef_35"
    features: "coef_36"
    features: "coef_37"
    features: "coef_38"
    features: "coef_39"
    features: "coef_4"
    features: "coef_40"
    features: "coef_41"
    features: "coef_42"
    features: "coef_43"
    features: "coef_44"
    features: "coef_45"
    features: "coef_46"
    features: "coef_47"
    features: "coef_48"
    features: "coef_49"
    features: "coef_5"
    features: "coef_50"
    features: "coef_51"
    features: "coef_52"
    features: "coef_53"
    features: "coef_54"
    features: "coef_55"
    features: "coef_56"
    features: "coef_57"
    features: "coef_58"
    features: "coef_59"
    features: "coef_6"
    features: "coef_60"
    features: "coef_61"
    features: "coef_62"
    features: "coef_63"
    features: "coef_64"
    features: "coef_65"
    features: "coef_66"
    features: "coef_67"
    features: "coef_68"
    features: "coef_69"
    features: "coef_7"
    features: "coef_70"
    features: "coef_71"
    features: "coef_72"
    features: "coef_73"
    features: "coef_74"
    features: "coef_75"
    features: "coef_8"
    features: "coef_9"
    label: "__LABEL"
    task: CLASSIFICATION
    random_seed: 123456
    [yggdrasil_decision_forests.model.gradient_boosted_trees.proto.gradient_boosted_trees_config] {
      num_trees: 300
      decision_tree {
        max_depth: 6
        min_examples: 5
        in_split_min_examples_check: true
        keep_non_leaf_label_distribution: true
        num_candidate_attributes: -1
        missing_value_policy: GLOBAL_IMPUTATION
        allow_na_conditions: false
        categorical_set_greedy_forward {
          sampling: 0.1
          max_num_items: -1
          min_item_frequency: 1
        }
        growing_strategy_local {
        }
        categorical {
          cart {
          }
        }
        axis_aligned_split {
        }
        internal {
          sorting_strategy: PRESORTED
        }
        uplift {
          min_examples_in_treatment: 5
          split_score: KULLBACK_LEIBLER
        }
      }
      shrinkage: 0.1
      loss: DEFAULT
      validation_set_ratio: 0.1
      validation_interval_in_trees: 1
      early_stopping: VALIDATION_LOSS_INCREASE
      early_stopping_num_trees_look_ahead: 30
      l2_regularization: 0
      lambda_loss: 1
      mart {
      }
      adapt_subsample_for_maximum_training_duration: false
      l1_regularization: 0
      use_hessian_gain: false
      l2_regularization_categorical: 1
      apply_link_function: true
      compute_permutation_variable_importance: false
      binary_focal_loss_options {
        misprediction_exponent: 2
        positive_sample_coefficient: 0.5
      }
    }
  }
  optimizer {
    optimizer_key: "RANDOM"
    [yggdrasil_decision_forests.model.hyperparameters_optimizer_v2.proto.random] {
      num_trials: 50
    }
  }
  search_space {
    fields {
      name: "min_examples"
      discrete_candidates {
        possible_values {
          integer: 2
        }
        possible_values {
          integer: 5
        }
        possible_values {
          integer: 7
        }
        possible_values {
          integer: 10
        }
      }
    }
    fields {
      name: "categorical_algorithm"
      discrete_candidates {
        possible_values {
          categorical: "CART"
        }
        possible_values {
          categorical: "RANDOM"
        }
      }
    }
    fields {
      name: "growing_strategy"
      discrete_candidates {
        possible_values {
          categorical: "LOCAL"
        }
        possible_values {
          categorical: "BEST_FIRST_GLOBAL"
        }
      }
      children {
        name: "max_depth"
        discrete_candidates {
          possible_values {
            integer: 3
          }
          possible_values {
            integer: 4
          }
          possible_values {
            integer: 5
          }
          possible_values {
            integer: 6
          }
          possible_values {
            integer: 8
          }
        }
        parent_discrete_values {
          possible_values {
            categorical: "LOCAL"
          }
        }
      }
      children {
        name: "max_num_nodes"
        discrete_candidates {
          possible_values {
            integer: 16
          }
          possible_values {
            integer: 32
          }
          possible_values {
            integer: 64
          }
          possible_values {
            integer: 128
          }
          possible_values {
            integer: 256
          }
        }
        parent_discrete_values {
          possible_values {
            categorical: "BEST_FIRST_GLOBAL"
          }
        }
      }
    }
    fields {
      name: "use_hessian_gain"
      discrete_candidates {
        possible_values {
          categorical: "true"
        }
        possible_values {
          categorical: "false"
        }
      }
    }
    fields {
      name: "shrinkage"
      discrete_candidates {
        possible_values {
          real: 0.02
        }
        possible_values {
          real: 0.05
        }
        possible_values {
          real: 0.1
        }
        possible_values {
          real: 0.15
        }
      }
    }
    fields {
      name: "num_candidate_attributes_ratio"
      discrete_candidates {
        possible_values {
          real: 0.2
        }
        possible_values {
          real: 0.5
        }
        possible_values {
          real: 0.9
        }
        possible_values {
          real: 1
        }
      }
    }
  }
}

[INFO kernel.cc:911] Deployment config:
cache_path: "/tmpfs/tmp/tmp89tz8qw2/working_cache"
num_threads: 32
try_resume_training: true

[INFO kernel.cc:940] Train model
[INFO hyperparameters_optimizer.cc:207] Hyperparameter search space:
fields {
  name: "min_examples"
  discrete_candidates {
    possible_values {
      integer: 2
    }
    possible_values {
      integer: 5
    }
    possible_values {
      integer: 7
    }
    possible_values {
      integer: 10
    }
  }
}
fields {
  name: "categorical_algorithm"
  discrete_candidates {
    possible_values {
      categorical: "CART"
    }
    possible_values {
      categorical: "RANDOM"
    }
  }
}
fields {
  name: "growing_strategy"
  discrete_candidates {
    possible_values {
      categorical: "LOCAL"
    }
    possible_values {
      categorical: "BEST_FIRST_GLOBAL"
    }
  }
  children {
    name: "max_depth"
    discrete_candidates {
      possible_values {
        integer: 3
      }
      possible_values {
        integer: 4
      }
      possible_values {
        integer: 5
      }
      possible_values {
        integer: 6
      }
      possible_values {
        integer: 8
      }
    }
    parent_discrete_values {
      possible_values {
        categorical: "LOCAL"
      }
    }
  }
  children {
    name: "max_num_nodes"
    discrete_candidates {
      possible_values {
        integer: 16
      }
      possible_values {
        integer: 32
      }
      possible_values {
        integer: 64
      }
      possible_values {
        integer: 128
      }
      possible_values {
        integer: 256
      }
    }
    parent_discrete_values {
      possible_values {
        categorical: "BEST_FIRST_GLOBAL"
      }
    }
  }
}
fields {
  name: "use_hessian_gain"
  discrete_candidates {
    possible_values {
      categorical: "true"
    }
    possible_values {
      categorical: "false"
    }
  }
}
fields {
  name: "shrinkage"
  discrete_candidates {
    possible_values {
      real: 0.02
    }
    possible_values {
      real: 0.05
    }
    possible_values {
      real: 0.1
    }
    possible_values {
      real: 0.15
    }
  }
}
fields {
  name: "num_candidate_attributes_ratio"
  discrete_candidates {
    possible_values {
      real: 0.2
    }
    possible_values {
      real: 0.5
    }
    possible_values {
      real: 0.9
    }
    possible_values {
      real: 1
    }
  }
}

[INFO hyperparameters_optimizer.cc:480] Start local tuner with 32 thread(s)
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on [INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD1420
[INFO gradient_boosted_trees.cc example(s) and 76 feature(s).:405
] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO [INFO gradient_boosted_trees.ccgradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO [INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
gradient_boosted_trees.cc[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:405] Default loss set to [INFO gradient_boosted_trees.cc:MULTINOMIAL_LOG_LIKELIHOOD[INFO [INFO
 [[INFO405 gradient_boosted_trees.cc] gradient_boosted_trees.cc:Default loss set to MULTINOMIAL_LOG_LIKELIHOOD1004
:gradient_boosted_trees.ccINFO:405[INFO] ] Training gradient boosted tree on 1420 example(s) and Default loss set to 405] Default loss set to  76gradient_boosted_trees.cc:1004MULTINOMIAL_LOG_LIKELIHOOD[  feature(s).gradient_boosted_trees.ccMULTINOMIAL_LOG_LIKELIHOOD:1004
] 
Training gradient boosted tree on 1420]  example(s) and 76INFO
[INFO  feature(s).gradient_boosted_trees.cc [:gradient_boosted_trees.ccTraining gradient boosted tree on 4051420: example(s) and INFO] Default loss set to  
100476] gradient_boosted_trees.cc feature(s).
:1004Training gradient boosted tree on MULTINOMIAL_LOG_LIKELIHOOD1420 example(s) and 76] Training gradient boosted tree on 1420 example(s) and 76
 feature(s).
 feature(s).[INFO [gradient_boosted_trees.cc:
INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
1004[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO [INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[[INFO gradient_boosted_trees.cc:1004] INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[[INFO gradient_boosted_trees.ccINFO[INFO :gradient_boosted_trees.cc :[405INFO gradient_boosted_trees.ccTraining gradient boosted tree on :405] [gradient_boosted_trees.ccgradient_boosted_trees.cc:405] Default loss set to :[1004INFO gradient_boosted_trees.cc:] INFOTraining gradient boosted tree on 4051420 1420gradient_boosted_trees.ccMULTINOMIAL_LOG_LIKELIHOOD example(s) and :
76 example(s) and 1004] ] ] Default loss set to 76 feature(s).[Default loss set to MULTINOMIAL_LOG_LIKELIHOODTraining gradient boosted tree on MULTINOMIAL_LOG_LIKELIHOOD
405 feature(s).
] 
[Default loss set to 1420Default loss set to 
 example(s) and INFO76 feature(s).MULTINOMIAL_LOG_LIKELIHOOD
[ INFO MULTINOMIAL_LOG_LIKELIHOODgradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD


[INFO gradient_boosted_trees.cc:gradient_boosted_trees.cc[:1004INFO[ gradient_boosted_trees.ccINFO[1004INFOINFO: gradient_boosted_trees.cc:10041004] ] Training gradient boosted tree on   gradient_boosted_trees.cc:gradient_boosted_trees.cc] ] Training gradient boosted tree on :14201004Training gradient boosted tree on Training gradient boosted tree on ] 14201420 example(s) and 1420 example(s) and  example(s) and 76 feature(s).
100476Training gradient boosted tree on  example(s) and 761420]  example(s) and 76 feature(s).76 feature(s).

 feature(s). feature(s).
Training gradient boosted tree on 1420 example(s) and 76 feature(s).[
[INFOINFO gradient_boosted_trees.cc:
 gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD405] Default loss set to 
MULTINOMIAL_LOG_LIKELIHOOD
[[INFO INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and gradient_boosted_trees.cc:1004[76INFO] Training gradient boosted tree on 1420 example(s) and 76 feature(s). feature(s). gradient_boosted_trees.cc

:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.638544 train-accuracy:0.734051 valid-loss:1.749113 valid-accuracy:0.638655
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO [INFO gradient_boosted_trees.cc:gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.885298 train-accuracy:0.568025 valid-loss:1.951454 valid-accuracy:0.504202
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.725335 train-accuracy:0.686395 valid-loss:1.807158 valid-accuracy:0.638655
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.854917 train-accuracy:0.730208 valid-loss:1.917172 valid-accuracy:0.647059
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.725225 train-accuracy:0.840123 valid-loss:1.820351 valid-accuracy:0.697479
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.521332 train-accuracy:0.801691 valid-loss:1.647648 valid-accuracy:0.697479
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.511938 train-accuracy:0.804766 valid-loss:1.650040 valid-accuracy:0.697479
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.420081 train-accuracy:0.853190 valid-loss:1.585621 valid-accuracy:0.655462
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.698993 train-accuracy:0.867794 valid-loss:1.842911 valid-accuracy:0.680672
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.716076 train-accuracy:0.848578 valid-loss:1.808865 valid-accuracy:0.705882
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.179394 train-accuracy:0.853190 valid-loss:2.211691 valid-accuracy:0.638655
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.003938 train-accuracy:0.831668 valid-loss:2.060782 valid-accuracy:0.680672
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.953477 train-accuracy:0.928517 valid-loss:2.056769 valid-accuracy:0.613445
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.375830 train-accuracy:0.903920 valid-loss:1.594810 valid-accuracy:0.680672
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.637385 train-accuracy:0.906995 valid-loss:1.786551 valid-accuracy:0.663866
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.165194 train-accuracy:0.889316 valid-loss:2.191730 valid-accuracy:0.663866
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.968487 train-accuracy:0.903151 valid-loss:2.044327 valid-accuracy:0.663866
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.167707 train-accuracy:0.903151 valid-loss:2.194467 valid-accuracy:0.705882
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.980560 train-accuracy:0.877018 valid-loss:2.039439 valid-accuracy:0.747899
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.962156 train-accuracy:0.912375 valid-loss:2.021629 valid-accuracy:0.722689
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.984918 train-accuracy:0.867025 valid-loss:2.043193 valid-accuracy:0.731092
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.620808 train-accuracy:0.925442 valid-loss:1.775139 valid-accuracy:0.689076
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.161208 train-accuracy:0.900077 valid-loss:2.192263 valid-accuracy:0.689076
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.651542 train-accuracy:0.889316 valid-loss:1.780750 valid-accuracy:0.663866
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.151670 train-accuracy:0.940046 valid-loss:2.191907 valid-accuracy:0.680672
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.157633 train-accuracy:0.905457 valid-loss:2.191650 valid-accuracy:0.680672
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.960765 train-accuracy:0.910838 valid-loss:2.029780 valid-accuracy:0.722689
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.169118 train-accuracy:0.892390 valid-loss:2.195639 valid-accuracy:0.747899
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.151746 train-accuracy:0.940815 valid-loss:2.186006 valid-accuracy:0.714286
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.967262 train-accuracy:0.896234 valid-loss:2.027062 valid-accuracy:0.747899
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.671270 train-accuracy:0.879324 valid-loss:1.809213 valid-accuracy:0.739496
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.362356 train-accuracy:0.895465 valid-loss:1.560552 valid-accuracy:0.714286
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.455902
[INFO gradient_boosted_trees.cc:230] Truncates the model to 250 tree(s) i.e. 25  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:25 valid-loss:0.455902 valid-accuracy:0.840336
[INFO hyperparameters_optimizer.cc:562] [1/50] Score: -0.455902 / -0.455902 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 4 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.166684 train-accuracy:0.888547 valid-loss:2.199236 valid-accuracy:0.680672
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.437367
[INFO gradient_boosted_trees.cc:230] Truncates the model to 280 tree(s) i.e. 28  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:28 valid-loss:0.437367 valid-accuracy:0.840336
[INFO hyperparameters_optimizer.cc:562] [2/50] Score: -0.437367 / -0.437367 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 4 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.459706
[INFO gradient_boosted_trees.cc:230] Truncates the model to 430 tree(s) i.e. 43  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:43 valid-loss:0.459706 valid-accuracy:0.848740
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO hyperparameters_optimizer.cc:562] [3/50] Score: -0.459706 / -0.437367 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.005777 train-accuracy:0.847041 valid-loss:2.069468 valid-accuracy:0.680672
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.569354 train-accuracy:0.961568 valid-loss:1.768937 valid-accuracy:0.697479
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.428818
[INFO gradient_boosted_trees.cc:230] Truncates the model to 280 tree(s) i.e. 28  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:28 valid-loss:0.428818 valid-accuracy:0.823529
[INFO hyperparameters_optimizer.cc:562] [4/50] Score: -0.428818 / -0.428818 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.572924 train-accuracy:0.963105 valid-loss:1.834257 valid-accuracy:0.630252
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.394704
[INFO gradient_boosted_trees.cc:230] Truncates the model to 630 tree(s) i.e. 63  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:63 valid-loss:0.394704 valid-accuracy:0.890756
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO hyperparameters_optimizer.cc:562] [5/50] Score: -0.394704 / -0.394704 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.014157 train-accuracy:0.568025 valid-loss:2.058656 valid-accuracy:0.504202
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.419142
[INFO gradient_boosted_trees.cc:230] Truncates the model to 420 tree(s) i.e. 42  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:42 valid-loss:0.419142 valid-accuracy:0.823529
[INFO hyperparameters_optimizer.cc:562] [6/50] Score: -0.419142 / -0.394704 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.410912
[INFO gradient_boosted_trees.cc:230] Truncates the model to 440 tree(s) i.e. 44  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:44 valid-loss:0.410912 valid-accuracy:0.848740
[INFO hyperparameters_optimizer.cc:562] [7/50] Score: -0.410912 / -0.394704 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.454031
[INFO gradient_boosted_trees.cc:230] Truncates the model to 210 tree(s) i.e. 21  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:21 valid-loss:0.454031 valid-accuracy:0.865546
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO hyperparameters_optimizer.cc:562] [8/50] Score: -0.454031 / -0.394704 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.387214
[INFO gradient_boosted_trees.cc:230] Truncates the model to 840 tree(s) i.e. 84  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:84 valid-loss:0.387214 valid-accuracy:0.865546
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.410860 train-accuracy:0.883167 valid-loss:1.653015 valid-accuracy:0.663866
[INFO hyperparameters_optimizer.cc:562] [9/50] Score: -0.387214 / -0.387214 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.447871 train-accuracy:0.859339 valid-loss:1.655336 valid-accuracy:0.638655
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.187255 train-accuracy:0.801691 valid-loss:2.206831 valid-accuracy:0.689076
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.168277 train-accuracy:0.902383 valid-loss:2.199005 valid-accuracy:0.697479
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.425877
[INFO gradient_boosted_trees.cc:230] Truncates the model to 410 tree(s) i.e. 41  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:41 valid-loss:0.425877 valid-accuracy:0.831933
[INFO hyperparameters_optimizer.cc:562] [10/50] Score: -0.425877 / -0.387214 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.643798 train-accuracy:0.905457 valid-loss:1.780205 valid-accuracy:0.714286
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.432679
[INFO gradient_boosted_trees.cc:230] Truncates the model to 450 tree(s) i.e. 45  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:45 valid-loss:0.432679 valid-accuracy:0.857143
[INFO hyperparameters_optimizer.cc:562] [11/50] Score: -0.432679 / -0.387214 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.448759
[INFO gradient_boosted_trees.cc:230] Truncates the model to 250 tree(s) i.e. 25  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:25 valid-loss:0.448759 valid-accuracy:0.798319
[INFO hyperparameters_optimizer.cc:562] [12/50] Score: -0.448759 / -0.387214 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.965129 train-accuracy:0.890853 valid-loss:2.031944 valid-accuracy:0.655462
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.174184 train-accuracy:0.875480 valid-loss:2.199853 valid-accuracy:0.714286
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.49016
[INFO gradient_boosted_trees.cc:230] Truncates the model to 330 tree(s) i.e. 33  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:33 valid-loss:0.490160 valid-accuracy:0.823529
[INFO hyperparameters_optimizer.cc:562] [13/50] Score: -0.49016 / -0.387214 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.458725
[INFO gradient_boosted_trees.cc:230] Truncates the model to 840 tree(s) i.e. 84  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:84 valid-loss:0.458725 valid-accuracy:0.831933
[INFO hyperparameters_optimizer.cc:562] [14/50] Score: -0.458725 / -0.387214 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.643410 train-accuracy:0.913144 valid-loss:1.764566 valid-accuracy:0.739496
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.621795 train-accuracy:0.919293 valid-loss:1.775435 valid-accuracy:0.672269
[INFO gradient_boosted_trees.cc:1432]     num-trees:101 train-loss:0.096974 train-accuracy:0.997694 valid-loss:0.415520 valid-accuracy:0.857143
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.41552
[INFO gradient_boosted_trees.cc:230] Truncates the model to 1010 tree(s) i.e. 101  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:101 valid-loss:0.415520 valid-accuracy:0.857143
[INFO hyperparameters_optimizer.cc:562] [15/50] Score: -0.41552 / -0.387214 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.626188 train-accuracy:0.921599 valid-loss:1.791896 valid-accuracy:0.705882
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.463903
[INFO gradient_boosted_trees.cc:230] Truncates the model to 460 tree(s) i.e. 46  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:46 valid-loss:0.463903 valid-accuracy:0.823529
[INFO hyperparameters_optimizer.cc:562] [16/50] Score: -0.463903 / -0.387214 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.462146
[INFO gradient_boosted_trees.cc:230] Truncates the model to 490 tree(s) i.e. 49  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:49 valid-loss:0.462146 valid-accuracy:0.831933
[INFO hyperparameters_optimizer.cc:562] [17/50] Score: -0.462146 / -0.387214 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.510047
[INFO gradient_boosted_trees.cc:230] Truncates the model to 380 tree(s) i.e. 38  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:38 valid-loss:0.510047 valid-accuracy:0.823529
[INFO hyperparameters_optimizer.cc:562] [18/50] Score: -0.510047 / -0.387214 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 128 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:2.163524 train-accuracy:0.913144 valid-loss:2.187975 valid-accuracy:0.739496
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.435778
[INFO gradient_boosted_trees.cc:230] Truncates the model to 360 tree(s) i.e. 36  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:36 valid-loss:0.435778 valid-accuracy:0.848740
[INFO hyperparameters_optimizer.cc:562] [19/50] Score: -0.435778 / -0.387214 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.952799 train-accuracy:0.907763 valid-loss:2.028851 valid-accuracy:0.705882
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.592175 train-accuracy:0.943121 valid-loss:1.790689 valid-accuracy:0.705882
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.465531
[INFO gradient_boosted_trees.cc:230] Truncates the model to 350 tree(s) i.e. 35  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:35 valid-loss:0.465531 valid-accuracy:0.840336
[INFO hyperparameters_optimizer.cc:562] [20/50] Score: -0.465531 / -0.387214 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.513542
[INFO gradient_boosted_trees.cc:230] Truncates the model to 310 tree(s) i.e. 31  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:31 valid-loss:0.513542 valid-accuracy:0.831933
[INFO hyperparameters_optimizer.cc:562] [21/50] Score: -0.513542 / -0.387214 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.424319
[INFO gradient_boosted_trees.cc:230] Truncates the model to 940 tree(s) i.e. 94  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:94 valid-loss:0.424319 valid-accuracy:0.840336
[INFO hyperparameters_optimizer.cc:562] [22/50] Score: -0.424319 / -0.387214 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.468989
[INFO gradient_boosted_trees.cc:230] Truncates the model to 320 tree(s) i.e. 32  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:32 valid-loss:0.468989 valid-accuracy:0.865546
[INFO hyperparameters_optimizer.cc:562] [23/50] Score: -0.468989 / -0.387214 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.519588
[INFO gradient_boosted_trees.cc:230] Truncates the model to 600 tree(s) i.e. 60  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:60 valid-loss:0.519588 valid-accuracy:0.798319
[INFO hyperparameters_optimizer.cc:562] [24/50] Score: -0.519588 / -0.387214 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.484521
[INFO gradient_boosted_trees.cc:230] Truncates the model to 300 tree(s) i.e. 30  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:30 valid-loss:0.484521 valid-accuracy:0.831933
[INFO hyperparameters_optimizer.cc:562] [25/50] Score: -0.484521 / -0.387214 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 256 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.387002
[INFO gradient_boosted_trees.cc:230] Truncates the model to 880 tree(s) i.e. 88  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:88 valid-loss:0.387002 valid-accuracy:0.857143
[INFO hyperparameters_optimizer.cc:562] [26/50] Score: -0.387002 / -0.387002 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.450161
[INFO gradient_boosted_trees.cc:230] Truncates the model to 720 tree(s) i.e. 72  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:72 valid-loss:0.450161 valid-accuracy:0.840336
[INFO hyperparameters_optimizer.cc:562] [27/50] Score: -0.450161 / -0.387002 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.445765
[INFO gradient_boosted_trees.cc:230] Truncates the model to 990 tree(s) i.e. 99  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:99 valid-loss:0.445765 valid-accuracy:0.840336
[INFO hyperparameters_optimizer.cc:562] [28/50] Score: -0.445765 / -0.387002 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 256 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.522104
[INFO gradient_boosted_trees.cc:230] Truncates the model to 600 tree(s) i.e. 60  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:60 valid-loss:0.522104 valid-accuracy:0.815126
[INFO hyperparameters_optimizer.cc:562] [29/50] Score: -0.522104 / -0.387002 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.441146
[INFO gradient_boosted_trees.cc:230] Truncates the model to 370 tree(s) i.e. 37  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:37 valid-loss:0.441146 valid-accuracy:0.865546
[INFO hyperparameters_optimizer.cc:562] [30/50] Score: -0.441146 / -0.387002 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.439972
[INFO gradient_boosted_trees.cc:230] Truncates the model to 700 tree(s) i.e. 70  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:70 valid-loss:0.439972 valid-accuracy:0.865546
[INFO hyperparameters_optimizer.cc:562] [31/50] Score: -0.439972 / -0.387002 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.392284
[INFO gradient_boosted_trees.cc:230] Truncates the model to 1210 tree(s) i.e. 121  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:121 valid-loss:0.392284 valid-accuracy:0.848740
[INFO hyperparameters_optimizer.cc:562] [32/50] Score: -0.392284 / -0.387002 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.450171
[INFO gradient_boosted_trees.cc:230] Truncates the model to 690 tree(s) i.e. 69  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:69 valid-loss:0.450171 valid-accuracy:0.840336
[INFO hyperparameters_optimizer.cc:562] [33/50] Score: -0.450171 / -0.387002 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.537393
[INFO gradient_boosted_trees.cc:230] Truncates the model to 290 tree(s) i.e. 29  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:29 valid-loss:0.537393 valid-accuracy:0.815126
[INFO hyperparameters_optimizer.cc:562] [34/50] Score: -0.537393 / -0.387002 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.45856
[INFO gradient_boosted_trees.cc:230] Truncates the model to 940 tree(s) i.e. 94  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:94 valid-loss:0.458560 valid-accuracy:0.823529
[INFO hyperparameters_optimizer.cc:562] [35/50] Score: -0.45856 / -0.387002 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.40477
[INFO gradient_boosted_trees.cc:230] Truncates the model to 490 tree(s) i.e. 49  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:49 valid-loss:0.404770 valid-accuracy:0.831933
[INFO hyperparameters_optimizer.cc:562] [36/50] Score: -0.40477 / -0.387002 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:1432]     num-trees:124 train-loss:0.082209 train-accuracy:1.000000 valid-loss:0.551178 valid-accuracy:0.764706
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.529507
[INFO gradient_boosted_trees.cc:230] Truncates the model to 760 tree(s) i.e. 76  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:76 valid-loss:0.529507 valid-accuracy:0.806723
[INFO hyperparameters_optimizer.cc:562] [37/50] Score: -0.529507 / -0.387002 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.466392
[INFO gradient_boosted_trees.cc:230] Truncates the model to 2390 tree(s) i.e. 239  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:239 valid-loss:0.466392 valid-accuracy:0.798319
[INFO hyperparameters_optimizer.cc:562] [38/50] Score: -0.466392 / -0.387002 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 4 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.420369
[INFO gradient_boosted_trees.cc:230] Truncates the model to 1690 tree(s) i.e. 169  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:169 valid-loss:0.420369 valid-accuracy:0.840336
[INFO hyperparameters_optimizer.cc:562] [39/50] Score: -0.420369 / -0.387002 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 128 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.398329
[INFO gradient_boosted_trees.cc:230] Truncates the model to 2290 tree(s) i.e. 229  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:229 valid-loss:0.398329 valid-accuracy:0.840336
[INFO hyperparameters_optimizer.cc:562] [40/50] Score: -0.398329 / -0.387002 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.434457
[INFO gradient_boosted_trees.cc:230] Truncates the model to 1670 tree(s) i.e. 167  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:167 valid-loss:0.434457 valid-accuracy:0.857143
[INFO hyperparameters_optimizer.cc:562] [41/50] Score: -0.434457 / -0.387002 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 128 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.426428
[INFO gradient_boosted_trees.cc:230] Truncates the model to 2060 tree(s) i.e. 206  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:206 valid-loss:0.426428 valid-accuracy:0.823529
[INFO hyperparameters_optimizer.cc:562] [42/50] Score: -0.426428 / -0.387002 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.54219
[INFO gradient_boosted_trees.cc:230] Truncates the model to 1540 tree(s) i.e. 154  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:154 valid-loss:0.542190 valid-accuracy:0.848740
[INFO hyperparameters_optimizer.cc:562] [43/50] Score: -0.54219 / -0.387002 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.476062
[INFO gradient_boosted_trees.cc:230] Truncates the model to 1830 tree(s) i.e. 183  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:183 valid-loss:0.476062 valid-accuracy:0.831933
[INFO hyperparameters_optimizer.cc:562] [44/50] Score: -0.476062 / -0.387002 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.406853
[INFO gradient_boosted_trees.cc:230] Truncates the model to 2030 tree(s) i.e. 203  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:203 valid-loss:0.406853 valid-accuracy:0.831933
[INFO hyperparameters_optimizer.cc:562] [45/50] Score: -0.406853 / -0.387002 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.620409
[INFO gradient_boosted_trees.cc:230] Truncates the model to 1380 tree(s) i.e. 138  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:138 valid-loss:0.620409 valid-accuracy:0.764706
[INFO hyperparameters_optimizer.cc:562] [46/50] Score: -0.620409 / -0.387002 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.478777
[INFO gradient_boosted_trees.cc:230] Truncates the model to 1680 tree(s) i.e. 168  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:168 valid-loss:0.478777 valid-accuracy:0.840336
[INFO hyperparameters_optimizer.cc:562] [47/50] Score: -0.478777 / -0.387002 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 128 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.496135
[INFO gradient_boosted_trees.cc:230] Truncates the model to 2140 tree(s) i.e. 214  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:214 valid-loss:0.496135 valid-accuracy:0.789916
[INFO hyperparameters_optimizer.cc:562] [48/50] Score: -0.496135 / -0.387002 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.441269
[INFO gradient_boosted_trees.cc:230] Truncates the model to 1560 tree(s) i.e. 156  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:156 valid-loss:0.441269 valid-accuracy:0.840336
[INFO hyperparameters_optimizer.cc:562] [49/50] Score: -0.441269 / -0.387002 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.464621
[INFO gradient_boosted_trees.cc:230] Truncates the model to 2480 tree(s) i.e. 248  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:248 valid-loss:0.464621 valid-accuracy:0.840336
[INFO hyperparameters_optimizer.cc:562] [50/50] Score: -0.464621 / -0.387002 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO hyperparameters_optimizer.cc:216] Best hyperparameters:
fields {
  name: "min_examples"
  value {
    integer: 10
  }
}
fields {
  name: "categorical_algorithm"
  value {
    categorical: "CART"
  }
}
fields {
  name: "growing_strategy"
  value {
    categorical: "BEST_FIRST_GLOBAL"
  }
}
fields {
  name: "max_num_nodes"
  value {
    integer: 16
  }
}
fields {
  name: "use_hessian_gain"
  value {
    categorical: "true"
  }
}
fields {
  name: "shrinkage"
  value {
    real: 0.05
  }
}
fields {
  name: "num_candidate_attributes_ratio"
  value {
    real: 1
  }
}

[INFO kernel.cc:957] Export model in log directory: /tmpfs/tmp/tmp89tz8qw2 with prefix 79ad6a30f5524ec2
[INFO kernel.cc:974] Save model in resources
[INFO kernel.cc:1176] Loading model from path /tmpfs/tmp/tmp89tz8qw2/model/ with prefix 79ad6a30f5524ec2
[INFO decision_forest.cc:639] Model loaded with 880 root(s), 27202 node(s), and 76 input feature(s).
[INFO abstract_model.cc:1246] Engine "GradientBoostedTreesGeneric" built
[INFO kernel.cc:1022] Use fast generic engine
Model trained in 0:01:17.822224
Compiling model...
Model compiled.
CPU times: user 26min 47s, sys: 8min 58s, total: 35min 46s
Wall time: 1min 19s
<keras.callbacks.History at 0x7fc7a2995ee0>
# Evaluate the model
tuned_model.compile(["accuracy"])
tuned_test_accuracy = tuned_model.evaluate(test_ds, return_dict=True, verbose=0)["accuracy"]
print(f"Test accuracy with the TF-DF hyper-parameter tuner: {tuned_test_accuracy:.4f}")
Test accuracy with the TF-DF hyper-parameter tuner: 0.8172

The hyper-parameters and objective scores of the trials are available in the model inspector. The score value is always maximized. In this example, the score is the negative log loss on the validation dataset (selected automatically).

# Display the tuning logs.
tuning_logs = tuned_model.make_inspector().tuning_logs()
tuning_logs.head()

The single rows with best=True is the one used in the final model.

# Best hyper-parameters.
tuning_logs[tuning_logs.best].iloc[0]
score                                     -0.387002
evaluation_time                           37.610261
best                                           True
min_examples                                     10
categorical_algorithm                          CART
growing_strategy                  BEST_FIRST_GLOBAL
max_depth                                       NaN
use_hessian_gain                               true
shrinkage                                      0.05
num_candidate_attributes_ratio                  1.0
max_num_nodes                                  16.0
Name: 25, dtype: object

Next, we plot the evoluation of the best score during the tuning.

plt.figure(figsize=(10, 5))
plt.plot(tuning_logs["score"], label="current trial")
plt.plot(tuning_logs["score"].cummax(), label="best trial")
plt.xlabel("Tuning step")
plt.ylabel("Tuning score")
plt.legend()
plt.show()

png

Training a model with Keras Tuner (Alternative appraoch)

TensorFlow Decision Forests is based on the Keras framework, and it is compatible with the Keras tuner.

Currently, the TF-DF Tuner and the Keras Tuner are complementary.

TF-DF Tuner

  • Automatic configuration of the objective.
  • Automatic extraction of validation dataset (if needed).
  • Support model self evaluation (e.g. out-of-bag evaluation).
  • Distributed hyper-parameter tuning.
  • Shared dataset access in between the trials: The tensorflow dataset is read only once, speeding-up tuning significantly on small datasets.

Keras Tuner

  • Support tuning of the pre-processing parameters.
  • Support hyper-band optimizer.
  • Support custom objectives.

Let's tune a TF-DF model using the Keras tuner.

# Install the Keras tuner
!pip install keras-tuner -U -qq
import keras_tuner as kt
%%time

def build_model(hp):
  """Creates a model."""

  model = tfdf.keras.GradientBoostedTreesModel(
      min_examples=hp.Choice("min_examples", [2, 5, 7, 10]),
      categorical_algorithm=hp.Choice("categorical_algorithm", ["CART", "RANDOM"]),
      max_depth=hp.Choice("max_depth", [4, 5, 6, 7]),
      # The keras tuner convert automaticall boolean parameters to integers.
      use_hessian_gain=bool(hp.Choice("use_hessian_gain", [True, False])),
      shrinkage=hp.Choice("shrinkage", [0.02, 0.05, 0.10, 0.15]),
      num_candidate_attributes_ratio=hp.Choice("num_candidate_attributes_ratio", [0.2, 0.5, 0.9, 1.0]),
  )

  # Optimize the model accuracy as computed on the validation dataset.
  model.compile(metrics=["accuracy"])
  return model

keras_tuner = kt.RandomSearch(
    build_model,
    objective="val_accuracy",
    max_trials=50,
    overwrite=True,
    directory="/tmp/keras_tuning")

# Important: The tuning should not be done on the test dataset.

# Extract a validation dataset from the training dataset. The new training
# dataset is called the "sub-training-dataset".
sub_train_df, sub_valid_df = split_dataset(train_df)
sub_train_ds = tfdf.keras.pd_dataframe_to_tf_dataset(sub_train_df, label=label)
sub_valid_ds = tfdf.keras.pd_dataframe_to_tf_dataset(sub_valid_df, label=label)

# Tune the model
keras_tuner.search(sub_train_ds, validation_data=sub_valid_ds)
Trial 50 Complete [00h 00m 06s]
val_accuracy: 0.8009049892425537

Best val_accuracy So Far: 0.8393664956092834
Total elapsed time: 00h 06m 31s
INFO:tensorflow:Oracle triggered exit
INFO:tensorflow:Oracle triggered exit
CPU times: user 27min 3s, sys: 11min 1s, total: 38min 5s
Wall time: 6min 31s

The best hyper-parameter are available with get_best_hyperparameters:

# Tune the model
best_hyper_parameters = keras_tuner.get_best_hyperparameters()[0].values
print("Best hyper-parameters:", keras_tuner.get_best_hyperparameters()[0].values)
Best hyper-parameters: {'min_examples': 7, 'categorical_algorithm': 'CART', 'max_depth': 6, 'use_hessian_gain': 1, 'shrinkage': 0.05, 'num_candidate_attributes_ratio': 0.9}

The model should be re-trained with the best hyper-parameters:

%set_cell_height 300
# Train the model
# The keras tuner convert automaticall boolean parameters to integers.
best_hyper_parameters["use_hessian_gain"] = bool(best_hyper_parameters["use_hessian_gain"])
best_model = tfdf.keras.GradientBoostedTreesModel(**best_hyper_parameters)
best_model.fit(train_ds, verbose=2)
<IPython.core.display.Javascript object>
Warning: The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
WARNING:absl:The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
Use /tmpfs/tmp/tmpza24xjxv as temporary training directory
Reading training dataset...
Training tensor examples:
Features: {'coef_0': <tf.Tensor 'data:0' shape=(None,) dtype=float64>, 'coef_1': <tf.Tensor 'data_1:0' shape=(None,) dtype=float64>, 'coef_2': <tf.Tensor 'data_12:0' shape=(None,) dtype=float64>, 'coef_3': <tf.Tensor 'data_23:0' shape=(None,) dtype=float64>, 'coef_4': <tf.Tensor 'data_34:0' shape=(None,) dtype=float64>, 'coef_5': <tf.Tensor 'data_45:0' shape=(None,) dtype=float64>, 'coef_6': <tf.Tensor 'data_56:0' shape=(None,) dtype=float64>, 'coef_7': <tf.Tensor 'data_67:0' shape=(None,) dtype=float64>, 'coef_8': <tf.Tensor 'data_74:0' shape=(None,) dtype=float64>, 'coef_9': <tf.Tensor 'data_75:0' shape=(None,) dtype=float64>, 'coef_10': <tf.Tensor 'data_2:0' shape=(None,) dtype=float64>, 'coef_11': <tf.Tensor 'data_3:0' shape=(None,) dtype=float64>, 'coef_12': <tf.Tensor 'data_4:0' shape=(None,) dtype=float64>, 'coef_13': <tf.Tensor 'data_5:0' shape=(None,) dtype=float64>, 'coef_14': <tf.Tensor 'data_6:0' shape=(None,) dtype=float64>, 'coef_15': <tf.Tensor 'data_7:0' shape=(None,) dtype=float64>, 'coef_16': <tf.Tensor 'data_8:0' shape=(None,) dtype=float64>, 'coef_17': <tf.Tensor 'data_9:0' shape=(None,) dtype=float64>, 'coef_18': <tf.Tensor 'data_10:0' shape=(None,) dtype=float64>, 'coef_19': <tf.Tensor 'data_11:0' shape=(None,) dtype=float64>, 'coef_20': <tf.Tensor 'data_13:0' shape=(None,) dtype=float64>, 'coef_21': <tf.Tensor 'data_14:0' shape=(None,) dtype=float64>, 'coef_22': <tf.Tensor 'data_15:0' shape=(None,) dtype=float64>, 'coef_23': <tf.Tensor 'data_16:0' shape=(None,) dtype=float64>, 'coef_24': <tf.Tensor 'data_17:0' shape=(None,) dtype=float64>, 'coef_25': <tf.Tensor 'data_18:0' shape=(None,) dtype=float64>, 'coef_26': <tf.Tensor 'data_19:0' shape=(None,) dtype=float64>, 'coef_27': <tf.Tensor 'data_20:0' shape=(None,) dtype=float64>, 'coef_28': <tf.Tensor 'data_21:0' shape=(None,) dtype=float64>, 'coef_29': <tf.Tensor 'data_22:0' shape=(None,) dtype=float64>, 'coef_30': <tf.Tensor 'data_24:0' shape=(None,) dtype=float64>, 'coef_31': <tf.Tensor 'data_25:0' shape=(None,) dtype=float64>, 'coef_32': <tf.Tensor 'data_26:0' shape=(None,) dtype=float64>, 'coef_33': <tf.Tensor 'data_27:0' shape=(None,) dtype=float64>, 'coef_34': <tf.Tensor 'data_28:0' shape=(None,) dtype=float64>, 'coef_35': <tf.Tensor 'data_29:0' shape=(None,) dtype=float64>, 'coef_36': <tf.Tensor 'data_30:0' shape=(None,) dtype=float64>, 'coef_37': <tf.Tensor 'data_31:0' shape=(None,) dtype=float64>, 'coef_38': <tf.Tensor 'data_32:0' shape=(None,) dtype=float64>, 'coef_39': <tf.Tensor 'data_33:0' shape=(None,) dtype=float64>, 'coef_40': <tf.Tensor 'data_35:0' shape=(None,) dtype=float64>, 'coef_41': <tf.Tensor 'data_36:0' shape=(None,) dtype=float64>, 'coef_42': <tf.Tensor 'data_37:0' shape=(None,) dtype=float64>, 'coef_43': <tf.Tensor 'data_38:0' shape=(None,) dtype=float64>, 'coef_44': <tf.Tensor 'data_39:0' shape=(None,) dtype=float64>, 'coef_45': <tf.Tensor 'data_40:0' shape=(None,) dtype=float64>, 'coef_46': <tf.Tensor 'data_41:0' shape=(None,) dtype=float64>, 'coef_47': <tf.Tensor 'data_42:0' shape=(None,) dtype=float64>, 'coef_48': <tf.Tensor 'data_43:0' shape=(None,) dtype=float64>, 'coef_49': <tf.Tensor 'data_44:0' shape=(None,) dtype=float64>, 'coef_50': <tf.Tensor 'data_46:0' shape=(None,) dtype=float64>, 'coef_51': <tf.Tensor 'data_47:0' shape=(None,) dtype=float64>, 'coef_52': <tf.Tensor 'data_48:0' shape=(None,) dtype=float64>, 'coef_53': <tf.Tensor 'data_49:0' shape=(None,) dtype=float64>, 'coef_54': <tf.Tensor 'data_50:0' shape=(None,) dtype=float64>, 'coef_55': <tf.Tensor 'data_51:0' shape=(None,) dtype=float64>, 'coef_56': <tf.Tensor 'data_52:0' shape=(None,) dtype=float64>, 'coef_57': <tf.Tensor 'data_53:0' shape=(None,) dtype=float64>, 'coef_58': <tf.Tensor 'data_54:0' shape=(None,) dtype=float64>, 'coef_59': <tf.Tensor 'data_55:0' shape=(None,) dtype=float64>, 'coef_60': <tf.Tensor 'data_57:0' shape=(None,) dtype=float64>, 'coef_61': <tf.Tensor 'data_58:0' shape=(None,) dtype=float64>, 'coef_62': <tf.Tensor 'data_59:0' shape=(None,) dtype=float64>, 'coef_63': <tf.Tensor 'data_60:0' shape=(None,) dtype=float64>, 'coef_64': <tf.Tensor 'data_61:0' shape=(None,) dtype=float64>, 'coef_65': <tf.Tensor 'data_62:0' shape=(None,) dtype=float64>, 'coef_66': <tf.Tensor 'data_63:0' shape=(None,) dtype=float64>, 'coef_67': <tf.Tensor 'data_64:0' shape=(None,) dtype=float64>, 'coef_68': <tf.Tensor 'data_65:0' shape=(None,) dtype=float64>, 'coef_69': <tf.Tensor 'data_66:0' shape=(None,) dtype=float64>, 'coef_70': <tf.Tensor 'data_68:0' shape=(None,) dtype=float64>, 'coef_71': <tf.Tensor 'data_69:0' shape=(None,) dtype=float64>, 'coef_72': <tf.Tensor 'data_70:0' shape=(None,) dtype=float64>, 'coef_73': <tf.Tensor 'data_71:0' shape=(None,) dtype=float64>, 'coef_74': <tf.Tensor 'data_72:0' shape=(None,) dtype=float64>, 'coef_75': <tf.Tensor 'data_73:0' shape=(None,) dtype=float64>}
Label: Tensor("data_76:0", shape=(None,), dtype=int64)
Weights: None
Normalized tensor features:
 {'coef_0': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast:0' shape=(None,) dtype=float32>), 'coef_1': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_1:0' shape=(None,) dtype=float32>), 'coef_2': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_2:0' shape=(None,) dtype=float32>), 'coef_3': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_3:0' shape=(None,) dtype=float32>), 'coef_4': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_4:0' shape=(None,) dtype=float32>), 'coef_5': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_5:0' shape=(None,) dtype=float32>), 'coef_6': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_6:0' shape=(None,) dtype=float32>), 'coef_7': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_7:0' shape=(None,) dtype=float32>), 'coef_8': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_8:0' shape=(None,) dtype=float32>), 'coef_9': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_9:0' shape=(None,) dtype=float32>), 'coef_10': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_10:0' shape=(None,) dtype=float32>), 'coef_11': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_11:0' shape=(None,) dtype=float32>), 'coef_12': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_12:0' shape=(None,) dtype=float32>), 'coef_13': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_13:0' shape=(None,) dtype=float32>), 'coef_14': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_14:0' shape=(None,) dtype=float32>), 'coef_15': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_15:0' shape=(None,) dtype=float32>), 'coef_16': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_16:0' shape=(None,) dtype=float32>), 'coef_17': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_17:0' shape=(None,) dtype=float32>), 'coef_18': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_18:0' shape=(None,) dtype=float32>), 'coef_19': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_19:0' shape=(None,) dtype=float32>), 'coef_20': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_20:0' shape=(None,) dtype=float32>), 'coef_21': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_21:0' shape=(None,) dtype=float32>), 'coef_22': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_22:0' shape=(None,) dtype=float32>), 'coef_23': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_23:0' shape=(None,) dtype=float32>), 'coef_24': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_24:0' shape=(None,) dtype=float32>), 'coef_25': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_25:0' shape=(None,) dtype=float32>), 'coef_26': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_26:0' shape=(None,) dtype=float32>), 'coef_27': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_27:0' shape=(None,) dtype=float32>), 'coef_28': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_28:0' shape=(None,) dtype=float32>), 'coef_29': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_29:0' shape=(None,) dtype=float32>), 'coef_30': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_30:0' shape=(None,) dtype=float32>), 'coef_31': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_31:0' shape=(None,) dtype=float32>), 'coef_32': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_32:0' shape=(None,) dtype=float32>), 'coef_33': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_33:0' shape=(None,) dtype=float32>), 'coef_34': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_34:0' shape=(None,) dtype=float32>), 'coef_35': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_35:0' shape=(None,) dtype=float32>), 'coef_36': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_36:0' shape=(None,) dtype=float32>), 'coef_37': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_37:0' shape=(None,) dtype=float32>), 'coef_38': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_38:0' shape=(None,) dtype=float32>), 'coef_39': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_39:0' shape=(None,) dtype=float32>), 'coef_40': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_40:0' shape=(None,) dtype=float32>), 'coef_41': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_41:0' shape=(None,) dtype=float32>), 'coef_42': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_42:0' shape=(None,) dtype=float32>), 'coef_43': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_43:0' shape=(None,) dtype=float32>), 'coef_44': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_44:0' shape=(None,) dtype=float32>), 'coef_45': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_45:0' shape=(None,) dtype=float32>), 'coef_46': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_46:0' shape=(None,) dtype=float32>), 'coef_47': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_47:0' shape=(None,) dtype=float32>), 'coef_48': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_48:0' shape=(None,) dtype=float32>), 'coef_49': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_49:0' shape=(None,) dtype=float32>), 'coef_50': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_50:0' shape=(None,) dtype=float32>), 'coef_51': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_51:0' shape=(None,) dtype=float32>), 'coef_52': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_52:0' shape=(None,) dtype=float32>), 'coef_53': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_53:0' shape=(None,) dtype=float32>), 'coef_54': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_54:0' shape=(None,) dtype=float32>), 'coef_55': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_55:0' shape=(None,) dtype=float32>), 'coef_56': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_56:0' shape=(None,) dtype=float32>), 'coef_57': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_57:0' shape=(None,) dtype=float32>), 'coef_58': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_58:0' shape=(None,) dtype=float32>), 'coef_59': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_59:0' shape=(None,) dtype=float32>), 'coef_60': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_60:0' shape=(None,) dtype=float32>), 'coef_61': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_61:0' shape=(None,) dtype=float32>), 'coef_62': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_62:0' shape=(None,) dtype=float32>), 'coef_63': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_63:0' shape=(None,) dtype=float32>), 'coef_64': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_64:0' shape=(None,) dtype=float32>), 'coef_65': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_65:0' shape=(None,) dtype=float32>), 'coef_66': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_66:0' shape=(None,) dtype=float32>), 'coef_67': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_67:0' shape=(None,) dtype=float32>), 'coef_68': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_68:0' shape=(None,) dtype=float32>), 'coef_69': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_69:0' shape=(None,) dtype=float32>), 'coef_70': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_70:0' shape=(None,) dtype=float32>), 'coef_71': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_71:0' shape=(None,) dtype=float32>), 'coef_72': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_72:0' shape=(None,) dtype=float32>), 'coef_73': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_73:0' shape=(None,) dtype=float32>), 'coef_74': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_74:0' shape=(None,) dtype=float32>), 'coef_75': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_75:0' shape=(None,) dtype=float32>)}
Training dataset read in 0:00:00.776876. Found 1420 examples.
Training model...
[INFO kernel.cc:809] Start Yggdrasil model training
[INFO kernel.cc:810] Collect training examples
[INFO kernel.cc:418] Number of batches: 2
[INFO kernel.cc:419] Number of examples: 1420
[INFO kernel.cc:832] Training dataset:
Number of records: 1420
Number of columns: 77

Number of columns by type:
    NUMERICAL: 76 (98.7013%)
    CATEGORICAL: 1 (1.2987%)

Columns:

NUMERICAL: 76 (98.7013%)
    0: "coef_0" NUMERICAL mean:0.185193 min:0.00492771 max:0.383341 sd:0.0928829
    1: "coef_1" NUMERICAL mean:0.374057 min:0.0090002 max:0.771281 sd:0.173715
    2: "coef_10" NUMERICAL mean:0.15629 min:0.00376415 max:0.40926 sd:0.0867228
    3: "coef_11" NUMERICAL mean:0.165407 min:0.00274976 max:0.506974 sd:0.0833784
    4: "coef_12" NUMERICAL mean:0.165901 min:0.00142364 max:0.471881 sd:0.0933784
    5: "coef_13" NUMERICAL mean:0.193121 min:0.00590215 max:0.635803 sd:0.109577
    6: "coef_14" NUMERICAL mean:0.143136 min:0.00647194 max:0.382988 sd:0.064732
    7: "coef_15" NUMERICAL mean:0.114066 min:0.00290088 max:0.366277 sd:0.0601661
    8: "coef_16" NUMERICAL mean:0.127982 min:0.00287311 max:0.462718 sd:0.0664961
    9: "coef_17" NUMERICAL mean:0.129778 min:0.00304677 max:0.396381 sd:0.0781749
    10: "coef_18" NUMERICAL mean:0.120858 min:0.00486002 max:0.407231 sd:0.0656658
    11: "coef_19" NUMERICAL mean:0.116757 min:0.00181387 max:0.345249 sd:0.0539648
    12: "coef_2" NUMERICAL mean:0.269064 min:0.0079055 max:0.665372 sd:0.137599
    13: "coef_20" NUMERICAL mean:0.108321 min:0.00243648 max:0.315663 sd:0.0530303
    14: "coef_21" NUMERICAL mean:0.108301 min:0.00261734 max:0.337783 sd:0.051511
    15: "coef_22" NUMERICAL mean:0.0986228 min:0.00263824 max:0.332622 sd:0.0585544
    16: "coef_23" NUMERICAL mean:0.115921 min:0.00418841 max:0.451662 sd:0.0652091
    17: "coef_24" NUMERICAL mean:0.106315 min:0.00182609 max:0.280131 sd:0.0473684
    18: "coef_25" NUMERICAL mean:0.0985301 min:0.00407151 max:0.312352 sd:0.0497109
    19: "coef_26" NUMERICAL mean:0.0955449 min:0.00480036 max:0.320455 sd:0.0474757
    20: "coef_27" NUMERICAL mean:0.090385 min:0.00293579 max:0.32458 sd:0.0500048
    21: "coef_28" NUMERICAL mean:0.0930036 min:0.00239058 max:0.312097 sd:0.0520835
    22: "coef_29" NUMERICAL mean:0.0943252 min:0.0044099 max:0.247468 sd:0.0445233
    23: "coef_3" NUMERICAL mean:0.263306 min:0.0101143 max:0.748012 sd:0.105796
    24: "coef_30" NUMERICAL mean:0.0931969 min:0.00282674 max:0.293574 sd:0.0457799
    25: "coef_31" NUMERICAL mean:0.088141 min:0.00174888 max:0.277819 sd:0.0430336
    26: "coef_32" NUMERICAL mean:0.0827424 min:0.00025893 max:0.292676 sd:0.0468864
    27: "coef_33" NUMERICAL mean:0.0829955 min:0.00256918 max:0.314669 sd:0.0438914
    28: "coef_34" NUMERICAL mean:0.0903238 min:0.00396516 max:0.278892 sd:0.0418604
    29: "coef_35" NUMERICAL mean:0.0864804 min:0.00158334 max:0.253353 sd:0.0426071
    30: "coef_36" NUMERICAL mean:0.0820904 min:0.00228121 max:0.257029 sd:0.041108
    31: "coef_37" NUMERICAL mean:0.0777956 min:0.00046895 max:0.291041 sd:0.0418326
    32: "coef_38" NUMERICAL mean:0.077968 min:0.00171066 max:0.251208 sd:0.040831
    33: "coef_39" NUMERICAL mean:0.0856294 min:0.00268666 max:0.247084 sd:0.0407453
    34: "coef_4" NUMERICAL mean:0.303393 min:0.00209035 max:0.721911 sd:0.15479
    35: "coef_40" NUMERICAL mean:0.0801986 min:0.00087734 max:0.243434 sd:0.0400049
    36: "coef_41" NUMERICAL mean:0.0773931 min:0.00206583 max:0.229024 sd:0.0380087
    37: "coef_42" NUMERICAL mean:0.0736305 min:0.00136188 max:0.23306 sd:0.0382668
    38: "coef_43" NUMERICAL mean:0.0712032 min:0.00113918 max:0.251665 sd:0.0383177
    39: "coef_44" NUMERICAL mean:0.083717 min:0.00308002 max:0.230327 sd:0.0409021
    40: "coef_45" NUMERICAL mean:0.0784619 min:0.00232565 max:0.225519 sd:0.0371967
    41: "coef_46" NUMERICAL mean:0.0745289 min:0.00223798 max:0.225919 sd:0.0394697
    42: "coef_47" NUMERICAL mean:0.0735969 min:0.00375832 max:0.234122 sd:0.0378476
    43: "coef_48" NUMERICAL mean:0.0858847 min:0.0029998 max:0.266345 sd:0.042262
    44: "coef_49" NUMERICAL mean:0.0834963 min:0.00115433 max:0.242921 sd:0.0411049
    45: "coef_5" NUMERICAL mean:0.158677 min:0.00267619 max:0.479913 sd:0.089747
    46: "coef_50" NUMERICAL mean:0.0806624 min:0.00280476 max:0.247757 sd:0.0425177
    47: "coef_51" NUMERICAL mean:0.0798156 min:0.00227356 max:0.235941 sd:0.0416608
    48: "coef_52" NUMERICAL mean:0.0948293 min:0.00345921 max:0.250377 sd:0.0453798
    49: "coef_53" NUMERICAL mean:0.0918011 min:0.00157472 max:0.257 sd:0.0439351
    50: "coef_54" NUMERICAL mean:0.0872454 min:0.00270626 max:0.256175 sd:0.0483413
    51: "coef_55" NUMERICAL mean:0.0850743 min:0.00156102 max:0.241507 sd:0.042891
    52: "coef_56" NUMERICAL mean:0.100832 min:0.00166723 max:0.277346 sd:0.0504132
    53: "coef_57" NUMERICAL mean:0.101564 min:0.00288512 max:0.284183 sd:0.0495377
    54: "coef_58" NUMERICAL mean:0.0906659 min:0.00173879 max:0.26817 sd:0.048872
    55: "coef_59" NUMERICAL mean:0.0947948 min:0.00237684 max:0.284695 sd:0.0531274
    56: "coef_6" NUMERICAL mean:0.367434 min:0.0119186 max:0.750938 sd:0.144447
    57: "coef_60" NUMERICAL mean:0.115679 min:0.00538156 max:0.300376 sd:0.0600483
    58: "coef_61" NUMERICAL mean:0.109685 min:0.00160526 max:0.319926 sd:0.0542184
    59: "coef_62" NUMERICAL mean:0.105632 min:0.00287046 max:0.31959 sd:0.0605518
    60: "coef_63" NUMERICAL mean:0.109943 min:0.00266793 max:0.351732 sd:0.0563731
    61: "coef_64" NUMERICAL mean:0.119207 min:0.00188382 max:0.427654 sd:0.0637886
    62: "coef_65" NUMERICAL mean:0.13092 min:0.00377491 max:0.371757 sd:0.0666428
    63: "coef_66" NUMERICAL mean:0.12369 min:0.00523224 max:0.394695 sd:0.0684848
    64: "coef_67" NUMERICAL mean:0.127944 min:0.00365267 max:0.418783 sd:0.0637265
    65: "coef_68" NUMERICAL mean:0.158167 min:0.00242352 max:0.47947 sd:0.0911508
    66: "coef_69" NUMERICAL mean:0.18051 min:0.0070905 max:0.490737 sd:0.0850876
    67: "coef_7" NUMERICAL mean:0.247664 min:0.00173686 max:0.52853 sd:0.123301
    68: "coef_70" NUMERICAL mean:0.148424 min:0.00377982 max:0.456282 sd:0.0866818
    69: "coef_71" NUMERICAL mean:0.165291 min:0.00778369 max:0.457965 sd:0.0910933
    70: "coef_72" NUMERICAL mean:0.231008 min:0.00397911 max:0.60137 sd:0.12293
    71: "coef_73" NUMERICAL mean:0.266279 min:0.0114335 max:0.620857 sd:0.13088
    72: "coef_74" NUMERICAL mean:0.144816 min:0.00113625 max:0.510432 sd:0.0856124
    73: "coef_75" NUMERICAL mean:0.216958 min:0.00457179 max:0.562397 sd:0.123788
    74: "coef_8" NUMERICAL mean:0.29034 min:0.010604 max:0.702383 sd:0.11784
    75: "coef_9" NUMERICAL mean:0.154272 min:0.00234154 max:0.584942 sd:0.0894681

CATEGORICAL: 1 (1.2987%)
    76: "__LABEL" CATEGORICAL integerized vocab-size:11 no-ood-item

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO kernel.cc:878] Configure learner
[WARNING gradient_boosted_trees.cc:1663] Subsample hyperparameter given but sampling method does not match.
[WARNING gradient_boosted_trees.cc:1676] GOSS alpha hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1685] GOSS beta hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1697] SelGB ratio hyperparameter given but SelGB is disabled.
[INFO kernel.cc:908] Training config:
learner: "GRADIENT_BOOSTED_TREES"
features: "coef_0"
features: "coef_1"
features: "coef_10"
features: "coef_11"
features: "coef_12"
features: "coef_13"
features: "coef_14"
features: "coef_15"
features: "coef_16"
features: "coef_17"
features: "coef_18"
features: "coef_19"
features: "coef_2"
features: "coef_20"
features: "coef_21"
features: "coef_22"
features: "coef_23"
features: "coef_24"
features: "coef_25"
features: "coef_26"
features: "coef_27"
features: "coef_28"
features: "coef_29"
features: "coef_3"
features: "coef_30"
features: "coef_31"
features: "coef_32"
features: "coef_33"
features: "coef_34"
features: "coef_35"
features: "coef_36"
features: "coef_37"
features: "coef_38"
features: "coef_39"
features: "coef_4"
features: "coef_40"
features: "coef_41"
features: "coef_42"
features: "coef_43"
features: "coef_44"
features: "coef_45"
features: "coef_46"
features: "coef_47"
features: "coef_48"
features: "coef_49"
features: "coef_5"
features: "coef_50"
features: "coef_51"
features: "coef_52"
features: "coef_53"
features: "coef_54"
features: "coef_55"
features: "coef_56"
features: "coef_57"
features: "coef_58"
features: "coef_59"
features: "coef_6"
features: "coef_60"
features: "coef_61"
features: "coef_62"
features: "coef_63"
features: "coef_64"
features: "coef_65"
features: "coef_66"
features: "coef_67"
features: "coef_68"
features: "coef_69"
features: "coef_7"
features: "coef_70"
features: "coef_71"
features: "coef_72"
features: "coef_73"
features: "coef_74"
features: "coef_75"
features: "coef_8"
features: "coef_9"
label: "__LABEL"
task: CLASSIFICATION
random_seed: 123456
metadata {
  framework: "TF Keras"
}
[yggdrasil_decision_forests.model.gradient_boosted_trees.proto.gradient_boosted_trees_config] {
  num_trees: 300
  decision_tree {
    max_depth: 6
    min_examples: 7
    in_split_min_examples_check: true
    keep_non_leaf_label_distribution: true
    missing_value_policy: GLOBAL_IMPUTATION
    allow_na_conditions: false
    categorical_set_greedy_forward {
      sampling: 0.1
      max_num_items: -1
      min_item_frequency: 1
    }
    growing_strategy_local {
    }
    categorical {
      cart {
      }
    }
    num_candidate_attributes_ratio: 0.9
    axis_aligned_split {
    }
    internal {
      sorting_strategy: PRESORTED
    }
    uplift {
      min_examples_in_treatment: 5
      split_score: KULLBACK_LEIBLER
    }
  }
  shrinkage: 0.05
  loss: DEFAULT
  validation_set_ratio: 0.1
  validation_interval_in_trees: 1
  early_stopping: VALIDATION_LOSS_INCREASE
  early_stopping_num_trees_look_ahead: 30
  l2_regularization: 0
  lambda_loss: 1
  mart {
  }
  adapt_subsample_for_maximum_training_duration: false
  l1_regularization: 0
  use_hessian_gain: true
  l2_regularization_categorical: 1
  apply_link_function: true
  compute_permutation_variable_importance: false
  binary_focal_loss_options {
    misprediction_exponent: 2
    positive_sample_coefficient: 0.5
  }
}

[INFO kernel.cc:911] Deployment config:
cache_path: "/tmpfs/tmp/tmpza24xjxv/working_cache"
num_threads: 32
try_resume_training: true

[INFO kernel.cc:940] Train model
[INFO gradient_boosted_trees.cc:405] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1004] Training gradient boosted tree on 1420 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1047] 1301 examples used for training and 119 examples used for validation
[INFO gradient_boosted_trees.cc:1430]     num-trees:1 train-loss:1.971125 train-accuracy:0.882398 valid-loss:2.031235 valid-accuracy:0.714286
[INFO gradient_boosted_trees.cc:1432]     num-trees:59 train-loss:0.061591 train-accuracy:1.000000 valid-loss:0.455276 valid-accuracy:0.831933
[INFO gradient_boosted_trees.cc:2863] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.455064
[INFO gradient_boosted_trees.cc:1475] Create final snapshot of the model at iteration 60
[INFO gradient_boosted_trees.cc:230] Truncates the model to 580 tree(s) i.e. 58  iteration(s).
[INFO gradient_boosted_trees.cc:264] Final model num-trees:58 valid-loss:0.455064 valid-accuracy:0.831933
[INFO kernel.cc:957] Export model in log directory: /tmpfs/tmp/tmpza24xjxv with prefix 9aef0a608b444d26
[INFO kernel.cc:974] Save model in resources
[INFO kernel.cc:1176] Loading model from path /tmpfs/tmp/tmpza24xjxv/model/ with prefix 9aef0a608b444d26
[INFO decision_forest.cc:639] Model loaded with 580 root(s), 23342 node(s), and 76 input feature(s).
[INFO abstract_model.cc:1246] Engine "GradientBoostedTreesGeneric" built
[INFO kernel.cc:1022] Use fast generic engine
Model trained in 0:00:04.133359
Compiling model...
Model compiled.
<keras.callbacks.History at 0x7fc610166ee0>

We can then evaluate the tuned model:

# Evaluate the model
best_model.compile(["accuracy"])
tuned_test_accuracy = best_model.evaluate(test_ds, return_dict=True, verbose=0)["accuracy"]
print(f"Test accuracy with the Keras Tuner: {tuned_test_accuracy:.4f}")
Test accuracy with the Keras Tuner: 0.8155