TensorFlow 2.0 Beta is available Learn more

tf.contrib.training.HParams

View source on GitHub

Class HParams

Class to hold a set of hyperparameters as name-value pairs.

A HParams object holds hyperparameters used to build and train a model, such as the number of hidden units in a neural net layer or the learning rate to use when training.

You first create a HParams object by specifying the names and values of the hyperparameters.

To make them easily accessible the parameter names are added as direct attributes of the class. A typical usage is as follows:

# Create a HParams object specifying names and values of the model
# hyperparameters:
hparams = HParams(learning_rate=0.1, num_hidden_units=100)

# The hyperparameter are available as attributes of the HParams object:
hparams.learning_rate ==> 0.1
hparams.num_hidden_units ==> 100

Hyperparameters have type, which is inferred from the type of their value passed at construction type. The currently supported types are: integer, float, boolean, string, and list of integer, float, boolean, or string.

You can override hyperparameter values by calling the parse() method, passing a string of comma separated name=value pairs. This is intended to make it possible to override any hyperparameter values from a single command-line flag to which the user passes 'hyper-param=value' pairs. It avoids having to define one flag for each hyperparameter.

The syntax expected for each value depends on the type of the parameter. See parse() for a description of the syntax.

Example:

# Define a command line flag to pass name=value pairs.
# For example using argparse:
import argparse
parser = argparse.ArgumentParser(description='Train my model.')
parser.add_argument('--hparams', type=str,
                    help='Comma separated list of "name=value" pairs.')
args = parser.parse_args()
...
def my_program():
  # Create a HParams object specifying the names and values of the
  # model hyperparameters:
  hparams = tf.contrib.training.HParams(
      learning_rate=0.1,
      num_hidden_units=100,
      activations=['relu', 'tanh'])

  # Override hyperparameters values by parsing the command line
  hparams.parse(args.hparams)

  # If the user passed `--hparams=learning_rate=0.3` on the command line
  # then 'hparams' has the following attributes:
  hparams.learning_rate ==> 0.3
  hparams.num_hidden_units ==> 100
  hparams.activations ==> ['relu', 'tanh']

  # If the hyperparameters are in json format use parse_json:
  hparams.parse_json('{"learning_rate": 0.3, "activations": "relu"}')

__init__

View source

__init__(
    hparam_def=None,
    model_structure=None,
    **kwargs
)

Create an instance of HParams from keyword arguments.

The keyword arguments specify name-values pairs for the hyperparameters. The parameter types are inferred from the type of the values passed.

The parameter names are added as attributes of HParams object, so they can be accessed directly with the dot notation hparams._name_.

Example:

# Define 3 hyperparameters: 'learning_rate' is a float parameter,
# 'num_hidden_units' an integer parameter, and 'activation' a string
# parameter.
hparams = tf.contrib.training.HParams(
    learning_rate=0.1, num_hidden_units=100, activation='relu')

hparams.activation ==> 'relu'

Note that a few names are reserved and cannot be used as hyperparameter names. If you use one of the reserved name the constructor raises a ValueError.

Args:

  • hparam_def: Serialized hyperparameters, encoded as a hparam_pb2.HParamDef protocol buffer. If provided, this object is initialized by deserializing hparam_def. Otherwise **kwargs is used.
  • model_structure: An instance of ModelStructure, defining the feature crosses to be used in the Trial.
  • **kwargs: Key-value pairs where the key is the hyperparameter name and the value is the value for the parameter.

Raises:

  • ValueError: If both hparam_def and initialization values are provided, or if one of the arguments is invalid.

Methods

__contains__

View source

__contains__(key)

add_hparam

View source

add_hparam(
    name,
    value
)

Adds {name, value} pair to hyperparameters.

Args:

  • name: Name of the hyperparameter.
  • value: Value of the hyperparameter. Can be one of the following types: int, float, string, int list, float list, or string list.

Raises:

  • ValueError: if one of the arguments is invalid.

del_hparam

View source

del_hparam(name)

Removes the hyperparameter with key 'name'.

Does nothing if it isn't present.

Args:

  • name: Name of the hyperparameter.

from_proto

View source

@staticmethod
from_proto(
    hparam_def,
    import_scope=None
)

get

View source

get(
    key,
    default=None
)

Returns the value of key if it exists, else default.

get_model_structure

View source

get_model_structure()

override_from_dict

View source

override_from_dict(values_dict)

Override existing hyperparameter values, parsing new values from a dictionary.

Args:

  • values_dict: Dictionary of name:value pairs.

Returns:

The HParams instance.

Raises:

  • KeyError: If a hyperparameter in values_dict doesn't exist.
  • ValueError: If values_dict cannot be parsed.

parse

View source

parse(values)

Override existing hyperparameter values, parsing new values from a string.

See parse_values for more detail on the allowed format for values.

Args:

  • values: String. Comma separated list of name=value pairs where 'value' must follow the syntax described above.

Returns:

The HParams instance.

Raises:

  • ValueError: If values cannot be parsed or a hyperparameter in values doesn't exist.

parse_json

View source

parse_json(values_json)

Override existing hyperparameter values, parsing new values from a json object.

Args:

  • values_json: String containing a json object of name:value pairs.

Returns:

The HParams instance.

Raises:

  • KeyError: If a hyperparameter in values_json doesn't exist.
  • ValueError: If values_json cannot be parsed.

set_from_map

View source

set_from_map(values_map)

DEPRECATED. Use override_from_dict. (deprecated)

set_hparam

View source

set_hparam(
    name,
    value
)

Set the value of an existing hyperparameter.

This function verifies that the type of the value matches the type of the existing hyperparameter.

Args:

  • name: Name of the hyperparameter.
  • value: New value of the hyperparameter.

Raises:

  • KeyError: If the hyperparameter doesn't exist.
  • ValueError: If there is a type mismatch.

set_model_structure

View source

set_model_structure(model_structure)

to_json

View source

to_json(
    indent=None,
    separators=None,
    sort_keys=False
)

Serializes the hyperparameters into JSON.

Args:

  • indent: If a non-negative integer, JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0, or negative, will only insert newlines. None (the default) selects the most compact representation.
  • separators: Optional (item_separator, key_separator) tuple. Default is (', ', ': ').
  • sort_keys: If True, the output dictionaries will be sorted by key.

Returns:

A JSON string.

to_proto

View source

to_proto(export_scope=None)

Converts a HParams object to a HParamDef protocol buffer.

Args:

  • export_scope: Optional string. Name scope to remove.

Returns:

A HParamDef protocol buffer.

values

View source

values()

Return the hyperparameter values as a Python dictionary.

Returns:

A dictionary with hyperparameter names as keys. The values are the hyperparameter values.