tf.tpu.experimental.AdamParameters
Stay organized with collections
Save and categorize content based on your preferences.
Optimization parameters for Adam with TPU embeddings.
tf.tpu.experimental.AdamParameters(
learning_rate, beta1=0.9, beta2=0.999, epsilon=1e-08, lazy_adam=True,
sum_inside_sqrt=True, use_gradient_accumulation=True, clip_weight_min=None,
clip_weight_max=None
)
Pass this to tf.estimator.tpu.experimental.EmbeddingConfigSpec via the
optimization_parameters argument to set the optimizer and its parameters.
See the documentation for tf.estimator.tpu.experimental.EmbeddingConfigSpec
for more details.
estimator = tf.estimator.tpu.TPUEstimator(
...
embedding_config_spec=tf.estimator.tpu.experimental.EmbeddingConfigSpec(
...
optimization_parameters=tf.tpu.experimental.AdamParameters(0.1),
...))
Args |
|---|
learning_rate
|
a floating point value. The learning rate.
|
beta1
|
A float value.
The exponential decay rate for the 1st moment estimates.
|
beta2
|
A float value.
The exponential decay rate for the 2nd moment estimates.
|
epsilon
|
A small constant for numerical stability.
|
lazy_adam
|
Use lazy Adam instead of Adam. Lazy Adam trains faster.
Please see optimization_parameters.proto for details.
|
sum_inside_sqrt
|
This improves training speed. Please see
optimization_parameters.proto for details.
|
use_gradient_accumulation
|
setting this to False makes embedding
gradients calculation less accurate but faster. Please see
optimization_parameters.proto for details.
for details.
|
clip_weight_min
|
the minimum value to clip by; None means -infinity.
|
clip_weight_max
|
the maximum value to clip by; None means +infinity.
|
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2020-10-01 UTC.
[null,null,["Last updated 2020-10-01 UTC."],[],[],null,["# tf.tpu.experimental.AdamParameters\n\n\u003cbr /\u003e\n\n|---------------------------------------------------------------------------------------------------------------------------------|\n| [View source on GitHub](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/python/tpu/tpu_embedding.py#L307-L373) |\n\nOptimization parameters for Adam with TPU embeddings.\n\n#### View aliases\n\n\n**Compat aliases for migration**\n\nSee\n[Migration guide](https://www.tensorflow.org/guide/migrate) for\nmore details.\n\n[`tf.compat.v1.tpu.experimental.AdamParameters`](/api_docs/python/tf/compat/v1/tpu/experimental/AdamParameters)\n\n\u003cbr /\u003e\n\n tf.tpu.experimental.AdamParameters(\n learning_rate, beta1=0.9, beta2=0.999, epsilon=1e-08, lazy_adam=True,\n sum_inside_sqrt=True, use_gradient_accumulation=True, clip_weight_min=None,\n clip_weight_max=None\n )\n\nPass this to [`tf.estimator.tpu.experimental.EmbeddingConfigSpec`](../../../tf/estimator/tpu/experimental/EmbeddingConfigSpec) via the\n`optimization_parameters` argument to set the optimizer and its parameters.\nSee the documentation for [`tf.estimator.tpu.experimental.EmbeddingConfigSpec`](../../../tf/estimator/tpu/experimental/EmbeddingConfigSpec)\nfor more details. \n\n estimator = tf.estimator.tpu.TPUEstimator(\n ...\n embedding_config_spec=tf.estimator.tpu.experimental.EmbeddingConfigSpec(\n ...\n optimization_parameters=tf.tpu.experimental.AdamParameters(0.1),\n ...))\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ---- ||\n|-----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `learning_rate` | a floating point value. The learning rate. |\n| `beta1` | A float value. The exponential decay rate for the 1st moment estimates. |\n| `beta2` | A float value. The exponential decay rate for the 2nd moment estimates. |\n| `epsilon` | A small constant for numerical stability. |\n| `lazy_adam` | Use lazy Adam instead of Adam. Lazy Adam trains faster. Please see `optimization_parameters.proto` for details. |\n| `sum_inside_sqrt` | This improves training speed. Please see `optimization_parameters.proto` for details. |\n| `use_gradient_accumulation` | setting this to `False` makes embedding gradients calculation less accurate but faster. Please see `optimization_parameters.proto` for details. for details. |\n| `clip_weight_min` | the minimum value to clip by; None means -infinity. |\n| `clip_weight_max` | the maximum value to clip by; None means +infinity. |\n\n\u003cbr /\u003e"]]
View source on GitHub