Warning: This project is deprecated. TensorFlow Addons has stopped development,
The project will only be providing minimal maintenance releases until May 2024. See the full
announcement here or on
github.
tfa.activations.rrelu
Stay organized with collections
Save and categorize content based on your preferences.
Randomized leaky rectified liner unit function.
tfa.activations.rrelu(
x: tfa.types.TensorLike,
lower: tfa.types.Number = 0.125,
upper: tfa.types.Number = 0.3333333333333333,
training: Optional[bool] = None,
seed: Optional[int] = None,
rng: Optional[tf.random.Generator] = None
) -> tf.Tensor
Computes rrelu function:
\[
\mathrm{rrelu}(x) =
\begin{cases}
x & \text{if } x > 0 \\
a x
\end{cases},
\]
where
\[
a \sim \mathcal{U}(\mathrm{lower}, \mathrm{upper})
\]
when training is True; or
\[
a = \frac{\mathrm{lower} + \mathrm{upper} }{2}
\]
when training is False.
See Empirical Evaluation of Rectified Activations in Convolutional Network.
Usage:
x = tf.constant([-1.0, 0.0, 1.0])
tfa.activations.rrelu(x, training=False)
<tf.Tensor: shape=(3,), dtype=float32, numpy=array([-0.22916667, 0. , 1. ], dtype=float32)>
tfa.activations.rrelu(x, training=True, seed=2020)
<tf.Tensor: shape=(3,), dtype=float32, numpy=array([-0.22631127, 0. , 1. ], dtype=float32)>
generator = tf.random.Generator.from_seed(2021)
tfa.activations.rrelu(x, training=True, rng=generator)
<tf.Tensor: shape=(3,), dtype=float32, numpy=array([-0.16031083, 0. , 1. ], dtype=float32)>
Args |
|---|
x
|
A Tensor. Must be one of the following types:
bfloat16, float16, float32, float64.
|
lower
|
float, lower bound for random alpha.
|
upper
|
float, upper bound for random alpha.
|
training
|
bool, indicating whether the call
is meant for training or inference.
|
seed
|
int, this sets the operation-level seed.
|
rng
|
A tf.random.Generator.
|
Returns |
|---|
result
|
A Tensor. Has the same type as x.
|
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2023-05-25 UTC.
[null,null,["Last updated 2023-05-25 UTC."],[],[],null,["# tfa.activations.rrelu\n\n|----------------------------------------------------------------------------------------------------------------------------|\n| [View source on GitHub](https://github.com/tensorflow/addons/blob/v0.20.0/tensorflow_addons/activations/rrelu.py#L21-L100) |\n\nRandomized leaky rectified liner unit function. \n\n tfa.activations.rrelu(\n x: ../../tfa/types/TensorLike,\n lower: ../../tfa/types/Number = 0.125,\n upper: ../../tfa/types/Number = 0.3333333333333333,\n training: Optional[bool] = None,\n seed: Optional[int] = None,\n rng: Optional[tf.random.Generator] = None\n ) -\u003e tf.Tensor\n\nComputes rrelu function:\n\n\\\\\\[\n\\\\mathrm{rrelu}(x) =\n\\\\begin{cases}\nx \\& \\\\text{if } x \\\u003e 0 \\\\\\\\\na x\n\\\\end{cases},\n\\\\\\]\n\nwhere\n\n\\\\\\[\na \\\\sim \\\\mathcal{U}(\\\\mathrm{lower}, \\\\mathrm{upper})\n\\\\\\]\n\nwhen `training` is `True`; or\n\n\\\\\\[\na = \\\\frac{\\\\mathrm{lower} + \\\\mathrm{upper} }{2}\n\\\\\\]\n\nwhen `training` is `False`.\n\nSee [Empirical Evaluation of Rectified Activations in Convolutional Network](https://arxiv.org/abs/1505.00853).\n\n#### Usage:\n\n x = tf.constant([-1.0, 0.0, 1.0])\n tfa.activations.rrelu(x, training=False)\n \u003ctf.Tensor: shape=(3,), dtype=float32, numpy=array([-0.22916667, 0. , 1. ], dtype=float32)\u003e\n tfa.activations.rrelu(x, training=True, seed=2020)\n \u003ctf.Tensor: shape=(3,), dtype=float32, numpy=array([-0.22631127, 0. , 1. ], dtype=float32)\u003e\n generator = tf.random.Generator.from_seed(2021)\n tfa.activations.rrelu(x, training=True, rng=generator)\n \u003ctf.Tensor: shape=(3,), dtype=float32, numpy=array([-0.16031083, 0. , 1. ], dtype=float32)\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ---- ||\n|------------|----------------------------------------------------------------------------------------------|\n| `x` | A `Tensor`. Must be one of the following types: `bfloat16`, `float16`, `float32`, `float64`. |\n| `lower` | `float`, lower bound for random alpha. |\n| `upper` | `float`, upper bound for random alpha. |\n| `training` | `bool`, indicating whether the `call` is meant for training or inference. |\n| `seed` | `int`, this sets the operation-level seed. |\n| `rng` | A [`tf.random.Generator`](https://www.tensorflow.org/api_docs/python/tf/random/Generator). |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Returns ------- ||\n|----------|---------------------------------------|\n| `result` | A `Tensor`. Has the same type as `x`. |\n\n\u003cbr /\u003e"]]
View source on GitHub