![]() |
Categorical distribution.
Inherits From: Distribution
tf.compat.v1.distributions.Categorical(
logits=None, probs=None, dtype=tf.dtypes.int32, validate_args=False,
allow_nan_stats=True, name='Categorical'
)
The Categorical distribution is parameterized by either probabilities or
log-probabilities of a set of K
classes. It is defined over the integers
{0, 1, ..., K}
.
The Categorical distribution is closely related to the OneHotCategorical
and
Multinomial
distributions. The Categorical distribution can be intuited as
generating samples according to argmax{ OneHotCategorical(probs) }
itself
being identical to argmax{ Multinomial(probs, total_count=1) }
.
Mathematical Details
The probability mass function (pmf) is,
pmf(k; pi) = prod_j pi_j**[k == j]
Pitfalls
The number of classes, K
, must not exceed:
- the largest integer representable by
self.dtype
, i.e.,2**(mantissa_bits+1)
(IEEE 754), - the maximum
Tensor
index, i.e.,2**31-1
.
In other words,
K <= min(2**31-1, {
tf.float16: 2**11,
tf.float32: 2**24,
tf.float64: 2**53 }[param.dtype])
Examples
Creates a 3-class distribution with the 2nd class being most likely.
dist = Categorical(probs=[0.1, 0.5, 0.4])
n = 1e4
empirical_prob = tf.cast(
tf.histogram_fixed_width(
dist.sample(int(n)),
[0., 2],
nbins=3),
dtype=tf.float32) / n
# ==> array([ 0.1005, 0.5037, 0.3958], dtype=float32)
Creates a 3-class distribution with the 2nd class being most likely. Parameterized by logits rather than probabilities.
dist = Categorical(logits=np.log([0.1, 0.5, 0.4])
n = 1e4
empirical_prob = tf.cast(
tf.histogram_fixed_width(
dist.sample(int(n)),
[0., 2],
nbins=3),
dtype=tf.float32) / n
# ==> array([0.1045, 0.5047, 0.3908], dtype=float32)
Creates a 3-class distribution with the 3rd class being most likely. The distribution functions can be evaluated on counts.
# counts is a scalar.
p = [0.1, 0.4, 0.5]
dist = Categorical(probs=p)
dist.prob(0) # Shape []
# p will be broadcast to [[0.1, 0.4, 0.5], [0.1, 0.4, 0.5]] to match counts.
counts = [1, 0]
dist.prob(counts) # Shape [2]
# p will be broadcast to shape [3, 5, 7, 3] to match counts.
counts = [[...]] # Shape [5, 7, 3]
dist.prob(counts) # Shape [5, 7, 3]
Args | |
---|---|
logits
|
An N-D Tensor , N >= 1 , representing the log probabilities
of a set of Categorical distributions. The first N - 1 dimensions
index into a batch of independent distributions and the last dimension
represents a vector of logits for each class. Only one of logits or
probs should be passed in.
|
probs
|
An N-D Tensor , N >= 1 , representing the probabilities
of a set of Categorical distributions. The first N - 1 dimensions
index into a batch of independent distributions and the last dimension
represents a vector of probabilities for each class. Only one of
logits or probs should be passed in.
|
dtype
|
The type of the event samples (default: int32). |
validate_args
|
Python bool , default False . When True distribution
parameters are checked for validity despite possibly degrading runtime
performance. When False invalid inputs may silently render incorrect
outputs.
|
allow_nan_stats
|
Python bool , default True . When True , statistics
(e.g., mean, mode, variance) use the value "NaN " to indicate the
result is undefined. When False , an exception is raised if one or
more of the statistic's batch members are undefined.
|
name
|
Python str name prefixed to Ops created by this class.
|
Attributes | |
---|---|
allow_nan_stats
|
Python bool describing behavior when a stat is undefined.
Stats return +/- infinity when it makes sense. E.g., the variance of a Cauchy distribution is infinity. However, sometimes the statistic is undefined, e.g., if a distribution's pdf does not achieve a maximum within the support of the distribution, the mode is undefined. If the mean is undefined, then by definition the variance is undefined. E.g. the mean for Student's T for df = 1 is undefined (no clear way to say it is either + or - infinity), so the variance = E[(X - mean)**2] is also undefined. |
batch_shape
|
Shape of a single sample from a single event index as a TensorShape .
May be partially defined or unknown. The batch dimensions are indexes into independent, non-identical parameterizations of this distribution. |
dtype
|
The DType of Tensor s handled by this Distribution .
|
event_shape
|
Shape of a single sample from a single batch as a TensorShape .
May be partially defined or unknown. |
event_size
|
Scalar int32 tensor: the number of classes.
|
logits
|
Vector of coordinatewise logits. |
name
|
Name prepended to all ops created by this Distribution .
|
parameters
|
Dictionary of parameters used to instantiate this Distribution .
|
probs
|
Vector of coordinatewise probabilities. |
reparameterization_type
|
Describes how samples from the distribution are reparameterized.
Currently this is one of the static instances
|
validate_args
|
Python bool indicating possibly expensive checks are enabled.
|
Methods
batch_shape_tensor
batch_shape_tensor(
name='batch_shape_tensor'
)
Shape of a single sample from a single event index as a 1-D Tensor
.
The batch dimensions are indexes into independent, non-identical parameterizations of this distribution.
Args | |
---|---|
name
|
name to give to the op |
Returns | |
---|---|
batch_shape
|
Tensor .
|
cdf
cdf(
value, name='cdf'
)
Cumulative distribution function.
Given random variable X
, the cumulative distribution function cdf
is:
cdf(x) := P[X <= x]
Args | |
---|---|
value
|
float or double Tensor .
|
name
|
Python str prepended to names of ops created by this function.
|
Returns | |
---|---|
cdf
|
a Tensor of shape sample_shape(x) + self.batch_shape with
values of type self.dtype .
|
copy
copy(
**override_parameters_kwargs
)
Creates a deep copy of the distribution.
Args | |
---|---|
**override_parameters_kwargs
|
String/value dictionary of initialization arguments to override with new values. |
Returns | |
---|---|
distribution
|
A new instance of type(self) initialized from the union
of self.parameters and override_parameters_kwargs, i.e.,
dict(self.parameters, **override_parameters_kwargs) .
|
covariance
covariance(
name='covariance'
)
Covariance.
Covariance is (possibly) defined only for non-scalar-event distributions.
For example, for a length-k
, vector-valued distribution, it is calculated
as,
Cov[i, j] = Covariance(X_i, X_j) = E[(X_i - E[X_i]) (X_j - E[X_j])]
where Cov
is a (batch of) k x k
matrix, 0 <= (i, j) < k
, and E
denotes expectation.
Alternatively, for non-vector, multivariate distributions (e.g.,
matrix-valued, Wishart), Covariance
shall return a (batch of) matrices
under some vectorization of the events, i.e.,
Cov[i, j] = Covariance(Vec(X)_i, Vec(X)_j) = [as above]
where Cov
is a (batch of) k' x k'
matrices,
0 <= (i, j) < k' = reduce_prod(event_shape)
, and Vec
is some function
mapping indices of this distribution's event dimensions to indices of a
length-k'
vector.
Args | |
---|---|
name
|
Python str prepended to names of ops created by this function.
|
Returns | |
---|---|
covariance
|
Floating-point Tensor with shape [B1, ..., Bn, k', k']
where the first n dimensions are batch coordinates and
k' = reduce_prod(self.event_shape) .
|
cross_entropy
cross_entropy(
other, name='cross_entropy'
)
Computes the (Shannon) cross entropy.
Denote this distribution (self
) by P
and the other
distribution by
Q
. Assuming P, Q
are absolutely continuous with respect to
one another and permit densities p(x) dr(x)
and q(x) dr(x)
, (Shanon)
cross entropy is defined as:
H[P, Q] = E_p[-log q(X)] = -int_F p(x) log q(x) dr(x)
where F
denotes the support of the random variable X ~ P
.
Args | |
---|---|
other
|
tfp.distributions.Distribution instance.
|
name
|
Python str prepended to names of ops created by this function.
|
Returns | |
---|---|
cross_entropy
|
self.dtype Tensor with shape [B1, ..., Bn]
representing n different calculations of (Shanon) cross entropy.
|
entropy
entropy(
name='entropy'
)
Shannon entropy in nats.
event_shape_tensor
event_shape_tensor(
name='event_shape_tensor'
)
Shape of a single sample from a single batch as a 1-D int32 Tensor
.
Args | |
---|---|
name
|
name to give to the op |
Returns | |
---|---|
event_shape
|
Tensor .
|