tf.compat.v1.distributions.Categorical

Categorical distribution.

Inherits From: Distribution

The Categorical distribution is parameterized by either probabilities or log-probabilities of a set of K classes. It is defined over the integers {0, 1, ..., K}.

The Categorical distribution is closely related to the OneHotCategorical and Multinomial distributions. The Categorical distribution can be intuited as generating samples according to argmax{ OneHotCategorical(probs) } itself being identical to argmax{ Multinomial(probs, total_count=1) }.

Mathematical Details

The probability mass function (pmf) is,

pmf(k; pi) = prod_j pi_j**[k == j]

Pitfalls

The number of classes, K, must not exceed:

  • the largest integer representable by self.dtype, i.e., 2**(mantissa_bits+1) (IEEE 754),
  • the maximum Tensor index, i.e., 2**31-1.

In other words,

K <= min(2**31-1, {
  tf.float16: 2**11,
  tf.float32: 2**24,
  tf.float64: 2**53 }[param.dtype])

Examples

Creates a 3-class distribution with the 2nd class being most likely.

dist = Categorical(probs=[0.1, 0.5, 0.4])
n = 1e4
empirical_prob = tf.cast(
    tf.histogram_fixed_width(
      dist.sample(int(n)),
      [0., 2],
      nbins=3),
    dtype=tf.float32) / n
# ==> array([ 0.1005,  0.5037,  0.3958], dtype=float32)

Creates a 3-class distribution with the 2nd class being most likely. Parameterized by logits rather than probabilities.

dist = Categorical(logits=np.log([0.1, 0.5, 0.4])
n = 1e4
empirical_prob = tf.cast(
    tf.histogram_fixed_width(
      dist.sample(int(n)),
      [0., 2],
      nbins=3),
    dtype=tf.float32) / n
# ==> array([0.1045,  0.5047, 0.3908], dtype=float32)

Creates a 3-class distribution with the 3rd class being most likely. The distribution functions can be evaluated on counts.

# counts is a scalar.
p = [0.1, 0.4, 0.5]
dist = Categorical(probs=p)
dist.prob(0)  # Shape []

# p will be broadcast to [[0.1, 0.4, 0.5], [0.1, 0.4, 0.5]] to match counts.
counts = [1, 0]
dist.prob(counts)  # Shape [2]

# p will be broadcast to shape [3, 5, 7, 3] to match counts.
counts = [[...]] # Shape [5, 7, 3]
dist.prob(counts)  # Shape [5, 7, 3]

logits An N-D Tensor, N >= 1, representing the log probabilities of a set of Categorical distributions. The first N - 1 dimensions index into a batch of independent distributions and the last dimension represents a vector of logits for each class. Only one of logits or probs should be passed in.
probs An N-D Tensor, N >= 1, representing the probabilities of a set of Categorical distributions. The first N - 1 dimensions index into a batch of independent distributions and the last dimension represents a vector of probabilities for each class. Only one of logits or probs should be passed in.
dtype The type of the event samples (default: int32).
validate_args Python bool, default False. When True distribution parameters are checked for validity despite possibly degrading runtime performance. When False invalid inputs may silently render incorrect outputs.
allow_nan_stats Python bool, default True. When True, statistics (e.g., mean, mode, variance) use the value "NaN" to indicate the result is undefined. When False, an exception is raised if one or more of the statistic's batch members are undefined.
name Python str name prefixed to Ops created by this class.

allow_nan_stats Python bool describing behavior when a stat is undefined.

Stats return +/- infinity when it makes sense. E.g., the variance of a Cauchy distribution is infinity. However, sometimes the statistic is undefined, e.g., if a distribution's pdf does not achieve a maximum within the support of the distribution, the mode is undefined. If the mean is undefined, then by definition the variance is undefined. E.g. the mean for Student's T for df = 1 is undefined (no clear way to say it is either + or - infinity), so the variance = E[(X - mean)**2] is also undefined.

batch_shape Shape of a single sample from a single event index as a TensorShape.

May be partially defined or unknown.

The batch dimensions are indexes into independent, non-identical parameterizations of this distribution.

dtype The DType of Tensors handled by this Distribution.
event_shape Shape of a single sample from a single batch as a TensorShape.

May be partially defined or unknown.

event_size Scalar int32 tensor: the number of classes.
logits Vector of coordinatewise logits.
name Name prepended to all ops created by this Distribution.
parameters Dictionary of parameters used to instantiate this Distribution.
probs Vector of coordinatewise probabilities.
reparameterization_type Describes how samples from the distribution are reparameterized.

Currently this is one of the static instances distributions.FULLY_REPARAMETERIZED or distributions.NOT_REPARAMETERIZED.

validate_args Python bool indicating possibly expensive checks are enabled.

Methods

batch_shape_tensor

View source

Shape of a single sample from a single event index as a 1-D Tensor.

The batch dimensions are indexes into independent, non-identical parameterizations of this distribution.

Args
name name to give to the op

Returns
batch_shape Tensor.

cdf

View source

Cumulative distribution function.

Given random variable X, the cumulative distribution function cdf is:

cdf(x) := P[X <= x]

Args
value float or double Tensor.
name Python str prepended to names of ops created by this function.

Returns
cdf a Tensor of shape sample_shape(x) + self.batch_shape with values of type self.dtype.

copy

View source

Creates a deep copy of the distribution.

Args
**override_parameters_kwargs String/value dictionary of initialization arguments to override with new values.

Returns
distribution A new instance of type(self) initialized from the union of self.parameters and override_parameters_kwargs, i.e., dict(self.parameters, **override_parameters_kwargs).

covariance

View source

Covariance.

Covariance is (possibly) defined only for non-scalar-event distributions.

For example, for a length-k, vector-valued distribution, it is calculated as,

Cov[i, j] = Covariance(X_i, X_j) = E[(X_i - E[X_i]) (X_j - E[X_j])]

where Cov is a (batch of) k x k matrix, 0 <= (i, j) < k, and E denotes expectation.

Alternatively, for non-vector, multivariate distributions (e.g., matrix-valued, Wishart), Covariance shall return a (batch of) matrices under some vectorization of the events, i.e.,

Cov[i, j] = Covariance(Vec(X)_i, Vec(X)_j) = [as above]

where Cov is a (batch of) k' x k' matrices, 0 <= (i, j) < k' = reduce_prod(event_shape), and Vec is some function mapping indices of this distribution's event dimensions to indices of a length-k' vector.

Args
name Python str prepended to names of ops created by this function.