model_remediation.min_diff.losses.AdjustedMMDLoss

Adjusted Maximum Mean Discrepancy between predictions on two groups of examples.

Inherits From: MMDLoss, MinDiffLoss

model_remediation.min_diff.losses.AdjustedMMDLoss(
    kernel='gaussian',
    predictions_transform=None,
    name: Optional[str] = None,
    enable_summary_histogram: Optional[bool] = True
)

kernel String (name of kernel) or losses.MinDiffKernel instance to be applied on the predictions. Defaults to 'gaussian' and it is recommended that this be either 'gaussian' (min_diff.losses.GaussianKernel) or 'laplacian' (min_diff.losses.LaplacianKernel).
predictions_transform Optional transform function to be applied to the predictions. This can be used to smooth out the distributions or limit the range of predictions.

The choice of whether to apply a transform to the predictions is task and data dependent. For example, for classifiers, it might make sense to apply a tf.sigmoid transform to the predictions (if this is not done already) so that MMD is calculated in probability space rather than on raw predictions. In some cases, such as regression, not having any transform is more likely to yield successful results.

name Name used for logging and tracking. Defaults to 'adjusted_mmd_loss'.
enable_summary_histogram Optional bool indicating if tf.summary.histogram should be included within the loss. Defaults to True.

The main motivation for adjusted MMDLoss is to capture variances of each membership's predictions. In the adjusted MMDLoss, we calculate the sum of variances of mean for each membership's prediction, and divide the original MMDLoss with the sum of variances. The adjustment works for any kernel.