tf_privacy.QuantileAdaptiveClipSumQuery

DPQuery for Gaussian sum queries with adaptive clipping.

Inherits From: SumAggregationDPQuery, DPQuery

tf_privacy.QuantileAdaptiveClipSumQuery(
    initial_l2_norm_clip, noise_multiplier, target_unclipped_quantile,
    learning_rate, clipped_count_stddev, expected_num_records, geometric_update=True
)

Clipping norm is tuned adaptively to converge to a value such that a specified quantile of updates are clipped, using the algorithm of Andrew et al. ( https://arxiv.org/abs/1905.03871). See the paper for details and suggested hyperparameter settings.

Args
initial_l2_norm_clip	The initial value of clipping norm.
noise_multiplier	The stddev of the noise added to the output will be this times the current value of the clipping norm.
target_unclipped_quantile	The desired quantile of updates which should be unclipped. I.e., a value of 0.8 means a value of l2_norm_clip should be found for which approximately 20% of updates are clipped each round. Andrew et al. recommends that this be set to 0.5 to clip to the median.
learning_rate	The learning rate for the clipping norm adaptation. With geometric updating, a rate of r means that the clipping norm will change by a maximum factor of exp(r) at each round. This maximum is attained when |actual_unclipped_fraction - target_unclipped_quantile| is 1.0. Andrew et al. recommends that this be set to 0.2 for geometric updating.
clipped_count_stddev	The stddev of the noise added to the clipped_count. Andrew et al. recommends that this be set to expected_num_records / 20 for reasonably fast adaptation and high privacy.
expected_num_records	The expected number of records per round, used to estimate the clipped count quantile.
geometric_update	If True, use geometric updating of clip (recommended).

Methods

accumulate_preprocessed_record

View source

accumulate_preprocessed_record(
    sample_state, preprocessed_record
)

Implements tensorflow_privacy.DPQuery.accumulate_preprocessed_record.

accumulate_record

View source

accumulate_record(
    params, sample_state, record
)

Accumulates a single record into the sample state.

This is a helper method that simply delegates to preprocess_record and accumulate_preprocessed_record for the common case when both of those functions run on a single device. Typically this will be a simple sum.

Args
params	The parameters for the sample. In standard DP-SGD training, the clipping norm for the sample's microbatch gradients (i.e., a maximum norm magnitude to which each gradient is clipped)
sample_state	The current sample state. In standard DP-SGD training, the accumulated sum of previous clipped microbatch gradients.
record	The record to accumulate. In standard DP-SGD training, the gradient computed for the examples in one microbatch, which may be the gradient for just one example (for size 1 microbatches).

Returns
The updated sample state. In standard DP-SGD training, the set of previous microbatch gradients with the addition of the record argument.

derive_metrics

View source

derive_metrics(
    global_state
)

Returns the current clipping norm as a metric.

derive_sample_params

View source

derive_sample_params(
    global_state
)

Implements tensorflow_privacy.DPQuery.derive_sample_params.

get_noised_result

View source

get_noised_result(
    sample_state, global_state
)

Implements tensorflow_privacy.DPQuery.get_noised_result.

initial_global_state

View source

initial_global_state()

Implements tensorflow_privacy.DPQuery.initial_global_state.

initial_sample_state

View source

initial_sample_state(
    template
)

Implements tensorflow_privacy.DPQuery.initial_sample_state.

merge_sample_states

View source

merge_sample_states(
    sample_state_1, sample_state_2
)

Implements tensorflow_privacy.DPQuery.merge_sample_states.

preprocess_record

View source

preprocess_record(
    params, record
)

Implements tensorflow_privacy.DPQuery.preprocess_record.