Accuracy

The library contains utilities to estimate accuracy at a given noise scale and statistical significance level or derive the necessary noise scale to meet a given target accuracy and statistical significance level.

Note

This confidence interval is specifically for the input to the noise addition mechanism. The library currently does not compensate for the bias introduced from clipping or other preprocessing ([KMRS+23] shows that this is somewhat unavoidable). There is a notebook demonstrating this limitation:

• Accuracy: Pitfalls and Edge Cases

The noise distribution may be either Laplace or Gaussian.

Laplacian:

Applies to any L1 noise addition mechanism.

make_base_laplace()

make_base_discrete_laplace()

make_base_laplace_threshold()

Gaussian:

Applies to any L2 noise addition mechanism.

make_base_gaussian()

make_base_discrete_gaussian()

The library provides the following functions for converting between noise scale and accuracy:

• opendp.accuracy.laplacian_scale_to_accuracy()

• opendp.accuracy.accuracy_to_laplacian_scale()

• opendp.accuracy.gaussian_scale_to_accuracy()

• opendp.accuracy.accuracy_to_gaussian_scale()

To demonstrate, the following snippet finds the necessary gaussian scale such that the input to make_base_gaussian(input_domain, input_metric, scale=1.) differs from the release by no more than 2 with 95% confidence.

>>> from opendp.accuracy import accuracy_to_gaussian_scale
>>> confidence = 95
>>> accuracy_to_gaussian_scale(accuracy=2., alpha=1. - confidence / 100)
1.020426913849308

You can generally plug the distribution (Laplace or Gaussian), scale, accuracy and alpha into the following statement to interpret these functions:

f"When the {distribution} scale is {scale}, "
f"the DP estimate differs from the true value by no more than {accuracy} "
f"at a statistical significance level alpha of {alpha}, "
f"or with (1 - {alpha})100% = {(1 - alpha) * 100}% confidence."