Skip to content

privacy_encoder

Module for the loss that trains a PrivacyEncoder.

A privacy encoder is a PostProcessor trained against a frozen Stained Glass Transform, jointly with adapters on the base model that recover the utility the encoder's extra distortion costs. Its loss combines two terms computed from two differently batched inputs:

  • A utility term, the temperature-scaled KL divergence from a teacher to a student over the positions selected by the loss mask, computed on full sequences from their final hidden states and the base model's lm_head weight, without materializing the logits. The teacher and student hidden states come from NoisyModel.teacher_and_student_forward.
  • A privacy term, the per-token sliced_wasserstein_distance between the encoder's output and a token-independent reference distribution, computed on many transformed samples of each of a batch of tokens. A sequence batch does not contain enough samples of any one token's distribution, so these are supplied separately.

The two are combined as KL / (1 + lambda) + lambda * SWD / (1 + lambda), where the ratio lambda is passed on every call so that it can follow a schedule.

Classes:

Name Description
PrivacyEncoderDistillationLossFunction

The loss function returned by

Functions:

Name Description
privacy_encoder_distillation_loss_factory

Create the loss function that trains the post-processor of a noisy model's noise layer, such as a

PrivacyEncoderDistillationLossFunction

Bases: Protocol

The loss function returned by privacy_encoder_distillation_loss_factory.

Methods:

Name Description
__call__

Compute the privacy encoder's composite loss.

__call__

Compute the privacy encoder's composite loss.

Parameters:

Name Type Description Default

teacher_output

Tensor

The teacher's final hidden states of shape (batch_size, sequence_length, hidden_size), i.e. the input of the base model's lm_head. Gradients never flow into it.

required

student_output

Tensor

The student's final hidden states, of the same shape as teacher_output.

required

loss_mask

Tensor

A boolean mask of shape (batch_size, sequence_length) selecting the positions to compute the utility term over.

required

token_samples

Tensor

Noise layer outputs, before the post-processor, of shape (num_tokens, num_samples, dim): num_samples transformed samples of each of num_tokens tokens.

required

sliced_wasserstein_distance_weight

float

The ratio lambda of the privacy term's weight to the utility term's weight. The terms are combined as KL / (1 + lambda) + lambda * SWD / (1 + lambda).

required

update_sliced_wasserstein_target

bool

Whether to update the factory's sliced_wasserstein_target from token_samples, and refresh its Cholesky factor if due, before drawing the reference. Pass False to compute the loss, e.g. on validation data, without changing the target. Calling the loss function under torch.no_grad does not prevent the update. Has no effect without a sliced_wasserstein_target.

True

Returns:

Type Description
torch.Tensor

The scalar composite loss.

privacy_encoder_distillation_loss_factory

privacy_encoder_distillation_loss_factory(
    noisy_model: NoisyModel[Any, ..., Any],
    kl_temperature: float,
    num_projections: int,
    *,
    use_jeffreys_divergence: bool,
    sliced_wasserstein_target: (
        MatchedGaussianTarget | None
    ) = None,
    fused_linear_divergence_process_group: (
        ProcessGroup | None
    ) = None
) -> tuple[
    PrivacyEncoderDistillationLossFunction,
    Callable[[], sg_transform_loss.ComponentLossesDict],
    Callable[[], sg_transform_loss.HyperparametersDict],
]

Create the loss function that trains the post-processor of a noisy model's noise layer, such as a PrivacyEncoder, against a frozen Stained Glass Transform.

The returned loss function computes:

  • The utility term kl_temperature ** 2 * KL(softmax(teacher / kl_temperature) || softmax(student / kl_temperature)), averaged over the positions loss_mask selects, or the Jeffreys divergence KL(teacher || student) + KL(student || teacher) in its place if use_jeffreys_divergence. The logits are the teacher's and the student's final hidden states projected by the base model's lm_head weight. They are computed in float32, a chunk of positions at a time, and never materialized in full; see fused_linear_masked_generalized_jsd.
  • The privacy term, the per-token sliced Wasserstein distance between the post-processor's output on token_samples and a fresh draw from the reference distribution: a standard normal, or, if sliced_wasserstein_target is given, a zero-mean Gaussian whose covariance matches the running covariance of token_samples. Every call updates the target from token_samples themselves, i.e. from the embeddings before the post-processor, never from its output, so that it does not chase the post-processor's own distribution. Pass update_sliced_wasserstein_target=False to compute the loss, e.g. on validation data, without updating it.
  • Their weighted sum KL / (1 + lambda) + lambda * SWD / (1 + lambda), where lambda is the sliced_wasserstein_distance_weight passed on each call. The privacy term is computed and recorded on every call, but only added to the sum if its weight is non-zero.
Note

The lm_head weight is captured by a forward hook on the base model's lm_head, which must therefore be called during the forward pass, e.g. with its projection replaced by the identity so that the base model returns the final hidden states. The hook is removed when the loss function is garbage collected. The lm_head must be a torch.nn.Linear without a bias, or a TiedLinear. The teacher and the student must use the same lm_head weight, and adapters on the lm_head must be merged into it.

Note

The post-processor is called directly on token_samples, outside of the noise layer's forward pass. Under fully sharded data parallel training it must therefore be wrapped as its own sharded unit, or its parameters will still be sharded when it is called.

Warning

The projection directions and reference samples are drawn from the global random number generator of the device token_samples live on. Under data-parallel training they, and the sliced_wasserstein_target statistics, are local to each rank; do not synchronize them. See MatchedGaussianTarget for how to checkpoint the target. The target's statistics summarize user embeddings and are as sensitive as any other Stained Glass Transform parameter: never log them.

Parameters:

Name Type Description Default

noisy_model

NoisyModel[Any, ..., Any]

The noisy model whose noise layer has the post-processor to train. Its noise layer is expected to be frozen.

required

kl_temperature

float

The softmax temperature of the utility term.

required

num_projections

int

The number of random directions the sliced Wasserstein distance projects onto.

required

use_jeffreys_divergence

bool

Whether the utility term is the Jeffreys divergence rather than the forward KL divergence.

required

sliced_wasserstein_target

MatchedGaussianTarget | None

The running Gaussian to draw the privacy term's reference samples from. If None, they are drawn from a standard normal. It must be on the device token_samples will be on.

None

fused_linear_divergence_process_group

ProcessGroup | None

An optional tensor-parallel process group over which to shard the token positions of the utility term. See fused_linear_masked_generalized_jsd.

None

Returns:

Type Description
tuple[PrivacyEncoderDistillationLossFunction, Callable[[], sg_transform_loss.ComponentLossesDict], Callable[[], sg_transform_loss.HyperparametersDict]]

A tuple of 3 functions: the loss function, a function to retrieve the loss components, and a function to retrieve the

tuple[PrivacyEncoderDistillationLossFunction, Callable[[], sg_transform_loss.ComponentLossesDict], Callable[[], sg_transform_loss.HyperparametersDict]]

hyperparameters. The latter two return the values recorded by the most recent call to the loss function, and may be called at most

tuple[PrivacyEncoderDistillationLossFunction, Callable[[], sg_transform_loss.ComponentLossesDict], Callable[[], sg_transform_loss.HyperparametersDict]]

once each after it.

Raises:

Type Description
ValueError

If the noisy model's noise layer has no post-processor.

ValueError

If kl_temperature or num_projections is not strictly positive.

ValueError

If the base model has no output embeddings, or its lm_head has a bias.

TypeError

If the base model's lm_head is neither a torch.nn.Linear nor a TiedLinear, e.g. because an adapter wraps it.

Examples:

One training step, with the lm_head's projection replaced by the identity so that the forward pass returns the final hidden states, and with token_samples holding many noise layer outputs of each of a batch of tokens:

>>> loss_function, get_losses, get_hyperparameters = (
...     privacy_encoder_distillation_loss_factory(
...         noisy_model,
...         kl_temperature=2.0,
...         num_projections=64,
...         use_jeffreys_divergence=False,
...     )
... )
>>> lm_head = noisy_model.base_model.get_output_embeddings()
>>> with unittest.mock.patch.object(
...     lm_head, "forward", lambda hidden_states: hidden_states
... ):
...     teacher_output, student_output = noisy_model.teacher_and_student_forward(
...         input_ids=input_ids,
...         attention_mask=attention_mask,
...         noise_mask=noise_mask,
...         teacher_context=frozen_teacher,
...     )
>>> loss = loss_function(
...     teacher_output.logits,
...     student_output.logits,
...     loss_mask,
...     token_samples,
...     sliced_wasserstein_distance_weight=10.0,
... )
>>> loss.backward()

Warning

This API is experimental and subject to change: The privacy encoder loss factory API is currently under development and is subject to change.

Added in version v4.7.0.