entropy

Functions:

Name	Description
gaussian_mixture_entropy	Compute the entropy of a Gaussian mixture on a mini batch via Monte Carlo integration.
square_mahalanobis_distance	Calculate the square Mahalanobis Distance between the features of x and y given the diagonal tensor of standard deviations.

gaussian_mixture_entropy

gaussian_mixture_entropy(
    clean_embeddings: Tensor,
    transformed_embeddings: Tensor,
    means: Tensor,
    stds: Tensor,
    mask: Tensor,
    losses: ComponentLossesDict | None = None,
) -> torch.Tensor

Compute the entropy of a Gaussian mixture on a mini batch via Monte Carlo integration.

Note: The transformed embeddings should be a different batch than the clean embeddings.

Parameters:

Name	Type	Description	Default
clean_embeddings	Tensor	The untransformed embeddings.	required
transformed_embeddings	Tensor	The stained glass transformed embeddings.	required
means	Tensor	The means tensor.	required
stds	Tensor	The standard deviations tensor.	required
mask	Tensor	The mask to apply to the reduction.	required
losses	ComponentLossesDict | None	The dictionary of component losses. If provided, the loss will be added to the losses dictionary.	None

Returns:

Type	Description
torch.Tensor	The computed Gaussian mixture entropy.

Warning

This API is experimental and subject to change: The entropy component of the mutual information loss is still under research.

square_mahalanobis_distance

square_mahalanobis_distance(
    x: Tensor, y: Tensor, stds: Tensor, mask: Tensor
) -> torch.Tensor

Calculate the square Mahalanobis Distance between the features of x and y given the diagonal tensor of standard deviations.

Notes

Adds a normalization factor of 0.5 to the square Mahalanobis distance calculation for the variance to align with the Gaussian distribution.

Parameters:

Name	Type	Description	Default
x	Tensor	The first tensor.	required
y	Tensor	The second tensor.	required
stds	Tensor	The tensor of standard deviations.	required
mask	Tensor	The mask to apply to the reduction.	required

Returns:

Type	Description
torch.Tensor	The computed square Mahalanobis distance between x and y.

Warning

This API is experimental and subject to change: The entropy component of the mutual information loss is still under research.