vision

ActivationImages

Bases: TypedDict

A dictionary of images, comprised of a grid of input images and their corresponding activations.

activation instance-attribute

activation: Tensor

An image tensor, formatted as a grid, where each row contains an input image and its corresponding noise layer output.

mask instance-attribute

mask: NotRequired[Tensor]

An image tensor, formatted as a grid, where each row contains an input image and its corresponding noise layer mask.

masked_std instance-attribute

masked_std: NotRequired[Tensor]

An image tensor, formatted as a grid, where each row contains an input image and its corresponding noise layer mask applied to the standard deviation.

mean instance-attribute

mean: Tensor

An image tensor, formatted as a grid, where each row contains an input image and its corresponding noise layer mean.

std instance-attribute

std: Tensor

An image tensor, formatted as a grid, where each row contains an input image and its corresponding noise layer standard deviation.

TensorsToVisualize

Bases: TypedDict

A dictionary of tensors to process into a visualization.

activation instance-attribute

activation: Tensor

The noise layer activation.

input instance-attribute

input: Tensor

The input to the model.

mask instance-attribute

mask: NotRequired[Tensor]

The noise layer mask.

masked_std instance-attribute

masked_std: NotRequired[Tensor]

The masked noise layer standard deviation.

mean instance-attribute

mean: Tensor

The noise layer mean.

std instance-attribute

std: Tensor

The noise layer standard deviation.

TransformedImageVisualizationManager

Captures NoisyModel input images and intermediate activations and creates visualizations, formatted as a grid of images where each row contains an input image and its corresponding activation.

grids property

grids: ActivationImages

The tensors representing the grid of images to visualize.

__init__

__init__(
    noisy_model: NoisyModel[ModuleT, ..., NoiseLayerT],
    input_name: str | None = None,
    max_examples: int = 4,
    max_color_channels: int = 3,
) -> None

Construct a new TransformedImageVisualizationManager.

Parameters:

Name	Type	Description	Default
noisy_model	NoisyModel[ModuleT, ..., NoiseLayerT]	The model to visualize input images and activations for.	required
input_name	str | None	The name of the noisy_model input Tensor argument. If noisy_model's target_parameter is set, it will be used by default, otherwise, this argument is required.	None
max_examples	int	The maximum number of rows to display in the visualizations.	4
max_color_channels	int	The maximum number of individual color channels to additionally display in each visualization. If the target layer has more than 3 color channels, these will be displayed in grayscale.	3

Raises:

Type	Description
ValueError	If noisy_model has no target_parameter and input_name is not provided.
ValueError	If noisy_model has a target_parameter and input_name is provided but does not match. In this case, you do not need to provide input_name.

prepare_activation_images

prepare_activation_images() -> ActivationImages

Collect the input and activation tensors from the most recent forward pass and populate them into a grid of images for each activation.

Returns:

Type	Description
ActivationImages	A dictionary of image tensors, each formatted as a grid of images, where each row corresponds to a specific example.