Skip to content

svd_coefficients

Wire format for prompt embeddings sent as the leading SVD coefficients of the output of a Stained Glass Core PrivacyEncoder.

The client compresses each embedding the noise layer transformed with the SVD of the privacy encoder's output (Stained Glass Core's PrivacyEncoderSVD): it projects the embedding onto the leading K singular vectors, (y - mean) @ singular_vectors[:K].mT, for a K it chooses, and transmits only those K coefficients. The vLLM plugin reconstructs hidden_size-dimensional embeddings from them as coefficients @ singular_vectors[:K] + mean, with the same SVD configured on the server (SG_PRIVACY_ENCODER_SVD_PATH). The singular vectors are the orthonormal rows of singular_vectors, sorted by decreasing singular value, so one SVD serves every K up to its number of singular vectors.

Coefficients only mean something with the SVD they were computed with. Reconstructed with another SVD of the same shape, they yield wrong embeddings, and the model's output degrades without any error. So the client sends the ID of its SVD, as compute_svd_id returns it, with the coefficients, and the server refuses coefficients computed with any SVD but its own.

Classes:

Name Description
SVDCoefficientsPayload

Wire format of a prompt_embeds payload that carries the leading SVD coefficients of each token.

Functions:

Name Description
compute_svd_id

Compute the ID of an SVD, which the client sends with the coefficients it computes with the SVD.

is_svd_coefficients_payload

Whether a deserialized prompt_embeds mapping is an SVD coefficients payload: one whose keys are svd_coefficients and svd_id.

is_svd_id

Whether a value is a well-formed SVD ID, as compute_svd_id returns.

Attributes:

Name Type Description
SVD_COEFFICIENTS_KEY Final[str]

The key of the coefficients in an SVDCoefficientsPayload.

SVD_ID_KEY Final[str]

The key of the ID of the SVD the coefficients were computed with, in an

SVD_COEFFICIENTS_KEY module-attribute

SVD_COEFFICIENTS_KEY: Final[str] = 'svd_coefficients'

The key of the coefficients in an SVDCoefficientsPayload.

SVD_ID_KEY module-attribute

SVD_ID_KEY: Final[str] = 'svd_id'

The key of the ID of the SVD the coefficients were computed with, in an SVDCoefficientsPayload.

SVDCoefficientsPayload

Bases: TypedDict

Wire format of a prompt_embeds payload that carries the leading SVD coefficients of each token.

Serialised with torch.save and sent as a base64-encoded prompt_embeds field, like a dense tensor or a TurboQuantPayload.

svd_coefficients holds the leading K coefficients of each token, either as a dense floating-point tensor of shape (num_tokens, K), or TurboQuant-compressed, with d=K. The number of coefficients K is the client's choice, and the server reads it from the payload. svd_id is the ID of the SVD they were computed with, as compute_svd_id returns it.

compute_svd_id

compute_svd_id(
    mean: Tensor, singular_vectors: Tensor
) -> str

Compute the ID of an SVD, which the client sends with the coefficients it computes with the SVD.

The ID is a digest of the SVD's mean and singular vectors, in float32: the two tensors both ends read from the SVD file. The file's other tensors and its metadata do not count, so two files that hold the same SVD have the same ID, however they were written. The ID only tells SVDs apart. It is no secret, and reveals nothing of the SVD.

Parameters:

Name Type Description Default

mean

Tensor

The mean of the privacy encoder's output, of shape (hidden_size,).

required

singular_vectors

Tensor

Its leading singular vectors, as rows, of shape (svd_rank, hidden_size), as the SVD file holds them.

required

Returns:

Type Description
str

The ID, as "blake3:<hex digest>".

Examples:

>>> svd_id = compute_svd_id(torch.zeros(4), torch.eye(4)[:2])
>>> is_svd_id(svd_id)
True
>>> svd_id == compute_svd_id(torch.zeros(4), torch.eye(4)[:3])
False

is_svd_coefficients_payload

is_svd_coefficients_payload(
    payload: Mapping[str, object],
) -> TypeGuard[SVDCoefficientsPayload]

Whether a deserialized prompt_embeds mapping is an SVD coefficients payload: one whose keys are svd_coefficients and svd_id.

The Inference Server decodes, and identifies, exactly the payloads this accepts. A payload with any other key is not one, rather than one with a key to ignore, so that whatever a sender meant by that key is never silently dropped. Nor is one whose coefficients are anything but a tensor or a TurboQuant payload: another SVD coefficients payload nested in one, in particular, would be reconstructed with the SVD twice. Only the keys and the form of the coefficients are checked, not their values or the SVD ID, which the Inference Server's decoder and prompt_embeds_id.compute_prompt_embeds_id check themselves.

Parameters:

Name Type Description Default

payload

Mapping[str, object]

The payload mapping, as torch.load returns it.

required

Returns:

Type Description
TypeGuard[SVDCoefficientsPayload]

Whether the keys of payload are exactly svd_coefficients and svd_id, and its svd_coefficients is a tensor or a mapping whose

TypeGuard[SVDCoefficientsPayload]

keys are exactly those of a TurboQuantPayload.

Examples:

>>> payload = {
...     "svd_coefficients": torch.zeros(3, 2),
...     "svd_id": "blake3:" + "0" * 64,
... }
>>> is_svd_coefficients_payload(payload)
True
>>> is_svd_coefficients_payload({**payload, "svd_rank": 2})
False
>>> is_svd_coefficients_payload({**payload, "svd_coefficients": payload})
False

is_svd_id

is_svd_id(value: object) -> TypeGuard[str]

Whether a value is a well-formed SVD ID, as compute_svd_id returns.

Lets the server refuse a malformed client-supplied ID before it compares it with its own, or names it in an error.

Parameters:

Name Type Description Default

value

object

Candidate ID, typically taken straight off the wire.

required

Returns:

Type Description
TypeGuard[str]

Whether value is a string of the form "blake3:<hex digest>".

Examples:

>>> is_svd_id("blake3:" + "0" * 64)
True
>>> is_svd_id("sha256:" + "0" * 64)
False