svd_coefficients
Wire format for prompt embeddings sent as the leading SVD coefficients of the output of a Stained Glass Core PrivacyEncoder.
The client compresses each embedding the noise layer transformed with the SVD of the privacy encoder's output (Stained Glass Core's
PrivacyEncoderSVD): it projects the embedding onto the leading K singular vectors, (y - mean) @ singular_vectors[:K].mT, for a K
it chooses, and transmits only those K coefficients. The vLLM plugin reconstructs hidden_size-dimensional embeddings from them as
coefficients @ singular_vectors[:K] + mean, with the same SVD configured on the server (SG_PRIVACY_ENCODER_SVD_PATH). The singular
vectors are the orthonormal rows of singular_vectors, sorted by decreasing singular value, so one SVD serves every K up to its number of
singular vectors.
Coefficients only mean something with the SVD they were computed with. Reconstructed with another SVD of the same shape, they yield wrong
embeddings, and the model's output degrades without any error. So the client sends the ID of its SVD, as
compute_svd_id returns it, with the coefficients, and the server refuses
coefficients computed with any SVD but its own.
Classes:
| Name | Description |
|---|---|
SVDCoefficientsPayload |
Wire format of a |
Functions:
| Name | Description |
|---|---|
compute_svd_id |
Compute the ID of an SVD, which the client sends with the coefficients it computes with the SVD. |
is_svd_coefficients_payload |
Whether a deserialized |
is_svd_id |
Whether a value is a well-formed SVD ID, as |
Attributes:
| Name | Type | Description |
|---|---|---|
SVD_COEFFICIENTS_KEY |
Final[str]
|
The key of the coefficients in an |
SVD_ID_KEY |
Final[str]
|
The key of the ID of the SVD the coefficients were computed with, in an |
SVD_COEFFICIENTS_KEY
module-attribute
¶
The key of the coefficients in an SVDCoefficientsPayload.
SVD_ID_KEY
module-attribute
¶
The key of the ID of the SVD the coefficients were computed with, in an
SVDCoefficientsPayload.
SVDCoefficientsPayload
¶
Bases: TypedDict
Wire format of a prompt_embeds payload that carries the leading SVD coefficients of each token.
Serialised with torch.save and sent as a base64-encoded prompt_embeds field, like a dense tensor or a
TurboQuantPayload.
svd_coefficients holds the leading K coefficients of each token, either as a dense floating-point tensor of shape (num_tokens, K),
or TurboQuant-compressed, with d=K. The number of coefficients K is the client's choice, and the server reads it from the payload.
svd_id is the ID of the SVD they were computed with, as
compute_svd_id returns it.
compute_svd_id
¶
compute_svd_id(
mean: Tensor, singular_vectors: Tensor
) -> str
Compute the ID of an SVD, which the client sends with the coefficients it computes with the SVD.
The ID is a digest of the SVD's mean and singular vectors, in float32: the two tensors both ends read from the SVD file. The file's other tensors and its metadata do not count, so two files that hold the same SVD have the same ID, however they were written. The ID only tells SVDs apart. It is no secret, and reveals nothing of the SVD.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
|
Tensor
|
The mean of the privacy encoder's output, of shape |
required |
|
Tensor
|
Its leading singular vectors, as rows, of shape |
required |
Returns:
| Type | Description |
|---|---|
str
|
The ID, as |
Examples:
is_svd_coefficients_payload
¶
Whether a deserialized prompt_embeds mapping is an SVD coefficients payload: one whose keys are svd_coefficients and svd_id.
The Inference Server decodes, and identifies, exactly the payloads this accepts. A payload with any other key is not one, rather than
one with a key to ignore, so that whatever a sender meant by that key is never silently dropped. Nor is one whose coefficients are
anything but a tensor or a TurboQuant payload: another SVD coefficients payload nested in one, in particular, would be reconstructed
with the SVD twice. Only the keys and the form of the coefficients are checked, not their values or the SVD ID, which the Inference
Server's decoder and prompt_embeds_id.compute_prompt_embeds_id check themselves.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
|
Mapping[str, object]
|
The payload mapping, as |
required |
Returns:
| Type | Description |
|---|---|
TypeGuard[SVDCoefficientsPayload]
|
Whether the keys of |
TypeGuard[SVDCoefficientsPayload]
|
keys are exactly those of a |
Examples:
is_svd_id
¶
Whether a value is a well-formed SVD ID, as compute_svd_id returns.
Lets the server refuse a malformed client-supplied ID before it compares it with its own, or names it in an error.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
|
object
|
Candidate ID, typically taken straight off the wire. |
required |
Returns:
| Type | Description |
|---|---|
TypeGuard[str]
|
Whether |
Examples: