prompt_embeds_caching_protocol

The prompt_embeds chat content part, as Output Protection extends it, and the errors a server answers it with.

This is the contract between a prompt-embeds client and an Output Protection server, and it lives here rather than on the server side because both ends have to agree on it. A client builds these parts; the server reads them. Pair it with compute_prompt_embeds_id, which produces the identifiers they carry.

vLLM already defines a content part for uploading embeddings:

class ChatCompletionContentPartPromptEmbedsParam(TypedDict, total=False):
    data: Required[str]
    type: Required[Literal["prompt_embeds"]]

Output Protection adds one key, id, and with it two shapes vLLM's type cannot express.

Why these are siblings of vLLM's type rather than subclasses of it

A reference carries no embeddings, so its data is None, but vLLM declares data: Required[str], and a TypedDict subclass may not change the type of an inherited item.

Type Aliases:

Name	Description
PromptEmbedsContentPart	Any prompt_embeds content part an Output Protection server accepts.

Classes:

Name	Description
ChatCompletionContentPartCachedPromptEmbedsParam	Embeddings named rather than sent, for a server that already holds them.
ChatCompletionContentPartUploadedPromptEmbedsParam	Embeddings sent in full, labelled with the identifier the client computed for them.
PromptEmbedsErrorBody	Envelope the errors above arrive in, as {"error": {...}}.
PromptEmbedsNotCachedError	Body of the 409 answering a reference the server cannot serve.

Functions:

Name	Description
cached_prompt_embeds_part	Build a part that references embeddings the server already holds.
uploaded_prompt_embeds_part	Build a part that uploads embeddings and labels them.

Attributes:

Name	Type	Description
INVALID_PROMPT_EMBEDS_PART_CODE	Final[str]	400. The part cannot be acted on: it carries neither data nor id, or an id that is not an identifier.
PROMPT_EMBEDS_ID_MISMATCH_CODE	Final[str]	400. An upload's claimed id is not the one the server computed, so the two ends disagree and references would never hit.
PROMPT_EMBEDS_NOT_CACHED_CODE	Final[str]	409. One or more referenced identifiers are not held. The client should re-upload them and retry.
PROMPT_EMBEDS_PART_TYPE	Final[str]	Value of a content part's type that marks it as carrying prompt embeddings. Unchanged from vLLM's.

INVALID_PROMPT_EMBEDS_PART_CODE module-attribute

INVALID_PROMPT_EMBEDS_PART_CODE: Final[str] = (
    "invalid_prompt_embeds_part"
)

400. The part cannot be acted on: it carries neither data nor id, or an id that is not an identifier.

PROMPT_EMBEDS_ID_MISMATCH_CODE module-attribute

PROMPT_EMBEDS_ID_MISMATCH_CODE: Final[str] = (
    "prompt_embeds_id_mismatch"
)

400. An upload's claimed id is not the one the server computed, so the two ends disagree and references would never hit.

PROMPT_EMBEDS_NOT_CACHED_CODE module-attribute

PROMPT_EMBEDS_NOT_CACHED_CODE: Final[str] = (
    "prompt_embeds_not_cached"
)

409. One or more referenced identifiers are not held. The client should re-upload them and retry.

PROMPT_EMBEDS_PART_TYPE module-attribute

PROMPT_EMBEDS_PART_TYPE: Final[str] = 'prompt_embeds'

Value of a content part's type that marks it as carrying prompt embeddings. Unchanged from vLLM's.

PromptEmbedsContentPart

PromptEmbedsContentPart = (
    ChatCompletionContentPartPromptEmbedsParam
    | ChatCompletionContentPartUploadedPromptEmbedsParam
    | ChatCompletionContentPartCachedPromptEmbedsParam
)

Any prompt_embeds content part an Output Protection server accepts.

ChatCompletionContentPartCachedPromptEmbedsParam

Bases: TypedDict

Embeddings named rather than sent, for a server that already holds them.

Both forms are accepted:

{"type": "prompt_embeds", "id": "blake3:..."}
{"type": "prompt_embeds", "data": null, "id": "blake3:..."}

Attributes:

Name	Type	Description
data	None	None when present, and may be left out entirely. The embeddings are named by id rather than carried.
id	Required[str]	Content identifier of the embeddings being referenced, as <algorithm>:<hex digest>.
type	Required[Literal['prompt_embeds']]	The type of the content part.

data instance-attribute

data: None

None when present, and may be left out entirely. The embeddings are named by id rather than carried.

id instance-attribute

id: Required[str]

Content identifier of the embeddings being referenced, as <algorithm>:<hex digest>.

type instance-attribute

type: Required[Literal['prompt_embeds']]

The type of the content part.

ChatCompletionContentPartUploadedPromptEmbedsParam

Bases: TypedDict

Embeddings sent in full, labelled with the identifier the client computed for them.

The first turn of a conversation, and every turn carrying a message the server has not seen. Identical to vLLM's part but for the id, which a server without the cache ignores.

Attributes:

Name	Type	Description
data	Required[str]	Base64-encoded bytes of a serialized torch.Tensor of shape (num_tokens, hidden_size), in the model's dtype.
id	Required[str]	Content identifier of those embeddings, as <algorithm>:<hex digest>.
type	Required[Literal['prompt_embeds']]	The type of the content part.

data instance-attribute

data: Required[str]

Base64-encoded bytes of a serialized torch.Tensor of shape (num_tokens, hidden_size), in the model's dtype.

id instance-attribute

id: Required[str]

Content identifier of those embeddings, as <algorithm>:<hex digest>.

type instance-attribute

type: Required[Literal['prompt_embeds']]

The type of the content part.

PromptEmbedsErrorBody

Bases: TypedDict

Envelope the errors above arrive in, as {"error": {...}}.

PromptEmbedsNotCachedError

Bases: TypedDict

Body of the 409 answering a reference the server cannot serve.

Attributes:

Name	Type	Description
missing_ids	Required[list[str]]	Every identifier the request referenced that the server does not hold, so one retry can carry them all.

missing_ids instance-attribute

missing_ids: Required[list[str]]

Every identifier the request referenced that the server does not hold, so one retry can carry them all.

cached_prompt_embeds_part

cached_prompt_embeds_part(
    identifier: str,
) -> ChatCompletionContentPartCachedPromptEmbedsParam

Build a part that references embeddings the server already holds.

Parameters:

Name	Type	Description	Default
identifier	str	What compute_prompt_embeds_id returned for the embeddings being referenced.	required

Returns:

Type	Description
ChatCompletionContentPartCachedPromptEmbedsParam	The content part.

Raises:

Type	Description
ValueError	If the identifier is not shaped like one, which would be rejected by the server.

uploaded_prompt_embeds_part

uploaded_prompt_embeds_part(
    payload: str, identifier: str
) -> ChatCompletionContentPartUploadedPromptEmbedsParam

Build a part that uploads embeddings and labels them.

Parameters:

Name	Type	Description	Default
payload	str	Base64-encoded bytes of the serialized tensor.	required
identifier	str	What compute_prompt_embeds_id returned for it.	required

Returns:

Type	Description
ChatCompletionContentPartUploadedPromptEmbedsParam	The content part.

Raises:

Type	Description
ValueError	If the payload is empty, or the identifier is not shaped like one. Either would be rejected or, worse, silently misread by the server.