decryption

Utilities to decrypt OpenAI-style Completion/Chat Completions responses.

Bare protocols are defined to represent the basic structure of the responses. They are compatible with both OpenAI's and vLLM's response pydantic models, and should be compatible with any custom response models that client code may define bespoke (as long as they're compatible with the API json responses)

These protocols should not be used directly in client code, but they are used implicitly by decrypt_completion_response and decrypt_chat_completion_response to ensure that the response is compatible with the expected structure.

Classes:

Name	Description
CannotDecryptFieldError	Raised when a field in a response object is not decryptable.
ChatCompletionMessageLike	Represents a message in a chat completion response from the API.
ChatCompletionResponseChoiceLike	Represents a choice in a chat completion response from the API.
ChatCompletionResponseLike	Represents a chat completion response from the API.
ChatCompletionStreamResponseChoiceLike	Represents a choice in a streaming chat completion response from the API.
ChatCompletionStreamResponseLike	Represents a streaming chat completion response from the API.
ChatCompletionTokenLogProbLike	Represents the log probability of a token in a chat completion response.
ChoiceDeltaLike	Represents a delta in a streaming chat completion response from the API.
ChoiceLogprobsLike	Represents the log probabilities of tokens in a chat completion response.
CompletionChoiceLike	Represents a choice in a completion response or stream from the API.
CompletionLogProbsLike	Represents the log probabilities of tokens in a completion response.
CompletionResponseLike	Represents a completion response from the API.
CompletionStreamResponseLike	Represents a streaming completion response from the API.
LogProbLike	Represents the log probability of a token in a completion response.
VllmChoiceLogprobsLike	Additional fields in vLLM's ChoiceLogprobs that are not part of the OpenAI API spec.
VllmCompletionChoiceLike	Additional fields in vLLM completion choices that are not part of the OpenAI API spec.
VllmCompletionResponseChoiceLike	Additional fields in vLLM's non-streaming completion choices.
VllmCompletionResponseStreamChoiceLike	Additional fields in vLLM's CompletionResponseStreamChoice that are not part of the OpenAI API spec.

Functions:

Name	Description
decrypt_chat_completion_response	Decrypts an OpenAI response.
decrypt_completion_response	Decrypts an OpenAI response.

CannotDecryptFieldError

Bases: ValueError

Raised when a field in a response object is not decryptable.

Methods:

Name	Description
__init__	Initialize the error with the field name, expected value, and actual object.

__init__

__init__(
    field_name: str,
    expected_value: str,
    actual_object: object,
) -> None

Initialize the error with the field name, expected value, and actual object.

Parameters:

Name	Type	Description	Default
field_name	str	The name of the field that is not decryptable.	required
expected_value	str	The expected value of the field.	required
actual_object	object	The actual object that contains the field.	required

ChatCompletionMessageLike

Bases: Protocol

Represents a message in a chat completion response from the API.

Attributes:

Name	Type	Description
content	str | None	The content of the message in the chat completion response.
function_call	Any | None	The function call of the message in the chat completion response, if available.
role	str | None	The role of the message in the chat completion response.
tool_calls	Sequence[Any] | None	The tool calls of the message in the chat completion response, if available.

content instance-attribute

content: str | None

The content of the message in the chat completion response.

function_call property

function_call: Any | None

The function call of the message in the chat completion response, if available.

This is not decryptable, so if this is defined in an encrypted ChatCompletionMessage, it is expected to be None.

role property

role: str | None

The role of the message in the chat completion response.

tool_calls property

tool_calls: Sequence[Any] | None

The tool calls of the message in the chat completion response, if available.

This is not decryptable, so if this is defined in an encrypted ChatCompletionMessage, it is expected to be None.

ChatCompletionResponseChoiceLike

Bases: Protocol

Represents a choice in a chat completion response from the API.

Attributes:

Name	Type	Description
logprobs	ChoiceLogprobsLike | VllmChoiceLogprobsLike | None	The log probabilities of the tokens in the choice, if available.
message	ChatCompletionMessageLike	The message of the choice in the chat completion response.

logprobs property

logprobs: ChoiceLogprobsLike | VllmChoiceLogprobsLike | None

The log probabilities of the tokens in the choice, if available.

These are not decryptable, so if these are defined in an encrypted ChatCompletionResponseChoice, they are expected to be all None.

message property

message: ChatCompletionMessageLike

The message of the choice in the chat completion response.

ChatCompletionResponseLike

Bases: Protocol

Represents a chat completion response from the API.

Attributes:

Name	Type	Description
choices	Sequence[ChatCompletionResponseChoiceLike]	The choices in the chat completion response.

choices property

choices: Sequence[ChatCompletionResponseChoiceLike]

The choices in the chat completion response.

ChatCompletionStreamResponseChoiceLike

Bases: Protocol

Represents a choice in a streaming chat completion response from the API.

Attributes:

Name	Type	Description
delta	ChoiceDeltaLike | None	The delta of the choice in the chat completion response.

delta property

delta: ChoiceDeltaLike | None

The delta of the choice in the chat completion response.

ChatCompletionStreamResponseLike

Bases: Protocol

Represents a streaming chat completion response from the API.

Attributes:

Name	Type	Description
choices	Sequence[ChatCompletionStreamResponseChoiceLike]	The choices in the chat completion response.

choices property

choices: Sequence[ChatCompletionStreamResponseChoiceLike]

The choices in the chat completion response.

ChatCompletionTokenLogProbLike

Bases: Protocol

Represents the log probability of a token in a chat completion response.

This is intentionally empty because we do not currently support decrypting logprobs, so we don't want to bother checking any fields.

ChoiceDeltaLike

Bases: Protocol

Represents a delta in a streaming chat completion response from the API.

Attributes:

Name	Type	Description
content	str | None	The content of the message in the chat completion response.
role	str | None	The role of the message in the chat completion response.
tool_calls	Sequence[Any] | None	The tool calls of the message in the chat completion response, if available.

content instance-attribute

content: str | None

The content of the message in the chat completion response.

role property

role: str | None

The role of the message in the chat completion response.

tool_calls property

tool_calls: Sequence[Any] | None

The tool calls of the message in the chat completion response, if available.

This is not decryptable, so if this is defined in an encrypted ChatCompletionMessage, it is expected to be None.

ChoiceLogprobsLike

Bases: Protocol

Represents the log probabilities of tokens in a chat completion response.

Attributes:

Name	Type	Description
content	Sequence[ChatCompletionTokenLogProbLike] | None	The log probabilities of the tokens in the choice, if available.
refusal	Sequence[ChatCompletionTokenLogProbLike] | None	The log probabilities of the refusal tokens in the choice, if available.

content property

content: Sequence[ChatCompletionTokenLogProbLike] | None

The log probabilities of the tokens in the choice, if available.

These are not decryptable, so if these are defined in an encrypted ChatCompletionResponseChoice, they are expected to be all None.

refusal property

refusal: Sequence[ChatCompletionTokenLogProbLike] | None

The log probabilities of the refusal tokens in the choice, if available.

These are not decryptable, so if these are defined in an encrypted ChatCompletionResponseChoice, they are expected to be all None.

CompletionChoiceLike

Bases: Protocol

Represents a choice in a completion response or stream from the API.

Attributes:

Name	Type	Description
index	int	The index of the choice in the completion response.
logprobs	CompletionLogProbsLike | None	The log probabilities of the tokens in the choice, if available.
text	str	The text of the choice in the completion response.

index instance-attribute

index: int

The index of the choice in the completion response.

logprobs property

logprobs: CompletionLogProbsLike | None

The log probabilities of the tokens in the choice, if available.

These are not decryptable, so if these are defined in an encrypted CompletionResponseChoice, they are expected to be all None.

text instance-attribute

text: str

The text of the choice in the completion response.

CompletionLogProbsLike

Bases: Protocol

Represents the log probabilities of tokens in a completion response.

Attributes:

Name	Type	Description
text_offset	Sequence[int] | None	The character offsets of the tokens in the completion response.
token_logprobs	Sequence[float | None] | None	The log probabilities of the tokens in the completion response.
tokens	Sequence[str] | None	The log probabilities of the tokens in the completion response.
top_logprobs	Sequence[Mapping[str, float] | None] | None	The log probabilities of the top tokens in the completion response.

text_offset property

text_offset: Sequence[int] | None

The character offsets of the tokens in the completion response.

token_logprobs property

token_logprobs: Sequence[float | None] | None

The log probabilities of the tokens in the completion response.

tokens property

tokens: Sequence[str] | None

The log probabilities of the tokens in the completion response.

top_logprobs property

top_logprobs: Sequence[Mapping[str, float] | None] | None

The log probabilities of the top tokens in the completion response.

CompletionResponseLike

Bases: Protocol

Represents a completion response from the API.

Attributes:

Name	Type	Description
choices	Sequence[CompletionChoiceLike | VllmCompletionResponseChoiceLike]	The choices in the completion response.
object	Literal['text_completion']	The object type of the response.

choices property

choices: Sequence[
    CompletionChoiceLike | VllmCompletionResponseChoiceLike
]

The choices in the completion response.

object instance-attribute

object: Literal['text_completion']

The object type of the response.

CompletionStreamResponseLike

Bases: Protocol

Represents a streaming completion response from the API.

Attributes:

Name	Type	Description
choices	Sequence[CompletionChoiceLike | VllmCompletionResponseStreamChoiceLike]	The choices in the completion response.
object	str | Literal['text_completion']	The object type of the response.

choices property

choices: Sequence[
    CompletionChoiceLike
    | VllmCompletionResponseStreamChoiceLike
]

The choices in the completion response.

object property

object: str | Literal['text_completion']

The object type of the response.

In practice, this is Literal["text_completion"], but vLLM doesn't type it that way.

LogProbLike

Bases: Protocol

Represents the log probability of a token in a completion response.

Attributes:

Name	Type	Description
decoded_token	str | None	The decoded token in the completion response, if available.
logprob	float	The log probability of the token in the completion response.
rank	int | None	The rank of the token in the completion response.

decoded_token instance-attribute

decoded_token: str | None

The decoded token in the completion response, if available.

logprob instance-attribute

logprob: float

The log probability of the token in the completion response.

rank instance-attribute

rank: int | None

The rank of the token in the completion response.

VllmChoiceLogprobsLike

Bases: Protocol

Additional fields in vLLM's ChoiceLogprobs that are not part of the OpenAI API spec.

Attributes:

Name	Type	Description
content	Sequence[ChatCompletionTokenLogProbLike] | None	The log probabilities of the tokens in the choice, if available.

content property

content: Sequence[ChatCompletionTokenLogProbLike] | None

The log probabilities of the tokens in the choice, if available.

These are not decryptable, so if these are defined in an encrypted ChatCompletionResponseChoice, they are expected to be all None.

VllmCompletionChoiceLike

Bases: CompletionChoiceLike, Protocol

Additional fields in vLLM completion choices that are not part of the OpenAI API spec.

Attributes:

Name	Type	Description
index	int	The index of the choice in the completion response.
logprobs	CompletionLogProbsLike | None	The log probabilities of the tokens in the choice, if available.
prompt_token_ids	Sequence[int] | None	The token IDs of the tokens in the prompt, if available.
text	str	The text of the choice in the completion response.
token_ids	Sequence[int] | None	The token IDs of the tokens in the choice, if available.

index instance-attribute

index: int

The index of the choice in the completion response.

logprobs property

logprobs: CompletionLogProbsLike | None

The log probabilities of the tokens in the choice, if available.

These are not decryptable, so if these are defined in an encrypted CompletionResponseChoice, they are expected to be all None.

prompt_token_ids property

prompt_token_ids: Sequence[int] | None

The token IDs of the tokens in the prompt, if available.

These are not decryptable, so if these are defined in an encrypted CompletionResponseChoice, they are expected to be all -1.

text instance-attribute

text: str

The text of the choice in the completion response.

token_ids property

token_ids: Sequence[int] | None

The token IDs of the tokens in the choice, if available.

These are not decryptable, so if these are defined in an encrypted CompletionResponseChoice, they are expected to be all -1.

VllmCompletionResponseChoiceLike

Bases: VllmCompletionChoiceLike, Protocol

Additional fields in vLLM's non-streaming completion choices.

Attributes:

Name	Type	Description
index	int	The index of the choice in the completion response.
logprobs	CompletionLogProbsLike | None	The log probabilities of the tokens in the choice, if available.
prompt_logprobs	Sequence[Mapping[int, LogProbLike] | None] | None	Log probabilities of the tokens in the prompt, if available.
prompt_token_ids	Sequence[int] | None	The token IDs of the tokens in the prompt, if available.
text	str	The text of the choice in the completion response.
token_ids	Sequence[int] | None	The token IDs of the tokens in the choice, if available.

index instance-attribute

index: int

The index of the choice in the completion response.

logprobs property

logprobs: CompletionLogProbsLike | None

The log probabilities of the tokens in the choice, if available.

These are not decryptable, so if these are defined in an encrypted CompletionResponseChoice, they are expected to be all None.

prompt_logprobs property

prompt_logprobs: (
    Sequence[Mapping[int, LogProbLike] | None] | None
)

Log probabilities of the tokens in the prompt, if available.

These are not decryptable, so if these are defined in an encrypted CompletionResponseChoice, they are expected to be all None.

prompt_token_ids property

prompt_token_ids: Sequence[int] | None

The token IDs of the tokens in the prompt, if available.

These are not decryptable, so if these are defined in an encrypted CompletionResponseChoice, they are expected to be all -1.

text instance-attribute

text: str

The text of the choice in the completion response.

token_ids property

token_ids: Sequence[int] | None

The token IDs of the tokens in the choice, if available.

These are not decryptable, so if these are defined in an encrypted CompletionResponseChoice, they are expected to be all -1.

VllmCompletionResponseStreamChoiceLike

Bases: VllmCompletionChoiceLike, Protocol

Additional fields in vLLM's CompletionResponseStreamChoice that are not part of the OpenAI API spec.

Attributes:

Name	Type	Description
index	int	The index of the choice in the completion response.
logprobs	CompletionLogProbsLike | None	The log probabilities of the tokens in the choice, if available.
prompt_token_ids	Sequence[int] | None	The token IDs of the tokens in the prompt, if available.
text	str	The text of the choice in the completion response.
token_ids	Sequence[int] | None	The token IDs of the tokens in the choice, if available.

index instance-attribute

index: int

The index of the choice in the completion response.

logprobs property

logprobs: CompletionLogProbsLike | None

The log probabilities of the tokens in the choice, if available.

These are not decryptable, so if these are defined in an encrypted CompletionResponseChoice, they are expected to be all None.

prompt_token_ids property

prompt_token_ids: Sequence[int] | None

The token IDs of the tokens in the prompt, if available.

This is not part of the OpenAI API spec, but vLLM includes it.

These are not decryptable, so if these are defined in an encrypted CompletionResponseChoice, they are expected to be all -1.

text instance-attribute

text: str

The text of the choice in the completion response.

token_ids property

token_ids: Sequence[int] | None

The token IDs of the tokens in the choice, if available.

These are not decryptable, so if these are defined in an encrypted CompletionResponseChoice, they are expected to be all -1.

decrypt_chat_completion_response

decrypt_chat_completion_response(
    response: T, shared_aes_key: bytes
) -> T

Decrypts an OpenAI response.

Parameters:

Name	Type	Description	Default
response	T	The OpenAI response to decrypt.	required
shared_aes_key	bytes	The shared AES key to use for decryption.	required

Returns:

Type	Description
T	The decrypted OpenAI response.

Raises:

Type	Description
CannotDecryptFieldError	If a field in the response is not decryptable.

decrypt_completion_response

decrypt_completion_response(
    response: T, shared_aes_key: bytes
) -> T

Decrypts an OpenAI response.

Parameters:

Name	Type	Description	Default
response	T	The OpenAI response to decrypt.	required
shared_aes_key	bytes	The shared AES key to use for decryption.	required

Returns:

Type	Description
T	The decrypted OpenAI response.

Raises:

Type	Description
CannotDecryptFieldError	If a field in the response is not decryptable.