request_json

Parse JSON request bodies faster, and at most once per request when a middleware has already parsed them.

A request carrying prompt embeddings can be many megabytes of JSON, almost all of it base64. Unpatched, it is parsed twice: once by PromptEmbedsMiddleware, which has to look inside it, and again by FastAPI when vLLM's endpoint asks for it through starlette.requests.Request.json.

This module removes both costs:

• loads parses with pydantic_core, which is several times faster than json.loads on large bodies.
• patch_request_json makes Request.json reuse a document a middleware has already parsed, handed down in the ASGI scope under PARSED_BODY_SCOPE_KEY, and otherwise parse with loads.

A scope is built by the server for each connection.

Warning

Under almost no circumstances should you need to import this module directly. If stainedglass_output_protection is installed, vLLM loads it automatically via the vllm.general_plugins entry point.

Functions:

Name	Description
loads	Decode a JSON request body, exactly as json.loads would but faster.
patch_request_json	Make starlette.requests.Request.json reuse a body parsed upstream, and parse any other with loads.

Attributes:

Name	Type	Description
PARSED_BODY_SCOPE_KEY	Final[str]	ASGI scope key under which a middleware hands down the request body it already parsed.

PARSED_BODY_SCOPE_KEY module-attribute

PARSED_BODY_SCOPE_KEY: Final[str] = (
    "stainedglass.parsed_body"
)

ASGI scope key under which a middleware hands down the request body it already parsed.

loads

loads(body: bytes) -> Any

Decode a JSON request body, exactly as json.loads would but faster.

pydantic_core agrees with json.loads on every document it accepts, but refuses a few that json.loads takes (a leading byte order mark, unpaired surrogate escapes, very deep nesting), and reports malformed input as a plain ValueError. Retrying anything it refuses with json.loads keeps both the documents accepted and the json.JSONDecodeError that FastAPI turns into a 422.

Parameters:

Name	Type	Description	Default
body	bytes	The raw request body.	required

Returns:

Type	Description
Any	The decoded document.

Raises:

Type	Description
JSONDecodeError	If the body is not valid JSON.
UnicodeDecodeError	If the body is not valid UTF-8, UTF-16 or UTF-32.

patch_request_json

patch_request_json() -> None

Make starlette.requests.Request.json reuse a body parsed upstream, and parse any other with loads.

Idempotent. Applies to every Starlette application in the process.