prompt_embeds

Enable vLLM's --enable-prompt-embeds by default under Output Protection (OP).

Stained Glass Proxy sends Stained Glass Transformed prompt embeddings, so the OP Inference Server must accept prompt_embeds. In vLLM this capability is gated behind enable_prompt_embeds, which is exposed only as the --enable-prompt-embeds CLI flag.

This module flips the flag's argparse default to True. Set SG_ENABLE_PROMPT_EMBEDS=0 to skip the patch and restore vLLM's default.

Warning

Under almost no circumstances should you need to import this module directly. If stainedglass_output_protection is installed, vLLM loads it automatically via the vllm.general_plugins entry point.

Functions:

Name	Description
enable_prompt_embeds_by_default	Default vLLM's --enable-prompt-embeds flag to True.

Attributes:

Name	Type	Description
SG_ENABLE_PROMPT_EMBEDS_ENV_VAR	Final[str]	Environment variable that disables our default flip when set to a falsy value.

SG_ENABLE_PROMPT_EMBEDS_ENV_VAR module-attribute

SG_ENABLE_PROMPT_EMBEDS_ENV_VAR: Final[str] = (
    "SG_ENABLE_PROMPT_EMBEDS"
)

Environment variable that disables our default flip when set to a falsy value.

enable_prompt_embeds_by_default

enable_prompt_embeds_by_default() -> None

Default vLLM's --enable-prompt-embeds flag to True.

Mutates the ModelConfig.enable_prompt_embeds dataclass field default. Must run before the serve parser is built. No-op when SG_ENABLE_PROMPT_EMBEDS is falsy.

Raises:

Type	Description
RuntimeError	If ModelConfig has no enable_prompt_embeds field.
RuntimeError	If the default did not take effect, which would leave the server silently rejecting prompt_embeds requests.