registry

User key registry for Stained Glass Output Protection in vLLM, shared across all vLLM processes.

Warning

Under almost no circumstances should you need to import this module directly. If stainedglass_output_protection is installed, and you launch vLLM via the alternative entrypoint, this module will be automatically applied.

Classes:

Name	Description
KeyRegistryAuthSecretNotDefinedError	The key registry auth secret has not been defined.
KeyRegistryNotYetInitializedError	The key registry has not yet been initialized.
KeyRegistryPreviouslyInitializedError	The key registry was previously initialized.

Functions:

Name	Description
get_shared_registry	Get the shared key registry.
registry_manager_main_process	Start the registry manager in the main process.
remove_key	Remove a key from the registry by first marking it as a sentinel allowing existing generation steps to not crash and then remove

KeyRegistryAuthSecretNotDefinedError

Bases: RuntimeError

The key registry auth secret has not been defined.

KeyRegistryNotYetInitializedError

Bases: RuntimeError

The key registry has not yet been initialized.

KeyRegistryPreviouslyInitializedError

Bases: RuntimeError

The key registry was previously initialized.

get_shared_registry

get_shared_registry() -> (
    multiprocessing.managers.DictProxy[str, bytes | None]
)

Get the shared key registry. Changes to this registry will be reflected across all processes.

Note

This can be safely run in any process, whether main or child.

Returns:

Type	Description
multiprocessing.managers.DictProxy[str, bytes | None]	The shared key registry.

Raises:

Type	Description
KeyRegistryNotYetInitializedError	If the key registry has not yet been initialized in the main process.
KeyRegistryAuthSecretNotDefinedError	If the authkey environment variable is not set.

registry_manager_main_process

registry_manager_main_process() -> Generator[
    multiprocessing.managers.SyncManager
]

Start the registry manager in the main process. Calling this function in children processes will silently fail and corrupt the registry manager in the main process.

Warning

The manager started using this function will stop as soon as it is garbage collected. Make sure to store the output of this function in a variable.

Yields:

Type	Description
Generator[multiprocessing.managers.SyncManager]	The registry manager.

Raises:

Type	Description
KeyRegistryPreviouslyInitializedError	If the registry manager has been previously initialized.
KeyRegistryAuthSecretNotDefinedError	If the authkey environment variable is not set.

remove_key async

remove_key(
    request_id: str,
    shared_registry: DictProxy[str, bytes | None],
) -> None

Remove a key from the registry by first marking it as a sentinel allowing existing generation steps to not crash and then remove after 60 seconds.

Parameters:

Name	Type	Description	Default
request_id	str	Request to remove from the registry.	required
shared_registry	DictProxy[str, bytes | None]	Shared registry from which to delete key.	required