sampling_params
Preserve special tokens in the raw model output under Output Protection (OP).
Output Protection returns the raw (encrypted) model output and delegates tool/reasoning parsing to the client.
For that delegated parse to succeed, the raw output must retain the structural markers the parsers depend on
([TOOL_CALLS], Harmony <|channel|>/<|message|>, ...). These markers are special tokens, and they are
dropped during de-tokenization when skip_special_tokens is True (the default).
vLLM normally recovers them two ways, neither of which works with Output-Protection:
- Non-streaming: The server re-decodes the generated token IDs with `skip_special_tokens=False` at parse time.
- Streaming: Most tool parsers set `skip_special_tokens=False` in `adjust_request`, and token-driven parsers
(Harmony / gpt-oss) read the raw `delta_token_ids` directly.
However, under Output-Protection the token IDs are scrubbed for privacy, and only the (encrypted) output.text survives,
so the client has nothing but that text to parse from. We identified that Harmony/gpt-oss in particular never set
skip_special_tokens=False, so their markers would be stripped and the delegated parsing would be impossible.
This module patches each request type's to_sampling_params so the resulting SamplingParams always has
skip_special_tokens=False. Detokenization then keeps every special token in output.text, which is what gets
encrypted and forwarded to the client.
This patch does not change what the model generates, only how the generated token IDs are detokenized into text.
Functions:
| Name | Description |
|---|---|
patch_skip_special_tokens |
Force |
Force skip_special_tokens=False for every generation request type so raw markers survive for the client.
Covers the chat, completion, and responses endpoints.