Skip to content

Stained Glass Transform (SGT) Proxy Logging

The SGT Proxy supports custom logging configuration as per python standards. We use alogging.yaml configuration file that follows the dictionary schema to enable users to customize logging features like formatters, handlers, filters, loggers, etc.

Configuring Logging via logging.yaml

Configuration File Breakdown

Let's walk through the default config file for SGT Proxy deployments.

---
version: 1
dsable_existing_loggers: false

formatters:
  standardFormatter:
    format: "%(asctime)s | %(name)-40s | %(levelname)-8s | %(correlation_id)-32s | %(message)s"
    datefmt: "%Y-%m-%d %H:%M:%S"

handlers:
  streamHandler:
    class: logging.StreamHandler
    formatter: standardFormatter
    filters: [correlationIdFilter]
    stream: ext://sys.stdout

filters:
  correlationIdFilter:
    (): asgi_correlation_id.CorrelationIdFilter

  endpointFilterLivez:
    (): stainedglass_proxy.app_logging.EndpointFilter
    endpoint: "/livez"

  endpointFilterReadyz:
    (): stainedglass_proxy.app_logging.EndpointFilter
    endpoint: "/readyz"

loggers:
  root:
    level: DEBUG
    handlers: [streamHandler]

  uvicorn.access:
    level: INFO
    handlers: [streamHandler]
    filters: [endpointFilterLivez, endpointFilterReadyz]
    propagate: false

  uvicorn.error:
    level: INFO
    handlers: [streamHandler]
    propagate: false

The logging.yaml contains the following top-level sections:

  • version: The version of the logging configuration format (currently 1).
  • disable_existing_loggers: A boolean flag that controls whether existing loggers should be disabled. The default is false, which means existing loggers will not be disabled.
  • formatters: Defines the format of log messages.
  • handlers: Specifies how log messages should be handled (e.g., written to stdout).
  • filters: Defines custom filters to be applied to log messages.
  • loggers: Configures individual loggers and their behavior.

Formatters

Formatters define the layout of log messages. The default configuration defines a single formatter called standardFormatter that includes the following elements:

  • format: Defines the structure of each log entry:
  • %(asctime)s: Timestamp of the log.
  • %(name)-40s: Logger name, aligned to 40 characters.
  • %(levelname)-8s: Log level, aligned to 8 characters.
  • %(correlation_id)-32s: Correlation ID, aligned to 32 characters.
  • %(message)s: Log message.
  • datefmt: Specifies the date format in logs ("%Y-%m-%d %H:%M:%S").

Handlers

Handlers determine where the logs are sent. The default configuration defines a single handler called streamHandler with the following elements:

  • class: Specifies logging.StreamHandler, which allows logs to be output to any file-like stream.
  • formatter: Uses standardFormatter for structured output.
  • filters: Applies correlationIdFilter to include correlation IDs in the logs.
  • stream: Sets the output stream, which in this case is set to ext://sys.stdout which is ideal for containerized environments where logs are usually aggregated from stdout and stderr.

Filters

Filters allow selective logging based on certain criteria. The default configuration defines three custom filters:

  • correlationIdFilter:
  • Class: asgi_correlation_id.CorrelationIdFilter ensures each log entry includes a correlation ID.

  • Purpose: Makes it easier to trace requests across services.

  • endpointFilterLivez and endpointFilterReadyz:

  • Class: stainedglass_proxy.app_logging.EndpointFilter which filters logs based on specific endpoints.
  • Purpose: Filters log entries based on specific endpoints (/livez and /readyz), helping to reduce log noise from frequent health checks.

Loggers

Loggers define the log levels, handlers, and filters for specific log sources. The default configuration defines the following loggers:

  • root:
  • level: Set to DEBUG to capture all logs of this level and above.
  • handlers: Uses streamHandler to output logs to stdout.
  • uvicorn.access:
  • level: Set to INFO, capturing informational logs for HTTP access requests.
  • filters: Applies endpointFilterLivez and endpointFilterReadyz to limit log output for health check endpoints.
  • propagate: Set to false to prevent logs from propagating up to the root logger.
  • uvicorn.error:
  • level: Set to INFO, capturing error-level logs.
  • handlers: Uses streamHandler for log output.
  • propagate: Set to false to avoid log duplication.

Warning

By default, the root logger level is set to INFO. Setting the level to DEBUG will enable certain sensitive logs such as chat input prompt and generated response. Please ensure that the DEBUG log level is only set when absolutely necessary and within protected environments to avoid data leakage.

Customizing the Logging Configuration

Python’s logging library provides flexibility to customize loggers, handlers, filters, and formatters to suit application needs. Here are some examples to demonstrate customization of these elements.

Adding Additional Handlers

You can add more handlers to route logs to different destinations (e.g., files, HTTP endpoints, external logging services):

handlers:
  fileHandler:
    class: logging.FileHandler
    formatter: standardFormatter
    filename: "/path/to/your/logfile.log"
    level: WARNING
  • logging.FileHandler: Directs logs to a file.
  • filename: Specifies the file path.
  • level: WARNING captures only warnings and errors.

Using Standard Python Filters

You can use Python’s built-in filters or create custom filters. For example, add a levelFilter to only allow specific log levels:

filters:
  levelFilter:
    (): logging.Filter
    level: ERROR
  • levelFilter: This custom filter (which you would implement) could restrict logs to specific levels, such as ERROR.

Modifying or Adding Loggers

Modify loggers to separate logging for different parts of the application:

loggers:
  uvicorn.access:
    level: DEBUG
    handlers: [streamHandler, fileHandler]
    propagate: false
  • uvicorn.access logger: Allows you to log uvicorn.access events, using streamHandler for console logs and fileHandler for file-based logs.
  • propagate: false prevents logs from appearing in other handlers.

Customizing Formatters

You can add more formatters to vary log formats for different handlers:

formatters:
  conciseFormatter:
    format: "%(levelname)s: %(message)s"
  detailedFormatter:
    format: "%(asctime)s | %(name)s | %(levelname)s | %(filename)s:%(lineno)d | %(message)s"
    datefmt: "%H:%M:%S"
  • conciseFormatter: A simple format displaying only the log level and message.
  • detailedFormatter: Adds filename and line number for more detailed debugging.

Configuring Rotating File Handlers

For instances that log extensively, a rotating file handler can manage log file sizes automatically:

handlers:
  rotatingFileHandler:
    class: logging.handlers.RotatingFileHandler
    formatter: standardFormatter
    filename: "/path/to/rotating.log"
    maxBytes: 10485760  # 10 MB
    backupCount: 5
    level: INFO
  • logging.handlers.RotatingFileHandler: Automatically rotates log files.
  • maxBytes: Specifies the maximum size of a log file before rotation.
  • backupCount: Keeps a maximum of 5 backup files.

Deploying SGT Proxy with logging.yaml

The SGT Proxy container application reads its logging configuration from the SGP_LOGGING_CONFIG_FILE environment variable which should point towards a logging config file on the container. You can deploy your SGT Proxy container instance with your logging config file as follows:

  1. Mount your logging config file on the container:

    ---
# version: '3.8'

services:
stainedglass:
    image: stainedglass-proxy:0.14.0-4edd93c
    env_file:
    - .env
    ports:
    - "8600:8600"
    volumes:
    - ./logging.yaml:/app/logging.yaml

  2. Set the SGP_LOGGING_CONFIG_FILE to the expected logging file path on the container. This defaults to logging.yaml on the app's run directory in the container.

  3. Start the container and tail logs:

    ✔ Container stained-glass-proxy-stainedglass-proxy-1  Created                                                                                                                                                                                                                   0.0s 
Attaching to stainedglass-proxy-1
stainedglass-proxy-1  | 2024-11-08 19:57:36 | uvicorn.error                            | INFO     | None                             | Started server process [1]
stainedglass-proxy-1  | 2024-11-08 19:57:36 | uvicorn.error                            | INFO     | None                             | Waiting for application startup.
stainedglass-proxy-1  | 2024-11-08 19:57:36 | stainedglass_proxy.dependencies          | INFO     | None                             | Initializing pre-run lifespan events.
stainedglass-proxy-1  | 2024-11-08 19:57:36 | stainedglass_proxy.utils.inference       | INFO     | None                             | Initializing HTTP session client.
stainedglass-proxy-1  | 2024-11-08 19:57:36 | stainedglass_proxy.utils.llm             | INFO     | None                             | Loading Stained Glass Transform model layer for inference.
stainedglass-proxy-1  | 2024-11-08 19:57:38 | uvicorn.error                            | INFO     | None                             | Application startup complete.
stainedglass-proxy-1  | 2024-11-08 19:57:38 | uvicorn.error                            | INFO     | None                             | Uvicorn running on http://0.0.0.0:8600 (Press CTRL+C to quit)
stainedglass-proxy-1  | 2024-11-08 19:57:38 | stainedglass_proxy.utils.llm             | INFO     | 9972a3996f384a2599a8bdd6d8bdfbda | Transforming request prompt.
stainedglass-proxy-1  | 2024-11-08 19:57:38 | stainedglass_proxy.utils.llm             | DEBUG    | 9972a3996f384a2599a8bdd6d8bdfbda | Request prompt: [RequestMessage(content='Write a short story about a robot', role='user', name=None, tool_calls=None, tool_call_id=None)]
stainedglass-proxy-1  | 2024-11-08 19:57:38 | stainedglass_proxy.utils.llm             | DEBUG    | 9972a3996f384a2599a8bdd6d8bdfbda | SGT input chat template : [{'content': 'Write a short story about a robot', 'role': 'user', 'name': None, 'tool_calls': None, 'tool_call_id': None}]
stainedglass-proxy-1  | 2024-11-08 19:57:39 | stainedglass_proxy.utils.llm             | INFO     | 9972a3996f384a2599a8bdd6d8bdfbda | Constructing `llm-api` request body.
stainedglass-proxy-1  | 2024-11-08 19:57:39 | uvicorn.access                           | INFO     | 9972a3996f384a2599a8bdd6d8bdfbda | 127.0.0.1:59128 - "POST /v1/chat/completions HTTP/1.1" 200
stainedglass-proxy-1  | 2024-11-08 19:57:39 | stainedglass_proxy.utils.inference       | INFO     | 9972a3996f384a2599a8bdd6d8bdfbda | Streaming response with ID '5bdeff62-f680-4a3e-b731-faf4ea412192' for request ID '9972a3996f384a2599a8bdd6d8bdfbda'.
stainedglass-proxy-1  | 2024-11-08 19:57:39 | stainedglass_proxy.utils.inference       | INFO     | 9972a3996f384a2599a8bdd6d8bdfbda | Sending POST request to LLM API endpoint with stream set to 'True'.
stainedglass-proxy-1  | 2024-11-08 19:57:39 | stainedglass_proxy.utils.llm             | INFO     | 664470d7281848248094ee98e4f99b2b | Transforming request prompt.