LiteLLM

LiteLLM is an open-source proxy that gives your team a single endpoint for multiple LLM providers. You can point it at Consus Gateway so all requests route through government-authorized infrastructure. For LiteLLM setup and deployment instructions, see the LiteLLM docs.

Authentication

Consus Gateway authenticates via the x-api-key header. LiteLLM’s built-in API key field is required but ignored by Consus Gateway. Your Consus API key must be passed via extra_headers.

Run with Docker Compose

This walkthrough gets LiteLLM Proxy running locally — with a Postgres database for virtual keys and spend tracking — routing every request through Consus Gateway. The only provider credential you need is your Consus API key.

Step 1 — Get the Docker Compose file

Download LiteLLM’s official docker-compose.yml into an empty directory:

curl -O https://raw.githubusercontent.com/BerriAI/litellm/main/docker-compose.yml

This starts three services: the LiteLLM proxy, a Postgres database, and Prometheus.

Step 2 — Add your Consus API key

Create a .env file next to docker-compose.yml. The first command uses > to create the file; the rest use >> to append:

# Admin key for the proxy + UI login — change this to your own value
echo 'LITELLM_MASTER_KEY="sk-1234"' > .env
# Encrypts keys stored in the DB — cannot be changed after you add a model
echo 'LITELLM_SALT_KEY="sk-1234"' >> .env
# Your Consus Gateway key — the only provider credential you need
echo 'CONSUS_API_KEY="your-consus-api-key"' >> .env

Unlike a standard LiteLLM setup, you do not need Azure, OpenAI, or any other provider keys — Consus Gateway brokers every model behind your single CONSUS_API_KEY.

Step 3 — Create your config.yaml

Create a config.yaml next to docker-compose.yml and paste the full model list below. Every entry already points api_base at https://api.consus.io/v1 and passes your key via extra_headers: x-api-key: os.environ/CONSUS_API_KEY, so there’s nothing else to edit.

Full config.yaml with all available models

model_list:
  - model_name: claude-sonnet-4-5:il5+itar
    litellm_params:
      model: openai/claude-sonnet-4-5:il5+itar
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: claude-sonnet-4-5:fedramp-high
    litellm_params:
      model: openai/claude-sonnet-4-5:fedramp-high
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: claude-opus-4-8:fedramp-high
    litellm_params:
      model: openai/claude-opus-4-8:fedramp-high
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: claude-opus-4-7:fedramp-high
    litellm_params:
      model: openai/claude-opus-4-7:fedramp-high
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: claude-opus-4-6:fedramp-high
    litellm_params:
      model: openai/claude-opus-4-6:fedramp-high
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: claude-sonnet-4-6:fedramp-high
    litellm_params:
      model: openai/claude-sonnet-4-6:fedramp-high
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: claude-haiku-4-5:fedramp-high
    litellm_params:
      model: openai/claude-haiku-4-5:fedramp-high
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: gemini-3-1-pro:il4
    litellm_params:
      model: openai/gemini-3-1-pro:il4
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: gemini-3-flash:il4
    litellm_params:
      model: openai/gemini-3-flash:il4
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: gpt-4.1:il5+itar
    litellm_params:
      model: openai/gpt-4.1:il5+itar
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: gpt-4.1-mini:il5+itar
    litellm_params:
      model: openai/gpt-4.1-mini:il5+itar
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: gpt-oss-120b:il5+itar
    litellm_params:
      model: openai/gpt-oss-120b:il5+itar
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: gpt-5.4:il5+itar
    litellm_params:
      model: openai/gpt-5.4:il5+itar
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

For the full list of available models and compliance levels, see the Model Explorer.

Step 4 — Enable the config mount

By default, LiteLLM’s docker-compose.yml ships with the config file commented out. This sed uncomments the volumes and command lines under the litellm service in place, so your config.yaml is loaded (it leaves a docker-compose.yml.bak backup you can delete):

sed -i.bak \
  -e 's|^    # volumes:|    volumes:|' \
  -e 's|^    #  - ./config.yaml:/app/config.yaml|      - ./config.yaml:/app/config.yaml|' \
  -e 's|^    # command:|    command:|' \
  -e 's|^    #  - "--config=/app/config.yaml"|      - "--config=/app/config.yaml"|' \
  docker-compose.yml

The compose file also mounts a prometheus.yml, so create a minimal one (it must exist before you start, or the stack won’t come up):

cat > prometheus.yml <<'EOF'
global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "litellm"
    static_configs:
      - targets: ["litellm:4000"]
EOF

Step 5 — Start the proxy

With docker-compose.yml, .env, config.yaml, and prometheus.yml all in the same directory, start everything:

docker compose up

Once the logs settle, send a test request through the proxy to a Consus model:

curl -X POST 'http://0.0.0.0:4000/chat/completions' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer sk-1234' \
  -d '{
    "model": "gpt-4.1:il5+itar",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

The Authorization: Bearer sk-1234 here is your LiteLLM master key (from .env), not your Consus key — LiteLLM injects CONSUS_API_KEY upstream via the extra_headers in your config.

Optional — Open the UI and create a virtual key

Visit http://localhost:4000/ui and log in with your master key (sk-1234). Under Virtual Keys → + Create New Key, you can mint scoped keys with per-key RPM limits and model access — all spend is tracked in the Postgres container that Compose already started.

Configure via the UI

Go to Models + Endpoints and click Add Model
Set Provider to OpenAI-Compatible Endpoints (Together AI, etc.)
Set LiteLLM Model Name to openai/<model>:<compliance-level> (e.g. openai/gpt-4.1:il5+itar)
Set API Base to https://api.consus.io/v1
Set OpenAI API Key to any value (required by LiteLLM, ignored by Consus)

Under Advanced Settings, set LiteLLM Params to:

{"extra_headers": {"x-api-key": "YOUR_CONSUS_API_KEY"}}

Click Add Model

Do not use the Custom OpenAI provider. It prepends openai/ to the model name sent to Consus Gateway, causing a 400 error. Use OpenAI-Compatible Endpoints instead.

Product

Getting Started

API Reference

Integrations

Authentication

Run with Docker Compose

Step 1 — Get the Docker Compose file

Step 2 — Add your Consus API key

Step 3 — Create your config.yaml

Step 4 — Enable the config mount

Step 5 — Start the proxy

Optional — Open the UI and create a virtual key

Configure via the UI

​Authentication

​Run with Docker Compose

​Step 1 — Get the Docker Compose file

​Step 2 — Add your Consus API key

​Step 3 — Create your config.yaml

​Step 4 — Enable the config mount

​Step 5 — Start the proxy

​Optional — Open the UI and create a virtual key

​Configure via the UI

Authentication

Run with Docker Compose

Step 1 — Get the Docker Compose file

Step 2 — Add your Consus API key

Step 3 — Create your config.yaml

Step 4 — Enable the config mount

Step 5 — Start the proxy

Optional — Open the UI and create a virtual key

Configure via the UI