Skip to main content
LiteLLM is an open-source proxy that gives your team a single endpoint for multiple LLM providers. You can point it at Consus Gateway so all requests route through government-authorized infrastructure. For LiteLLM setup and deployment instructions, see the LiteLLM docs.

​
Authentication

Consus Gateway authenticates via the x-api-key header. LiteLLM’s built-in API key field is required but ignored by Consus Gateway. Your Consus API key must be passed via extra_headers.

​
Run with Docker Compose

This walkthrough gets LiteLLM Proxy running locally — with a Postgres database for virtual keys and spend tracking — routing every request through Consus Gateway. The only provider credential you need is your Consus API key.

​
Step 1 — Get the Docker Compose file

Download LiteLLM’s official docker-compose.yml into an empty directory: 
curl -O https://raw.githubusercontent.com/BerriAI/litellm/main/docker-compose.yml
This starts three services: the LiteLLM proxy, a Postgres database, and Prometheus.

​
Step 2 — Add your Consus API key

Create a .env file next to docker-compose.yml. The first command uses > to create the file; the rest use >> to append: 
# Admin key for the proxy + UI login — change this to your own value
echo 'LITELLM_MASTER_KEY="sk-1234"' > .env
# Encrypts keys stored in the DB — cannot be changed after you add a model
echo 'LITELLM_SALT_KEY="sk-1234"' >> .env
# Your Consus Gateway key — the only provider credential you need
echo 'CONSUS_API_KEY="your-consus-api-key"' >> .env
Unlike a standard LiteLLM setup, you do not need Azure, OpenAI, or any other provider keys — Consus Gateway brokers every model behind your single CONSUS_API_KEY.

​
Step 3 — Create your config.yaml

Create a config.yaml next to docker-compose.yml and paste the full model list below. Every entry already points api_base at https://api.consus.io/v1 and passes your key via extra_headers: x-api-key: os.environ/CONSUS_API_KEY, so there’s nothing else to edit. 
model_list:
  - model_name: claude-3-7-sonnet:il5+itar
    litellm_params:
      model: openai/claude-3-7-sonnet:il5+itar
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: claude-sonnet-4-5:il5+itar
    litellm_params:
      model: openai/claude-sonnet-4-5:il5+itar
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: claude-sonnet-4-5:fedramp-high
    litellm_params:
      model: openai/claude-sonnet-4-5:fedramp-high
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: claude-opus-4-6:fedramp-high
    litellm_params:
      model: openai/claude-opus-4-6:fedramp-high
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: claude-sonnet-4-6:fedramp-high
    litellm_params:
      model: openai/claude-sonnet-4-6:fedramp-high
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: claude-opus-4-5:fedramp-high
    litellm_params:
      model: openai/claude-opus-4-5:fedramp-high
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: claude-haiku-4-5:fedramp-high
    litellm_params:
      model: openai/claude-haiku-4-5:fedramp-high
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: claude-opus-4-1:fedramp-high
    litellm_params:
      model: openai/claude-opus-4-1:fedramp-high
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: claude-opus-4:fedramp-high
    litellm_params:
      model: openai/claude-opus-4:fedramp-high
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: claude-sonnet-4:fedramp-high
    litellm_params:
      model: openai/claude-sonnet-4:fedramp-high
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: gemini-2-5-pro:il5
    litellm_params:
      model: openai/gemini-2-5-pro:il5
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: gemini-2-5-flash:il5
    litellm_params:
      model: openai/gemini-2-5-flash:il5
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: gpt-4.1:il5+itar
    litellm_params:
      model: openai/gpt-4.1:il5+itar
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: gpt-4.1-mini:il5+itar
    litellm_params:
      model: openai/gpt-4.1-mini:il5+itar
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: gpt-oss-120b:il5+itar
    litellm_params:
      model: openai/gpt-oss-120b:il5+itar
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY

  - model_name: gpt-5.4:il5+itar
    litellm_params:
      model: openai/gpt-5.4:il5+itar
      api_base: https://api.consus.io/v1
      api_key: dummy
      extra_headers:
        x-api-key: os.environ/CONSUS_API_KEY
For the full list of available models and compliance levels, see the Model Explorer.

​
Step 4 — Enable the config mount

By default, LiteLLM’s docker-compose.yml ships with the config file commented out. This sed uncomments the volumes and command lines under the litellm service in place, so your config.yaml is loaded (it leaves a docker-compose.yml.bak backup you can delete): 
sed -i.bak \
  -e 's|^    # volumes:|    volumes:|' \
  -e 's|^    #  - ./config.yaml:/app/config.yaml|      - ./config.yaml:/app/config.yaml|' \
  -e 's|^    # command:|    command:|' \
  -e 's|^    #  - "--config=/app/config.yaml"|      - "--config=/app/config.yaml"|' \
  docker-compose.yml
The compose file also mounts a prometheus.yml, so create a minimal one (it must exist before you start, or the stack won’t come up): 
cat > prometheus.yml <<'EOF'
global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "litellm"
    static_configs:
      - targets: ["litellm:4000"]
EOF

​
Step 5 — Start the proxy

With docker-compose.yml, .env, config.yaml, and prometheus.yml all in the same directory, start everything: 
docker compose up
Once the logs settle, send a test request through the proxy to a Consus model: 
curl -X POST 'http://0.0.0.0:4000/chat/completions' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer sk-1234' \
  -d '{
    "model": "gpt-4.1:il5+itar",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
The Authorization: Bearer sk-1234 here is your LiteLLM master key (from .env), not your Consus key — LiteLLM injects CONSUS_API_KEY upstream via the extra_headers in your config.

​
Optional — Open the UI and create a virtual key

Visit http://localhost:4000/ui and log in with your master key (sk-1234). Under Virtual Keys → + Create New Key, you can mint scoped keys with per-key RPM limits and model access — all spend is tracked in the Postgres container that Compose already started.

​
Configure via the UI

  1. Go to Models + Endpoints and click Add Model
  2. Set Provider to OpenAI-Compatible Endpoints (Together AI, etc.)
  3. Set LiteLLM Model Name to openai/<model>:<compliance-level> (e.g. openai/gpt-4.1:il5+itar)
  4. Set API Base to https://api.consus.io/v1
  5. Set OpenAI API Key to any value (required by LiteLLM, ignored by Consus)
  6. Under Advanced Settings, set LiteLLM Params to: 
    {"extra_headers": {"x-api-key": "YOUR_CONSUS_API_KEY"}}
  7. Click Add Model
Do not use the Custom OpenAI provider. It prepends openai/ to the model name sent to Consus Gateway, causing a 400 error. Use OpenAI-Compatible Endpoints instead.