> ## Documentation Index
> Fetch the complete documentation index at: https://gateway.consus.io/llms.txt
> Use this file to discover all available pages before exploring further.

# LiteLLM

> Connect LiteLLM Proxy to Consus Gateway for a unified AI gateway in government environments.

[LiteLLM](https://litellm.ai) is an open-source proxy that gives your team a single endpoint for multiple LLM providers. You can point it at Consus Gateway so all requests route through government-authorized infrastructure.

For LiteLLM setup and deployment instructions, see the [LiteLLM docs](https://docs.litellm.ai/docs/).

<video width="100%" style={{ aspectRatio: "16/9" }} controls>
  <source src="https://mintcdn.com/consusindustries/R0PY0LO7eUs6NpFk/assets/litellm.mp4?fit=max&auto=format&n=R0PY0LO7eUs6NpFk&q=85&s=0dce52306f9cfe11196d122f9f4387f9" type="video/mp4" data-path="assets/litellm.mp4" />
</video>

## Authentication

Consus Gateway authenticates via the `x-api-key` header. LiteLLM's built-in API key field is required but ignored by Consus Gateway. Your Consus API key must be passed via `extra_headers`.

## Run with Docker Compose

This walkthrough gets LiteLLM Proxy running locally — with a Postgres database for virtual keys and spend tracking — routing every request through Consus Gateway. The only provider credential you need is your Consus API key.

### Step 1 — Get the Docker Compose file

Download LiteLLM's official `docker-compose.yml` into an empty directory:

```bash theme={null}
curl -O https://raw.githubusercontent.com/BerriAI/litellm/main/docker-compose.yml
```

This starts three services: the LiteLLM proxy, a Postgres database, and Prometheus.

### Step 2 — Add your Consus API key

Create a `.env` file next to `docker-compose.yml`. The first command uses `>` to create the file; the rest use `>>` to append:

```bash theme={null}
# Admin key for the proxy + UI login — change this to your own value
echo 'LITELLM_MASTER_KEY="sk-1234"' > .env
# Encrypts keys stored in the DB — cannot be changed after you add a model
echo 'LITELLM_SALT_KEY="sk-1234"' >> .env
# Your Consus Gateway key — the only provider credential you need
echo 'CONSUS_API_KEY="your-consus-api-key"' >> .env
```

Unlike a standard LiteLLM setup, you do **not** need Azure, OpenAI, or any other provider keys — Consus Gateway brokers every model behind your single `CONSUS_API_KEY`.

### Step 3 — Create your config.yaml

Create a `config.yaml` next to `docker-compose.yml` and paste the full model list below. Every entry already points `api_base` at `https://api.consus.io/v1` and passes your key via `extra_headers: x-api-key: os.environ/CONSUS_API_KEY`, so there's nothing else to edit.

<Accordion title="Full config.yaml with all available models">
  ```yaml theme={null}
  model_list:
    - model_name: claude-sonnet-4-5:il5+itar
      litellm_params:
        model: openai/claude-sonnet-4-5:il5+itar
        api_base: https://api.consus.io/v1
        api_key: dummy
        extra_headers:
          x-api-key: os.environ/CONSUS_API_KEY

    - model_name: claude-sonnet-4-5:fedramp-high
      litellm_params:
        model: openai/claude-sonnet-4-5:fedramp-high
        api_base: https://api.consus.io/v1
        api_key: dummy
        extra_headers:
          x-api-key: os.environ/CONSUS_API_KEY

    - model_name: claude-opus-4-8:fedramp-high
      litellm_params:
        model: openai/claude-opus-4-8:fedramp-high
        api_base: https://api.consus.io/v1
        api_key: dummy
        extra_headers:
          x-api-key: os.environ/CONSUS_API_KEY

    - model_name: claude-opus-4-7:fedramp-high
      litellm_params:
        model: openai/claude-opus-4-7:fedramp-high
        api_base: https://api.consus.io/v1
        api_key: dummy
        extra_headers:
          x-api-key: os.environ/CONSUS_API_KEY

    - model_name: claude-opus-4-6:fedramp-high
      litellm_params:
        model: openai/claude-opus-4-6:fedramp-high
        api_base: https://api.consus.io/v1
        api_key: dummy
        extra_headers:
          x-api-key: os.environ/CONSUS_API_KEY

    - model_name: claude-sonnet-4-6:fedramp-high
      litellm_params:
        model: openai/claude-sonnet-4-6:fedramp-high
        api_base: https://api.consus.io/v1
        api_key: dummy
        extra_headers:
          x-api-key: os.environ/CONSUS_API_KEY

    - model_name: claude-haiku-4-5:fedramp-high
      litellm_params:
        model: openai/claude-haiku-4-5:fedramp-high
        api_base: https://api.consus.io/v1
        api_key: dummy
        extra_headers:
          x-api-key: os.environ/CONSUS_API_KEY

    - model_name: gemini-3-1-pro:il4
      litellm_params:
        model: openai/gemini-3-1-pro:il4
        api_base: https://api.consus.io/v1
        api_key: dummy
        extra_headers:
          x-api-key: os.environ/CONSUS_API_KEY

    - model_name: gemini-3-flash:il4
      litellm_params:
        model: openai/gemini-3-flash:il4
        api_base: https://api.consus.io/v1
        api_key: dummy
        extra_headers:
          x-api-key: os.environ/CONSUS_API_KEY

    - model_name: gpt-4.1:il5+itar
      litellm_params:
        model: openai/gpt-4.1:il5+itar
        api_base: https://api.consus.io/v1
        api_key: dummy
        extra_headers:
          x-api-key: os.environ/CONSUS_API_KEY

    - model_name: gpt-4.1-mini:il5+itar
      litellm_params:
        model: openai/gpt-4.1-mini:il5+itar
        api_base: https://api.consus.io/v1
        api_key: dummy
        extra_headers:
          x-api-key: os.environ/CONSUS_API_KEY

    - model_name: gpt-oss-120b:il5+itar
      litellm_params:
        model: openai/gpt-oss-120b:il5+itar
        api_base: https://api.consus.io/v1
        api_key: dummy
        extra_headers:
          x-api-key: os.environ/CONSUS_API_KEY

    - model_name: gpt-5.4:il5+itar
      litellm_params:
        model: openai/gpt-5.4:il5+itar
        api_base: https://api.consus.io/v1
        api_key: dummy
        extra_headers:
          x-api-key: os.environ/CONSUS_API_KEY
  ```
</Accordion>

For the full list of available models and compliance levels, see the [Model Explorer](/models).

### Step 4 — Enable the config mount

By default, LiteLLM's `docker-compose.yml` ships with the config file commented out. This `sed` uncomments the `volumes` and `command` lines under the `litellm` service in place, so your `config.yaml` is loaded (it leaves a `docker-compose.yml.bak` backup you can delete):

```bash theme={null}
sed -i.bak \
  -e 's|^    # volumes:|    volumes:|' \
  -e 's|^    #  - ./config.yaml:/app/config.yaml|      - ./config.yaml:/app/config.yaml|' \
  -e 's|^    # command:|    command:|' \
  -e 's|^    #  - "--config=/app/config.yaml"|      - "--config=/app/config.yaml"|' \
  docker-compose.yml
```

The compose file also mounts a `prometheus.yml`, so create a minimal one (it must exist before you start, or the stack won't come up):

```bash theme={null}
cat > prometheus.yml <<'EOF'
global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "litellm"
    static_configs:
      - targets: ["litellm:4000"]
EOF
```

### Step 5 — Start the proxy

With `docker-compose.yml`, `.env`, `config.yaml`, and `prometheus.yml` all in the same directory, start everything:

```bash theme={null}
docker compose up
```

Once the logs settle, send a test request through the proxy to a Consus model:

```bash theme={null}
curl -X POST 'http://0.0.0.0:4000/chat/completions' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer sk-1234' \
  -d '{
    "model": "gpt-4.1:il5+itar",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

The `Authorization: Bearer sk-1234` here is your **LiteLLM master key** (from `.env`), not your Consus key — LiteLLM injects `CONSUS_API_KEY` upstream via the `extra_headers` in your config.

### Optional — Open the UI and create a virtual key

Visit [http://localhost:4000/ui](http://localhost:4000/ui) and log in with your master key (`sk-1234`). Under **Virtual Keys → + Create New Key**, you can mint scoped keys with per-key RPM limits and model access — all spend is tracked in the Postgres container that Compose already started.

## Configure via the UI

1. Go to **Models + Endpoints** and click **Add Model**
2. Set **Provider** to `OpenAI-Compatible Endpoints (Together AI, etc.)`
3. Set **LiteLLM Model Name** to `openai/<model>:<compliance-level>` (e.g. `openai/gpt-4.1:il5+itar`)
4. Set **API Base** to `https://api.consus.io/v1`
5. Set **OpenAI API Key** to any value (required by LiteLLM, ignored by Consus)
6. Under **Advanced Settings**, set **LiteLLM Params** to:
   ```json theme={null}
   {"extra_headers": {"x-api-key": "YOUR_CONSUS_API_KEY"}}
   ```
7. Click **Add Model**

<Warning>
  Do **not** use the `Custom OpenAI` provider. It prepends `openai/` to the model name sent to Consus Gateway, causing a `400` error. Use `OpenAI-Compatible Endpoints` instead.
</Warning>