Skip to main content
Agent harnesses — Claude Code, the Claude Agent SDK, OpenAI-SDK tool loops — grow a conversation on every turn. Long sessions eventually approach the model’s real context ceiling on the platform serving it, which is not always the number the model’s commercial documentation advertises. A harness that believes the wrong ceiling never compacts proactively; it discovers the truth as a 400 on a live request.

​
Effective context windows

The limit that matters is the input (prompt) budget enforced by the gov cloud platform behind the route — the context_window in the Model Explorer is the authoritative per-model figure. The ceilings agent sessions most commonly reach:

​
What hitting the ceiling looks like

A request over the input limit returns 400 invalid_request_error with code: "context_window_exceeded" — including on stream: true requests, where it arrives as a real HTTP 400 before any events. When the provider reports them, the message carries the attempted and maximum token counts. This error is terminal for the request as sent: retrying it verbatim can never succeed. Trim, summarize, or compact the conversation and resend. See Errors for the envelope.

​
Claude Code and the Claude Agent SDK

Claude Code determines when to auto-compact from what it believes the model’s context window is — via a built-in table of stock Anthropic model ids. The gateway’s composite ids (claude-sonnet-4-5:il5+itar, claude-opus-4-8:fedramp-moderate, …) are not in that table, so calibrate the client explicitly:
With the window set, proactive auto-compaction fires before the ceiling and sessions stop bouncing off 400s. Two more rules:
  • Only claim the window the route actually serves. Appending [1m] to a model id makes the client assume a 1M-token window — on a 200K route (e.g. claude-sonnet-4-5:il5+itar) that disables proactive compaction entirely, and on the 1M models it adds nothing over setting the window explicitly.
  • Pre-count when precision matters. POST /v1/messages/count_tokens is supported on the Claude routes (both Bedrock- and Vertex-served) and returns the exact input count for a candidate request.

​
Custom loops (OpenAI or Anthropic SDK)

If you run your own agent loop, two behaviors keep it healthy against the gateway:
  1. Track context from response usage. Each response’s usage block reports actual input tokens; budget the next turn against the table above, leaving headroom for max_tokens and any thinking budget.
  2. Classify before retrying. Retry 429, 502, 503, and timeouts with backoff. Never retry a 400 — and specifically treat code: "context_window_exceeded" as a signal to compact, not resend. Streamed requests now surface pre-stream failures with their real HTTP status, so ordinary SDK status handling works; only errors arriving mid-stream (after events have flowed) come as in-band SSE error frames.

​
Compaction options