POST /v1/chat/completions
Creates a chat completion. This is the primary endpoint for generating AI responses. Requests are routed to the appropriate government cloud provider based on the model you specify.
Supports text, multi-turn conversations, tool use, and image input (vision).
Request
Headers
Body Parameters
Message Object
Response
Non-Streaming
Streaming
Setstream: true to receive the response as Server-Sent Events (SSE). Streaming is real and incremental — chunks are relayed as the model generates them, so tokens arrive over time (first-token latency reflects the model, not the full completion). Each event is a JSON chunk prefixed with data: , ending with data: [DONE].
stream_options: {"include_usage": true} to receive token usage in the final chunk. Streaming requests are not bound by the 5-minute non-streaming ceiling; they run up to ~15 minutes (see Request Timeout), which lets long reasoning turns complete.
Finish Reasons
Reasoning
Some models support extended thinking before producing the final response. The model spends additional output tokens reasoning through the problem internally, then returns the visible answer. This generally improves accuracy on harder tasks at the cost of latency and tokens. Passreasoning_effort on the request to control it. Reasoning is currently supported on all Claude models (Bedrock and Vertex), GPT-5.1 and GPT-5.6 Sol (Azure), GPT-OSS-120B, GPT-5.4, GPT-5.6 Luna and GPT-5.6 Terra, and Grok 4.6 (Bedrock), and Gemini 3.5 Flash, Gemini 3.8 Flash, and Gemini 4 Argon (Vertex). Other models silently ignore the parameter.
Effort levels
On Gemini models these levels map to Google’s native
thinking_level. Gemini thinking cannot be fully turned off, so none maps to the lowest level each model accepts, not zero — the model may still think briefly on hard prompts. On Gemini 3.5 Flash the mapping is none→minimal, low→low, medium→medium, high→high. Gemini 3.8 Flash and Gemini 4 Argon do not support the minimal thinking level, so none maps to low instead. xhigh is not a Gemini tier; it is clamped to high.
If you omit reasoning_effort, the gateway applies the per-model default from the catalog. Today that is medium on Claude (except Claude Sonnet 5.5 and Claude Haiku 5.5), GPT-OSS-120B, GPT-5.4, GPT-5.6 Luna/Terra/Sol, Gemini 3.5 Flash, Gemini 3.8 Flash, and Gemini 4 Argon, high on GPT-5.1, Claude Sonnet 5.5, and Claude Haiku 5.5 (Anthropic’s own default for those models), and low on Grok 4.6 (xAI’s own default — Grok always reasons; low through xhigh set how much). To explicitly disable reasoning on a capable model, send "reasoning_effort": "none".
Some models always reason and cannot be turned off. On Claude Opus 5.5 and Claude Sonnet 5.5, extended thinking is always on upstream — none means the gateway sends no thinking configuration at all, and the model still reasons at its own default depth (use low to minimize the spend). Sending "thinking": {"type": "disabled"} on Messages returns a 400 from the provider. Gemini behaves the same way, as described above.
Claude Haiku 5.5 also keeps reasoning when you send none on this endpoint, but it can run with thinking off: send "thinking": {"type": "disabled"} on Messages with effort low, medium, or high (xhigh with thinking disabled returns a 400).
Token budget
The thinking budget counts againstmax_tokens. On Claude, the gateway clamps the budget to at most max_tokens - 1024 so room remains for the visible answer, with a floor of 1,024 thinking tokens. If you do not set max_tokens, the gateway sets it to budget + 4096 so the request always has headroom. Setting max_tokens very low will reduce the effective thinking budget.
GPT-5.1, GPT-5.6 Sol, GPT-OSS-120B, GPT-5.4, GPT-5.6 Luna/Terra, Grok 4.6, and Gemini 3.5 Flash, 3.8 Flash, and 4 Argon manage their own reasoning internally from reasoning_effort; the gateway does not clamp their thinking budget against max_tokens the way it does on Claude.
Sampling parameters
On Claude (Bedrock and Vertex), GPT-5.1 and GPT-5.6 Sol (Azure), and GPT-5.4 / GPT-5.6 (Bedrock),temperature and top_p are not compatible with reasoning: when reasoning_effort is a non-none value, both are stripped before forwarding upstream to avoid 400 errors. GPT-OSS-120B (Bedrock) and Gemini 3.5 Flash and 3.8 Flash (Vertex) accept temperature and top_p alongside reasoning, so they are preserved. Grok 4.6 (Bedrock) does not accept temperature or top_p at all, with or without reasoning, so the gateway always strips both for that model rather than returning a 400.
Tool choice
When reasoning is enabled on a Claude model, a forcedtool_choice (required or a specific tool) is coerced to auto. The model needs latitude to think before deciding whether to call a tool. On Claude Opus 5.5 and Claude Sonnet 5.5 this coercion always applies, because these models reason unconditionally and reject a forced tool_choice upstream; ask for the tool in your prompt when you need it called. Tool definitions still pass through unchanged. GPT-5.1, GPT-5.6 Sol, GPT-OSS-120B, GPT-5.4, GPT-5.6, and Grok 4.6 have no equivalent restriction. On Gemini models (reasoning or not), tool definitions pass through but a forced tool_choice is not translated to Vertex AI — the model always selects tools automatically.
Reasoning token usage
When the model used reasoning, the OpenAI-shapeusage object includes a completion_tokens_details.reasoning_tokens count. Reasoning tokens are already included in completion_tokens; the field is surfaced separately so you can attribute spend.
completion_tokens.
Example
Tool Use
Pass atools array to let the model call functions. The model will respond with tool_calls when it wants to use a tool, and you send back the result in a tool message.
Tool use is supported on all available Claude, Gemini, and GPT models. See Reasoning > Tool choice for how tool_choice interacts with reasoning_effort on Claude.
Tool Definition
Functionname must match ^[a-zA-Z0-9_-]{1,64}$ (ASCII letters, digits, underscore, hyphen; 1 to 64 characters), matching the OpenAI and Anthropic tool name specs. Names outside this pattern return 400 invalid_request_error. description is limited to 65,536 characters.
Tool Call Response
When the model calls a tool, the response includestool_calls instead of (or alongside) content:
Sending Tool Results
Include the tool call result in a follow-up message withrole: "tool":
Tool history without tools
You can send a conversation that contains earliertool_calls and tool messages without a tools array, or with tool_choice: "none". The OpenAI API allows this, and agent frameworks commonly do it on a summarizing or final-answer turn.
On models served through AWS Bedrock’s Converse API (Claude at the :itar and :il5 levels, Grok 4.6, and GPT-OSS-120B), Bedrock does not accept tool calls and results in a request that defines no tools. On those models the gateway passes the earlier tool calls and results to the model as plain text, so the model still sees what happened but cannot call a tool on that turn. To let the model call tools again, include tools. When tools is present (and tool_choice is not none), the history is sent in its native form, unchanged.
Rejected Tool Schemas
The gateway rejects tool definitions whose parameter schemas include property names that clearly describe an outbound destination. These schemas are the shape of a data exfiltration tool, and accepting them at the gateway would be careless regardless of what the caller intends to do with the result. Rejected property names (case insensitive):- Destination names:
destination_url,dest_url,dst_url - Webhook names:
webhook,webhook_url,webhooks - Callback names:
callback,callback_url - Send and forward names:
forward_to,forward_url,send_to,post_to,push_to - Target names:
target_url,target_host - Named sinks:
upload_url,ingest_url,notification_url,notify_url,report_url,sink_url - Obvious intent:
exfil_url,exfiltrate
items schema, a $defs entry, or a oneOf branch will not bypass it.
Ambiguous names that can legitimately read as well as write are not rejected. url, uri, endpoint, host, and hostname all pass at this layer. A database connection tool with host and port, or a tool that reads a record from an internal API by url, continues to work. The runtime check described in the next section handles what actually appears in the arguments.
destination on its own is also allowed. Agent tools commonly use it for file paths, and a URL placed in it is still flagged by the runtime check below.
A rejected request returns 400 invalid_request_error and identifies the offending tool and parameter:
Tool Call Governance Metadata
When a model returns tool calls, the gateway scans eacharguments payload for outbound destinations (URLs with a scheme like https://, ftp://, s3://, data:, mailto:, and raw IPv4 addresses). When any are found, the response includes an advisory field called x_consus_governance alongside the standard OpenAI body. The tool call itself is not modified. You still receive the real arguments so your application can run.
Response shape when destinations are detected:
finish_reason). Clients that already parse the final chunk for usage can pick up x_consus_governance from the same place.
Examples
Basic Completion
Streaming
With System Prompt and Parameters
Multi-Turn Conversation
Image Input (Vision)
All current Claude and Gemini models accept image input, as do the Azure GPT models (GPT-5.1, GPT-5.6 Sol, GPT-4.1) and GPT-5.4, GPT-5.6 Luna, GPT-5.6 Terra, and Grok 4.6 (Bedrock). GPT-OSS-120B is text-only and does not accept image input. See the models page for the live list, or readinput_modalities from GET /v1/models.
An image_url part sent to a model whose input_modalities do not include image is rejected before it reaches the provider with a 400 invalid_request_error, code: "unsupported_input_modality", that names the model — for example Model 'gpt-oss-120b' does not accept image input (accepted: text). Remove the image_url content part or choose a model whose input_modalities on GET /v1/models include "image". Do not retry it as sent; see Errors.
Supported formats: jpeg, png, gif, webp
How images are sent
To include an image, setcontent to an array instead of a plain string. Each element is a content part with a type field, either "text" or "image_url".
TheA base64 data URI looks like this:image_urlfield name is inherited from OpenAI’s API format. Despite the name, you do not pass a URL. You pass the raw image bytes encoded as a base64 data URI. External URLs (https://...) are rejected with400to prevent data exfiltration.
data:<mime-type>;base64,, and put the whole string in the url field.
Request Format
contentcan contain any number oftextandimage_urlparts in any order- Images are only valid in
usermessages.systemandassistantmessages must use a plain string forcontent. - Multiple images per message are supported (up to 20)
Size Limits
Requests exceeding these limits are rejected with
400.
Integrations handle this for you
If you’re using an integration like OpenCode or Cline, you don’t need to do any of this manually. Those tools encode images automatically when you paste a screenshot. Just paste and send. The base64 encoding and data URI formatting is handled behind the scenes. If you’re calling the API directly, read on.Complete Examples
bash (curl)Document Input (PDF)
All Claude and Gemini models support PDF document inputs. GPT models do not — neither the Azure GPT models (GPT-5.1, GPT-5.6 Sol, GPT-4.1) nor GPT-OSS-120B, GPT-5.4, GPT-5.6, and Grok 4.6 on Bedrock. Afile part sent to a model whose input_modalities do not include pdf is rejected the same way as an unsupported image: a 400 with code: "unsupported_input_modality" naming the model and "pdf".
Supported file types: application/pdf
How documents are sent
To include a PDF, add a content part with"type": "file" to the content array. The file_data field must be a base64 data URI. External URLs are rejected for the same no-egress reason as images.
- Documents are only valid in
usermessages.systemandassistantmessages must use plain strings. filenameis metadata passed to the model; it is treated as untrusted input and sanitized before use- Text, images, and files can be mixed in a single message’s
contentarray
Size Limits
The combined image + file budget is shared at 4.5 MB per message, enforced by the gateway’s request payload limit.