Skip to main content

​
API Rate Limits

Each API key is assigned a usage plan with the following default limits: These are enforced at the API Gateway level before your request reaches the application. Requests that exceed these limits receive a 429 response. Limits can be adjusted per API key. Book a call to discuss your needs.

​
Model Rate Limits

Each model has its own upstream provider limits. These are shared across all Consus Gateway customers and are separate from your API key limits.
Model rate limits are set by the upstream provider and may change. If you receive a 429 error but are within your API key limits, you’ve hit a model-level limit. Consus Gateway automatically retries these once before returning the error.Gemini 3.x models run on Google’s Dynamic Shared Quota: Vertex AI enforces no fixed per-project RPM/TPM for these models (verified for Gemini 3.5 Flash against the Cloud Quotas API). Capacity is shared elastically across Google Cloud customers, so a 429 reflects transient regional saturation rather than a fixed cap.

​
What Happens When You Hit a Limit

  1. API key limit exceeded: You get an immediate 429 from the API Gateway (no JSON body).
  2. Model limit exceeded: Consus Gateway retries once with a 2 second delay. If the retry also fails, you receive a 429 with type: "rate_limit_error" and code: "upstream_rate_limit".
The code field lets you tell the difference: if code is "upstream_rate_limit", it’s a model-level limit. If there’s no JSON body at all, it’s your API key limit. For client-side handling, retry on 429 with exponential backoff: start at 1 second, double each attempt, and cap at 5 retries.

​
Request Timeout

In addition to rate limits, every non-streaming request has a maximum duration of 300 seconds (5 minutes). If the model has not finished generating within that window, the request is terminated and you receive a 504 with type: "timeout":
Streaming requests get a longer budget. With stream: true the gateway relays tokens incrementally, which keeps the connection alive, so streaming requests run up to about 15 minutes instead of the 5-minute non-streaming ceiling. This is the recommended way to run long reasoning turns — stream them and they complete instead of timing out. The 5-minute limit above applies only to non-streaming (buffered) requests. Long-running requests are almost always reasoning workloads. If you hit the timeout consistently:
  • Stream the request (stream: true). This is the simplest fix — streaming raises the ceiling to ~15 minutes, so most long reasoning turns that fail buffered will complete when streamed.
  • Lower the reasoning effort (reasoning_effort for GPT models, thinking.budget_tokens for Claude). xhigh/large-budget requests on complex prompts can exceed 5 minutes buffered.
  • Reduce max_tokens if you’re requesting very long outputs.
  • Split the task into smaller calls — several shorter requests are also more resilient than one long one.
A request that timed out at the 5-minute ceiling will usually time out again if retried unchanged. Retry 504s only after reducing the work in the request. You are not billed for requests that time out.