retryable and errors.
Every response carries
x-request-id; include it when you contact support.
Status codes
Rate limits
Limits apply per organization and depend on its plan. They cover the inference endpoints:POST /v1/chat/completions, /v1/images/generations, /v1/images/edits and /v1/videos/generations.
-
Requests per minute. Every inference request returns the same headers as OpenAI:
Over the limit, the request fails with
429 rate_limit_exceeded, andremainingis0. -
Requests in progress. Each accepted request, including an asynchronous job, holds a slot until it finishes. When all slots are taken, new requests fail with
429 concurrency_limit_exceeded.
429 responses carry a retry-after header in seconds. Rejected requests are not charged.
Retrying
- Retry only when
retryableistrue, typically429,500and503, with exponential backoff. For429, wait at leastretry-afterseconds. - Don’t retry other
4xxerrors without changing the request. - For asynchronous requests, send an
Idempotency-Keyso a retry never creates a second job.
Failed generations
An asynchronous job that fails after being accepted hasstatus: "failed" and an error_code such as content_policy_violation, input_unreachable, upstream_rejected_request, model_overloaded or generation_timeout. Failed jobs are not charged.
Failed workflow steps
A workflow step that stops or fails has anerror_code on the step, and a failed run repeats it in the run’s error_code.
A step can also carry the code of the API error that stopped it, such as
content_policy_violation. Steps that fail with a retryable error are retried automatically before they are marked failed.