This page lists the operating facts a production team can plan around today: concurrency and rate limits, request size and per-request timeouts, what each error code means and what to do about it, the structured-output contract, and how change is managed. Every figure on this page is the behavior of the running service, not a target.
Pareta is a verified-inference provider built for repeatable
production AI workloads. Through one OpenAI-compatible endpoint and one model
ID, "auto", your application can access Pareta’s fleet of
open-weight specialists, including several distilled in-house. Pareta checks
every result against the workload’s quality bar and automatically escalates
to a frontier model whenever the specialist path cannot meet it. You are
charged only for the model that produces the final response. For workloads
handled by Pareta specialists, serving costs can be 30–140× lower than the
comparable frontier model, and you can verify the quality and economics on
your own production data.
The ceilings and deadlines in effect on the service today. Concurrency and rate ceilings apply per account on every metered route; the body cap applies to every route.
| Limit | Value | Definition |
|---|---|---|
| Concurrency | 8 requests in flight | Metered requests one account may have in flight per serving instance; the fleet-wide ceiling is higher and scales with instances (the 120/min rate limit is fleet-wide). A request beyond the eighth is refused with HTTP 429 and Retry-After: 5; nothing is served or billed. |
| Rate limit | 120 requests / minute | Metered requests admitted for one account in a rolling one-minute window. Excess requests are refused with HTTP 429 and Retry-After: 15; nothing is served or billed. |
| Request body | 30 MB | Maximum request body on every route. A larger body is refused with HTTP 413. |
| Per-request timeout | 45 s default 120 s ICD-10 coding 180 s long-document extraction | The deadline for the specialist path of one request through auto. When the deadline passes, the request is escalated to a frontier model rather than left waiting on the specialist. A cold specialist can add up to 60 s of boot time before that deadline applies. A request that no path can serve returns an explicit error and is not billed. |
| Exhausted balance | HTTP 402 | A request the account balance cannot cover is not served and returns 402. Nothing is billed. |
| SDK defaults | 600 s timeout · 10 s connect (Python) 2 automatic retries | Defaults of the Pareta Python and TypeScript SDKs; the TypeScript client uses a single 600 s budget. Retries cover 408, 409, 429, 500, 502, 503 and 504 with backoff, honoring Retry-After; stable 4xx errors are raised immediately. |
| Receipts | X-Pareta-Billed X-Pareta-Frontier-Would-Have-Cost | Carried on every chat-completion response (and as SSE comment lines on streams). auto responses also carry the route diagnostics X-Pareta-Served-By, X-Pareta-Specialist, X-Pareta-Escalated, X-Pareta-Route-Version, and X-Pareta-Input-Required on clarification responses; tool-calling turns carry the receipt headers only. Direct endpoints report cost in the dashboard. X-Pareta-Billed is the exact amount billed, in micro-dollars. X-Pareta-Frontier-Would-Have-Cost is an estimate: usage tokens × the frontier provider’s list price; the frontier model is not run on live requests. |
Control-plane reads and authentication do not count against the concurrency or rate ceilings. The per-request timeout is the deadline Pareta applies on the serving side; set your client’s timeout above it (the SDK defaults already are), because long-document requests legitimately run for minutes.
Latency is reported per workload as p50 and p95. Each figure is the time to the last byte of a non-streaming request, measured from a client in the United States against warm serving pools.
| Workload | p50 | p95 | Requests |
|---|---|---|---|
| Intent classification | 1.32 s | 1.45 s | 200 of 200 OK |
| ICD-10 coding | 5.07 s | 10.56 s | 200 of 201 OK · 1 non-200 |
| Invoice extraction | 33.01 s | 64.14 s | 200 of 200 OK |
| Text embedding | 8.08 s | 9.73 s | 200 of 200 OK |
| Document reranking | 10.76 s | 11.52 s | 200 of 200 OK |
Measured 2026-08-22: 200 non-streaming requests per workload through the production API, client in the United States, warm serving pools, 4 requests in flight, throttled under the account rate limit; p50/p95 of time to last byte. Inputs drawn from the benchmark test sets. Non-200 responses are excluded from the percentiles and counted in the last column. Latency varies with input length and routing; this is a point-in-time measurement, not a commitment.
With response_format set, a streamed response
delivers the JSON as a single buffered chunk after validation rather than
token by token.
Every failure is an explicit HTTP status with a JSON body,
{"detail": "…"}, that names the cause. A request that cannot be
served returns an error, not a silent empty answer, and a request that fails
is not billed.
| Status | Meaning | What to do |
|---|---|---|
| 400 | Request validation failed: a malformed body or parameter, a malformed response_format, tools combined with response_format, or stream combined with tools. | Fix the request; it is not retried. The detail names the problem. |
| 401 | The API key is missing, invalid, or revoked. | Check the key. Not retried. |
| 402 | The account balance cannot cover the request. It is not served and nothing is billed. | Top up in the dashboard. Not retried. |
| 413 | The request body exceeds 30 MB. | Reduce the payload. Not retried. |
| 429 | The account is over its concurrency ceiling (8 in flight; Retry-After: 5) or its rate ceiling (120 per minute; Retry-After: 15). Nothing is served or billed. | Wait the Retry-After seconds and retry. The SDKs do this automatically. |
| 502 | auto could not complete the request: every serving path failed, or a structured request produced no schema-conformant answer. Nothing is billed. | Retry; the SDKs retry automatically. If one input fails repeatedly, treat that input as unservable rather than retrying further. |
| 503 | A serving backend behind auto is warming or briefly unavailable, or an upstream model provider is unavailable. | Retry shortly; the SDKs retry automatically. There is nothing to start or fix on your side. |
The Pareta SDKs retry 408, 409, 429, 500, 502, 503 and 504 up to
two times by default, waiting the server’s Retry-After when present
(capped at 30 s) and otherwise backing off exponentially with jitter. Retries
cover the initial handshake only: a stream that drops mid-way raises and
should be re-issued from the top. The OpenAI client’s built-in retries handle
429 and 5xx the same way. Full exception map and tuning:
Errors, retries & timeouts →
If your code parses the output, send the schema on every
request. response_format is the contract; the prompt is not.
response_format with type: "json_schema" is the supported contract for structured output and is what constrains decoding on Pareta specialists. Schema instructions given only in the prompt are not enforced. The root schema must be an object; local $ref / $defs are supported, external $ref URLs are rejected with a 400, and schemas are capped at 50 KB.type: "json_object" guarantees the content parses as JSON, with no schema check. A malformed response_format returns HTTP 400 with a specific message; it is never silently ignored.response_format set, temperature is not applied, and a streamed response arrives as one buffered chunk after validation.response_format (json_object, json_schema), tools / tool_choice, image inputs on document tasks, /v1/embeddings, /v1/rerank, audio transcription and speech, and image generation. tools cannot be combined with response_format or with stream (400). Unsupported parameters are ignored or rejected as listed in the compatibility section of the docs.What Pareta holds constant, and what it does not.
auto.response_format conformance is enforced on every served response, including frontier-served ones.Dedicated GPU capacity, custom specialists, and support commitments are by arrangement. Pareta does not publish an SLA today.
detail message, and the time of the request.Use the OpenAI client already in your application, set the
model ID to auto, and compare Pareta with your current frontier
model on representative traffic before switching production.