Rate Limits

Concurrency thresholds, requests per minute (RPM), tokens per minute (TPM), and high-throughput scaling.

Toknix provides production-grade concurrency limits by default, preventing unexpected starvation across multi-tenant deployments.

Default Limits

TierConcurrent RequestsRequests / Min (RPM)Tokens / Min (TPM)
Standard Account100 concurrent3,000 RPM5,000,000 TPM
High-Volume Enterprise500+ concurrent10,000+ RPM25,000,000+ TPM

Handling 429 Responses

When your request frequency exceeds the concurrency ceiling, Toknix responds with HTTP 429 Too Many Requests along with a Retry-After header indicating seconds to wait:

json
{
  "error": {
    "message": "Rate limit exceeded. Concurrency ceiling reached (100 active connections).",
    "type": "rate_limit_error",
    "param": null,
    "code": "rate_limit_exceeded"
  }
}

High-Volume Scaling

If your production workload requires higher concurrency thresholds or dedicated throughput pools, reach out via the console or support channels to adjust your account limits.