Skip to main content
Docs
Rate Limits

Rate Limits

How throughput and spend limits work, and how to handle being limited.

Two separate mechanisms bound what a key can do: a spend limit (how much model usage your organization can consume) and a throughput limit (how fast requests are accepted).

Spend limits

Your plan carries a monthly usage allowance (see Plans). Once it's spent, requests are refused until you add a credit pack or the cycle rolls over.

This is the limit most people actually hit. It returns 429 with a message about quota rather than rate.

Throughput limits

Throughput rises with your plan — Free is the most constrained, Ultimate the least, and Enterprise is negotiated.

Fixed requests-per-minute figures aren't published, because the real ceiling moves with upstream provider capacity and with the model you're calling: a frontier model's throughput is far lower than a small model's on any plan. If you need a guaranteed floor, that's an Enterprise conversation.

Per-key limits

Each key can carry its own ceiling, so one runaway service can't drain the whole organization:

  1. Go to Settings > API Keys
  2. Click a key to edit it
  3. Set a max budget and, optionally, restrict which models it can call
  4. Save
SettingDescription
Max budgetSpending cap for this key
Model restrictionsLimit the key to specific models

Giving each environment its own key with its own budget is the cheapest insurance against a bad deploy.

Handling a 429

{
  "error": {
    "message": "Rate limit exceeded. Please slow down.",
    "type": "rate_limit_error",
    "code": "rate_limit_exceeded"
  }
}

Retry with exponential backoff and jitter:

import time
import random
 
def request_with_backoff(make_request, max_retries=5):
    for attempt in range(max_retries):
        try:
            return make_request()
        except RateLimitError:
            if attempt == max_retries - 1:
                raise
            time.sleep((2 ** attempt) + random.uniform(0, 1))

Reducing pressure

Fewer requests — cache responses, batch similar work, and don't re-send conversation history you don't need.

Fewer tokens — set max_tokens when you know the answer should be short, keep prompts tight, and let Alvin route so small prompts land on small models.

Spread the load — queue rather than burst, use separate keys per service so one can't starve another, and schedule batch jobs off-peak.

Monitoring

Analytics > Usage shows requests over time per key. If you're regularly close to a ceiling, that's the signal to optimize or upgrade before it becomes an outage.

Rate Limits | Alvin