What does an API rate limit error (429) mean and how do I fix it?

If you build with AI APIs, sooner or later you will see a 429 error, usually with a message like "rate limit exceeded" or "too many requests." It means the provider is temporarily refusing requests because you have sent more than your allowed amount within a time window.

AI providers set rate limits to share capacity fairly and protect their systems. Limits are usually measured in several ways:

Requests per minute.
Tokens per minute, covering both input and output.
Sometimes tokens per day or concurrent requests.

Limits depend on your account tier. New accounts typically start with lower limits, which increase as your usage and spending grow.

Common causes:

1. Sending many requests at once, such as processing a whole file of records in a fast loop.
2. Large prompts that use up the tokens-per-minute allowance quickly.
3. Several applications or team members sharing the same API key.
4. A bug that sends requests in an infinite loop or retries too aggressively.
5. A sudden traffic spike in your application.

Immediate fixes:

1. Implement retries with exponential back-off. When you get a 429, wait a short time and retry, increasing the wait each time, such as 1 second, then 2, then 4, with a little randomness. Many official SDKs already do this automatically.
2. Read the retry headers. Responses often tell you how long to wait before retrying.
3. Slow down. Add a delay between requests or limit how many run in parallel.

Long-term fixes:

1. Reduce tokens. Shorten prompts, trim conversation history and cap output length.
2. Use caching. Reuse results for repeated questions, and use prompt caching for repeated long instructions.
3. Use batch processing. For non-urgent bulk jobs, many providers offer batch APIs with separate, higher limits and lower prices.
4. Queue requests. Use a job queue that processes requests at a controlled rate.
5. Spread load across models, using smaller models for simple tasks.
6. Request a higher limit. Check your provider's dashboard for your current tier and how to increase it.

Also distinguish 429 from other errors. A 429 can also appear when you have run out of credits or reached a spending cap, so check your billing. Server errors like 500 or 529 indicate the provider is overloaded, which also calls for retries with back-off.

Good error handling for rate limits is essential for any production AI application.