API Rate Limit Backoff Strategy That Never Drops Data

A good API rate limit backoff strategy runs every call from a queue, sorts each error into retry, wait, re-authorize, or give up, and backs off exponentially with random jitter and a hard cap. It honors Retry-After or the provider's usage headers, locks per account so one job never runs twice, and alerts when retries pile up.

TL;DR: Never retry inside a user's request. Queue the call, classify the error, then wait a random time between zero and an exponentially growing ceiling, capped (Pullsy caps at 30 minutes). Treat expired access as its own state, make writes idempotent, put each provider behind one adapter, and alert on the oldest stuck job.

Why do naive retries make rate limits worse?

Naive retries make things worse because every worker hits the limit together and retries together. "If it fails, try again" keeps the burst alive, so the limit stays tripped. A fixed delay doesn't help either: all the retries line up and arrive at the same moment, just a few seconds later.

Three ideas fix it. Retry only errors that can succeed later. Wait longer after each failure. Randomize the wait so retries spread out. AWS's Builders' Library article on the subject describes exponential backoff as the most common pattern and says jitter "adds some amount of randomness to the backoff to spread the retries around in time" (AWS, Timeouts, retries, and backoff with jitter).

We learned this on Pullsy, our analytics product for creators. It syncs Instagram insights through Meta's API on a schedule, and Meta keeps account metrics for only up to 90 days, so a sync that fails quietly for too long loses data for good. Why that matters to creators is covered in keeping Instagram insights history. This post is about the mechanics.

How should you classify API errors before retrying?

Classify by what will fix the error: time, a new token, or nothing. Pullsy maps Meta's error codes to specific handling instead of treating every failure the same. The table below is the general version we'd use for any partner API.

Error kind	Examples	What to do
Temporary	Timeouts, 5xx responses, connection resets	Retry with backoff
Rate limited	HTTP 429, provider throttle codes	Wait for the provider's signal, then retry
Auth expired or revoked	401, "token invalid" codes	Stop; mark the account as needing sign-in
Permanent	400 for bad input, 404 for deleted objects	Don't retry; record and move on
Unknown	Anything not mapped yet	Retry a few times, then park for review

The auth row deserves emphasis. In Pullsy, re-authorization is its own state. Retrying a revoked token forever burns quota and never succeeds. Mark the account, tell the user, and resume when they sign in again.

What does a worked rate limit backoff schedule look like?

A worked schedule starts small, doubles, and stops growing at a cap. With full jitter, each actual wait is a random value between zero and the ceiling for that attempt. Here's an example with a 5-second base and the 30-minute cap Pullsy uses.

Attempt	Ceiling (base × 2ⁿ)	Actual wait with full jitter
1	5 s	random 0 to 5 s
2	10 s	random 0 to 10 s
3	20 s	random 0 to 20 s
4	40 s	random 0 to 40 s
5	80 s	random 0 to 80 s
6	160 s	random 0 to 160 s
7	320 s	random 0 to 320 s
8	640 s	random 0 to 640 s
9	1,280 s	random 0 to 1,280 s
10 and later	1,800 s (cap)	random 0 to 30 min

The base and the attempt count are illustrative; the 30-minute cap is Pullsy's real setting. Why cap at all? Without one, a job that failed fifteen times might sleep for days, and for data that expires, days are too long. The AWS article makes the same point: implementations "typically cap their backoff to a maximum value."

import random

def backoff_seconds(attempt: int, base: float = 5, cap: float = 1800) -> float:
    ceiling = min(cap, base * (2 ** (attempt - 1)))
    return random.uniform(0, ceiling)  # full jitter

Should you trust Retry-After or the provider's usage headers?

Trust whatever signal the provider actually sends, and only fall back to your own schedule when it sends nothing. HTTP 429 "MAY include a Retry-After header" telling you how long to wait (RFC 6585, section 4). "May" is the important word: many APIs don't send it.

Meta is a good example. Its Graph API reports usage in headers such as X-App-Usage and X-Business-Use-Case-Usage, and the business use case header includes estimated_time_to_regain_access, in minutes. Throttling arrives as error codes, such as 4, 17, 613, and 80002 for the Instagram limit (Meta, rate limiting). Read those headers on every response, slow down before you hit the ceiling, and use the regain-access estimate as your wait when it's present.

Run third-party calls from a queue

Calls that can fail belong in background jobs because a queue gives a delayed retry somewhere to live. The user's request just enqueues work and returns. They see "syncing" instead of an error page, and a retry 20 minutes later doesn't hold a web worker open.

Pullsy uses a reliable Redis queue: an atomic move-with-ack so a job isn't lost if a worker dies mid-call, a delayed-retry set that holds jobs until their backoff expires, and a scheduler that recovers crashed jobs from missed heartbeats. If you'd rather not run Redis, Postgres can do the same job; we cover that design, including leases and reclaiming orphaned jobs, in a Postgres job queue with SKIP LOCKED.

How do you stop the same job running twice?

Lock per account. When syncs are scheduled and retried, two copies of one account's sync can end up running together. That doubles API usage and can corrupt data with overlapping writes. Pullsy uses per-connection locks so one account never syncs twice at the same time. A lock with an expiry, keyed by account, is enough.

Spend the remaining quota on purpose:

• Spread scheduled work. Don't sync every account at midnight. Stagger start times.
• Fetch only what changed when the API allows it.
• Prioritize. A user waiting on screen comes before a background refresh.
• Track usage against the limit, so you see the ceiling before you hit it.

How do you make retried writes safe?

Make them idempotent, so sending the same write twice has the effect of sending it once. Reads are harmless to repeat. A duplicated payment or message isn't.

Stripe's API shows the standard approach. The client sends an Idempotency-Key header, ideally a V4 UUID, and "if a connection error occurs, you can safely repeat the request without risk of creating a second object." Keys can be up to 255 characters and may be pruned after they're at least 24 hours old (Stripe, idempotent requests). Where a provider has no such header, record what you sent and check before resending. For incoming webhooks, store each event ID and ignore repeats.

Timeouts come first

Set timeouts before anything else, because backoff only helps when a call actually fails. A call that hangs never fails; it just holds a worker. AWS calls setting a timeout on any remote call a best practice at Amazon, and warns that too low a timeout causes extra retries while too high a timeout wastes resources.

Set a connect timeout and a total timeout on every external request, and send a timeout through the same backoff path as any temporary error. For long model calls, time out on silence rather than total duration, so a slow but working stream isn't cut off. We show that for streamed AI responses in SSE streaming for AI chat.

One adapter per provider

An adapter gives each provider one home for its auth, error map, and limits. When the provider changes its API or has an outage, the fix lives in one file. Pullsy's connector registry works this way and already reaches beyond Instagram.

The same idea shows up elsewhere in our work. Kraftr has pluggable providers for image generation and for storage, swappable by config. SigmaCPM calls models through OpenRouter, so changing a model doesn't mean rewriting the pipeline.

How do you know retries are hiding a problem?

You know because you measure them. Retries hide problems by design, so someone has to see the count. Four signals catch most trouble:

1. Retries per provider and per error kind.
2. The age of the oldest queued job, with an alert above what you can tolerate.
3. Accounts moving into the needs-sign-in state in unusual numbers, which often means the provider changed something.
4. A list of jobs that ran out of retries, with the last error.

Of the four, the oldest-job age is the one we'd set up first. Retry counts rise and fall with normal traffic. A job that's been waiting six hours is always news. Our reliability and monitoring work sets these alerts up.

FAQ

What is full jitter in exponential backoff?

Full jitter means each wait is a random value between zero and the current exponential ceiling, instead of the ceiling itself. If the ceiling for the third attempt is 20 seconds, the job waits anywhere from zero to 20. That spreads retries from many workers across time, so they don't hit the provider together.

How many times should a job retry before giving up?

It depends on how long the data stays available. For data that expires, retry on a capped schedule until the deadline gets close, then alert a person. For permanent errors, don't retry at all. A fixed attempt count without a cap or deadline is the setting that most often loses data.

What if the API documentation doesn't list its error codes?

Log every response and error code you see, map them as you learn, and keep unknown errors in a review list rather than retrying forever. Within a few weeks of real traffic, most codes are mapped. The unknown bucket then becomes a useful early warning that the provider changed something.

Does a cap on backoff ever cause problems?

A cap can be too low. If a provider's limit window is an hour and your cap is one minute, you'll keep hitting the limit. Pick a cap near the provider's reset window, and prefer the provider's own wait signal, such as Meta's regain-access estimate, when it sends one.

Losing data to a fragile third-party API, or about to integrate one? See our API development work and send us the integration that keeps failing.