Async FastAPI Best Practices That Hold Up in Production

The async FastAPI best practices that matter most come down to one rule: never block the event loop. Use async def only with libraries you can await, plain def for blocking ones, one shared client per upstream, a timeout on every await, a cap on concurrent calls to slow services, and a queue for long work.

TL;DR: Most "FastAPI is slow" reports trace back to a blocked event loop. Pick def or async def by whether the library can be awaited, create HTTP and database clients once in the lifespan handler, put a timeout on every await, cap fan-out with a semaphore, send long jobs to a real queue, and measure each route's latency before tuning anything.

This post is the pattern reference. Each section gives the rule, a short code shape, and the failure it prevents. For the story of how these played out in four specific systems, read FastAPI in production: four systems.

Why does one blocking call slow down every request?

An async FastAPI process serves many requests on one event loop. While a handler awaits a database or HTTP call, the loop runs other requests. A blocking call, such as a synchronous driver, requests.get, or a tight CPU loop, stops the whole loop until it returns, so every other request on that process waits behind it.

That's the whole mechanism. The fix is choosing the right kind of route function, and FastAPI's async guide spells out the rule:

1. If the library tells you to call it with await, declare the route with async def and await it.
2. If the library blocks (the guide notes this is still true of most database libraries), declare the route with plain def. FastAPI runs it in an external threadpool and awaits that, so the loop stays free.
3. If you don't know, use plain def.

The damaging case is mixing the two: an async def route that calls a blocking library. It looks async, and it behaves worse than a plain sync route, because nothing moves it off the loop. The same guide warns against the opposite habit too. Declaring a trivial compute-only route as def for speed has the reverse effect in FastAPI, so leave those as async def.

When should a route be def, not async def?

Use plain def whenever the route calls something that blocks and has no awaitable version: an ORM with a sync driver, a PDF parser, an image library, a vendor SDK. Use async def when every slow call inside can be awaited. You can mix both kinds in one app; FastAPI handles each correctly.

Sometimes you're inside an async route and need just one blocking call. Push that call to a thread instead of rewriting the route:

import asyncio

@app.post("/extract")
async def extract(file: UploadFile):
    data = await file.read()
    result = await asyncio.to_thread(parse_pdf, data)  # blocking parser in a thread
    return result

Threads have a limit, though. Python's asyncio docs say that because of the GIL, asyncio.to_thread() can typically only make IO-bound functions non-blocking. Real CPU work, such as model inference or video decoding, belongs in a separate worker process or service. Acasa Sentinel, our hackathon build, runs its detection pipeline that way: a worker per camera under a supervisor, with the FastAPI layer only answering queries.

Why share one client per upstream?

A new HTTP client per request throws away connection pooling and adds a fresh handshake to every call. Create clients once at startup and close them at shutdown. FastAPI's lifespan events give you one place for both: code before yield runs before the app starts, code after it runs once the app has finished.

from contextlib import asynccontextmanager
import httpx

@asynccontextmanager
async def lifespan(app):
    app.state.http = httpx.AsyncClient(timeout=httpx.Timeout(10.0, connect=3.0))
    yield
    await app.state.http.aclose()

app = FastAPI(lifespan=lifespan)

Do the same for database pools, Redis clients, and vector store clients. The lifespan handler is also where clean restarts live. Don't report the service as ready until its pool and key clients are connected, and close them after in-flight requests drain. Deploys, autoscaling and crashes all restart processes, so this code runs more often than people expect.

How long should an await be allowed to wait?

Never forever. An await without a limit is a promise to wait as long as the other side likes, and upstream APIs hang, model providers stall, and cameras stop sending frames. Set a timeout on the client, then wrap anything critical in an overall deadline as well.

HTTPX already helps here. Its timeout docs say the default is to raise after 5 seconds of network inactivity, and httpx.Timeout lets you set the connect timeout separately, as in the lifespan example above. For a whole operation, including retries and parsing, wrap it:

async with asyncio.timeout(20):
    answer = await call_model(prompt)  # asyncio.timeout was added in Python 3.11

For streams, the useful limit is on silence, not total length. A long answer is fine; a connection that has said nothing for a minute is dead. Sentinel applies bounded stream timeouts so a stalled camera returns in seconds instead of wedging a thread. A camera that goes quiet becomes an ordinary, logged error rather than a stuck worker.

How do you stop async code from flooding a slow service?

Cap how many calls are in flight with a semaphore. Async makes it trivial to start a thousand requests at once, and your model provider, database or partner API may not survive that. Python's asyncio.Semaphore keeps a counter, blocks callers when it hits zero, and releases even if the block raises.

model_slots = asyncio.Semaphore(20)

async def call_model_limited(prompt):
    async with model_slots:
        return await call_model(prompt)

This turns a burst into a short queue inside the process and protects the upstream from you. It doesn't replace respecting the provider's own limits. When the provider answers 429, you still need classified errors and jittered backoff, which our rate-limit post covers step by step.

What number goes in the semaphore? Start from the provider's documented limit divided by the number of processes you run, then adjust from measurements.

What should happen when the client disconnects mid-stream?

Stop the work you're paying for. If a user closes the tab during a streamed model answer, every further token is wasted money. Starlette often cancels a streaming response when the client goes away, but only at the points where your generator yields. If it does slow work between yields, check explicitly:

async def stream_answer(request: Request, prompt: str):
    async for chunk in model_stream(prompt):
        if await request.is_disconnected():
            break
        yield f"data: {chunk}\n\n"

That's the async pattern. The full streaming setup, including the event format, auth, proxy buffering and reconnects, lives in streaming AI chat with server-sent events. NoteTube is where we learned it: its chat streams over server-sent events with inactivity timeouts, so dead connections don't hold resources.

Are FastAPI background tasks enough, or do you need a queue?

For small follow-up work, background tasks are enough. FastAPI's BackgroundTasks docs describe them as running after the response is returned, in the same process, and point to bigger tools like Celery for heavy work that can run in separate processes or servers. In practice that also means no retries and nothing survives a restart.

Anything that must survive a restart, retry on failure, or run for minutes belongs in a real queue. Pullsy hands every Instagram sync to a Redis queue with atomic move-with-ack and a scheduler that recovers crashed jobs from missed heartbeats, so the API only enqueues and returns. If you'd rather not run a broker, a Postgres table works too; see Postgres as a job queue.

Work	Where it goes
Send a confirmation email	BackgroundTasks
Write an audit log row	BackgroundTasks or inline
Sync data from a third-party API	Queue with retries
Extract data from a large PDF	Queue, worker process
Generate images or video	Queue, separate worker service

Why should a fan-out buffer drop data instead of blocking?

Because a slow listener should never slow the producer. When a service pushes updates to many consumers, such as a dashboard over WebSockets, give each buffer a size and decide in advance what happens when it fills: drop the oldest update, drop the newest, or disconnect the consumer. Blocking is the one answer that's never right.

The clearest case we've built is the options trading platform, a confidential client system. Its publisher is bounded and drops when full, and it never blocks the trading path; market-data decode runs on its own thread and is handed to the engine loop thread-safely. To be clear, the trading engine itself isn't a FastAPI app, even though FastAPI is part of the platform's wider stack. The rule still carries over to any async service: the core loop must never wait on a consumer.

How do you find out where the time goes?

Measure before you tune. Averages hide the problem, so look at percentiles per route. Then check four things: p50 and p99 latency per route, event-loop lag (if the loop wakes late, something is blocking it), time spent waiting on each upstream, and threadpool saturation if you run many plain def routes.

The trading platform measures latency across 12 stages and exports the percentiles to Prometheus and Grafana with alert rules; its published figures were measured in tape replay. Most APIs need far less than that. The habit is what transfers: time each stage separately, so you know whether the slow part is your code or someone else's service.

In the FastAPI codebases we're asked to review, the first look usually turns up the same short list:

• async def routes that call blocking libraries
• HTTP or database clients created per request
• Awaits with no timeout
• Unbounded fan-out to model providers
• Long jobs in BackgroundTasks
• No per-route latency metrics

Fixing those usually matters more than any server setting. If you're still choosing a framework, our FastAPI vs Flask page covers the async difference.

FAQ

Does adding more Uvicorn workers fix a blocked event loop?

It hides it for a while. Each extra process has its own loop, so a blocking call stalls fewer requests, but every process still freezes when it hits that call. More workers help with CPU-bound load across cores. For a blocking call inside an async route, the fix is plain def, a thread, or a separate worker.

Can I call a sync ORM from an async FastAPI route?

Not directly. A synchronous ORM query inside async def blocks the event loop for its full duration. Either declare that route with plain def so FastAPI runs it in the threadpool, wrap the query with asyncio.to_thread, or move to the ORM's async driver if it has one. Don't mix styles inside one route.

What concurrency limit should a semaphore use for an LLM provider?

Start from the provider's published rate limit, divide it by the number of processes you run, and leave headroom for retries. Then watch for 429 responses and queueing time inside the process. If requests wait long on the semaphore but the provider is calm, raise it; if 429s appear, lower it.

Should health checks be async def?

Yes, if they only await things. A readiness check that pings the database pool and Redis with awaitable clients should be async def and return quickly. Keep a separate, cheaper liveness check that touches nothing external, so a slow database doesn't make the orchestrator restart healthy processes.

Is your FastAPI service slower than it should be? See our FastAPI performance work and send us the slowest route; we'll measure it, find what's blocking, and tell you what to change first.