FastAPI backends for AI apps
The service between your app and the model.
An AI product needs a backend that holds the model keys, runs retrieval, streams the answer, and records what it cost. Acasa Labs builds that backend in FastAPI. NoteTube's AI runs this way: the web and mobile apps call one FastAPI service, and the keys never leave it.
The problem
Calling the model from the app works until someone opens the network tab.
A prototype calls OpenAI from the browser. A product cannot. The key leaks, every client reimplements the prompt, nobody knows what a conversation cost, and switching models means shipping three apps. The fix is a backend service that owns every model call. FastAPI suits it: async by default, streaming built in, and Python, where the AI libraries are.
What we build with FastAPI
- 01
One place for every model call
Keys, prompts, model choice, and fallbacks in one service. Web, mobile, browser extension, and partner clients all call the same endpoints.
- 02
Streaming answers
Server-sent events from FastAPI to the client, with timeouts and cancellation so a closed tab stops the model call too.
- 03
Retrieval over your data
Ingestion, chunking, embeddings, and hybrid search in Qdrant or pgvector, scoped per user or per workspace so people only retrieve what they may read.
- 04
Cost, limits, and evaluation
Token use and cost logged per call, usage limits per plan, and a test set of real questions scored on every change.
From production
What this looks like in systems we run.
Keys stay server-side
NoteTubeNoteTube's AI keys live only in its FastAPI service. A Next.js web app, an Expo app, a Chrome extension, and a remote MCP server all go through it.
Model choice per conversation
NoteTubeStudents can pick OpenAI, Gemini, Claude, Llama, or Mistral per conversation. Because every call goes through one service, adding a provider is a backend change, not an app release.
Retrieval scoped per user and session
NoteTubeUploads are parsed, chunked, embedded, and stored in Qdrant with per-user, per-session scoping, and chat streams back grounded in that material over server-sent events.
Local inference when data cannot leave
Acasa SentinelAcasa Sentinel runs detection, OCR, and embeddings locally behind FastAPI, with pgvector for appearance search. No footage or number plate is sent to a third-party API.
Honest trade-offs
When a separate AI service is overkill
If the product makes one simple model call, such as a summary on save, a serverless function next to your existing backend may be enough. We split the AI into its own FastAPI service when more than one client needs it, when calls stream or run long, or when retrieval is involved.
What we reach for
- FastAPI, server-sent events
- OpenAI, Anthropic, Gemini, open-weight models
- Qdrant, pgvector
- Hybrid BM25 + vector search, reranking
- Redis for rate limits and queues
- Evaluation sets and tracing
Proof
Shipped, not pitched.
01Our product
NoteTube
A lecture becomes a study system: chat, flashcards, quizzes, all grounded in the source.
An AI learning platform that turns PDFs, YouTube lectures, recordings, and notes into a structured notebook with grounded chat, spaced-repetition flashcards, and graded quizzes.
Web · iOS · Android · Chrome extension · MCP server
02Case study
Acasa Sentinel
Give it a number plate. It returns the vehicle's route across a CCTV network.
A vendor-neutral video analytics system built for the Gujarat Police Hackathon Innovation Challenge 2026. It reads number plates, re-identifies vehicles by appearance across cameras, reconstructs routes, and raises watchlist alerts for an operator to confirm.
Operator console (web) · API
Questions
FastAPI for AI apps questions we get asked.
Why put the AI in a separate backend service?
So the model keys never reach a client, every app shares one prompt and one retrieval pipeline, and you can change models or add usage limits without shipping a new app version.
Can the model run on our own servers?
Yes. The FastAPI service can call an open-weight model in your cloud or on your hardware. Acasa Sentinel runs all of its inference locally.
How do you control what the AI costs?
Every call records tokens and cost, plans get usage limits enforced in the backend, and retrieval narrows what the model reads, which is usually the biggest saving.
Do you use LangChain?
Only where it is clearer. Most of our pipelines are plain Python around the model SDKs, which keeps them easy to debug and to hand over.
FastAPI backends for AI apps
Bring the prototype that calls the model from the browser.
We will show you where the backend should start and what it would take to make it a product.