AI Tutor App Development: Keep Answers in the Course

In AI tutor app development, keeping answers inside the course takes three things: index every lecture, PDF, and note with its location; retrieve only from the student's own material for each question; and instruct the model to answer from those passages, cite them, and say so when the course is silent. Then test that it actually does.

TL;DR: A tutor stays inside the course through retrieval, not training. Index the material with page and timestamp locations, scope search to each student on the server, require citations, and allow "the course doesn't cover this". Instructions reduce made-up answers but don't eliminate them, so test with questions the course can't answer.

NoteTube is our AI learning platform that turns PDFs, YouTube lectures, recordings, and notes into a notebook with grounded chat, spaced-repetition flashcards, and graded quizzes. This post covers how a tutor like that is built to stay inside the material, and how to check that it does before students rely on it.

Why is a general chatbot the wrong tutor?

Because it answers from the internet, not from the lecture the exam is about. Its answer can be right in general and wrong for this course: a different notation, a different definition, a topic the syllabus treats another way. A tutor has to answer the way the course does and show where.

Students already have more material than time. A two-hour lecture or a 300-page PDF is easy to save and hard to learn from. What helps is an answer from that lecture, with a pointer to the minute or page it came from, so the student can go back and read the real thing.

The parts of a grounded tutor

There are six parts, each feeding the next: ingest, index, scope, answer, study tools, and the surfaces students use. In NoteTube, all of them share one index, so the chat and the flashcards can't drift apart.

Part	What it does in NoteTube
Ingest	Videos, PDFs, DOCX, images, articles, and audio are parsed
Index	Text is chunked, embedded, and stored in Qdrant
Scope	Retrieval is limited per user and per session
Answer	Chat streams back grounded in the student's material
Study tools	Summaries, chapters, mind maps, podcasts, quizzes, SM-2 flashcards
Surfaces	Web, iOS, Android, a Chrome extension, and a remote MCP server

How should course material be ingested and chunked?

Parse each format its own way, keep the location of every piece, and split on the material's own boundaries. Transcribe video and audio, extract text from documents, and recognise text in images of slides or notes. The location you store here becomes the citation later.

A citation that says "lecture 3, 42 minutes in" is worth far more to a student than "from your materials". So store the file, page, heading, or timestamp with each chunk. NoteTube handles videos, PDFs, DOCX files, images, articles, and audio, and its Chrome extension can transcribe Google Meet tab audio, so a live online class becomes a source too.

Chunking decides what the tutor can find, and study material has structure worth keeping:

1. Split on chapters, headings, slides, and topic changes, not only on character counts.
2. Keep worked examples whole. A solved problem split across two chunks is useless to retrieve.
3. Keep formulas and tables with their labels, so a retrieved formula arrives with what its symbols mean.
4. Re-index when material changes, replacing old chunks so outdated notes stop appearing.

How do you keep each student's material separate?

Filter every search by the student, on the server, from the authenticated user. NoteTube stores embeddings in Qdrant with per-user, per-session scoping, so a question in one study session retrieves only from that session's sources. That keeps answers focused and uploads private.

Qdrant's own guidance for many small tenants is a single collection partitioned by a payload field, with a keyword index on that field and a filter on every query (Qdrant docs). The important part is who sets the filter. The client never chooses which data it searches. Our post on tenant isolation in vector search covers the failure modes in detail.

How does a tutor stay inside the course?

By instruction, backed by retrieval and checked by tests. The model receives only the retrieved passages and is told to answer from them, cite them, and say plainly when they don't cover the question. That last permission matters most: a model told it may say "I don't know" fills fewer gaps with guesses.

Anthropic's documentation lists the same techniques for reducing hallucinations: let the model say "I don't know", ground answers in direct quotes, verify claims with citations and retract any claim without a supporting quote, and restrict it to the provided documents. It also notes these techniques reduce hallucinations but "don't eliminate them entirely" (Claude docs).

So treat "answers only from the course" as a target you measure, not a property you declare. For a tutor, three behaviours define it:

1. A citation on every answer, pointing to the source and its location.
2. A clear "the course doesn't cover this" when retrieval finds nothing relevant.
3. The course's own terms and notation, which come naturally from answering out of its text.

Answers stream to the student as they're generated; our server-sent events guide shows how.

How do you test that the tutor stays grounded?

Build a small test set from one real course and run it on every change. Most of the questions should come from what students actually ask. The important additions are questions the course can't answer, because those are where a tutor is tempted to guess.

1. Answerable questions. For each, note the page or timestamp that holds the answer, and check that the tutor cites it.
2. Unanswerable questions. Topics close to the course but not in it. The only correct response is "not covered".
3. Trap questions. Where the course defines something differently from common usage, check the tutor uses the course's version.
4. Isolation checks. Ask in one student's session about material only another student uploaded. Nothing should come back.
5. Quiz review. Compare generated quiz questions against the syllabus.

How many questions, and how to score retrieval hits, is covered in our post on building a RAG evaluation set. Run the unanswerable set after every prompt or model change. It's the first thing to break.

Why not fine-tune a model on the course instead?

Because fine-tuning changes how a model writes, not what it can point to. OpenAI's model optimization guide describes fine-tuning as a way to make a model format responses consistently or handle new kinds of input, and says information from private databases or up-to-date sources belongs in the prompt (OpenAI docs).

For a tutor, that's decisive. Retrieval keeps each answer tied to the current text and makes a citation possible; a fine-tuned model can't tell you which page it learned something from. When the teacher updates the notes, retrieval picks up the change on re-index. Fine-tuning can still help with tone or format. We compare the two on the RAG vs fine-tuning page.

Quizzes and flashcards from the same index

A tutor should quiz, not only answer. Answering questions helps a student understand; practice helps them remember. NoteTube generates summaries, chapters, mind maps, podcasts, quizzes, and SM-2 spaced-repetition flashcards from the student's own sources, using the same index as the chat.

SM-2 is the scheduling method Piotr Wozniak described for SuperMemo: each card's next interval grows by an easiness factor that starts at 2.5 and shifts after every review, based on a 0 to 5 recall grade, and a failed card restarts its sequence (SuperMemo). Cards built from the course and scheduled this way turn a pile of material into a few minutes of review a day.

Quizzes from the same index test what the course actually taught, and their results tell a student which sections to reread.

Keys, models, and apps

Everything sensitive lives on the server. In NoteTube, AI keys live only in a separate FastAPI service, and model choice across OpenAI, Gemini, Claude, Llama, and Mistral is made per conversation and enforced there; our API keys post explains why. Students reach it from a Next.js web app and an Expo app for iOS and Android.

If teachers will sell material too, decide early whether they're users, sellers, or both, because it changes accounts, payments, and permissions. We cover wider course platforms on the e-learning app page.

FAQ

What should an AI tutor say when the course doesn't cover a question?

It should say so plainly, then point to the nearest related section if retrieval found one. It shouldn't answer from general knowledge without flagging it, because students can't tell which parts came from the course. Test this with a set of questions you know the material doesn't answer.

Can citations point to a moment in a video lecture?

Yes, if timestamps are kept when the video is transcribed. Store the start time with each chunk, and the citation can link to that point in the lecture. The same idea works for PDFs with page numbers and for slides with slide numbers, so students can check the source.

How often should course material be re-indexed?

Whenever it changes. Re-index the changed file, replace its old chunks rather than adding new ones beside them, and rerun your test questions for that section. If outdated chunks stay in the index, the tutor will keep citing notes the teacher has already corrected.

Building an AI tutor or adding one to your course platform? See our AI tutor app work and bring one course's material, plus ten questions it can't answer, to test it on.