Regulatory Document RAG
Build a question-answering system over real regulatory documents. It retrieves the relevant passages and answers with citations. This is the foundation that Labs 4, 6 and 7 extend.
The Colab button reads the notebook from the GitHub repository, so it works once the latest changes are pushed. Until then use the download.
What you will build
A pipeline in five steps: load PDFs, split them into chunks, embed the chunks into ChromaDB, retrieve the best matches for a question, and have an LLM answer only from them with citations. You then compare two chunking strategies by measuring retrieval.
Prerequisites
- ▸Read Module 3 sections 1 to 6 (embeddings, RAG, chunking). Module 2 section 7 helps for grounding.
- ▸Python basics. No earlier lab is required.
- ▸Access to one LLM provider (below). Embeddings need no key.
Documents and sources
All documents are public. Put the PDFs in a data/ folder next to the notebook.
| Document | Publisher | How to get it |
|---|---|---|
| Board memorandum on assigning responsibility for AI tool use (Dec 2024). A proposal paper, not a final circular. | SEBI | The notebook downloads it for you from sebi.gov.in, or get a copy here. |
| Master Direction on IT Governance, Risk, Controls and Assurance Practices, 2023. | RBI | Download in your browser from the RBI Master Directions page. RBI blocks scripted downloads. |
| Integrated Annual Report 2024-25 (large, optional). | HDFC Bank | Download from the bank's investor relations page. Pick the 2024-25 report. |
The SEBI file was downloaded on 2026-10-31. The RBI and HDFC links are the publishers' pages. Document versions on those sites can change, so check you have the current one.
Exercises
Choose your LLM provider
Why we do this: The rest of the lab calls one function, ask_llm(). You pick the provider once, and nothing else changes. This lets the same lab run on Ollama, Groq or Anthropic.
What the code does: Builds ask_llm(system, user) for the provider you set. Ollama and Groq both use the OpenAI-style API, so they share code. Anthropic uses its own SDK.
- ▸Provider: the service that runs the model (Ollama locally, Groq or Anthropic hosted).
- ▸Temperature 0: asks the model for the most repeatable answer, which makes results easier to compare.
PROVIDER = "groq" # "ollama" | "groq" | "anthropic"
def ask_llm(system: str, user: str) -> str:
... # routes to the provider you selected
print(ask_llm("Be brief.", "Say hello in five words."))Expected output: A short greeting. An authentication or connection error means your key is wrong (Groq, Anthropic) or Ollama is not running with the model pulled.
What to notice: Ollama runs on your own computer, and Google Colab cannot reach it. If you choose Ollama, run the notebook locally. Model names change over time, so update MODEL if one is rejected.
Load the documents
Why we do this: RAG quality starts with clean source text. We keep the page number with every piece of text so answers can cite it later.
What the code does: load_pdf reads each PDF page with pypdf and returns (page_number, text) pairs. We then print the page and character count of each document.
- ▸Corpus: your collection of documents.
def load_pdf(path):
return [(i + 1, (p.extract_text() or ""))
for i, p in enumerate(PdfReader(path).pages)]Expected output: One line per PDF with page and character counts. The SEBI memo has 6 pages. A PDF with almost no characters is a scanned image and needs OCR.
What to notice: Real PDFs contain headers, page numbers and odd line breaks. That noise ends up inside your chunks and can affect retrieval.
Finance link: Regulatory text is only useful if you can point back to the exact page. Keeping page numbers from the start is what makes that possible.
Chunk the text two ways
Why we do this: A model cannot read a whole document for every question, and one embedding of a whole document is too vague to search. We split text into chunks, and the splitting method strongly affects retrieval quality (Module 3, section 5).
What the code does: chunk_fixed cuts every 800 characters with a 150-character overlap. You complete chunk_sentences, which packs whole sentences into chunks of up to 800 characters.
- ▸Fixed-size chunking: cut every N characters. Simple, but can cut a sentence in half.
- ▸Overlap: characters repeated between neighbouring chunks, so a cut sentence still appears whole once.
- ▸Sentence chunking: group whole sentences up to a size limit, so no sentence is split.
for s in sentences:
# YOUR CODE HERE: if current + s fits in size, append it;
# otherwise save current and start a new chunk with s.
passExpected output: A count of fixed chunks and sentence chunks, plus one sample of each. Fixed samples often start or end mid-sentence.
What to notice: Sentence chunks read more cleanly but vary in length. Neither is always better, which is why Exercise 5 measures them.
Finance link: A split clause such as 'the entity shall not ... unless' can reverse its meaning if the second half is lost.
Embed and store in ChromaDB
Why we do this: An embedding turns text into a list of numbers so that similar meanings sit close together. A vector database stores those numbers and finds the nearest ones to a question.
What the code does: Creates one Chroma collection per chunking strategy so we can compare them. Chroma's built-in local embedding model is used, so no API key is needed. It downloads a small model the first time.
- ▸Embedding: a vector (list of numbers) representing the meaning of text.
- ▸Collection: a named set of stored chunks in Chroma.
- ▸Metadata: extra fields stored with each chunk, here the document name and page.
col = client.get_or_create_collection(name)
col.add(ids=[...], documents=[c["text"] for c in chunks],
metadatas=[{"doc": c["doc"], "page": c["page"]} for c in chunks])Expected output: A line saying how many chunks were stored.
What to notice: The metadata travels with each chunk. That is how the answer step will later show where each passage came from.
Finance link: Because embedding runs locally, document text is not sent to an embedding service. That matters for confidential client documents.
Retrieve and answer with citations
Why we do this: This is the retrieval and generation half of RAG. We fetch the most relevant chunks, put them in the prompt, and tell the model to answer only from them.
What the code does: retrieve asks Chroma for the top-k closest chunks. answer numbers them as sources, instructs the model to cite like [1], and tells it to reply 'Not found in the provided documents.' when the sources do not contain the answer.
- ▸Top-k: how many chunks to retrieve (here 4).
- ▸Grounding: forcing the answer to rely on supplied text rather than the model's memory.
SYSTEM = ("You answer using ONLY the numbered sources. Cite like [1]. "
"If the answer is not in the sources, reply exactly: "
"'Not found in the provided documents.'")Expected output: An answer citing [1], [2] and so on, saying regulated entities remain responsible for AI tool output, followed by the document and page of each retrieved chunk. Wording varies by model, so check each citation against the printed pages.
What to notice: Ask something the documents do not cover, such as 'What is the repo rate?'. A well-grounded system says it was not found rather than guessing. Smaller models follow this rule less reliably.
Finance link: Citations and a 'not found' rule make an answer auditable, which is the requirement in Module 2, section 7.
Compare chunking strategies
Why we do this: You will measure which chunking retrieves better instead of guessing.
What the code does: For each test question you give a keyword that a correct chunk must contain. hit_rate is the share of questions where at least one top-k chunk contains that keyword. You write four more test questions from your own documents.
- ▸Recall@k: how often the right passage appears in the top-k results. hit_rate here is a crude proxy for it.
TESTS = [
("Who is solely responsible for AI tool output?", "responsib"),
# YOUR CODE HERE: add 4 more (question, keyword) pairs
]Expected output: Two scores between 0 and 1, one per strategy. The numbers will differ by documents and questions.
What to notice: With only five questions the scores are noisy, and a one-question difference is not conclusive. Try changing size, overlap and k, then re-run and see what moves.
Finance link: Retrieval that misses the right clause produces confident but wrong answers. Measuring retrieval separately from answer quality tells you which part to fix.
Stretch goals
- ▸Add the RBI and HDFC documents and ask questions that span more than one.
- ▸Use Chroma metadata filtering (
where={"doc": ...}) to limit a query to one document. - ▸OCR a scanned PDF and add it to the corpus.
Common errors
- ▸ModuleNotFoundError: re-run the install cell, then restart the runtime.
- ▸Connection refused (Ollama): run
ollama serveand pull the model. Use a local notebook, not Colab. - ▸Rate limit (Groq free tier): wait a minute or lower k. Limits change, so check Groq's documentation.