{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab 4: Financial Research Agent\n",
    "**Goal:** turn the Lab 3 RAG system into an *agent* that decides for itself which tool to use, then pauses for human approval on high-stakes answers.\n",
    "\n",
    "**Why:** plain RAG always does the same thing: retrieve, then answer. An agent can choose between searching documents, doing arithmetic, or summarising a page, and can chain them. This is Module 4, sections 1, 2 and 4 (tools, function calling, ReAct) and section 8 (human-in-the-loop controls).\n",
    "\n",
    "This notebook is self-contained: Exercise 1 rebuilds the small Lab 3 index, so you do not need to keep the Lab 3 notebook open."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 0: Choose your model provider\n",
    "**Why we do this:** agents need a model that supports *tool calling* (also called function calling): the model replies with a request like \"call calculate_ratio with these numbers\" instead of plain text. Not every model does this well, so the model choice matters more here than in Lab 3.\n",
    "\n",
    "**What the code does:** defines one `chat(messages, tools)` function that works with six providers and always returns the reply in the same shape. Five providers use the OpenAI-style API (so they share code); Anthropic has its own API, so a small adapter converts the messages.\n",
    "\n",
    "| Provider | Cost | Notes |\n",
    "|---|---|---|\n",
    "| `ollama` | Free, local | Run this notebook locally; Colab cannot reach your machine. Use a model tagged \"tools\" in the Ollama library. Small models call tools less reliably. |\n",
    "| `groq` | Free tier | Fast hosted open models. Rate-limited. |\n",
    "| `gemini` | Free tier | Google AI Studio key. Used here through Google's OpenAI-compatible endpoint. |\n",
    "| `openrouter` | Free models with limits | One key, many models. The `openrouter/free` router picks a free model; free limits are small, and an agent makes several calls per question. |\n",
    "| `openai` | Paid | Works well with tools. |\n",
    "| `anthropic` | Paid | Works well with tools. |\n",
    "\n",
    "**Important:** model names and free-tier limits change often. If a model is rejected, check the provider's current model list and edit `MODEL`. Limits count every model call, and an agent makes several per question."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "%pip install -q chromadb pypdf openai anthropic"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import os, json, getpass\n",
    "\n",
    "PROVIDER = \"groq\"   # ollama | groq | gemini | openrouter | openai | anthropic\n",
    "\n",
    "PROVIDERS = {\n",
    "    \"ollama\":     {\"base_url\": \"http://localhost:11434/v1\",                                \"key_env\": None,                \"model\": \"llama3.1:8b\"},\n",
    "    \"groq\":       {\"base_url\": \"https://api.groq.com/openai/v1\",                          \"key_env\": \"GROQ_API_KEY\",       \"model\": \"llama-3.3-70b-versatile\"},\n",
    "    \"gemini\":     {\"base_url\": \"https://generativelanguage.googleapis.com/v1beta/openai/\", \"key_env\": \"GEMINI_API_KEY\",     \"model\": \"gemini-2.5-flash\"},\n",
    "    \"openrouter\": {\"base_url\": \"https://openrouter.ai/api/v1\",                            \"key_env\": \"OPENROUTER_API_KEY\", \"model\": \"openrouter/free\"},\n",
    "    \"openai\":     {\"base_url\": None,                                                      \"key_env\": \"OPENAI_API_KEY\",     \"model\": \"gpt-4o-mini\"},\n",
    "    \"anthropic\":  {\"base_url\": None,                                                      \"key_env\": \"ANTHROPIC_API_KEY\",  \"model\": \"claude-sonnet-5-5\"},\n",
    "}\n",
    "cfg = PROVIDERS[PROVIDER]\n",
    "MODEL = cfg[\"model\"]\n",
    "if cfg[\"key_env\"] and not os.getenv(cfg[\"key_env\"]):\n",
    "    os.environ[cfg[\"key_env\"]] = getpass.getpass(cfg[\"key_env\"] + \": \")\n",
    "\n",
    "def to_anthropic(messages):\n",
    "    # Convert OpenAI-style messages into Anthropic's format.\n",
    "    system = messages[0][\"content\"]\n",
    "    out = []\n",
    "    for m in messages[1:]:\n",
    "        if m[\"role\"] == \"user\":\n",
    "            out.append({\"role\": \"user\", \"content\": m[\"content\"]})\n",
    "        elif m[\"role\"] == \"assistant\":\n",
    "            blocks = []\n",
    "            if m.get(\"content\"):\n",
    "                blocks.append({\"type\": \"text\", \"text\": m[\"content\"]})\n",
    "            for tc in m.get(\"tool_calls\", []):\n",
    "                blocks.append({\"type\": \"tool_use\", \"id\": tc[\"id\"], \"name\": tc[\"function\"][\"name\"],\n",
    "                               \"input\": json.loads(tc[\"function\"][\"arguments\"] or \"{}\")})\n",
    "            out.append({\"role\": \"assistant\", \"content\": blocks})\n",
    "        elif m[\"role\"] == \"tool\":\n",
    "            block = {\"type\": \"tool_result\", \"tool_use_id\": m[\"tool_call_id\"], \"content\": m[\"content\"]}\n",
    "            if out and out[-1][\"role\"] == \"user\" and isinstance(out[-1][\"content\"], list):\n",
    "                out[-1][\"content\"].append(block)\n",
    "            else:\n",
    "                out.append({\"role\": \"user\", \"content\": [block]})\n",
    "    return system, out\n",
    "\n",
    "def chat(messages, tools=None):\n",
    "    # Returns {\"role\": \"assistant\", \"content\": str, \"tool_calls\": [...] (optional)}\n",
    "    if PROVIDER == \"anthropic\":\n",
    "        import anthropic\n",
    "        system, msgs = to_anthropic(messages)\n",
    "        kw = dict(model=MODEL, max_tokens=1000, temperature=0, system=system, messages=msgs)\n",
    "        if tools:\n",
    "            kw[\"tools\"] = [{\"name\": t[\"function\"][\"name\"], \"description\": t[\"function\"][\"description\"],\n",
    "                            \"input_schema\": t[\"function\"][\"parameters\"]} for t in tools]\n",
    "        r = anthropic.Anthropic().messages.create(**kw)\n",
    "        msg = {\"role\": \"assistant\", \"content\": \"\".join(b.text for b in r.content if b.type == \"text\")}\n",
    "        calls = [{\"id\": b.id, \"type\": \"function\", \"function\": {\"name\": b.name, \"arguments\": json.dumps(b.input)}}\n",
    "                 for b in r.content if b.type == \"tool_use\"]\n",
    "        if calls:\n",
    "            msg[\"tool_calls\"] = calls\n",
    "        return msg\n",
    "    from openai import OpenAI\n",
    "    key = os.getenv(cfg[\"key_env\"]) if cfg[\"key_env\"] else \"ollama\"\n",
    "    client = OpenAI(base_url=cfg[\"base_url\"], api_key=key) if cfg[\"base_url\"] else OpenAI(api_key=key)\n",
    "    kw = dict(model=MODEL, temperature=0, messages=messages)\n",
    "    if tools:\n",
    "        kw[\"tools\"] = tools\n",
    "    m = client.chat.completions.create(**kw).choices[0].message\n",
    "    msg = {\"role\": \"assistant\", \"content\": m.content or \"\"}\n",
    "    if m.tool_calls:\n",
    "        msg[\"tool_calls\"] = [{\"id\": tc.id, \"type\": \"function\",\n",
    "                              \"function\": {\"name\": tc.function.name, \"arguments\": tc.function.arguments}}\n",
    "                             for tc in m.tool_calls]\n",
    "    return msg\n",
    "\n",
    "print(chat([{\"role\": \"system\", \"content\": \"Be brief.\"}, {\"role\": \"user\", \"content\": \"Say hello in five words.\"}])[\"content\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected output:** a short greeting. An authentication or connection error means a wrong key, or (Ollama) that the server is not running or the model is not pulled."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1: Rebuild the Lab 3 index (recap)\n",
    "**Why we do this:** the agent's first tool searches documents, so we need the same ChromaDB index as Lab 3. This is a shortened version; if you finished Lab 3 you have seen every line.\n",
    "\n",
    "**What the code does:** downloads the SEBI memo, reads each page, cuts it into overlapping chunks, and stores them in Chroma with the document name and page as metadata. We also keep the raw page text in `PAGES`, because the third tool needs whole pages.\n",
    "\n",
    "Put any extra PDFs (for example the RBI Master Direction) in the `data/` folder before running; they are loaded too."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import re, urllib.request\n",
    "import chromadb\n",
    "from pypdf import PdfReader\n",
    "\n",
    "os.makedirs(\"data\", exist_ok=True)\n",
    "SEBI_URL = \"https://www.sebi.gov.in/sebi_data/meetingfiles/dec-2024/1735042007618_1.pdf\"\n",
    "if not os.path.exists(\"data/sebi_ai_memo.pdf\"):\n",
    "    req = urllib.request.Request(SEBI_URL, headers={\"User-Agent\": \"Mozilla/5.0\"})\n",
    "    open(\"data/sebi_ai_memo.pdf\", \"wb\").write(urllib.request.urlopen(req).read())\n",
    "\n",
    "PAGES = {}   # {file name: {page number: text}}\n",
    "for f in os.listdir(\"data\"):\n",
    "    if f.lower().endswith(\".pdf\"):\n",
    "        PAGES[f] = {i + 1: re.sub(r\"\\s+\", \" \", p.extract_text() or \"\").strip()\n",
    "                    for i, p in enumerate(PdfReader(f\"data/{f}\").pages)}\n",
    "\n",
    "def chunk(text, size=800, overlap=150):\n",
    "    return [text[s:s + size] for s in range(0, max(len(text), 1), size - overlap) if text[s:s + size].strip()]\n",
    "\n",
    "col = chromadb.Client().get_or_create_collection(\"lab4\")\n",
    "ids, docs, metas = [], [], []\n",
    "for f, pages in PAGES.items():\n",
    "    for pg, text in pages.items():\n",
    "        for j, piece in enumerate(chunk(text)):\n",
    "            ids.append(f\"{f}-{pg}-{j}\"); docs.append(piece); metas.append({\"doc\": f, \"page\": pg})\n",
    "col.add(ids=ids, documents=docs, metadatas=metas)\n",
    "print(\"documents:\", list(PAGES), \"| chunks stored:\", col.count())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected output:** the document names and a chunk count. If `PAGES` is empty, no PDF was found in `data/`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2: Define three tools\n",
    "**Why we do this:** a tool is just a normal Python function plus a short description the model can read. The model never runs code itself: it only *asks* for a tool by name with arguments, and our program runs it. The description is how the model decides which tool fits (Module 4, section 2).\n",
    "\n",
    "**The three tools:**\n",
    "- `search_documents(query, k)` looks up passages in the Chroma index (retrieval).\n",
    "- `calculate_ratio(numerator, denominator)` does arithmetic in plain Python. Models are unreliable at arithmetic, so we never let them do it in their head.\n",
    "- `summarise_section(doc, page)` fetches one whole page and asks the model to summarise it.\n",
    "\n",
    "**Key terms:** *Tool schema* = the JSON description (name, purpose, parameters) sent to the model. *Deterministic* = the same input always gives the same output, like `calculate_ratio`.\n",
    "\n",
    "**Your task:** write `run_tool`, which looks up the tool by name, runs it with the model's arguments, and returns the result as text. If anything fails it must return an error message instead of crashing, so the model can recover. Hint: use a dictionary mapping names to functions, and `try/except`."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "def search_documents(query: str, k: int = 3) -> str:\n",
    "    r = col.query(query_texts=[query], n_results=int(k))\n",
    "    return \"\\n\\n\".join(f\"[{m['doc']} p.{m['page']}] {t}\" for t, m in zip(r[\"documents\"][0], r[\"metadatas\"][0]))\n",
    "\n",
    "def calculate_ratio(numerator: float, denominator: float, as_percent: bool = True) -> str:\n",
    "    if float(denominator) == 0:\n",
    "        return \"Error: denominator is zero.\"\n",
    "    value = float(numerator) / float(denominator)\n",
    "    return f\"{value * 100:.2f}%\" if as_percent else f\"{value:.4f}\"\n",
    "\n",
    "def summarise_section(doc: str, page: int) -> str:\n",
    "    text = PAGES.get(doc, {}).get(int(page))\n",
    "    if not text:\n",
    "        return f\"Error: no page {page} in '{doc}'. Documents: {list(PAGES)}\"\n",
    "    reply = chat([{\"role\": \"system\", \"content\": \"Summarise in 3 bullet points. Use only the given text.\"},\n",
    "                  {\"role\": \"user\", \"content\": text}])\n",
    "    return reply[\"content\"]\n",
    "\n",
    "TOOLS = [\n",
    "    {\"type\": \"function\", \"function\": {\n",
    "        \"name\": \"search_documents\",\n",
    "        \"description\": \"Search the loaded regulatory documents and return the most relevant passages with document name and page.\",\n",
    "        \"parameters\": {\"type\": \"object\", \"properties\": {\n",
    "            \"query\": {\"type\": \"string\", \"description\": \"What to look for\"},\n",
    "            \"k\": {\"type\": \"integer\", \"description\": \"Number of passages (default 3)\"}}, \"required\": [\"query\"]}}},\n",
    "    {\"type\": \"function\", \"function\": {\n",
    "        \"name\": \"calculate_ratio\",\n",
    "        \"description\": \"Divide numerator by denominator exactly. Use for ANY ratio or percentage; never calculate by hand.\",\n",
    "        \"parameters\": {\"type\": \"object\", \"properties\": {\n",
    "            \"numerator\": {\"type\": \"number\"}, \"denominator\": {\"type\": \"number\"},\n",
    "            \"as_percent\": {\"type\": \"boolean\", \"description\": \"Return as a percentage (default true)\"}},\n",
    "            \"required\": [\"numerator\", \"denominator\"]}}},\n",
    "    {\"type\": \"function\", \"function\": {\n",
    "        \"name\": \"summarise_section\",\n",
    "        \"description\": \"Summarise one whole page of a loaded document. Loaded documents: \" + \", \".join(PAGES),\n",
    "        \"parameters\": {\"type\": \"object\", \"properties\": {\n",
    "            \"doc\": {\"type\": \"string\", \"description\": \"Exact file name\"},\n",
    "            \"page\": {\"type\": \"integer\", \"description\": \"Page number, starting at 1\"}}, \"required\": [\"doc\", \"page\"]}}},\n",
    "]\n",
    "\n",
    "def run_tool(name: str, arguments: str) -> str:\n",
    "    # YOUR CODE HERE:\n",
    "    # 1. Map tool names to the functions above.\n",
    "    # 2. Parse `arguments` (a JSON string) with json.loads.\n",
    "    # 3. Call the function with those arguments and return the result as a string.\n",
    "    # 4. If the name is unknown or anything raises, return \"Error: ...\" instead of crashing.\n",
    "    raise NotImplementedError"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details><summary>Solution (try first)</summary>\n",
    "\n",
    "```python\n",
    "def run_tool(name, arguments):\n",
    "    registry = {\"search_documents\": search_documents,\n",
    "                \"calculate_ratio\": calculate_ratio,\n",
    "                \"summarise_section\": summarise_section}\n",
    "    if name not in registry:\n",
    "        return f\"Error: unknown tool '{name}'.\"\n",
    "    try:\n",
    "        return str(registry[name](**json.loads(arguments or \"{}\")))\n",
    "    except Exception as e:\n",
    "        return f\"Error: {e}\"\n",
    "```\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "print(run_tool(\"calculate_ratio\", '{\"numerator\": 1350, \"denominator\": 10000}'))\n",
    "print(run_tool(\"calculate_ratio\", '{\"numerator\": 1, \"denominator\": 0}'))\n",
    "print(run_tool(\"no_such_tool\", \"{}\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected output:** `13.50%`, then an error about a zero denominator, then an error about an unknown tool. The errors are returned as text, not raised.\n",
    "\n",
    "**What to notice:** returning the error as text lets the model read it and try something else, instead of the whole agent crashing.\n",
    "\n",
    "**Finance link:** `calculate_ratio` keeps every number reproducible. An auditor can re-run the exact calculation, which a model's mental arithmetic does not allow."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3: The ReAct loop\n",
    "**Why we do this:** ReAct means *Reason, then Act, then Observe, repeat* (Module 4, section 4). The model reasons about what it needs, requests a tool (act), we run it and show the result (observe), and the model decides whether it has enough to answer or needs another tool.\n",
    "\n",
    "**What the code does:** keeps a growing `messages` list. Each round it asks the model what to do. If the reply contains tool calls, we run each one and append the result as a `tool` message, then loop. If the reply has no tool calls, that reply is the final answer. `max_steps` stops the loop if the model never finishes.\n",
    "\n",
    "**Key terms:** *Step limit* = a hard cap on rounds, which prevents infinite loops and runaway cost. *Trace* = the printed record of each tool call and result.\n",
    "\n",
    "**Why the cap matters:** each round is one paid or rate-limited model call."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "SYSTEM = (\n",
    "    \"You are a financial regulation research assistant. Use the tools to ground every factual claim. \"\n",
    "    \"Use calculate_ratio for any arithmetic. Cite sources as [document p.N]. \"\n",
    "    \"If the documents do not contain the answer, say so plainly rather than guessing.\"\n",
    ")\n",
    "\n",
    "def run_agent(question: str, max_steps: int = 6, verbose: bool = True) -> str:\n",
    "    messages = [{\"role\": \"system\", \"content\": SYSTEM}, {\"role\": \"user\", \"content\": question}]\n",
    "    for step in range(1, max_steps + 1):\n",
    "        msg = chat(messages, TOOLS)\n",
    "        messages.append(msg)\n",
    "        calls = msg.get(\"tool_calls\")\n",
    "        if not calls:\n",
    "            return msg[\"content\"]\n",
    "        for call in calls:\n",
    "            name, args = call[\"function\"][\"name\"], call[\"function\"][\"arguments\"]\n",
    "            result = run_tool(name, args)\n",
    "            if verbose:\n",
    "                print(f\"step {step}: {name}({args})\\n   -> {result[:200]}{'...' if len(result) > 200 else ''}\")\n",
    "            messages.append({\"role\": \"tool\", \"tool_call_id\": call[\"id\"], \"content\": result})\n",
    "    return \"Stopped: step limit reached without a final answer.\"\n",
    "\n",
    "print(run_agent(\"According to the SEBI memo, who is responsible for the output of AI tools?\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected output:** a trace showing a `search_documents` call, then a final answer citing the SEBI memo. The exact query the model writes will vary.\n",
    "\n",
    "**What to notice:** the model chose to search without being told to. Now ask: *\"A bank has total capital of 1,350 and risk-weighted assets of 10,000. What is its capital adequacy ratio?\"* You should see `calculate_ratio` instead, and no document search.\n",
    "\n",
    "**If it fails:** a model that answers without calling any tool, or writes the call as plain text, does not support tool calling well. Try a larger model or another provider."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 4: Human-in-the-loop checkpoint\n",
    "**Why we do this:** some answers should not go out without a person checking them. In finance that includes anything that sounds like a compliance conclusion, an approval, or a recommendation (Module 4, section 8, \"autonomy and control\").\n",
    "\n",
    "**What the code does:** `is_high_stakes` flags a question using keywords. `run_with_approval` runs the agent, then, for flagged questions, shows the draft and asks a human to approve before returning it. A rejected answer is withheld.\n",
    "\n",
    "**Your task:** complete `run_with_approval`. Hint: call `run_agent`, test `is_high_stakes`, ask with `input()`, and return the draft only if the reply starts with \"y\".\n",
    "\n",
    "**Important:** a keyword list is a crude screen. It will miss some high-stakes questions and flag harmless ones. Real systems use risk tiers, classifiers and policy review. Treat this as the simplest possible version of the idea."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "HIGH_STAKES = [\"approve\", \"reject\", \"loan\", \"recommend\", \"complian\", \"minimum\", \"breach\", \"penalt\"]\n",
    "\n",
    "def is_high_stakes(question: str) -> bool:\n",
    "    q = question.lower()\n",
    "    return any(word in q for word in HIGH_STAKES)\n",
    "\n",
    "AUTO_APPROVE = False   # set True only for unattended test runs\n",
    "\n",
    "def run_with_approval(question: str) -> str:\n",
    "    # YOUR CODE HERE:\n",
    "    # 1. draft = run_agent(question)\n",
    "    # 2. if the question is not high-stakes, return draft\n",
    "    # 3. otherwise print the draft, ask the human to approve, return draft if they type y,\n",
    "    #    else return \"Answer withheld: reviewer did not approve.\"\n",
    "    # (honour AUTO_APPROVE: if True, skip input() and approve)\n",
    "    raise NotImplementedError"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details><summary>Solution (try first)</summary>\n",
    "\n",
    "```python\n",
    "def run_with_approval(question):\n",
    "    draft = run_agent(question)\n",
    "    if not is_high_stakes(question):\n",
    "        return draft\n",
    "    print(\"\\n--- HIGH-STAKES: DRAFT FOR REVIEW ---\\n\" + draft + \"\\n-------------------------------------\")\n",
    "    ok = True if AUTO_APPROVE else input(\"Approve this answer? (y/n): \").strip().lower().startswith(\"y\")\n",
    "    return draft if ok else \"Answer withheld: reviewer did not approve.\"\n",
    "```\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 5: Test the agent\n",
    "**Why we do this:** an agent can fail in new ways: picking the wrong tool, looping, or answering without evidence. Run it on three different tasks and read the trace each time.\n",
    "\n",
    "**Test questions:**\n",
    "1. A document question, which should call `search_documents`.\n",
    "2. A calculation with a compliance angle, which should call `calculate_ratio` and then pause for approval. The 9% figure is supplied in the question; verify the real minimum against the RBI's Basel III capital regulations before relying on it.\n",
    "3. A summary request, which should call `summarise_section`."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "tests = [\n",
    "    \"According to the SEBI memo, who is responsible for the output of AI tools?\",\n",
    "    \"A bank has total capital of 1,350 and risk-weighted assets of 10,000. What is its capital adequacy ratio, and does it meet a 9% minimum?\",\n",
    "    \"Summarise page 3 of the SEBI memo (file sebi_ai_memo.pdf).\",\n",
    "]\n",
    "for q in tests:\n",
    "    print(\"\\nQ:\", q)\n",
    "    print(\"A:\", run_with_approval(q))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected output:** question 1 uses search, question 2 uses `calculate_ratio` (13.50%) and then asks for your approval, question 3 uses `summarise_section`. If a different tool is chosen, read the tool descriptions: they are what the model relies on.\n",
    "\n",
    "**What to notice:** change a tool's `description` to something vague and re-run. Tool selection usually gets worse, which shows how much the description matters.\n",
    "\n",
    "## Stretch goals\n",
    "1. Add a fourth tool, for example `list_documents()`, and see whether the agent uses it.\n",
    "2. Add the RBI Master Direction and ask a question that needs both documents.\n",
    "3. Count tool calls and rough token use per question to estimate cost.\n",
    "\n",
    "## Common errors\n",
    "- **The model never calls a tool (Ollama or small models):** use a model tagged \"tools\" in the Ollama library, or switch provider.\n",
    "- **400 error mentioning tools or thought signatures (Gemini):** use `gemini-2.5-flash`; newer Gemini versions need extra fields passed back that this notebook does not handle.\n",
    "- **429 or rate limit (Groq, Gemini, OpenRouter free):** wait, or lower `max_steps`. Each round is one request.\n",
    "- **`NotImplementedError`:** you have not completed `run_tool` or `run_with_approval` yet."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}