{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Lab 2: Financial Prompt Library\n", "**Goal:** build five reusable, versioned prompt templates for finance, test which prompting style works best on a credit task, measure what prompt caching saves, and save everything as a JSON library your team can reuse.\n", "\n", "**Why:** prompts that live in someone's chat history are a risk: nobody knows which version produced which output. Treating prompts as small, tested, versioned assets is how teams make LLM output repeatable (Module 2, sections 2 to 7).\n", "\n", "**What you need:** one model provider (free tiers are fine). The caching exercise needs **Anthropic or OpenAI**; every other exercise works with any provider." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exercise 0: Choose your provider\n", "**Why we do this:** all exercises share one helper, `ask(system, user)`, so the provider is a single setting.\n", "\n", "**What the code does:** defines `chat_with(...)` (reply plus token usage for six providers) and `ask(...)`, which returns just the text. You set `PROVIDER` and enter its key when asked.\n", "\n", "**Providers:** `ollama` (local and free; run this notebook locally), `groq`, `gemini` and `openrouter` (free tiers with limits), `openai` and `anthropic` (paid). Model names and limits change, so edit `PROVIDERS` if a model is rejected." ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "%pip install -q openai anthropic pypdf" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import os, json, getpass, time\n", "\n", "PROVIDERS = {\n", " \"ollama\": {\"base_url\": \"http://localhost:11434/v1\", \"key_env\": None, \"model\": \"llama3.1:8b\"},\n", " \"groq\": {\"base_url\": \"https://api.groq.com/openai/v1\", \"key_env\": \"GROQ_API_KEY\", \"model\": \"llama-3.3-70b-versatile\"},\n", " \"gemini\": {\"base_url\": \"https://generativelanguage.googleapis.com/v1beta/openai/\", \"key_env\": \"GEMINI_API_KEY\", \"model\": \"gemini-2.5-flash\"},\n", " \"openrouter\": {\"base_url\": \"https://openrouter.ai/api/v1\", \"key_env\": \"OPENROUTER_API_KEY\", \"model\": \"openrouter/free\"},\n", " \"openai\": {\"base_url\": None, \"key_env\": \"OPENAI_API_KEY\", \"model\": \"gpt-4o-mini\"},\n", " \"anthropic\": {\"base_url\": None, \"key_env\": \"ANTHROPIC_API_KEY\", \"model\": \"claude-sonnet-5-5\"},\n", "}\n", "\n", "def ensure_key(provider):\n", " env = PROVIDERS[provider][\"key_env\"]\n", " if env and not os.getenv(env):\n", " os.environ[env] = getpass.getpass(env + \": \")\n", "\n", "def chat_with(provider, system, user, temperature=0.0, max_tokens=700, cache_system=False):\n", " \"\"\"Returns (text, usage). usage = {\"in\": total input tokens, \"out\": output tokens, \"cached\": tokens read from cache}.\"\"\"\n", " cfg = PROVIDERS[provider]\n", " if provider == \"anthropic\":\n", " import anthropic\n", " sys_arg = [{\"type\": \"text\", \"text\": system, \"cache_control\": {\"type\": \"ephemeral\"}}] if cache_system else system\n", " r = anthropic.Anthropic().messages.create(model=cfg[\"model\"], max_tokens=max_tokens, temperature=temperature,\n", " system=sys_arg, messages=[{\"role\": \"user\", \"content\": user}])\n", " u = r.usage\n", " created = getattr(u, \"cache_creation_input_tokens\", 0) or 0\n", " read = getattr(u, \"cache_read_input_tokens\", 0) or 0\n", " usage = {\"in\": u.input_tokens + created + read, \"out\": u.output_tokens, \"cached\": read}\n", " return \"\".join(b.text for b in r.content if b.type == \"text\"), usage\n", " from openai import OpenAI\n", " key = os.getenv(cfg[\"key_env\"]) if cfg[\"key_env\"] else \"ollama\"\n", " client = OpenAI(base_url=cfg[\"base_url\"], api_key=key) if cfg[\"base_url\"] else OpenAI(api_key=key)\n", " r = client.chat.completions.create(model=cfg[\"model\"], temperature=temperature, max_tokens=max_tokens,\n", " messages=[{\"role\": \"system\", \"content\": system}, {\"role\": \"user\", \"content\": user}])\n", " u = r.usage\n", " details = getattr(u, \"prompt_tokens_details\", None) if u else None\n", " usage = {\"in\": getattr(u, \"prompt_tokens\", 0) or 0, \"out\": getattr(u, \"completion_tokens\", 0) or 0,\n", " \"cached\": (getattr(details, \"cached_tokens\", 0) or 0) if details else 0}\n", " return r.choices[0].message.content or \"\", usage" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "PROVIDER = \"groq\" # ollama | groq | gemini | openrouter | openai | anthropic\n", "ensure_key(PROVIDER)\n", "\n", "def ask(system: str, user: str, temperature: float = 0.0) -> str:\n", " return chat_with(PROVIDER, system, user, temperature)[0]\n", "\n", "print(ask(\"Be brief.\", \"Say hello in five words.\"))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Expected output:** a short greeting. An error means a wrong key, a retired model name, or (Ollama) a server that is not running." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exercise 1: Build the template library\n", "**Why we do this:** a template with named blanks (like `$text`) turns a one-off prompt into a reusable tool. Storing it with a version and an owner makes it auditable (Module 2, section 6).\n", "\n", "**What the code does:** the library `LIB` is a dictionary. Each entry has a `version`, an `owner`, a `system` prompt, a `template` with `$variables`, and the list of `vars` it needs. Three templates are provided: earnings summary, credit analysis and regulatory Q&A. You write the other two and a `render` function that fills a template and refuses to run if a variable is missing.\n", "\n", "**Key terms:**\n", "- *System prompt:* standing instructions that set the model's role and rules.\n", "- *Template variable:* a named blank such as `$text`, replaced when the template is used.\n", "- *Python `string.Template`:* the tool we use for filling blanks. It uses `$name`, which means JSON braces `{}` in a template need no escaping.\n", "\n", "**Your tasks:**\n", "1. Write the `risk_flags` template. It must ask for **only** a JSON list where each item has `flag`, `severity` (`low`, `medium` or `high`) and `evidence` (a short quote from the text), and say what to return if there are no risks. Variable: `$text`.\n", "2. Write the `executive_brief` template: at most 120 words, with a headline, exactly three bullets, and a line \"Decision needed:\". Variable: `$text`.\n", "3. Complete `render(name, **values)`. Hint: `Template(entry[\"template\"]).substitute(values)` raises `KeyError` for a missing variable; turn that into a clear `ValueError`." ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "from string import Template\n", "\n", "LIB = {\n", " \"earnings_summary\": {\n", " \"version\": \"1.0\", \"owner\": \"research\", \"vars\": [\"text\"],\n", " \"system\": \"You are a careful financial analyst. Use only the text provided. Never invent figures.\",\n", " \"template\": \"Summarise the results below in exactly three bullets (growth, profitability, balance sheet) and one bullet on risks or open questions. If a figure is not stated, write 'not stated'.\\n\\nTEXT:\\n$text\",\n", " },\n", " \"credit_analysis\": {\n", " \"version\": \"1.0\", \"owner\": \"credit-risk\", \"vars\": [\"borrower_facts\"],\n", " \"system\": \"You are a credit analyst preparing an advisory note. You do not make the decision.\",\n", " \"template\": \"From the facts below give: Strengths (2 bullets), Weaknesses (2 bullets), Missing information (a list), and a suggested risk band (LOW, MEDIUM or HIGH) labelled 'Advisory only'.\\n\\nFACTS:\\n$borrower_facts\",\n", " },\n", " \"regulatory_qa\": {\n", " \"version\": \"1.0\", \"owner\": \"compliance\", \"vars\": [\"context\", \"question\"],\n", " \"system\": \"You answer using ONLY the provided context and quote the phrase you rely on. If the context does not contain the answer, reply exactly: 'Not found in the provided context.'\",\n", " \"template\": \"CONTEXT:\\n$context\\n\\nQUESTION: $question\",\n", " },\n", " \"risk_flags\": {\n", " \"version\": \"1.0\", \"owner\": \"risk\", \"vars\": [\"text\"],\n", " \"system\": \"\", # YOUR CODE HERE: role and rules (JSON only, no prose, no invention)\n", " \"template\": \"\", # YOUR CODE HERE: ask for a JSON list of {flag, severity, evidence}; say what to return if none; include $text\n", " },\n", " \"executive_brief\": {\n", " \"version\": \"1.0\", \"owner\": \"strategy\", \"vars\": [\"text\"],\n", " \"system\": \"\", # YOUR CODE HERE\n", " \"template\": \"\", # YOUR CODE HERE: <=120 words, headline, exactly three bullets, 'Decision needed:' line; include $text\n", " },\n", "}\n", "\n", "def render(name: str, **values):\n", " # YOUR CODE HERE: look up LIB[name]; raise ValueError naming any missing variable;\n", " # return (system_prompt, user_prompt) with the template filled in.\n", " raise NotImplementedError\n", "\n", "def validate_library():\n", " for name, e in LIB.items():\n", " assert e[\"system\"].strip() and e[\"template\"].strip(), f\"template '{name}' is still blank\"\n", " for v in e[\"vars\"]:\n", " assert \"$\" + v in e[\"template\"] or \"${\" + v + \"}\" in e[\"template\"], f\"'{name}' never uses ${v}\"\n", " print(\"library OK:\", list(LIB))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Solution (try first)\n", "\n", "```python\n", "LIB[\"risk_flags\"][\"system\"] = (\"You extract risk flags from financial text. Reply with JSON only, no prose. \"\n", " \"Use only facts in the text; never invent evidence.\")\n", "LIB[\"risk_flags\"][\"template\"] = (\"List every risk in the text as a JSON array of objects with keys \"\n", " \"flag, severity (low, medium or high) and evidence (a short quote). \"\n", " \"If there are no risks, return []. Return only the JSON.\\n\\nTEXT:\\n$text\")\n", "LIB[\"executive_brief\"][\"system\"] = \"You write for a busy executive. Be exact, plain and brief. Use only the text.\"\n", "LIB[\"executive_brief\"][\"template\"] = (\"Write a brief of at most 120 words: a one-line headline, exactly three bullets, \"\n", " \"and a final line starting 'Decision needed:'.\\n\\nTEXT:\\n$text\")\n", "\n", "def render(name, **values):\n", " e = LIB[name]\n", " missing = [v for v in e[\"vars\"] if v not in values]\n", " if missing:\n", " raise ValueError(f\"template '{name}' is missing variables: {missing}\")\n", " return e[\"system\"], Template(e[\"template\"]).substitute(values)\n", "```\n", "
" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "validate_library()\n", "print(render(\"regulatory_qa\", context=\"Fees are waived after 5 years.\", question=\"When are fees waived?\")[1])\n", "try:\n", " render(\"earnings_summary\")\n", "except ValueError as e:\n", " print(\"caught:\", e)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Expected output:** `library OK` with five names, the filled-in Q&A prompt, and a clear error for the missing `text` variable.\n", "\n", "**What to notice:** failing loudly on a missing variable prevents the worst template bug: a prompt silently sent with `$text` still in it.\n", "\n", "**Finance link:** `version` and `owner` mean that when an output is questioned, you can say which wording produced it and who is accountable for it (Module 5, section 2)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exercise 2: Run all five templates\n", "**Why we do this:** a template is only trusted once it has been run on realistic input. We run each one on a small sample and check the output has the right *shape*.\n", "\n", "**What the code does:** `SAMPLES` holds a short, **fictional or public** input for each template. `run_template` renders and runs one. The risk-flag template must return valid JSON, so you write `parse_json`, which tolerates the code fences models often add.\n", "\n", "**Your task:** complete `parse_json(text)`. Hint: strip whitespace, remove a leading triple-backtick fence (with an optional `json` label) and a trailing fence, then call `json.loads`.\n", "\n", "**Important:** models sometimes break format rules. Never pass model output into other code without validating it, as `parse_json` does here." ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import json, re\n", "\n", "SAMPLES = {\n", " \"earnings_summary\": {\"text\": \"Zenith Textiles Ltd (fictional) reported Q2 results. Revenue was Rs 1,250 crore, up 8% year on year. EBITDA margin was 14.2%. Net profit was Rs 96 crore. Debt-to-equity was 0.9. An interim dividend of Rs 2 per share was declared.\"},\n", " \"credit_analysis\": {\"borrower_facts\": \"Borrower: Zenith Textiles (fictional). Revenue Rs 1,250 crore. EBITDA margin 14.2%. Net profit Rs 96 crore. Debt/equity 0.9. DSCR 1.4. Largest customer is 35% of sales. Audited accounts available. Collateral details: not provided.\"},\n", " \"regulatory_qa\": {\"context\": \"The person regulated by SEBI would be solely responsible for the privacy, security and integrity of investors' and stakeholders' data; for the output arising from the usage of such tools and techniques it relies upon; and for compliance with applicable laws in force.\", \"question\": \"Who is responsible for the output of AI tools under the proposal?\"},\n", " \"risk_flags\": {\"text\": \"Lender memo (fictional): 35% of revenue comes from one customer. Two covenant waivers were granted in the last 12 months. Inventory days rose from 80 to 120. Management says demand is strong.\"},\n", " \"executive_brief\": {\"text\": \"Zenith Textiles (fictional) grew revenue 8% to Rs 1,250 crore with a 14.2% EBITDA margin and debt-to-equity of 0.9. One customer is 35% of sales and inventory days rose from 80 to 120. The lender must decide whether to renew a working-capital line this quarter.\"},\n", "}\n", "\n", "def parse_json(text: str):\n", " # YOUR CODE HERE: remove optional ``` or ```json fences around `text`, then return json.loads(...)\n", " raise NotImplementedError\n", "\n", "def run_template(name):\n", " system, user = render(name, **SAMPLES[name])\n", " return ask(system, user)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Solution (try first)\n", "\n", "```python\n", "def parse_json(text):\n", " fence = \"`\" * 3 # three backticks, built this way so it cannot break this notebook's formatting\n", " t = text.strip()\n", " if t.startswith(fence):\n", " t = t.split(\"\\n\", 1)[1] if \"\\n\" in t else t[3:] # drop the opening fence line (and any 'json' label)\n", " if t.endswith(fence):\n", " t = t[:-3]\n", " return json.loads(t.strip())\n", "```\n", "
" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "outputs = {}\n", "for name in LIB:\n", " outputs[name] = run_template(name)\n", " print(f\"=== {name} ===\\n{outputs[name]}\\n\")\n", "\n", "flags = parse_json(outputs[\"risk_flags\"])\n", "print(\"parsed\", len(flags), \"risk flags; keys ok:\", all({\"flag\", \"severity\", \"evidence\"} <= set(f) for f in flags))\n", "print(\"brief word count:\", len(outputs[\"executive_brief\"].split()), \"(target <= 120)\")\n", "print(\"QA mentions 'solely':\", \"solely\" in outputs[\"regulatory_qa\"].lower())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Expected output:** five outputs. The risk flags should parse as a JSON list with the three keys, the brief should be close to 120 words or fewer, and the Q&A answer should quote the \"solely responsible\" wording.\n", "\n", "**What to notice:** check each output against its instructions. Which template does the model follow least reliably? That is the one to tighten first, and to add a test for.\n", "\n", "**Finance link:** the Q&A template's exact \"Not found\" reply gives an auditable way to say \"the document does not say\". Try it with a question the context cannot answer." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exercise 3: Zero-shot, few-shot or chain-of-thought?\n", "**Why we do this:** you can often make a model better at a task by *how* you ask. We test three styles on one credit-rating task and measure accuracy, length and speed (Module 2, section 3).\n", "\n", "**The task:** rate eight fictional borrowers LOW, MEDIUM or HIGH risk from two numbers: *DSCR* (debt service coverage ratio: cash available divided by debt payments due) and *debt to EBITDA* (leverage). The bank has a **house policy** that the model is *not* told in numbers. Whether a style works depends on whether the model can pick the rule up.\n", "\n", "**The three styles:**\n", "- *Zero-shot:* just the question and the policy in words.\n", "- *Few-shot:* the same plus a few solved examples (**in-context learning**).\n", "- *Chain-of-thought (CoT):* ask the model to reason step by step before answering.\n", "\n", "**Your task:** complete the three prompt builders. Each takes a case dictionary and returns the user prompt. Zero-shot and few-shot must ask for one word; CoT must ask for brief reasoning and then a final line `RATING: `. Use `EXAMPLES` for few-shot.\n", "\n", "**Important:** with only eight test cases, one wrong answer moves accuracy by 12.5 points. Treat differences of one or two cases as noise. Real comparisons need far more cases." ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "POLICY = (\"House policy: rate HIGH if the borrower cannot comfortably service its debt or is heavily leveraged; \"\n", " \"LOW if debt service is comfortably covered and leverage is moderate; MEDIUM for everything in between.\")\n", "\n", "EXAMPLES = [\n", " {\"dscr\": 2.0, \"lev\": 1.5, \"rating\": \"LOW\"},\n", " {\"dscr\": 0.7, \"lev\": 4.5, \"rating\": \"HIGH\"},\n", " {\"dscr\": 1.2, \"lev\": 3.6, \"rating\": \"MEDIUM\"},\n", " {\"dscr\": 1.8, \"lev\": 3.8, \"rating\": \"MEDIUM\"},\n", " {\"dscr\": 1.2, \"lev\": 5.8, \"rating\": \"HIGH\"},\n", "]\n", "CASES = [\n", " {\"dscr\": 1.8, \"lev\": 2.1, \"rating\": \"LOW\"}, {\"dscr\": 0.9, \"lev\": 3.5, \"rating\": \"HIGH\"},\n", " {\"dscr\": 1.3, \"lev\": 3.8, \"rating\": \"MEDIUM\"}, {\"dscr\": 2.2, \"lev\": 5.5, \"rating\": \"HIGH\"},\n", " {\"dscr\": 1.6, \"lev\": 2.9, \"rating\": \"LOW\"}, {\"dscr\": 1.1, \"lev\": 4.2, \"rating\": \"MEDIUM\"},\n", " {\"dscr\": 1.7, \"lev\": 3.4, \"rating\": \"MEDIUM\"}, {\"dscr\": 1.4, \"lev\": 2.5, \"rating\": \"MEDIUM\"},\n", "]\n", "RATER_SYSTEM = \"You are a credit analyst applying your bank's house policy.\"\n", "\n", "def describe(case):\n", " return f\"DSCR {case['dscr']}, debt/EBITDA {case['lev']}\"\n", "\n", "def zero_shot(case):\n", " # YOUR CODE HERE: POLICY + the borrower line + 'Reply with exactly one word: LOW, MEDIUM or HIGH.'\n", " raise NotImplementedError\n", "\n", "def few_shot(case):\n", " # YOUR CODE HERE: POLICY + each solved example from EXAMPLES + the new borrower; reply with one word\n", " raise NotImplementedError\n", "\n", "def chain_of_thought(case):\n", " # YOUR CODE HERE: POLICY + borrower; think step by step briefly; finish with 'RATING: '\n", " raise NotImplementedError\n", "\n", "def parse_rating(text):\n", " found = re.findall(r\"\\b(LOW|MEDIUM|HIGH)\\b\", text.upper())\n", " return found[-1] if found else None" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Solution (try first)\n", "\n", "```python\n", "def zero_shot(case):\n", " return f\"{POLICY}\\n\\nBorrower: {describe(case)}\\nReply with exactly one word: LOW, MEDIUM or HIGH.\"\n", "\n", "def few_shot(case):\n", " shots = \"\\n\".join(f\"Borrower: {describe(e)}\\nRating: {e['rating']}\" for e in EXAMPLES)\n", " return f\"{POLICY}\\n\\nExamples:\\n{shots}\\n\\nBorrower: {describe(case)}\\nRating (one word):\"\n", "\n", "def chain_of_thought(case):\n", " return (f\"{POLICY}\\n\\nBorrower: {describe(case)}\\n\"\n", " \"Think step by step in a few short lines, then finish with a line: RATING: \")\n", "```\n", "
" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "def evaluate_style(builder, name):\n", " correct, out_tokens, t0 = 0, 0, time.time()\n", " for case in CASES:\n", " text, usage = chat_with(PROVIDER, RATER_SYSTEM, builder(case), 0.0, max_tokens=300)\n", " correct += parse_rating(text) == case[\"rating\"]\n", " out_tokens += usage[\"out\"]\n", " secs = time.time() - t0\n", " print(f\"{name:18} accuracy {correct}/{len(CASES)} avg output tokens {out_tokens / len(CASES):6.1f} {secs:5.1f}s total\")\n", "\n", "evaluate_style(zero_shot, \"zero-shot\")\n", "evaluate_style(few_shot, \"few-shot\")\n", "evaluate_style(chain_of_thought, \"chain-of-thought\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Expected output:** three lines. Usually few-shot scores best because the examples reveal the thresholds the policy only describes, chain-of-thought produces far more output tokens (so it is slower and costs more), and zero-shot is cheapest but least reliable. Strong models may score well on all three.\n", "\n", "**Reveal.** The labels came from this hidden rule: HIGH if DSCR is below 1.0 or debt/EBITDA is above 5; LOW if DSCR is at least 1.5 and debt/EBITDA is at most 3; otherwise MEDIUM. No style was given those numbers. Few-shot works when the examples let the model infer them.\n", "\n", "**What to notice:** more reasoning is not always better. If you had put the numeric rule in the prompt, all three styles would likely score highly and the cheapest would win.\n", "\n", "**Finance link:** a rating produced this way is advisory. In a real process the numeric policy should be applied by code, and the model should explain or draft, not decide (Module 4, section 8, autonomy and control)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exercise 4: Prompt caching, what does it save?\n", "**Why we do this:** many finance prompts start with the same long material (a policy, a filing) followed by a short, changing question. *Prompt caching* lets the provider remember the long shared start so you pay less, and wait less, for the repeats (Module 2, latest advancements).\n", "\n", "**What the code does:** builds a long shared context from two real SEBI documents, then asks three different short questions against it and records how many input tokens were served from cache. You write `cached_share`, the fraction of input tokens served from cache on the later calls.\n", "\n", "**How it works by provider:**\n", "- **Anthropic:** you mark the shared part with a cache setting (the notebook does this). The first call writes the cache, later calls read it.\n", "- **OpenAI:** caching is automatic for long enough prompts that start identically. Nothing to switch on.\n", "- **Others:** this notebook does not demonstrate caching for them, and the cell skips with a message. Some providers cache without telling you in a form we read.\n", "\n", "**Important:**\n", "- Caches are only used for prompts above a minimum length (it varies by model) and expire after a short time.\n", "- The first call on some providers can cost slightly *more* than normal because the cache is being written. Savings come from repeats.\n", "- Prices and discount sizes change. Enter yours from the provider's pricing page into `PRICE_IN_PER_M` and `CACHED_PRICE_FRACTION` (the share of the normal input price charged for cached tokens). Left at 0 and 1, the cell reports tokens only.\n", "\n", "**Your task:** complete `cached_share(usages)`. Hint: skip the first call, then divide the sum of `cached` by the sum of `in` over the rest." ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import urllib.request\n", "from pypdf import PdfReader\n", "\n", "DOCS = {\n", " \"memo.pdf\": \"https://www.sebi.gov.in/sebi_data/meetingfiles/dec-2024/1735042007618_1.pdf\",\n", " \"circular.pdf\": \"https://www.sebi.gov.in/sebi_data/attachdocs/feb-2025/1738665456458.pdf\",\n", "}\n", "os.makedirs(\"data\", exist_ok=True)\n", "texts = []\n", "for name, url in DOCS.items():\n", " if not os.path.exists(f\"data/{name}\"):\n", " req = urllib.request.Request(url, headers={\"User-Agent\": \"Mozilla/5.0\"})\n", " open(f\"data/{name}\", \"wb\").write(urllib.request.urlopen(req).read())\n", " texts.append(re.sub(r\"\\s+\", \" \", \" \".join(p.extract_text() or \"\" for p in PdfReader(f\"data/{name}\").pages)))\n", "LONG_CONTEXT = \"You answer questions about these two SEBI documents.\\n\\n\" + \"\\n\\n---\\n\\n\".join(texts)\n", "print(\"shared context: about\", len(LONG_CONTEXT) // 4, \"tokens (rough estimate)\")\n", "\n", "PRICE_IN_PER_M = 0.0 # dollars per million input tokens, from your provider's pricing page\n", "CACHED_PRICE_FRACTION = 1.0 # share of the normal price charged for cached tokens (1.0 = no discount known)\n", "\n", "def cached_share(usages):\n", " # YOUR CODE HERE: ignore the first call; return sum(cached) / sum(in) over the remaining calls\n", " raise NotImplementedError\n", "\n", "QUESTIONS = [\"Summarise the first document in two sentences.\",\n", " \"List the dates mentioned in either document.\",\n", " \"Which kinds of regulated entities are addressed?\"]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Solution (try first)\n", "\n", "```python\n", "def cached_share(usages):\n", " rest = usages[1:]\n", " total = sum(u[\"in\"] for u in rest)\n", " return sum(u[\"cached\"] for u in rest) / total if total else 0.0\n", "```\n", "
" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "if PROVIDER not in (\"anthropic\", \"openai\"):\n", " print(\"Caching is demonstrated only for anthropic and openai. Set PROVIDER to one of them to run this cell.\")\n", "else:\n", " usages = []\n", " for q in QUESTIONS:\n", " text, usage = chat_with(PROVIDER, LONG_CONTEXT, q, 0.0, max_tokens=300, cache_system=True)\n", " usages.append(usage)\n", " print(f\"in={usage['in']:>6} cached={usage['cached']:>6} out={usage['out']:>4} | {q[:48]}\")\n", " print(f\"\\ncached share of input on calls 2 and 3: {cached_share(usages):.0%}\")\n", " if PRICE_IN_PER_M > 0:\n", " full = sum(u[\"in\"] for u in usages) * PRICE_IN_PER_M / 1e6\n", " paid = sum((u[\"in\"] - u[\"cached\"]) + u[\"cached\"] * CACHED_PRICE_FRACTION for u in usages) * PRICE_IN_PER_M / 1e6\n", " print(f\"input cost without caching: ${full:.5f} with caching: ${paid:.5f}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Expected output:** on Anthropic, call 1 shows `cached=0` (it wrote the cache) and calls 2 and 3 show a large `cached` figure. On OpenAI, caching is automatic, so call 1 usually shows `cached=0` and the later calls show cached tokens if the prompt was long enough. If you see `cached=0` everywhere, the prompt may be below the model's minimum, the calls may have been too slow apart, or the provider may not report it.\n", "\n", "**What to notice:** the *answers* are the same with or without caching. Only cost and speed change. That makes caching one of the safest optimisations when a long, stable prefix is reused.\n", "\n", "**Finance link:** caching a long policy or filing across many analyst questions is an easy cost control, but check the provider's data-retention terms for cached content before using it with confidential material." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exercise 5: Save and reload the library\n", "**Why we do this:** the library is only useful if others can use the same versions. A JSON file can live in a shared repository, be reviewed like code, and be loaded by any application.\n", "\n", "**What the code does:** you write `save_library(path)` and `load_library(path)`. The cell saves the library, reloads it, checks it is identical, and renders one template from the reloaded copy to prove it still works.\n", "\n", "**Your task:** complete both functions with `json.dump` and `json.load` (use `indent=2` and `ensure_ascii=False` when saving, and encoding `utf-8`).\n", "\n", "**Governance idea:** treat a prompt change like a code change. Bump `version`, run the Exercise 2 checks, and record who approved it." ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "def save_library(path):\n", " # YOUR CODE HERE: write LIB to `path` as readable JSON (utf-8)\n", " raise NotImplementedError\n", "\n", "def load_library(path):\n", " # YOUR CODE HERE: read and return the JSON library from `path`\n", " raise NotImplementedError" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Solution (try first)\n", "\n", "```python\n", "def save_library(path):\n", " with open(path, \"w\", encoding=\"utf-8\") as f:\n", " json.dump(LIB, f, indent=2, ensure_ascii=False)\n", "\n", "def load_library(path):\n", " with open(path, encoding=\"utf-8\") as f:\n", " return json.load(f)\n", "```\n", "
" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "save_library(\"prompt_library.json\")\n", "reloaded = load_library(\"prompt_library.json\")\n", "print(\"round-trip identical:\", reloaded == LIB, \"| templates:\", list(reloaded))\n", "system, user = render(\"earnings_summary\", **SAMPLES[\"earnings_summary\"])\n", "print(user[:120], \"...\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Expected output:** `round-trip identical: True`, the five template names, and the start of a rendered prompt. Open `prompt_library.json` in a text editor to see exactly what a teammate would receive.\n", "\n", "## Stretch goals\n", "1. Add an automatic test per template (for example, \"the brief is at most 120 words\") and run them whenever a version changes.\n", "2. Add a `changelog` field to each template and write a short entry when you improve one.\n", "3. Test the three prompt styles on a different task, for example sentiment on earnings headlines, and see if the ranking changes.\n", "4. Put `prompt_library.json` in a Git repository and review a template change as a pull request.\n", "\n", "## Common errors\n", "- **`NotImplementedError` or `AssertionError: template ... is still blank`:** you have not completed that exercise's code or one of the two templates yet.\n", "- **`KeyError` or `ValueError` when rendering:** a variable name in the template does not match the `vars` list or the values you passed.\n", "- **`JSONDecodeError` in Exercise 2:** the model added prose around the JSON. Tighten the template (\"Return only the JSON\") and check `parse_json` handles code fences.\n", "- **Caching shows `cached=0` everywhere:** see Exercise 4. The prompt must be long and the calls close together, and the provider must report it.\n", "- **429 or rate limit:** Exercise 3 makes 24 calls. Wait, use a provider with higher limits, or shorten `CASES`." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 5 }