Skip to content

Add Citation Faithfulness Check notebook (RAG cookbook) #783

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tonydzi wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add Citation Faithfulness Check notebook (RAG cookbook) #783

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
202 changes: 202 additions & 0 deletions notebooks/Citation_Faithfulness_Check.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "cecb1dd6",
"metadata": {},
"source": [
"# Check Citation Faithfulness in Cohere RAG\n",
"\n",
"Cohere's Chat API grounds answers in the `documents` you pass and returns\n",
"**citations** — spans of the reply, each linked to the source document(s) it came\n",
"from. That is exactly the metadata you need to catch a citation *before* it\n",
"reaches a user. This notebook adds a small, deterministic faithfulness check on\n",
"top of Cohere citations.\n",
"\n",
"The failure modes worth catching are the ones that read as authoritative:\n",
"\n",
"- **fabricated** — a cited span that appears in none of the source documents;\n",
"- **frankenquote** — every word is real, but the exact span was never written\n",
" contiguously in the source;\n",
"- **misattributed** — a real span, but the citation points at the wrong document.\n",
"\n",
"A judge model asked \"does this support the claim?\" waves all three through — they\n",
"look fluent and supportive. So we ask the cheaper, prior question first, with no\n",
"model and no tokens: **does the cited text appear verbatim in the document it is\n",
"attributed to?**\n",
"\n",
"This is the standalone, framework-agnostic gate from\n",
"[`verbatim-citation-gate`](https://github.com/Palo-Alto-AI-Research-Lab/verbatim-citation-gate),\n",
"inlined here so the notebook has no extra dependency."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8b9793df",
"metadata": {},
"outputs": [],
"source": [
"import re\n",
"\n",
"\n",
"def normalize(text: str) -> str:\n",
" \"\"\"Case/typography/whitespace-insensitive form for verbatim matching.\"\"\"\n",
" text = text.lower()\n",
" text = re.sub(r\"[‘’]\", \"'\", text)\n",
" text = re.sub(r\"[“”]\", '\"', text)\n",
" text = re.sub(r\"[–—]\", \"-\", text)\n",
" text = re.sub(r\"[^a-z0-9%.]+\", \" \", text)\n",
" return \" \".join(text.split())\n",
"\n",
"\n",
"def gate(cited_text: str, cited_doc_id: str, docs: dict) -> str:\n",
" \"\"\"Return 'found' | 'misattributed' | 'not_found'. Fails closed on empty text.\"\"\"\n",
" q = normalize(cited_text)\n",
" if not q:\n",
" return \"not_found\"\n",
" cited = docs.get(cited_doc_id)\n",
" if cited is not None and q in normalize(cited):\n",
" return \"found\"\n",
" if any(q in normalize(t) for d, t in docs.items() if d != cited_doc_id):\n",
" return \"misattributed\"\n",
" return \"not_found\""
]
},
{
"cell_type": "markdown",
"id": "4ffd1ead",
"metadata": {},
"source": [
"## 1. Run it offline on Cohere-shaped citations\n",
"\n",
"So the notebook is runnable in CI without an API key, here is a small knowledge\n",
"base and a set of citations in the shape Cohere's Chat API returns them\n",
"(`start`, `end`, `text`, and the `sources` they are attributed to). One citation\n",
"is faithful; the others are the three planted failure modes."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c5349f75",
"metadata": {},
"outputs": [],
"source": [
"# id -> document text (as you would pass to co.chat(documents=...))\n",
"DOCS = {\n",
" \"doc_0\": \"The Aptera solar EV has a claimed range of 400 miles on a single charge.\",\n",
" \"doc_1\": \"Its roof-mounted solar array adds up to 40 miles of range per day in ideal sun.\",\n",
"}\n",
"\n",
"# Citations in the shape Cohere returns (text = the exact span, sources = doc ids).\n",
"CITATIONS = [\n",
" {\"text\": \"range of 400 miles on a single charge\", \"doc_id\": \"doc_0\", # faithful\n",
" \"claim\": \"The car goes 400 miles per charge.\"},\n",
" {\"text\": \"adds up to 40 miles of range per day\", \"doc_id\": \"doc_0\", # real span, wrong doc\n",
" \"claim\": \"Solar adds 40 miles/day.\"},\n",
" {\"text\": \"400 miles of range per day from solar\", \"doc_id\": \"doc_1\", # frankenquote\n",
" \"claim\": \"Solar alone gives 400 miles/day.\"},\n",
" {\"text\": \"a top speed of 110 miles per hour\", \"doc_id\": \"doc_0\", # fabricated\n",
" \"claim\": \"Top speed is 110 mph.\"},\n",
"]\n",
"\n",
"for c in CITATIONS:\n",
" status = gate(c[\"text\"], c[\"doc_id\"], DOCS)\n",
" faithful = \"OK \" if status == \"found\" else \"FLAG\"\n",
" print(f\"{faithful} [{status:>13}] {c['claim']}\")"
]
},
{
"cell_type": "markdown",
"id": "353c11aa",
"metadata": {},
"source": [
"`found` citations are safe to surface; `misattributed`, `not_found` (fabricated\n",
"or frankenquote) should be flagged or dropped before the answer reaches a user —\n",
"all decided deterministically, for zero tokens."
]
},
{
"cell_type": "markdown",
"id": "6cdb1b87",
"metadata": {},
"source": [
"## 2. Wire it to the live Cohere Chat API\n",
"\n",
"With an API key, ground a real answer in documents, then run the same gate over\n",
"the citations Cohere returns. This cell needs `COHERE_API_KEY` and is not run in\n",
"CI."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f48b9d72",
"metadata": {},
"outputs": [],
"source": [
"# pip install cohere\n",
"import os\n",
"\n",
"if not os.getenv(\"COHERE_API_KEY\"):\n",
" print(\"Set COHERE_API_KEY to run the live example.\")\n",
"else:\n",
" import cohere\n",
"\n",
" co = cohere.ClientV2()\n",
" documents = [\n",
" {\"id\": \"doc_0\", \"data\": {\"text\": DOCS[\"doc_0\"]}},\n",
" {\"id\": \"doc_1\", \"data\": {\"text\": DOCS[\"doc_1\"]}},\n",
" ]\n",
" resp = co.chat(\n",
" model=\"command-r-plus\",\n",
" messages=[{\"role\": \"user\", \"content\": \"What is the Aptera's range, and how much does solar add per day?\"}],\n",
" documents=documents,\n",
" )\n",
"\n",
" # Build the id -> text map from the same documents we grounded on.\n",
" doc_text = {d[\"id\"]: d[\"data\"][\"text\"] for d in documents}\n",
"\n",
" for cit in (resp.message.citations or []):\n",
" span = cit.text\n",
" for src in cit.sources:\n",
" # ChatV2 source ids look like \"doc_0\", matching the document ids above.\n",
" doc_id = getattr(src, \"document\", {}).get(\"id\") if hasattr(src, \"document\") else src.id\n",
" status = gate(span, doc_id, doc_text)\n",
" flag = \"OK \" if status == \"found\" else \"FLAG\"\n",
" print(f\"{flag} [{status:>13}] {span!r} -> {doc_id}\")"
]
},
{
"cell_type": "markdown",
"id": "5aa1d879",
"metadata": {},
"source": [
"## Notes\n",
"\n",
"- Cohere cites **spans of the generated reply**, so the gate's \"does this text\n",
" exist verbatim in the cited document?\" question maps directly onto\n",
" `citation.text` vs the document it points at — no offset bookkeeping needed.\n",
"- The gate is the cheap first stage. For the harder, genuinely ambiguous case —\n",
" a real, correctly-attributed span that may still not *support* the claim at full\n",
" strength — pair it with the burden-of-proof LLM judge in\n",
" [`verbatim-citation-gate`](https://github.com/Palo-Alto-AI-Research-Lab/verbatim-citation-gate),\n",
" which fails closed on unparseable output."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.10"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
1 change: 1 addition & 0 deletions notebooks/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ This section provides a deep dive into various techniques in the following topic
| [Creating a QA Bot From Technical Documentation](https://github.com/cohere-ai/cohere-developer-experience/blob/main/notebooks/guides/Creating_a_QA_bot_from_technical_documentation.ipynb) | Chat, Embed, Rerank, LlamaIndex | [<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>](https://colab.research.google.com/github/cohere-ai/cohere-developer-experience/blob/main/notebooks/guides/Creating_a_QA_bot_from_technical_documentation.ipynb) |
| [Analysis of Form 10-K/10-Q Using Cohere and RAG](https://github.com/cohere-ai/cohere-developer-experience/blob/main/notebooks/guides/Analysis_of_Form_10_K_Using_Cohere_and_RAG.ipynb) | Cohere, Embed, Rerank, LlamaIndex, Langchain | [<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>](https://colab.research.google.com/github/cohere-ai/cohere-developer-experience/blob/main/notebooks/guides/Analysis_of_Form_10_K_Using_Cohere_and_RAG.ipynb) |
| [Adaptive RAG](https://github.com/cohere-ai/cohere-developer-experience/blob/main/notebooks/agents/Multi_Step_Tool_Use.ipynb) | Chat, LangChain | [<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>](https://colab.research.google.com/github/cohere-ai/cohere-developer-experience/blob/main/notebooks/agents/Multi_Step_Tool_Use.ipynb) |
| [Check Citation Faithfulness](https://github.com/cohere-ai/cohere-developer-experience/blob/main/notebooks/Citation_Faithfulness_Check.ipynb) | Chat | [<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>](https://colab.research.google.com/github/cohere-ai/cohere-developer-experience/blob/main/notebooks/Citation_Faithfulness_Check.ipynb) |


## Agents
Expand Down