How do you make an assistant cite its sources, and how do you check the citations are real?

“I split this into producing citations and verifying them, and the model owns neither a source’s identity nor how it’s rendered.

Producing. Retrieval returns passages with stable IDs from the index, such as policy_812#p4. I put them in the prompt under short aliases, [S1] through [S8], and keep the alias map in code; short aliases are easy for the model to copy and trivial to validate. The model returns JSON: a list of statements, each with sources, a list of aliases, and quote, a short span copied from one of them. The UI renders title, link and date from our source store by ID, after checking the user may open that source, so a citation never shows a document the user isn’t allowed to see. The model never writes a URL.

If the provider’s API returns citations as spans of documents you pass in, such as Anthropic’s citations feature, I’d use it. It guarantees the cited span exists in a document I supplied, so it replaces the alias and quote checks, but not the number and entailment checks. On that API it can’t be combined with structured outputs, so it replaces the JSON format too.

Verifying. A validator runs on every response before anyone sees it:

import re
import unicodedata

MIN_QUOTE_WORDS = 5
NUM = re.compile(r"\d+(?:[.,]\d+)*")

def normalize(s: str) -> str:
    s = unicodedata.normalize("NFKC", s).lower()
    s = re.sub(r"[^\w\s]", " ", s)
    return " ".join(s.split())

def numbers_supported(claim: str, passage: str) -> bool:
    wanted = set(NUM.findall(claim))
    return wanted <= set(NUM.findall(passage))

def verify(
    answer: Answer,
    retrieved: dict[str, Passage],
    judge: Judge,
) -> list[Statement]:
    kept = []
    for s in answer.statements:
        passages = [retrieved.get(a) for a in s.sources]
        if not passages or None in passages:
            log_failure("unknown_source", s)
            continue
        q = normalize(s.quote)
        # the passage the quote came from
        source = next(
            (p for p in passages if q in normalize(p.text)),
            None,
        )
        too_short = len(q.split()) < MIN_QUOTE_WORDS
        if too_short or source is None:
            log_failure("quote_not_found", s)
            continue
        if not numbers_supported(s.text, source.text):
            log_failure("number_mismatch", s)
            continue
        verdict = judge.label(
            premise=source.text, claim=s.text
        )
        if verdict != "supported":
            log_failure("not_entailed", s)
            continue
        kept.append(s)
    return kept

Four statements show what each check catches:

The first check catches invented sources. The second catches a real source with an invented quote; an empty or tiny quote fails, or the check proves nothing. The third is cheap and runs before any model: every number in the statement, including the digits of a date, must appear in the passage, because a wrong figure is the error a legal or finance customer can least afford, and an entailment model can let $5,000 pass against $50,000. The fourth is for the hard case, where the citation and quote are both real but the statement goes beyond them: the passage says the policy covers water damage from a burst pipe, and the statement says it covers flooding. Only an entailment check sees that. I’d use an NLI model or a small LLM judge with a narrow prompt: given this passage, is the statement supported, contradicted, or not addressed? The judge reads the passage the quote came from, so every check is about the same passage, and a single passage fits a typical NLI model, which reads about 512 tokens and would truncate joined text. A statement only two passages support together fails this check, so the prompt asks for statements small enough that one passage supports each. Either way I calibrate the judge against a few hundred human-labeled pairs from this customer’s documents and report agreement, because the judge is a model too and can be wrong.

Checking before display rules out streaming, so I send every statement and passage pair to the judge in one batch and show the passages while it runs. For a low-stakes internal tool, I’d stream the answer and mark each statement as unverified until its check passes.

On failure. Unsupported statements are dropped. If that removes the core of the answer, I regenerate once, naming the statement that failed; if it fails again, the assistant says the documents don’t answer the question and shows the closest passages. For a legal or financial customer, any failure means no generated answer, only the passages.

Knowing it keeps working. Offline, a labeled question set with gold passages, scored on citation precision and recall, with the judge spot-checked by a person. Online, the failure rate for each check goes on a dashboard: a jump in quote_not_found after a model or prompt change is a regression I want to see the same day. A weekly sample of shown answers goes to a reviewer, since the judge misses things as well.

When their legal team asks what the check mark next to a sentence means, I say: ‘It means our checker found that the cited passage supports that sentence. We tested the checker on a few hundred sentences from your own documents that people had labeled, and its agreement with them is on this dashboard. Every week a person reviews a sample of the answers it passed.’“