AI answers without sources: how do you control citations?

At the retrieval layer, query the knowledge base and return chunks with their original metadata. Azure AI Search describes RAG as grounding an LLM with retrieved content; its current agentic retrieval can return structured grounding data, citations, and execution metadata. With classic RAG, the application can still preserve the metadata as it passes context to the model.

At the generation layer, constrain the model to supplied evidence and request structured output—for example, claims carrying citation_ids that reference only identifiers in the retrieval set. The backend resolves those IDs into title, location, and URL metadata. A new URL written by the model is not accepted as a valid source.

At the verification layer, split the response into evidence-bearing claims and evaluate each claim ↔ cited chunk pair. The verifier may use rules, a separate evaluation model, or layered checks. For important claims, a compact verdict such as supported | unsupported | insufficient is easier to connect to policy than open-ended reviewer prose.

1. Normalize the question and establish tenant, authorization, query type, and freshness requirements.
2. Retrieve candidate chunks from authorized sources; hybrid search and reranking can improve retrieval quality where the workload benefits from them.
3. Apply a retrieval gate: if no sufficiently relevant evidence is found, stop or use another retrieval strategy instead of allowing an ungrounded model-only answer.
4. Generate a structured answer from evidence with explicit claims and citation IDs.
5. Resolve citation IDs in the backend to stored metadata; an ID outside the retrieval set is invalid.
6. Verify claim-evidence pairs for faithfulness, completeness, and sufficiency according to the use case's risk.
7. Apply policy: remove or rewrite unsupported claims, retrieve again when evidence is weak, and abstain when the system cannot establish enough evidence.
8. Render citations at a useful granularity—document, section/page, or deep link where the source supports it.
9. Persist the trace including source version, retrieved chunks, answer, verifier verdicts, and model/config version for reproducible audits.

Citations are rendered near the end of the pipeline, but source identity must already exist at ingestion and retrieval time. A user-interface fix cannot repair missing provenance.

For an internal knowledge assistant, a sensible policy can require evidence for important factual claims, while greetings or simple interface navigation may not need citations. If retrieval is empty, the source is expired, authorization cannot be verified, attribution metadata is lost, or the verifier marks a claim unsupported, the system should not silently switch to model background knowledge without telling the user.

The OWASP RAG Security Cheat Sheet recommends fail-closed behavior throughout the RAG pipeline: if retrieval fails, do not fall back to model memory; where attribution is required, failure to produce source attribution should block the response. OWASP also recommends returning attribution down to retrieved documents/chunks with provenance metadata and protecting attribution integrity.

Make the policy risk-tiered. HR policies, prices, contractual terms, operational procedures, and customer data deserve stricter controls than brainstorming. For actions with side effects, even excellent citations are not authorization; permission to execute the action must be enforced separately.

Build an evaluation set from real questions, including answerable cases, missing-data cases, conflicting sources, stale documents, and attempts to reach unauthorized content. Each case should define expected evidence or expected abstention. At minimum, track citation faithfulness, citation coverage for claims that require sources, unsupported-claim rate, abstention quality, retrieval recall on the test set, and the rate at which citations resolve to real sources.

Avoid declaring one universal threshold safe for every domain. Calibrate gates to the cost of error. If the verifier uses an LLM, retain the model version, prompt/rubric version, and representative samples so drift is observable. The NIST GenAI Profile recommends reviewing and verifying citations in pre-deployment measurement and ongoing monitoring; NIST's evaluation-probe project also demonstrates running probes in-workflow or post-hoc and recording structured audit trails.

In production, alert when unsupported, unresolved citation, retrieval empty, or abstain rates deviate materially from the established baseline. These signals help distinguish knowledge-base, indexing, permission, prompt, and model-change problems from the vague report that “the AI is answering strangely today.”

Step 1 — Audit provenance. Check whether every current chunk carries document_id, source location, version/hash, and enough ACL context to trace it safely.

Step 2 — Define the citation contract. Specify the schema between retriever and generator; citations may reference only IDs present in context.

Step 3 — Use structured generation. Make answer, claims, and citation IDs machine-validatable instead of parsing free-form [1] markers from Markdown.

Step 4 — Add a verifier. Start with important claims and three questions: does the source support the claim, is critical context omitted, and is the evidence sufficient for the strength of the statement?

Step 5 — Add abstention and re-retrieval. When evidence is weak, search again or return a controlled insufficient-evidence response; do not silently use model memory when policy requires grounding.

Step 6 — Build a golden set and CI gate. Run evaluations when chunking, embeddings, retrievers, rerankers, prompts, or models change.

Step 7 — Monitor production. Retain audit traces according to policy and watch citation failures, permission failures, and retrieval drift.

If the current system only has a prompt saying “cite your sources,” prioritize steps 1–3. They turn citation from decorative text into a data contract the application can validate.

1. Is a prompt saying “always cite sources” enough?

No. A prompt cannot prove that a URL exists or that a source supports a claim. Source identity should come from retrieval metadata and be validated by the application.

2. Should citations point to a whole document or an exact passage?

When the source supports it, preserve both document identity and a finer location such as chunk, page, or section. Users need to open the source, while verifiers need evidence narrow enough to evaluate the claim.

3. Does every sentence need a citation?

Not necessarily. Define which claim types require evidence by risk tier. Important factual claims grounded in the knowledge base should be cited; navigation-only or conversational text can use a different policy.

4. If two sources conflict, should AI simply choose one?

Not silently. Authority, version, and freshness can drive an explicit precedence policy; if the conflict remains meaningful, the answer should expose the disagreement or abstain rather than create false certainty.

5. Does RAG eliminate hallucinations?

No. RAG grounds answers in retrieved data, but the system can still retrieve the wrong content, cite the wrong chunk, or overstate what evidence supports. Retrieval quality, citation verification, evaluation, and fail-closed policy are still necessary.