How we measure whether the evidence is right

Most clinical AI tools ask you to trust a summary. CliniAtlas is built to be checked, and we measure how well it holds up. Our evaluation harness reports three metrics on a fixed, clinician-reviewed question set.

Citation faithfulness

This is our primary anti-hallucination metric. For every non-abstained answer, each cited paragraph is checked against the text of its cited sources and labelled supported, partially supported, or unsupported. Alongside it we report citation coverage — the share of substantive claims that carry a valid citation — and the invalid-citation rate. Because the check uses titles, abstracts, and snippets rather than licensed full text, the reported score is a deliberate lower bound.

Retrieval recall

This is deterministic: for questions with expected key references (identified by PMID or DOI), it measures the fraction that retrieval actually surfaces. It targets evidence sufficiency — whether the system finds the papers a clinician would expect.

Answer correctness

This compares each answer to a clinician-reviewed reference answer and its key points, and is abstention-aware: for questions where the honest answer is “the evidence doesn’t support a conclusion,” full marks require the system to abstain rather than invent one.

Human calibration and AI Act alignment

To keep the judged metrics honest, a clinician hand-labels a portion of the faithfulness and correctness items and we calibrate the automated judge against those human ratings before trusting its scores. The methodology is designed to support EU AI Act technical documentation: measurable accuracy and groundedness, dataset provenance, versioned results, and human oversight.