The Science of Confabulation Elimination. Toward Hallucination-Free AI-Generated Clinical Notes

Abridge

Davis Liang, Michael Oberst, Chenhao Tan, Zachary C. Lipton

01. Introduction

Ambient AI has rapidly become ubiquitous in healthcare workflows. The rapid adoption of this technology speaks to the tangible benefits experienced by clinicians.

In this paper, we focus on a topic that is top-of-mind for adoption of any AI-powered product: “hallucinations.”There is now an opportunity to reduce errors in clinical documentation, even to the point of near-elimination.

With the accelerating adoption of ambient AI, there is now an opportunity to reduce these errors in clinical documentation, even to the point of near-elimination.

Continued advances in AI technology present an opportunity to exceed prior standards for accuracy, ensuring that every factual claim made in an AI-drafted clinical note is supported by the clinical conversation and existing medical record.

This whitepaper outlines progress that Abridge has made towards ensuring that every claim in Abridge-generated documents is faithful to the underlying clinical context. At the outset, we stress that even amidst this progress, clinician review of AI-generated notes remains essential. Clinicians are required to review and verify their own notes before filing in the electronic health record (EHR).

Not all unsupported claims (colloquially called “hallucinations") are created equal

We first discuss how we characterize and categorize the degree to which claims are supported by context and the severity of unsupported claims. Many claims sit at the boundary of “reasonable inferences,” since clinical conversations can leave room for interpretation.

We then reveal the inner workings of our note-generation pipeline to discuss how we approach the challenge of ensuring that every claim is faithful to clinical context. We focus in particular on the guardrails we have developed, supported by over one thousand hours of human validation, to automatically revise AI-generated “first drafts” in real time, preemptively detecting and correcting unsupported claims before a clinician reviews their draft documentation.

02. Assessing Factuality of AI-Generated Documentation

When considering unsupported claims, it is essential to provide clear definitions to distinguish among distinct phenomena. Discussing “hallucinations” in black-and-white terms can be misleading and uninformative, especially when conducting fine-grained evaluations of systems, and developing auxiliary AI models that can precisely detect and correct unsupported claims.

For example, text models often draw inferences that are likely but not explicitly stated from context: Consider a situation where a note references “diabetes” when the conversation included discussion of continuing a patient's dose of metformin and monitoring HbA1c levels, even if the word “diabetes” is never mentioned in the conversation.

With these distinctions in mind, we present in this document our current internal guidelines for categorizing unsupported claims.

Note that these categories are independent of clinical severity, which is assessed in the “severity” axis.

Support - Is the claim supported by the transcript?

For each factual claim in the generated note, we assign one of the following categories to reflect the extent to which the claim is supported by the transcript of the conversation.

Severity - What is the expected impact of a claim that is not directly supported?

For each factual claim in the generated note, we assign one of the following categories to reflect the extent to which the claim is supported by the transcript of the conversation.

03. Purpose-built Guardrails for Factual Accuracy, Tuned on Clinical Data

Generating a high-quality note requires good context: the transcript and any relevant information from the EHR.

In our previous whitepaper, Pioneering the Science of AI Evaluation, we described in depth how we ensure that our automatic speech recognition (ASR) system generates transcripts that are faithful to the original conversation, far exceeding performance of off-the-shelf models.

Now, we focus on the second component of our system, which goes from a transcript to a draft of the clinical documentation. Just like our ASR system, our note-generation system consists of multiple models. Once a “first draft” of clinical documentation is produced, our system conducts further automatic refinements to convert this “first draft” into a “final draft” for the clinician to review. The system we describe below is one core part of our note generation and refinement pipeline, and we continue to iterate and improve upon it.

The Abridge system for ensuring accuracy in AI-drafted documentation has two components:

01
A proprietary, task-specific AI model for detecting unsupported claims in draft documentation
02
An automated system for correcting unsupported claims, leveraging the conversation transcript and EHR context

DETECTING UNSUPPORTED CLAIMS

An important component of our guardrail system is an in-house, task-specific language model trained on a curated set of over 50,000 training examples. Our data combine both open-source datasets for hallucination detection in the general domain and domain-specific examples (transcripts, EHR context, and example notes) that cover the breadth of scenarios where unsupported claims can arise in practice. These training examples are informed by our hands-on experience (and observations / feedback) building models for real-world clinical applications. This model is designed to detect and assess the level of support of each claim in the note, and to produce detailed reasoning for its assessment.

1. Detect and assess the level of support for each claim from the drafted note

2. Our in-house, task specific model produces detailed reasoning for its assessment

✓ The statement is supported by the conversation because...

✗ The statement is not supported by the conversation because...

AUTOMATIC SELF-CORRECTION

Once a claim is detected and assessed as being unsupported, then we correct it using another automated system. This system examines the identified claim and associated reasoning from the detection system to determine the best way to resolve the confabulation.

→ It may correct the claim to align with the clinical context

→ Delete the claim entirely

→ Decide that it is a false alarm given the reasoning

CLINICIAN REVIEW

In addition to system-level automation, we also believe in clinician review. All clinician notes generated by Abridge are reviewed and edited (if necessary) by the clinicians who conducted the patient encounter before being entered into the EHR. No guardrail is perfect, and clinical conversations can contain legitimate sources of ambiguity, for instance, where the “correct” note may rely on clinical judgements that were not verbalized during the encounter.

To make clinician review as efficient as possible, every user has access to proprietary Abridge capabilities developed as part of our foundational commitment to trust and verification of AI-generated outputs (including Linked Evidence, which enables clinicians to see evidence for each AI-generated summary). This tool empowers clinicians to perform their own review and validation of the clinical document.

Validating the automated system. Setting New Benchmarks for Accuracy

The automated components of our system—catching and correcting unsupported claims in “first draft” clinical documentation—are backed by careful curation of challenging internal benchmarks as well as over a thousand hours of annotation and validation by board-certified physicians. For instance, one of our internal benchmark datasets contains over 10,000 realistic clinical encounters, including examples of realistic claims in clinical notes as well as modified versions of those claims that are unsupported to varying degrees.

These internal datasets help us compare to standard off-the-shelf solutions. On the benchmark discussed above, our technology outperforms GPT-4o by a wide margin, catching 97% of the confabulations, while GPT-4o only catches 82%.

In other words, a standard off-the-shelf model misses six times as many confabulations as the Abridge system. These results highlight the value of a purpose-built model trained on carefully curated realistic data for this task.

Percentage of all unsupported claims caught
Percentage of all unsupported claims caught

Conclusion

Our vision for ambient AI is not just about faster and easier documentation:

We believe that the adoption of ambient AI has the potential to nearly eliminate all mistakes in medical documentation.

THE OPPORTUNITY IS CREAL:

A 2024 review of 48 research studies that examined medical records to detect errors found that in 47 of the 48 studies, errors were present, suggesting that errors can commonly be found in medical documentation. These mistakes are not harmless; from the review: “Various studies have shown that poor documentation contributes to medical errors, malpractice claims, and even patient mortality.”

We launched Abridge in 2018 to power deeper understanding in healthcare. Understanding must start with accuracy, and trust to deliver accuracy must be earned—over and over again.

“That’s why it’s imperative that the deployment of ambient AI does not just maintain the status quo, but actually improves the accuracy of documentation—both to engender trust from clinicians, patients, and other stakeholders, as well as to ensure that we introduce meaningful progress when we add new technology to workflows.”

By sharing how we understand and address the problem of “hallucinations,” we hope to set a higher standard for transparency and trust. We also hope to spark a broader conversation on measuring progress towards the goal of perfect accuracy: Not all “hallucinations” are the same, and understanding the subtle-but-important differences in type and severity is crucial to our long-term aim to reduce inaccuracies in clinical documentation to near zero. As discussed above, our current system is equipped with purpose-built guardrails that already out-perform standard off-the-shelf solutions. Our ambition is to continue to measure, adjust, and build AI technology that draws us ever closer to clinical notes free of documentation errors.