Real-world evidence data quality: FDA requirements for RWE regulatory submission

FDA rejected a supplemental new drug application in 2023 partly because the sponsor could not demonstrate that its real-world data source reliably captured the outcomes it claimed to measure. The study design was sound. The statistical methods were appropriate. The data itself was the problem.

This pattern repeats across the agency's growing body of RWE-related decisions. Since the 21st Century Cures Act mandated FDA to evaluate the use of real-world evidence in regulatory decision-making, the agency has published multiple guidance documents, a formal RWE framework, and dozens of pilot projects. The consistent finding: data quality is the bottleneck, not regulatory willingness.

What FDA means by real-world evidence and real-world data

FDA distinguishes between real-world data (RWD) and real-world evidence (RWE). RWD is the raw material: electronic health records, claims and billing data, patient registries, data from mobile devices, and other sources that capture what happens to patients outside of controlled clinical trial settings.

RWE is the clinical evidence derived from analysis of RWD. A claims database is RWD. A comparative effectiveness study built on that claims database is RWE.

This distinction matters because FDA does not regulate the data source directly. It evaluates whether the evidence generated from a data source is fit for a specific regulatory question. That evaluation rests almost entirely on whether the underlying RWD meets defined quality thresholds.

The FDA RWE framework: three pillars of data fitness

FDA's Framework for Real-World Evidence Program, published in December 2018 and updated through subsequent guidance documents, establishes three core criteria for evaluating whether RWD can support regulatory decisions.

Relevance. Does the data capture the right population, exposure, and outcomes for the regulatory question? A claims database might capture drug dispensing but miss whether the patient actually took the medication. An EHR system might record diagnoses but lack structured outcome assessments. Relevance requires that the data source actually measures what the sponsor claims it measures.

Reliability. Can the data generation process be trusted to produce consistent, accurate results? This includes data accrual (how data enters the system), data completeness (whether records contain the fields needed for analysis), and data quality assurance (whether systematic checks exist to identify errors, duplicates, and inconsistencies).

Regulatory context. Is the RWE being used to support an appropriate type of decision? FDA has accepted RWE for label expansions, post-marketing safety commitments, and as historical controls in single-arm trials. The bar differs depending on whether the evidence supports an efficacy claim versus a safety signal.

Most submissions that fail on RWE do so on reliability. The sponsor cannot demonstrate an unbroken chain from data capture to analysis dataset.

Key statistics

RWE data curation: manual vs DTI-scored processing (imaware case study)

FDA has considered RWE in more than 120 regulatory decisions since 2018, spanning drugs, biologics, and medical devices.

The agency's 2023 guidance on RWE for medical devices specifies that sponsors must document data provenance, including the original source, any transformations applied, and the completeness of key variables.

A 2022 Duke-Margolis analysis found that 65% of RWE submissions that received FDA feedback were asked to provide additional documentation on data quality or data source fitness, more than any other category of deficiency.

SuperTruth's work with imaware standardized 105,000 diagnostic records in 2 hours, down from 3 weeks manually, achieving provenance-level documentation that meets the data lineage requirements FDA expects for RWD sources.

FDA's draft guidance on electronic health records as RWD sources identifies at least 14 specific data quality dimensions that sponsors must address, including coding accuracy, temporal completeness, and cross-source concordance.

What FDA actually audits in an RWE submission

When a sponsor submits RWE to support a regulatory decision, FDA reviewers evaluate a specific set of data quality attributes. These are not abstract concepts. They map to concrete documentation requirements.

Data provenance and lineage. Where did each record originate? What system captured it? What transformations were applied between capture and the analytic dataset? FDA expects sponsors to maintain a complete audit trail. If a diagnosis code was mapped from ICD-9 to ICD-10 during the study period, that mapping must be documented and validated.

Completeness assessment. What percentage of records have missing values for key variables? How does missingness correlate with patient characteristics? FDA treats differential missingness (where data is more likely to be missing for certain subgroups) as a potential source of bias that must be addressed quantitatively.

Outcome validation. If the study uses diagnosis codes as proxies for clinical outcomes, what is the positive predictive value of those codes against chart-confirmed diagnoses? FDA has required chart validation studies for claims-based outcome definitions, sometimes demanding PPV above 80% before accepting the outcome measure.

Temporal integrity. Are dates of service, prescription fills, and outcomes recorded with sufficient precision? Claims data often has 30 to 90 day reporting lags that distort the temporal relationship between exposure and outcome. FDA expects sponsors to quantify and adjust for these lags.

Cross-source concordance. When multiple data sources contribute to a study (for example, linking EHR data to claims data to capture both clinical detail and healthcare utilization), FDA evaluates linkage accuracy, overlap rates, and whether linked records show concordant information for shared variables.

These requirements align closely with what any trust-scored data infrastructure must deliver. Provenance, completeness, recency, concordance: these are not just FDA terms. They are the dimensions that determine whether data is fit for any high-stakes use.

How FDA RWE requirements differ for drugs versus devices

FDA applies different evidentiary standards depending on the product type.

For drugs and biologics, RWE has been accepted primarily as supportive evidence: historical controls for single-arm trials, external control arms for rare diseases, and post-marketing commitments. The Cures Act specifically directed FDA to evaluate how RWE could support new indications for approved drugs and post-approval study requirements.

For medical devices, FDA has been more permissive. The agency's 2017 and 2023 guidance documents on real-world evidence for medical devices explicitly describe using RWE for premarket clearance (510(k)), de novo classification, and premarket approval (PMA) applications. The agency has accepted registry data from the National Cardiovascular Data Registry (NCDR) and the Society of Thoracic Surgeons (STS) database to support device approvals.

The data quality requirements are functionally similar for both product types, but device submissions more frequently rely on registry data, which introduces its own set of quality challenges around standardized data collection forms, site-level variability, and follow-up completeness.

The gap between guidance and practice

FDA guidance documents describe what good data quality looks like. They do not prescribe a specific scoring system or certification process. This creates a gap that sponsors must fill themselves.

In practice, most sponsors rely on ad hoc documentation: data dictionaries, validation reports, and narrative descriptions of data quality procedures. These documents are typically assembled late in the submission process, after the analysis is complete, by teams who did not control data capture.

This approach fails in predictable ways. A sponsor might discover during FDA review that their EHR data source changed its coding system mid-study. Or that their claims data vendor applies proprietary cleaning algorithms that cannot be fully described. Or that their patient registry has a 25% loss-to-follow-up rate that disproportionately affects the sickest patients.

The alternative is to score data quality prospectively, before analysis begins, using a systematic framework that maps to the dimensions FDA evaluates. This is what trust-scored data infrastructure enables.

What FDA expects from EHR data used as real-world data

FDA's draft guidance on EHRs as a source of RWD, published in 2024, is the most specific document the agency has released on data quality for a particular RWD source type. It addresses several areas where EHR data frequently fails regulatory scrutiny.

Structured versus unstructured data. EHRs contain both structured fields (coded diagnoses, lab values, medication orders) and unstructured text (clinical notes, radiology reports). FDA expects sponsors to describe how they extracted information from unstructured data, what NLP methods were used, and what the error rate of that extraction process was. This connects directly to the challenge of NLP on clinical notes, where trust scoring must precede any text extraction.

Coding accuracy. ICD-10 codes in EHRs are entered for billing purposes, not research. The same clinical condition may be coded differently across sites, providers, or time periods. FDA expects sponsors to validate that their code-based definitions actually identify the intended patient population. This is the same ICD-10 coding accuracy problem that affects any downstream use of billing data.

Data completeness across care settings. A patient's EHR at one health system captures only the care delivered at that system. If the patient receives specialist care, fills prescriptions, or is hospitalized elsewhere, those events are invisible. FDA evaluates whether the data source captures a sufficiently complete picture of the patient's relevant healthcare experience.

Site-level variability. Multi-site EHR studies must account for differences in documentation practices, EHR vendor configurations, and clinical workflows. Two sites using the same EHR platform may capture the same clinical concept in different fields or with different levels of detail.

Claims data, registries, and other RWD sources

Claims data offers broad population coverage and longitudinal follow-up, but FDA reviewers are acutely aware of its limitations. Claims capture billing events, not clinical events. A claim for a procedure confirms that a provider billed for it, not that the patient experienced a specific clinical outcome.

FDA expects sponsors using claims data to document the lag between service delivery and claim adjudication, validate that procedure codes reflect clinical intent, and address the fundamental gap between administrative data and clinical reality.

Patient registries receive somewhat more favorable treatment because they are typically designed for research or quality measurement with standardized data collection forms. But FDA still evaluates registry data quality rigorously, particularly around enrollment completeness, follow-up rates, and whether the registry population is representative of the broader patient population.

Patient-generated health data from wearables and mobile apps represents the newest RWD source under FDA evaluation. The agency's requirements here are still evolving, but early guidance emphasizes device validation, data transmission integrity, and patient adherence to data collection protocols.

How data trust scoring maps to FDA RWE requirements

DTI dimensions mapped to FDA RWE data quality requirements

The eight dimensions of the Data Trust Index map directly to what FDA evaluates in RWE submissions.

Provenance (25% weight in DTI). FDA's data lineage requirements ask the same question DTI's provenance dimension answers: where did this record come from, who created it, and what happened to it between creation and analysis?

Consent (20%). FDA requires that RWD used in regulatory submissions complies with applicable privacy regulations and that patients' data is used within the scope of authorized purposes. For international data sources, this includes GDPR considerations.

Recency (15%). FDA evaluates whether the study period is relevant to the current treatment landscape. Using data from 2015 to support a 2025 submission raises questions about whether care patterns, coding practices, and the competitive treatment environment have changed.

Quality (10%). This maps to FDA's data accuracy and completeness requirements, including coding validation, missingness analysis, and error rate documentation.

Concordance (10%). When sponsors link multiple data sources, FDA evaluates cross-source agreement. DTI's concordance dimension quantifies exactly this.

Validation (10%). FDA frequently requires external validation of outcome definitions, algorithm performance, and study replicability. DTI's validation dimension tracks whether records have been independently verified.

Breadth (5%). The clinical breadth of a data source determines what questions it can answer. A source that captures only inpatient encounters cannot reliably support a study of outpatient treatment patterns.

Stability (5%). Data sources that change their capture methods, coding systems, or population coverage over time create analytic challenges that FDA expects sponsors to address.

What a submission-ready data infrastructure looks like

Sponsors who succeed with RWE submissions share common infrastructure characteristics.

They score data quality before study design, not after analysis. They maintain automated provenance documentation that tracks every transformation from source to analytic dataset. They validate outcome definitions against gold-standard clinical assessments before committing to a study design. They quantify completeness and missingness at the variable level, not just the record level.

This is not manual work. At the scale of modern RWE studies (often hundreds of thousands of patients across multiple data sources), systematic scoring is the only feasible approach.

The imaware case demonstrates what this looks like in practice. SuperTruth's DTI Engine processed 105,000 diagnostic records, standardizing variable names, mapping codes to canonical terminologies, and scoring each record across trust dimensions. What previously required 3 weeks of manual curation took 2 hours. The resulting dataset carried provenance documentation that could withstand regulatory scrutiny because every transformation was logged, scored, and auditable.

The regulatory direction is clear

FDA has signaled repeatedly that RWE will play an expanding role in regulatory decisions. The PDUFA VII commitments include specific milestones for developing RWE standards. The agency's partnership with Sentinel for safety surveillance generates RWE at massive scale. International harmonization through ICH is incorporating real-world data quality standards into global regulatory expectations.

But expanding acceptance of RWE does not mean relaxing quality standards. If anything, as RWE moves from supportive evidence to primary evidence, the data quality bar will rise. Sponsors who build trust-scored data infrastructure now will have a structural advantage over those who continue to rely on post hoc documentation.

The difference between an RWE submission that succeeds and one that triggers a refuse-to-file letter often comes down to a single question: can you prove that your data is what you say it is? That proof requires systematic, auditable, dimension-level trust scoring applied before the first analysis runs.

SuperTruth's trust layer turns RWE from a compliance liability into a competitive asset. If your team needs audit-ready provenance for FDA submission or payer negotiation, contact Louis Simeonidis at louis@supertruth.ai or (215) 918-4140.

Further reading:

Pharma solution

DTI™ Engine

What makes health data Platinum-grade for FDA regulatory submission

How the FDA will audit your health AI's training data

Registry data quality for rare disease: what FDA expects from patient registries