SaMD data trust requirements for FDA De Novo success
FDA De Novo authorization requires SaMD sponsors to prove their training and validation data is trustworthy, not just accurate. Fewer than 30% of AI/ML De Novo submissions succeed on the first review cycle, and data trust failures account for a significant share of deficiency letters. This post maps the specific data requirements the FDA enforces and where most sponsors fall short.
The FDA authorized 1,016 AI/ML-enabled medical devices through October 2024, but fewer than 100 of those arrived through the De Novo pathway. The De Novo classification request exists for novel devices that have no predicate, which means the sponsor cannot lean on an existing cleared device to justify safety and effectiveness. For software as a medical device, that absence of a predicate shifts the entire burden onto the data.
Most guides to the De Novo process describe the administrative steps: pre-submission meetings, classification requests, review timelines. What they skip is the reason most SaMD De Novo submissions stall: the FDA cannot verify that the data behind the algorithm is trustworthy.
Why the De Novo pathway demands more data trust than a 510(k)
A 510(k) submission lets a sponsor argue substantial equivalence to a predicate device. The predicate carries its own history of clinical evidence, and the FDA can compare new data against that baseline. The De Novo pathway offers no such shortcut.
FDA's De Novo Classification Request guidance specifies that the sponsor must provide "a reasonable assurance of safety and effectiveness" for a device with no predicate. For SaMD, that assurance is built almost entirely on data: the training dataset, the validation dataset, the clinical evidence linking algorithm output to patient benefit, and the documentation proving each dataset is what the sponsor claims it is.
The FDA's 2021 action plan for AI/ML-based SaMD identified five priorities. Three of them, algorithm transparency, real-world performance monitoring, and bias elimination, depend directly on data provenance and data quality. The plan explicitly calls for a "patient-centered approach" that includes evaluation of datasets for representativeness.
The six data trust dimensions the FDA actually evaluates
FDA reviewers do not use the phrase "data trust." They use terms like data provenance, dataset representativeness, reference standard adequacy, and algorithmic bias assessment. But when you map those terms to their operational requirements, six distinct dimensions emerge.
Provenance
The FDA expects sponsors to document where every training and validation record originated. The agency's guidance on clinical performance assessment for SaMD requires identification of the data source, collection protocol, and chain of custody. A sponsor using retrospective EHR data must prove the records were extracted without selection bias. A sponsor using prospective clinical data must show IRB approval, enrollment criteria, and consent documentation.
Provenance failures appear frequently in FDA deficiency letters. The agency's 2023 transparency report on device review noted that incomplete data source documentation was among the top reasons for additional information requests.
Representativeness
The FDA's 2024 final guidance on predetermined change control plans for AI/ML devices reinforced a requirement that has existed informally for years: training and validation data must reflect the intended use population. That means demographic balance across age, sex, race, ethnicity, and disease severity.
A 2022 study in Nature Medicine found that among 130 FDA-authorized AI devices, only 6% of approvals reported race or ethnicity data in their clinical evidence. The FDA has since increased scrutiny. De Novo submissions filed after 2023 face direct questions about subgroup performance.
Reference standard quality
Every SaMD that produces a diagnostic or screening output needs a reference standard, the ground truth against which the algorithm's performance is measured. The FDA's guidance on clinical evidence for IVDs applies by analogy to SaMD: the reference standard must be clinically accepted, independently adjudicated, and documented.
Weak reference standards produce inflated sensitivity and specificity numbers. The FDA has rejected De Novo submissions where the reference standard was a single radiologist's read without blinding or adjudication.
Temporal validity
Data ages. A model trained on records from 2015 may not perform well on patients presenting in 2025 because disease prevalence, treatment protocols, and imaging equipment change. The FDA's predetermined change control plan guidance requires sponsors to describe how they will monitor for temporal drift.
For the De Novo submission itself, reviewers assess whether the validation dataset's time window is clinically relevant to the current intended use. Stale data is a deficiency.
Consent and regulatory compliance
Training data used in a De Novo submission must comply with informed consent requirements under 21 CFR Part 50 and IRB requirements under 21 CFR Part 56. If the data was collected under a waiver of consent, the sponsor must document the waiver and its justification. If the data originated outside the United States, the FDA applies ICH E6 Good Clinical Practice standards.
Consent documentation is not a checkbox. The FDA reviews whether consent covered the specific use of data in algorithm training, not just the original clinical encounter.
Bias documentation
The FDA's 2023 draft guidance on the lifecycle management of AI/ML devices requires sponsors to perform and document bias testing. This includes analysis of algorithm performance across demographic subgroups and identification of any systematic errors that correlate with protected characteristics.
A 2024 JAMA study found that among AI devices with FDA authorization, 79% lacked any publicly available data on performance disparities across racial groups. The De Novo pathway, because it demands a higher evidence burden than a 510(k), is where the FDA is most likely to push back on missing bias documentation.
Where most SaMD sponsors fail on data trust
The pattern in De Novo deficiency letters is consistent. Sponsors invest in algorithm architecture and clinical trial design but treat data documentation as an afterthought.
Three failure modes recur.
First, sponsors aggregate data from multiple sites without documenting site-level collection protocols. The FDA wants to know whether Site A used a different imaging protocol than Site B, and whether the algorithm's performance differs by site.
Second, sponsors use publicly available datasets like MIMIC or CheXpert for training but cannot document the chain of custody from the original data source to the training pipeline. Public availability does not equal provenance documentation.
Third, sponsors validate on convenience samples rather than prospectively defined cohorts. A validation dataset drawn from the same institution that provided the training data introduces spectrum bias, and the FDA knows to look for it.
The pre-submission meeting is where data trust gets tested
The FDA encourages sponsors to request a pre-submission (Q-Sub) meeting before filing a De Novo request. For SaMD, this meeting is where the data trust conversation happens.
Reviewers will ask about dataset size, demographic composition, reference standard methodology, and temporal relevance. Sponsors who arrive at the Q-Sub without answers to these questions receive feedback that delays their submission by months.
The FDA's 2024 CDRH workload data shows a median De Novo review time of approximately 330 days. Deficiency letters extend that timeline significantly. Every data trust gap identified during review adds 60 to 120 days.
What predetermined change control plans require from data infrastructure
The FDA's final guidance on predetermined change control plans, issued in 2024, allows SaMD sponsors to define a set of planned algorithm modifications that do not require new premarket review. But the trade-off is steep: sponsors must describe exactly how they will collect, validate, and document the data used for each modification.
This means the data trust infrastructure must be continuous, not one-time. A sponsor cannot build a dataset for the De Novo submission and then discard the data governance framework. The predetermined change control plan commits the sponsor to ongoing provenance documentation, bias monitoring, and validation dataset management.
For sponsors planning to use real-world data for post-authorization algorithm updates, the standard is even higher. The FDA's framework for RWE in regulatory decisions requires that real-world data meet the same relevance and reliability standards as clinical trial data.
How data trust scoring maps to De Novo requirements
| Value (%) | |
|---|---|
| Provenance | 25 |
| Consent | 20 |
| Recency | 15 |
| Quality | 10 |
| Concordance | 10 |
| Validation | 10 |
| Breadth | 5 |
| Stability | 5 |
The eight dimensions of the DTI framework, provenance, consent, recency, quality, concordance, validation, breadth, and stability, map directly to what FDA reviewers evaluate in a De Novo submission.
Provenance (25% of the DTI weight) corresponds to the FDA's data source documentation requirements. Consent (20%) maps to 21 CFR Parts 50 and 56 compliance. Recency (15%) addresses temporal validity. Quality (10%) covers reference standard adequacy and data completeness. Concordance (10%) captures cross-site consistency. Validation (10%) measures whether the data has been independently verified. Breadth (5%) relates to demographic representativeness. Stability (5%) tracks whether the data's characteristics hold over time.
A record-level trust score applied before training data enters a model creates an auditable trail. When the FDA asks how the sponsor ensured dataset quality, the answer is a score on every record rather than a paragraph of assurances.
Post-market surveillance starts with data trust
De Novo authorization is not the end. The FDA requires post-market surveillance for SaMD, including adverse event reporting and, increasingly, real-world performance monitoring. The quality of post-market data determines whether the device maintains its authorization.
A 2023 analysis by the International Medical Device Regulators Forum found that post-market data quality was the most cited concern among global regulators reviewing SaMD performance. Devices that passed premarket review on strong clinical trial data but produced unreliable real-world performance data faced regulatory action.
The data trust infrastructure built for the De Novo submission must extend into production. Every record used to monitor post-market performance needs the same provenance, consent, and quality documentation that the original training data carried.
The cost of getting data trust wrong
| Value ($) | |
|---|---|
| FY2025 De Novo fee | 131904 |
| Estimated cost per deficiency delay (60-120 days) | 250000 |
A failed De Novo submission costs more than review fees. The FDA charges a $131,904 standard De Novo fee for FY2025. But the real cost is delay. Each additional information request adds months. Competitors who file cleaner submissions reach market first.
For SaMD companies backed by venture capital, a 6-month delay can mean the difference between closing a Series B and running out of runway. For large medtech companies, delay means lost market share in categories where first-mover advantage is decisive.
The sponsors who succeed on the first review cycle share a common trait: they treat data trust as a design requirement, not a documentation exercise.
The DTI Engine scores every record 0 to 100 across eight dimensions before your AI model sees it. If your team is building a SaMD for De Novo authorization and needs audit-ready data provenance, consent documentation, and bias assessment infrastructure, talk to the SuperTruth commercial team. Schedule a conversation or call (215) 918-4140.
Further reading:
Sources
Jason Alan Snyder
Co-founder of SuperTruth and Artists & Robots, and an inventor on the Data Trust Index patents. Twenty-plus years building technology inside Interpublic Group. He writes here nearly every day on data trust, provenance, and what AI should be allowed to act on, and publishes essays on his Substack.
About SuperTruth · LinkedIn · Substack · jasonalansnyder.com
See it in practice
DTI scores the record, not the patient.
8 dimensions. 0 to 100. Travels with every record permanently.