Health technology assessment data quality: what ICER models require
ICER cost-effectiveness models depend on clinical, economic, and quality-of-life data that most organizations cannot verify for accuracy, completeness, or provenance. Without structured data trust requirements, health technology assessments inherit every upstream data quality failure. The gap between what ICER models require and what manufacturers actually submit is where billion-dollar coverage decisions go wrong.
ICER's cost-effectiveness models influence billions of dollars in drug pricing and coverage decisions annually. The Institute for Clinical and Economic Review published 23 evidence reports in 2023 alone, each one shaping how payers, manufacturers, and health systems think about the value of new therapies. Yet the data feeding these models rarely undergoes formal trust verification before it enters the analytic pipeline.
This is not a critique of ICER's methodology. It is a structural observation about the data supply chain underneath health technology assessment. When input data lacks verified provenance, validated recency, and measurable completeness, the resulting cost-per-QALY estimates carry uncertainty that no sensitivity analysis can fully capture.
What is an ICER in health economics?
The incremental cost-effectiveness ratio is a calculation that compares two interventions by dividing the difference in their costs by the difference in their health outcomes. The formula is straightforward: ICER = (Costnew - Costcomparator) / (Effectnew - Effectcomparator). The result is typically expressed as cost per quality-adjusted life year (QALY) gained.
ICER (the organization, the Institute for Clinical and Economic Review) uses this ratio as a centerpiece of its value assessments. When ICER publishes a report stating that a therapy costs $150,000 per QALY gained, that number directly affects whether insurers cover the drug and at what price manufacturers negotiate.
The critical point most analyses miss: both the numerator and denominator of this ratio depend entirely on the quality of underlying data. A 10% error in cost data or a systematic bias in outcomes measurement can shift the ICER by tens of thousands of dollars per QALY. At the $100,000 to $150,000 per QALY willingness-to-pay threshold that ICER commonly references, those shifts determine whether a therapy is deemed cost-effective or not.
What is healthcare data technology?
Healthcare data technology encompasses the systems, standards, and infrastructure that collect, store, transmit, and analyze health information. This includes electronic health records, claims processing systems, clinical trial databases, patient registries, wearable device platforms, and the interoperability standards (HL7, FHIR, ICD-10, SNOMED CT) that connect them.
For HTA purposes, healthcare data technology serves as the pipeline that converts raw clinical observations into the structured inputs that economic models consume. The problem is that most of this technology was built for billing, documentation, or care delivery. None of it was built with cost-effectiveness modeling as a primary use case.
When an ICER model pulls hospitalization rates from claims data, those rates carry the coding inconsistencies, lag times, and coverage gaps inherent to the claims system. When it uses quality-of-life scores from clinical trials, those scores reflect the narrow demographics and controlled conditions of trial enrollment. The technology that produces the data shapes the data's limitations.
The five data categories ICER models require
Every ICER evidence report draws from at least five distinct data categories, each with its own quality challenges.
Clinical efficacy data. Sourced primarily from randomized controlled trials, this includes response rates, progression-free survival, overall survival, and adverse event rates. The quality challenge here is generalizability. Trial populations skew younger, whiter, and healthier than real-world patient populations. ICER's own reports have acknowledged this gap repeatedly.
Natural history data. Models need to understand what happens to patients without the new intervention. This requires long-term observational data from registries, cohort studies, and claims databases. Recency is a major concern; natural history data from a 2015 registry may not reflect 2024 treatment patterns.
Cost data. Direct medical costs (hospitalizations, procedures, drug administration), indirect costs (lost productivity, caregiver burden), and future healthcare costs all feed the economic model. Cost data typically comes from claims databases, Medicare fee schedules, and published literature. The lag between service delivery and claims adjudication (often 30 to 90 days) introduces systematic recency problems.
Health utility data. Quality-of-life weights (utilities) assigned to different health states are essential for QALY calculations. These come from patient surveys using instruments like EQ-5D or SF-6D. The trust challenge is that utility values vary significantly by instrument, population, and collection method. A utility value of 0.72 for moderate disease from one study and 0.65 from another creates meaningful ICER differences.
Epidemiological data. Prevalence, incidence, and demographic distribution data determine the budget impact portion of ICER assessments. These data often come from national surveys, cancer registries, or administrative databases with their own completeness and coverage issues.
Why is data quality important in healthcare?
Data quality in healthcare is not an abstract concern. It is the difference between a therapy being covered or denied, a price being negotiated at $80,000 or $120,000 per year, and a patient population gaining or losing access to treatment.
In the context of HTA, data quality failures compound through the model. A study published in Value in Health found that varying input parameters by their reported confidence intervals changed ICER results by 40% to 200% in oncology models. That is not rounding error. That is the difference between a drug being deemed cost-effective and being labeled low-value.
Three specific consequences of poor data quality in HTA:
How is quality of healthcare data defined in terms of accuracy, completeness, or relevance?
Healthcare data quality is conventionally defined across several dimensions. Accuracy measures whether data values correctly represent the real-world facts they describe. Completeness assesses whether all required data elements are present. Relevance evaluates whether the data actually pertains to the question being asked.
But these three dimensions are insufficient for HTA purposes. ICER models also need:
Provenance. Where did this data originate? Was it collected in a clinical trial, extracted from an EHR, or derived from a claims database? The source determines the appropriate level of confidence.
Recency. When was this data collected? A five-year-old cost study may not reflect current pricing. A clinical trial completed before a new standard of care emerged may overstate the comparator's effectiveness.
Concordance. Do multiple independent data sources agree? If three registries report different prevalence rates for the same condition, the model needs to account for that disagreement rather than arbitrarily selecting one source.
Consent and governance. Was the data collected under consent frameworks that allow its use in economic modeling? Patient-reported outcomes used without proper consent governance face regulatory and ethical challenges.
Validation. Has the data been independently verified? Self-reported manufacturer data that has not been validated against external sources carries different weight than registry data audited by an independent body.
The SuperTruth Data Trust Index scores health data records across eight dimensions that map directly to these HTA requirements: Provenance (25%), Consent (20%), Recency (15%), Quality (10%), Concordance (10%), Validation (10%), Breadth (5%), and Stability (5%). Each dimension addresses a specific failure mode that ICER models are vulnerable to.
Key statistics
ICER published 23 evidence reports in 2023, each influencing coverage and pricing decisions worth hundreds of millions of dollars.
Varying input parameters within their reported confidence intervals changes oncology ICER results by 40% to 200%, according to research published in Value in Health.
Claims data used in cost-effectiveness models carries a 30 to 90 day reporting lag, meaning cost inputs systematically underrepresent recent pricing changes.
SuperTruth's DTI Engine reduced data standardization time for imaware's 105,000 diagnostic records from 3 weeks to 2 hours, a 95% reduction, demonstrating that trust scoring at scale is operationally feasible.
Health utility values measured by EQ-5D versus SF-6D for the same disease state can differ by 0.05 to 0.15 points, enough to shift a cost-per-QALY estimate by $20,000 to $50,000.
Where ICER models break: the data trust gap
ICER's modeling process includes multiple rounds of stakeholder feedback, draft reports, public comment periods, and revised analyses. This iterative process is well-designed for methodological transparency. But it does not include a formal data trust verification step.
Consider what happens when a manufacturer submits clinical trial data to support a new oncology therapy. ICER's modelers review the trial design, assess the statistical methods, and evaluate the clinical endpoints. What they do not systematically assess is whether the underlying patient-level data meets provenance, recency, and concordance standards.
The manufacturer may have pooled data from trials conducted across different countries with different standard-of-care comparators. The cost data may come from a U.S. claims database that excludes uninsured patients, systematically underestimating the true cost of the disease in vulnerable populations. The utility data may have been collected using a different instrument than what ICER's reference case recommends.
Each of these issues introduces bias that propagates through the model. And because ICER typically presents results as point estimates with probabilistic sensitivity analyses, the structural data quality issues get absorbed into confidence intervals rather than being flagged as distinct trust failures.
Health equity and the data representation problem
ICER has published a white paper on advancing HTA methods that support health equity. This is a meaningful step. But health equity in HTA is fundamentally a data quality problem.
If clinical trials enroll 85% white patients and the therapy will be used by a population that is 40% non-white, the efficacy estimates carry a representativeness gap. If cost data comes from commercially insured populations and the budget impact analysis applies to Medicaid beneficiaries, the cost estimates carry a coverage gap.
These are not methodological limitations. They are data trust failures. The data does not represent the population it claims to represent, and no statistical adjustment can fully compensate for missing populations.
SuperTruth's approach to this problem is to score data for Breadth, one of the eight DTI dimensions. Breadth measures the demographic and clinical representativeness of a data set. A clinical trial data set that covers only three racial categories and excludes patients over 75 would receive a lower Breadth score than a registry data set that captures six racial categories and includes geriatric patients.
When HTA modelers can see a Breadth score alongside efficacy data, they can make informed decisions about which populations the model's conclusions actually apply to.
Real-world evidence and the trust threshold
ICER increasingly incorporates real-world evidence (RWE) into its models, particularly for long-term outcomes that clinical trials cannot capture. This is the right direction. But RWE carries its own data quality challenges that are distinct from trial data.
RWE from EHR systems inherits every coding error, documentation gap, and interoperability failure in the source system. ICD-10 coding inaccuracies, which affect up to 20% of diagnosis codes in some studies, directly corrupt disease prevalence and comorbidity inputs. Claims data lag means that RWE cost estimates are systematically stale.
The FDA has published guidance on RWE for regulatory submissions, and those requirements increasingly emphasize data provenance and quality documentation. ICER's models should adopt similar standards. A DTI score applied to RWE sources before they enter an HTA model would provide modelers with a quantified measure of how much to trust each input.
For a deeper look at FDA requirements for real-world evidence, see Real-world evidence data quality: FDA requirements for RWE regulatory submission.
What a trust-scored HTA pipeline looks like
Imagine an ICER model built on data that has been scored before it enters the analytic pipeline. Each input source carries a DTI score that tells the modeler:
With these scores visible, the modeler can set minimum trust thresholds for each input category. Clinical efficacy data below a DTI of 70 triggers additional sensitivity analysis. Cost data below a DTI of 75 requires supplementary sources. Utility data below a DTI of 60 gets flagged for potential equity impact.
This is not a hypothetical framework. The DTI Engine already scores health data records across these dimensions. The application to HTA is a natural extension of what the engine already does for clinical AI, regulatory submission, and payer analytics.
The manufacturer submission problem
Manufacturers submitting data to ICER face a structural incentive to present their therapy in the most favorable light. This is not fraud. It is rational behavior within a system that does not require independent data trust verification.
A manufacturer might choose to submit utility data from the instrument that yields higher values, cost data from the database that shows lower treatment costs, or efficacy data from the trial subgroup that shows the greatest benefit. Each of these choices is defensible individually but collectively creates a systematic bias toward favorable ICER results.
Independent data trust scoring would not eliminate manufacturer discretion. But it would make the choices visible. If a manufacturer submits utility data with a Concordance score of 45 when alternative data with a score of 78 exists, that gap becomes part of the public record.
What this means for payers and health systems
Payers who use ICER reports to inform formulary decisions are making billion-dollar bets on data they have not independently verified. Health systems that adopt ICER-endorsed pricing benchmarks are building budgets on assumptions they cannot trace back to source.
The fix is not to abandon ICER's framework. It is to add a trust layer underneath it. Every data source that feeds an HTA model should carry a verifiable trust score that quantifies its provenance, recency, accuracy, completeness, concordance, and representativeness.
This is the same principle that applies to clinical AI, regulatory submission, and population health analytics. The model is only as trustworthy as its least trusted input. And right now, most HTA inputs have never been scored at all.
SuperTruth's trust layer turns RWE from a compliance liability into a competitive asset. If your team needs audit-ready provenance for FDA submission, payer negotiation, or HTA data preparation, contact Louis Simeonidis at louis@supertruth.ai or (215) 918-4140.
Further reading:
Jason Alan Snyder
Co-founder of SuperTruth and Artists & Robots, and an inventor on the Data Trust Index patents. Twenty-plus years building technology inside Interpublic Group. He writes here nearly every day on data trust, provenance, and what AI should be allowed to act on, and publishes essays on his Substack.
About SuperTruth · LinkedIn · Substack · jasonalansnyder.com
See it in practice
DTI scores the record, not the patient.
8 dimensions. 0–100. Travels with every record permanently.