Race and ethnicity data quality: what self-reported vs inferred data means for AI
Up to 30% of race and ethnicity fields in EHR data are either missing or inferred from surnames and geocoding rather than self-reported by patients. When AI models train on this data without distinguishing source method, they replicate and amplify the very disparities health equity programs aim to fix. Understanding the difference between self-reported and inferred demographic data is a prerequisite for any clinical AI deployment.
Between 20% and 40% of race and ethnicity data in U.S. electronic health records is missing entirely. When it does exist, much of it was never reported by the patient. It was inferred by an algorithm, guessed by a registrar, or pulled from a surname database. That distinction matters enormously for any AI system trained on this data.
Why the source of race and ethnicity data determines AI outcomes
Race and ethnicity data enters health systems through two fundamentally different paths. The first is self-report: a patient selects their own identity from a set of categories, typically during intake or registration. The second is inference: an algorithm, a billing system, or an administrative staff member assigns a category based on indirect signals like surname, zip code, language preference, or insurance type.
These two paths produce data that looks identical in a structured field. A race column in a database table does not carry metadata explaining whether the value came from the patient or from a Bayesian Improved Surname Geocoding (BISG) algorithm. But the accuracy gap between them is massive.
Studies comparing self-reported race to BISG-imputed race show disagreement rates of 10% to 30% depending on the population. For multiracial individuals, the error rate climbs even higher. For Indigenous and Pacific Islander populations, imputation methods often fail entirely because the training data for those algorithms underrepresents them.
When an AI model trains on a dataset where 25% of race values are inferred and 75% are self-reported, but the model treats all values as equivalent, every downstream output carries embedded measurement error. Risk scores skew. Disparity analyses report false gaps or miss real ones. Clinical decision support tools calibrated on this mixed data produce recommendations that reflect the biases of surname databases rather than the biology or social context of actual patients.
What is the difference between racial and ethnic data?
Race and ethnicity are related but distinct concepts, and most health data systems conflate them.
Race, as used by the U.S. Census Bureau and OMB Directive 15, refers to categories based on physical characteristics and geographic ancestry: White, Black or African American, American Indian or Alaska Native, Asian, and Native Hawaiian or Other Pacific Islander. Ethnicity is a separate question, typically binary: Hispanic or Latino, or Not Hispanic or Latino.
The OMB two-question format asks ethnicity first, then race. Many EHR systems collapse these into a single field or allow only one selection. This creates data quality problems before any AI model enters the picture.
When a patient identifies as Black and Hispanic, a single-field system forces a choice or defaults to one value. That lost granularity propagates through every analysis that uses the data. Health equity research that cannot distinguish between Hispanic Black patients and non-Hispanic Black patients misses well-documented differences in disease prevalence, treatment response, and social determinants of health.
What is the purpose of collecting race and ethnicity data?
Race and ethnicity data serves three primary functions in healthcare: clinical care, population health management, and regulatory compliance.
Clinically, certain conditions have prevalence differences across racial and ethnic groups. Sickle cell disease disproportionately affects Black patients. Tay-Sachs disease has higher prevalence in Ashkenazi Jewish populations. Pharmacogenomic responses vary across ancestry groups. Ignoring these patterns in clinical AI leads to worse outcomes.
For population health, race and ethnicity data is required to identify and address disparities. CMS quality measures, HEDIS metrics, and state health department reporting all require stratification by race and ethnicity. Without accurate data, health systems cannot measure whether their interventions are reaching the populations that need them most.
Regulatory requirements are tightening. The CMS Health Equity Index, introduced in 2022, requires hospitals to report quality measures stratified by race and ethnicity. Section 1557 of the ACA mandates collection of demographic data. The 2024 CMS Interoperability and Prior Authorization final rule requires payers to collect and maintain standardized demographic data.
But collection without quality assurance creates a false sense of compliance. Reporting race data that is 30% inferred satisfies the checkbox. It does not satisfy the purpose.
Is AI a race or ethnicity?
No. AI is a technology, not a demographic category. But this question, which appears frequently in public search queries, points to a real concern: whether AI systems themselves carry racial or ethnic bias.
The answer is that AI systems reflect the data they train on. An AI model has no race. But if its training data contains race fields that were disproportionately inferred for Black and Hispanic patients (because those populations have higher rates of missing self-reported data in many health systems), then the model's outputs will systematically differ for those groups in ways that have nothing to do with biology or clinical need.
The Epic sepsis prediction model controversy illustrated this. The model performed differently across racial groups in part because the underlying data, including demographic variables and utilization patterns, reflected systemic differences in how care was documented and delivered across populations. The model did not "have" a race. But it produced racially disparate results because the data carried racial bias.
Key statistics
What should you report for race and ethnicity in the methods section of research?
Research methods sections should specify the source of race and ethnicity data, the classification system used, and the completeness rate.
The NIH and JAMA have both published guidance requiring researchers to state whether race and ethnicity data was self-reported or assigned by observers, extracted from administrative records, or imputed using algorithms. Simply stating "race was obtained from the EHR" is insufficient because it obscures the measurement method.
A rigorous methods section should include:
Studies that use imputed race data without disclosing the imputation method produce results that other researchers cannot replicate or validate. This is a data provenance failure, and it undermines the entire evidence chain from raw data to clinical application.
As we have written about in the context of real-world evidence data quality, the FDA increasingly requires documentation of data lineage for any dataset used to support regulatory decisions. Race and ethnicity data is no exception.
Self-reported data is better but not perfect
Self-reported race and ethnicity data is the gold standard for a reason: it reflects the patient's own identity. But it carries its own quality problems.
Patients may select different categories at different visits. A study in the Journal of General Internal Medicine found that 6% of patients changed their self-reported race between consecutive visits in the same health system. This is not necessarily error; identity is contextual and can shift based on how questions are framed, what options are available, and who is asking.
Intake processes introduce additional noise. As we have explored in health literacy and data quality, patients with lower health literacy may not understand the distinctions between categories, may feel pressured to select a single option, or may skip the question entirely. Non-English-speaking patients may encounter forms only available in English, leading to higher rates of "unknown" or "declined" responses.
Registration staff sometimes fill in race fields based on visual assessment rather than asking the patient. This observer-assigned data gets stored in the same field as self-reported data with no distinguishing metadata. One study at a large academic medical center found that 15% of race values were entered by staff without patient input.
Imputed data has a compounding error problem
BISG and similar imputation methods were designed for population-level analyses where individual-level accuracy matters less than aggregate distributions. Using them for individual-level AI model training is a category error.
The compounding problem works like this: BISG assigns a probability distribution of race based on surname and census tract. The algorithm might assign a 72% probability of "Asian" and a 28% probability of "White" for a given patient. But the EHR field requires a single value. So the system selects the highest probability category, and the uncertainty disappears.
Now an AI model training on this data treats that imputed "Asian" value with the same confidence as a self-reported "Asian" value from another patient. The model cannot distinguish between them. When the model makes predictions for this patient, it uses clinical patterns associated with Asian patients, even though there was a 28% chance the patient identifies as White.
Multiply this across thousands of records and the aggregate bias becomes substantial. Populations that are hardest to impute accurately (multiracial individuals, immigrants from countries with naming conventions unfamiliar to U.S. surname databases, adopted individuals with surnames that do not match their ancestry) receive the worst predictions.
How trust scoring addresses the race and ethnicity data quality gap
The fundamental problem is not that inferred data exists. For some applications, imputed race data is the only available option, and it can be useful at the population level. The problem is that health AI systems cannot currently distinguish between self-reported and inferred values when they consume demographic data.
This is a provenance problem. The DTI framework scores every data element across eight dimensions, and Provenance carries the highest weight at 25%. For race and ethnicity fields specifically, provenance scoring answers: Where did this value come from? Was it entered by the patient, assigned by staff, imputed by an algorithm, or inherited from a prior record?
A race value with confirmed self-report provenance and a timestamp from the current visit scores differently than a value imputed by BISG three years ago. An AI developer who sets a DTI floor of 70 for demographic fields would automatically exclude the low-provenance inferred values from model training, without needing to build custom data cleaning pipelines.
This approach also addresses the concordance dimension: does the race value in the EHR match the value in claims data, in the patient portal, and in research consent forms? Discordance across sources is a strong signal of either imputation or data entry error, and it should reduce the trust score accordingly.
The regulatory trajectory is clear
Federal agencies are moving toward requiring demographic data quality, not just demographic data collection.
The CMS Health Equity Index penalizes hospitals with incomplete race and ethnicity data. The ONC's United States Core Data for Interoperability (USCDI) v3 includes race and ethnicity as required data elements with specific value sets. The FDA's 2023 guidance on diversity in clinical trials requires sponsors to report enrollment demographics and explain gaps.
State-level mandates are even more specific. California's AB 1204 requires health plans to collect self-reported race, ethnicity, and language data using standardized categories that go beyond the OMB minimum. Illinois, Massachusetts, and New York have similar requirements.
For AI developers, the implication is straightforward: race and ethnicity data quality will be audited. If your model was trained on a dataset where 30% of race values were inferred by BISG and you cannot demonstrate that you accounted for this in your validation, regulators will have questions. The FDA's evolving AI guidance increasingly focuses on training data provenance, and demographic variables are among the most scrutinized.
What health systems should do now
Three actions reduce demographic data quality risk before it reaches an AI model.
First, add source metadata to race and ethnicity fields. Every value should carry a tag indicating whether it was self-reported, observer-assigned, or imputed. This requires EHR configuration changes but is not technically complex. HL7 FHIR R4 already supports extension elements for data provenance.
Second, re-collect self-reported data at regular intervals. A patient's self-reported race from 2015 may not reflect their current identity, and it certainly does not carry the same trust level as a value confirmed in 2024. Recency is an underrated dimension in health data trust, and demographic data is no exception.
Third, score demographic data before it enters any AI pipeline. The DTI Engine applies provenance, recency, concordance, and completeness scoring to every field, including race and ethnicity. AI developers can then set minimum trust thresholds for demographic variables, ensuring that models train only on data that meets a defined quality standard.
The alternative is to build AI that perpetuates the measurement biases of 1990s surname databases. That is not a health equity strategy. It is a liability.
The DTI Engine scores every health data record 0 to 100 across 8 trust dimensions before your AI model sees it. For demographic data quality, provenance scoring distinguishes self-reported values from inferred ones, giving AI developers the ability to set trust floors that prevent bias from entering the training pipeline. If your team is evaluating data for training, compliance, or clinical use, contact Louis Simeonidis at louis@supertruth.ai or (215) 918-4140.
Further reading:
Jason Alan Snyder
Co-founder of SuperTruth and Artists & Robots, and an inventor on the Data Trust Index patents. Twenty-plus years building technology inside Interpublic Group. He writes here nearly every day on data trust, provenance, and what AI should be allowed to act on, and publishes essays on his Substack.
About SuperTruth · LinkedIn · Substack · jasonalansnyder.com
See it in practice
DTI scores the record, not the patient.
8 dimensions. 0 to 100. Travels with every record permanently.