Why health system CTOs are wrong about what makes AI trustworthy
Most health system CTOs define AI trustworthiness by model accuracy, HIPAA compliance, and vendor certifications. None of those address the actual failure point: unscored, unverified training data flowing into production AI without provenance, consent validation, or recency checks. The trust problem is not in the model. It is in the data the model never should have seen.
Most health system CTOs will tell you their AI is trustworthy because it passed a validation study, runs inside a HIPAA-compliant environment, and came from a reputable vendor. They are wrong. Not because those things are irrelevant, but because they address the wrong layer of the problem.
The question is not whether the model works. The question is whether the data underneath it was ever verified before the model trained on it.
The CTO trust checklist that misses the point
Ask a health system CTO what makes their AI trustworthy and you will hear a predictable list. Model accuracy above 90%. SOC 2 certification. HIPAA BAA in place. FDA clearance for the specific use case. Maybe an explainability layer that shows feature importance weights.
Every item on that list describes the model or the infrastructure around it. None of them describe the data. And the data is where healthcare AI actually breaks.
A model can be 95% accurate on a validation set and still produce dangerous outputs in production because the training data contained records that were stale, duplicated, unconsented, or sourced from systems with broken provenance chains. AI explainability solves the wrong problem when the data underneath has never been scored.
Why do people not trust AI in healthcare?
Patient distrust of healthcare AI is well documented. Research shows 65.8% of people express low trust in their healthcare systems' use of AI. But the reasons most often cited, fear of errors, lack of transparency, job displacement, obscure a deeper structural issue.
People do not trust healthcare AI because nobody can tell them where the data came from. Patients cannot verify whether their records were used with proper consent. Clinicians cannot trace which data points trained the model recommending a treatment. CTOs cannot prove their training sets excluded poisoned, synthetic, or outdated records.
The trust deficit is not emotional. It is evidentiary. There is no chain of custody. There is no score. There is no standard. As we have written before, infrastructure trust and data trust are fundamentally different problems, and most platforms only solve the first one.
What can go wrong with AI in healthcare?
The failure modes are specific and documented. Sepsis prediction models at major health systems have generated false alert rates above 80%, burning out clinical staff who learn to ignore them. Dermatology AI trained predominantly on lighter skin tones has missed melanoma in darker-skinned patients. Readmission models have encoded socioeconomic bias because the training data reflected insurance status, not clinical acuity.
Every one of these failures traces back to a data problem, not a model problem. The training data lacked breadth. It lacked recency. It lacked concordance across sources. Nobody scored it before it entered the pipeline. The result is what happens when unscored health data reaches production: systematic, confident, invisible errors.
Can you trust AI with monitoring your health?
You can trust a health monitoring AI exactly as far as you can trust the data feeding it. A wearable device generating heart rate variability data is only clinically useful if that data has provenance (which device, which firmware version, which calibration), recency (was this from today or from a cached batch three weeks old), and consent (did the patient agree to clinical use, not just app functionality).
Most health monitoring AI fails on at least one of these dimensions. The wearable data trust problem is real: consumer-grade signals need clinical-grade verification before any model should act on them.
What actually makes an AI system trustworthy
Trustworthy AI requires trustworthy data. That means every record entering a training pipeline or inference engine needs to be scored across measurable dimensions before the model sees it.
The Data Trust Index scores health data 0 to 100 across eight dimensions: Provenance (25%), Consent (20%), Recency (15%), Quality (10%), Concordance (10%), Validation (10%), Breadth (5%), and Stability (5%). These weights reflect where healthcare data actually breaks. Provenance and consent alone account for 45% of the score because they represent the two dimensions most often missing entirely from CTO trust frameworks.
This is not a theoretical exercise. When SuperTruth standardized 105,000 diagnostic records for imaware, we reduced processing time from three weeks to two hours and identified a patient segment driving 20% of revenue that had been invisible in unscored data. The data was already there. It had never been trusted.
The governance gap CTOs cannot close with vendor contracts
A vendor contract guarantees the vendor's behavior. It does not guarantee the data's integrity. Health system CTOs sign BAAs that protect against unauthorized disclosure but say nothing about whether the disclosed data was accurate, current, or properly consented in the first place.
Audit trails are the foundation of health AI accountability, but most health systems do not have them at the data level. They have system access logs. They have model version histories. They do not have record-by-record provenance chains showing where each data point originated, how it was transformed, and whether consent covered the specific AI use case.
This is the gap the FDA is closing. Emerging FDA guidance on AI training data audits will require exactly the kind of data-level documentation that most health system CTO offices cannot produce today.
Key statistics
What CTOs should do differently
Stop evaluating AI trustworthiness at the model layer. Start evaluating it at the data layer. Before any model trains or infers, every input record should have a trust score that is auditable, reproducible, and dimension-specific.
The question is not "Is this model accurate?" The question is "Can I prove, record by record, that the data underneath this model meets a defined trust floor?" If the answer is no, the model is not trustworthy. It is just unaudited.
SuperTruth scores incoming EHR data at the point of ingestion, before it reaches a model. If your system is deploying clinical AI and needs to answer an auditor's questions, schedule a conversation with the SuperTruth commercial team or (215) 918-4140.
Further reading:
Jason Alan Snyder
Co-founder of SuperTruth and Artists & Robots, and an inventor on the Data Trust Index patents. Twenty-plus years building technology inside Interpublic Group. He writes here nearly every day on data trust, provenance, and what AI should be allowed to act on, and publishes essays on his Substack.
About SuperTruth · LinkedIn · Substack · jasonalansnyder.com
See it in practice
EHR data scored before any AI model sees it.
DTI integrates with Epic, Oracle Health, and all major EHR systems.