A review of 22 studies warns that algorithmic fairness cannot be measured through accuracy alone, as patients face additional barriers involving trust, language, and participation in healthcare.
Medical artificial intelligence systems may reproduce healthcare inequalities even when developers focus on improving technical accuracy, according to newly published research examining how algorithmic bias affects patients.
A November 9 scoping review in the Online Journal of Public Health Informatics analyzed 22 studies exploring the relationship between medical AI bias, patient empowerment, and digital inequality.
The researchers found that bias-related problems extended beyond inaccurate predictions to include difficulties understanding health information, participating in medical decisions, and accessing digital services.
The findings highlight a growing responsibility for healthcare technology developers: evaluating whether AI systems work fairly for the people who depend on them.
Researchers Identify Multiple Forms of Bias
The review, conducted by Lisa-Maria Norz, Elizaveta Beeck, and Elske Ammenwerth from Austria’s UMIT TIROL university, examined research published from 2015 onward.
Of the 22 included studies, 21 addressed social bias, 16 discussed algorithmic bias, 14 identified structural bias, and 12 examined design-related bias.
Meanwhile, 19 studies considered factors contributing to the digital divide, including usability problems and limited digital literacy.
These figures describe how frequently issues appeared in the reviewed literature, not the percentage of medical AI products proven to be biased.
The researchers found that patient involvement in care was examined more frequently than broader questions about whether individuals could meaningfully understand or challenge AI-supported decisions.
Clinical Recommendations Can Change With Patient Characteristics
The concern is supported by separate experimental evidence.
A 2025 Nature Medicine study evaluated nine large language models using more than 1.7 million generated outputs derived from 1,000 emergency department cases.
Researchers varied demographic information while keeping clinical details constant.
The study found that certain patient identities, including being Black, unhoused, or LGBTQIA+, influenced recommendations for urgent treatment, invasive procedures, or mental health evaluations.
In some scenarios, LGBTQIA+ subgroup labels were associated with mental health assessment recommendations approximately six to seven times more frequently than clinically indicated.
These were controlled model evaluations, not evidence that those recommendations caused actual patient harm.
Healthcare Data Can Carry Bias Before AI Training
Problems may also originate in clinical documentation.
A Rutgers University study examined 754 hospital discharge notes, finding stigmatizing language in 62% of notes involving gender-expansive patients, compared with 26% of comparison notes.
The findings illustrate how biased language can already exist within healthcare information systems.
Although that study did not establish that the notes were used to train commercial AI models, such documentation raises concerns about the quality and representativeness of potential training data.
The November review similarly emphasized that addressing technical bias alone may be insufficient when wider healthcare inequalities remain unresolved.
AI Developers Face a More Demanding Safety Standard
The World Health Organization’s guidance identifies biased training data, automation bias, and unequal access as important risks in healthcare AI.
WHO Director for Digital Health and Innovation Dr Alain Labrique has emphasized the need for coordinated governance as increasingly capable AI systems enter medical settings.
For developers, these findings support evaluating accuracy across demographic groups while also testing accessibility, language support, explanations, and opportunities for human review.
Medical technology companies must distinguish technical performance from actual patient outcomes.
The November review also found that relatively few studies empirically tested whether proposed bias-reduction strategies improved patient empowerment or digital equity.
The central challenge is no longer simply making medical AI more accurate. It is establishing whether that accuracy translates into healthcare experiences that are equally understandable, accessible, and trustworthy for different patients.
