THESE “Doctors” Get It Wrong 1 Out of 4 Times?! (SCARY)
Dear Reader,
It’s 3 a.m. and you have chest pain…
Instead of getting up and going to the ER or urgent care, you can just grab your phone and get on the internet to figure out what’s going on.
And who could blame you?
It’s fast. It’s free. And it always has an answer.
But there’s one problem…
That answer is WRONG 25% of the time.
And if you act on it, the consequences could be devastating.
The tech industry wants you to believe artificial intelligence is ready to revolutionize healthcare.
Some experts even claim AI could replace doctors for many routine questions.
But a new study suggests we’re nowhere near that reality.
Researchers asked four popular AI chatbots—including ChatGPT, Gemini, and Llama—to answer more than 200 real-world medical questions.
Then they handed those responses to a panel of board-certified physicians for grading.
What happened next should concern anyone using AI for medical advice.
Doctors found that roughly 1 out of every 4 answers were WRONG.
Think about that for a moment.
Would you board an airplane if there was a 1-in-4 chance the pilot was wrong?
Would you take a medication that failed 25% of the time?
Of course not.
Yet millions of people are trusting AI with their health every single day.
Even the best-performing model, ChatGPT-4o, got it wrong about 15% of the time.
And some systems performed far worse.
One model delivered valid answers only 50% of the time.
That’s basically a coin flip.
But here’s what really alarmed physicians…
Mental health questions.
Several doctors warned that AI responses during mental health crises could be actively dangerous.
Imagine someone struggling with severe depression… experiencing suicidal thoughts… or facing a psychiatric emergency…
And receiving bad advice from a chatbot programmed to sound confident even when it’s wrong.
That’s a recipe for disaster.
Yet the mainstream keeps pushing AI as the next great healthcare revolution.
The researchers even tested whether giving AI access to medical textbooks and clinical guidelines improved performance.
Surprisingly, it often didn’t.
In some cases, doctors actually preferred the standard version over the medically enhanced version.
In other words, even when fed expert information, AI still struggled to consistently deliver safe medical guidance.
Now, does that mean AI is useless?
Not at all.
It can be a helpful starting point. It can explain medical terms. It can help you organize your thoughts before a doctor’s appointment.
But it should never replace professional medical care.
Because when it comes to your health…
“Mostly right” isn’t good enough.
And a technology that gets medical questions wrong 1 out of every 4 times isn’t ready to be your doctor.
View Sources
Mingole, B., et al. (2025). Dr. GPT will see you now, but should it? Exploring the benefits and harms of large language models in medical diagnosis using crowdsourced clinical cases. arXiv. https://arxiv.org/abs/2506.13805
