That’s roughly how many chatbot-generated answers to real cancer treatment questions turned out to be wrong when oncologists actually checked.
The finding comes from a 2023 study published in JAMA Oncology, led by Michael Foote, MD, a gastrointestinal medical oncologist at Memorial Sloan Kettering Cancer Center (MSK). A follow-up study published in the American Journal of Clinical Oncology in 2024, conducted by James Janopaul-Naylor, MD, an MSK radiation oncologist, found that many chatbot responses were accurate, but others were flawed in ways that are easy to miss. “One of the challenges is that patients may latch on to the misleading parts, especially if they confirm what they already believe,” Dr. Janopaul-Naylor says.
But everyone is using AI these days, which can make understanding anything, including medical jargon, much easier. “It’s very common for patients to use chatbots to educate themselves about their medical conditions,” Dr. Foote says. So how do you know when to trust what a chatbot tells you? The good news is that accuracy has come a long way in a short time, which makes sense, given that AI operates at warp speed. In the three years since that first study, chatbot technology has improved significantly, and both doctors say patients can and should use these tools, as long as they know the pitfalls.
We asked Dr. Foote and Dr. Janopaul-Naylor to break down exactly where these tools still get it wrong and how to properly use the technology to your benefit.
What AI chatbots can get wrong about cancer
Small input mistakes can throw off an entire answer. “User error can affect the results you get,” Dr. Janopaul-Naylor says. “I had one patient who was concerned about his prognosis for prostate cancer, but it turned out that when he entered his test results into a chatbot, he had typed his PSA number wrong.” A PSA number, short for prostate-specific antigen, is a key blood test result doctors use to monitor prostate cancer, so even a mistyped digit there can completely change the picture a chatbot paints.
Chatbots also don’t know anything about you beyond what you tell them. A 35-year-old and an 80-year-old with the same diagnosis may need very different treatment approaches, and a chatbot working from limited context will hand you a limited, less relevant answer, delivered with the same confidence either way.
There’s also a hidden expiration date on what a chatbot knows. These tools have a training cutoff, after which they stop learning new information. “If a new treatment has recently been approved, it could be missing from the results you get,” Dr. Foote explains, which matters enormously in a field like oncology, where treatment guidelines often shift.
Then there’s the bias problem. Chatbots tend to shape their answers around what you say you’re hoping to hear. Mention that you’d like to avoid surgery, or you want herbs and supplements to be a part of your plan, and the computer is more likely to agree with you than challenge you, which is exactly the opposite of what you need when the stakes are this high.
And a chatbot’s confidence is not the same thing as accuracy. These tools tend to deliver findings with a lot of certainty, even when the real answer is far more nuanced, which matters a lot when the topic is especially sensitive and dependent on an individual case, like survival rates.
How to stress-test an AI chatbot’s cancer answer
Once you know where the traps are, both doctors agree there are things you can do to avoid falling into them.
Start by asking the AI to show its work. “When a chatbot tells you something, you can ask it for references,” Dr. Foote says. “You should then go check those references and make sure they are reliable or if they are even real.” Chatbots scan the entire internet to build their answers, which means the information could come from a peer-reviewed medical journal, or from a website hawking supplements. Asking for sources, and actually clicking through to check them, is the fastest way to confirm accurate information.
It’s well documented that these tools sometimes make up answers, especially when they’re unsure. “If something doesn’t seem right, there’s a good chance it’s not,” Dr. Janopaul-Naylor says. “One good way to check this is to ask the same question in different ways or to ask more than one chatbot. If you get different responses, that information is less likely to be accurate.”
And don’t overlook privacy concerns. Chatbot companies often use your conversations to train their models and may share that data with third parties, including insurers. If you’re sharing medical details or uploading documents, strip out anything that could identify you, including your full name, birthdate, and medical record number.
Where chatbots are worth using when it comes to cancer-related questions
None of this means you should avoid using these new tools altogether. “Think of a chatbot as a research assistant rather than a decision-maker,” Dr. Janopaul-Naylor says. “Any medical decisions you make should always involve your care team.”
That research-assistant role is especially helpful before an appointment. Walking into an oncology visit can be overwhelming, and it’s easy to blank on what you meant to ask. A chatbot can help you build that list ahead of time. “A lot of times the chatbot may suggest things you didn’t think of,” Dr. Foote says. “Patients who consult AI before they come to see me often have a better understanding of their disease and ask more in-depth questions.”
It’s also useful for cutting through medical jargon. Early in a cancer journey, everything can feel like it’s written in a foreign language, and a chatbot can translate dense medical terminology into something that makes sense to you.
Whatever you find, bring it back to your medical team, especially anything that worries you or doesn’t sit right. “I appreciate when my patients or their family members come to an appointment with printouts or screenshots they’ve gotten from a chatbot,” Dr. Janopaul-Naylor says. “It helps them to ask more thoughtful questions and allows me to provide more detailed answers.” Used smartly, AI stops being a source of anxiety and becomes exactly what it should be: a tool that makes you a more informed patient, not a self-diagnosed one.
To stay up to date on the latest news about cancer, subscribe to MSK Now.
