The Risks of AI-Generated Health Information: A Wake-Up Call for Public Safety
Have you ever wondered if your persistent fatigue is a sign of something more serious? Or whether that chest pain is a cause for concern? For many, Google has been the go-to source for quick medical insights. But with the rise of AI Overviews, a new era of health information access has emerged, and it's not without its controversies.
In May 2024, Sundar Pichai, Google's CEO, unveiled a bold plan to integrate AI into its search engine, a move that would revolutionize the way we access information. By July 2025, this technology had reached a global audience, with AI Overviews becoming a monthly feature for over 2 billion people across 200 countries and 40 languages.
But here's where it gets controversial: AI Overviews, while efficient, carry inherent risks. These summaries, generated by AI, provide quick snapshots of information, often missing crucial context and accuracy. And when it comes to health, these oversights can have serious consequences.
Within weeks of its launch, users reported AI Overviews containing untruths and inaccuracies. For instance, one Overview stated that Andrew Jackson, the seventh US president, graduated from college in 2005! Google's response? They acknowledged some errors but emphasized the scale of the web, suggesting that oddities and mistakes were inevitable.
And this is the part most people miss: when it comes to health queries, accuracy is non-negotiable. A Guardian investigation revealed that Google's AI Overviews were providing false and misleading health information, putting people at risk. In one alarming case, Google advised pancreatic cancer patients to avoid high-fat foods, which experts say is the exact opposite of what should be recommended and could increase the risk of death from the disease.
Another example involved liver function tests, where AI Overviews provided information that could lead people with serious liver disease to believe they were healthy. Experts warn that these summaries could lead to seriously ill patients ignoring follow-up appointments, thinking their test results were normal.
But here's the catch: Google initially downplayed these concerns, stating that its AI Overviews were reliable and linked to reputable sources. However, within days of the investigation, they removed some of the problematic AI Overviews for health queries.
Vanessa Hebditch, from the British Liver Trust, expressed concern, saying, "It's not tackling the bigger issue of AI Overviews for health." Sue Farrington, from the Patient Information Forum, added, "There are still too many examples of Google AI Overviews giving inaccurate health information."
A recent study has only added to these concerns. Researchers found that AI Overviews relied heavily on YouTube, a platform not designed for medical publishing. This raises questions about the accuracy and reliability of the information presented in AI Overviews.
"With AI Overviews, users are presented with a single, confident answer that exhibits medical authority," says Hannah van Kolfschooten, a researcher at the University of Basel. "This restructures health information online, creating a new form of unregulated medical authority."
Google maintains that AI Overviews are built to surface information backed up by top web results and include links to supporting web content. However, experts argue that these single blocks of text can cause confusion and prevent users from critically evaluating the information.
"Users are less likely to research further, depriving them of the opportunity to compare and assess information critically," says Nicole Gross, an associate professor at the National College of Ireland.
Even when AI Overviews provide accurate facts, they may not distinguish between strong and weak evidence, and they can miss important caveats. This can give the impression that some claims are better established than they actually are, and answers can change over time, even without scientific shifts.
"People are getting different answers depending on when they search, and that's not good enough," says Athena Lamnisos, CEO of the Eve Appeal cancer charity.
The biggest worry, according to Gross, is that bogus and dangerous medical information in AI Overviews can influence patients' daily practices and routines, potentially with life-threatening consequences.
So, what's the solution? How can we ensure that AI-generated health information is accurate and reliable? Join the discussion in the comments and share your thoughts on this critical issue!
