Google AI accused of giving incorrect medical advice that could be fatal

The investigation, published on Jan. 2, found that inaccurate and confusing medical information appearing in Google’s artificial intelligence (AI) summaries may expose users to serious health risks.

In one case described by experts as "really dangerous", Google advised pancreatic cancer patients to avoid foods high in fat. Medical specialists said this guidance directly contradicts recommended treatment protocols and could increase the risk of death.

Anna Jewell, Director of Services, Research & Influencing at Pancreatic Cancer U.K., warned: "If someone followed what the search result told them, then they might not take in enough calories, struggle to put on weight, and be unable to tolerate either chemotherapy or potentially life-saving surgery."

Another example involved the AI tool providing explanations of key liver function indicators that experts deemed "dangerous" as they could lead some people with severe liver disease to mistakenly believe they were healthy.

Google search results related to gynecological cancer screening tests were also found to contain information that doctors described as "completely wrong."

Experts told The Guardian that such inaccuracies could cause patients to become complacent and ignore genuine warning signs.

They added that prioritizing misleading information in the most prominent position on search results pages poses a direct threat to public health.

In response to the allegations, a Google spokesperson said the examples cited were merely "incomplete screenshots". The technology company said its search results always cite reputable sources and include recommendations to consult medical professionals.

Google maintained that AI Overviews, its generative AI feature designed to summarize key information on a topic or question, are mostly accurate and helpful, and that the company continually improves quality, according to Digital Watch.