{"id":10534,"date":"2026-04-26T17:30:00","date_gmt":"2026-04-26T17:30:00","guid":{"rendered":"https:\/\/musictechohio.online\/site\/top-medical-journal-warning-against-medical-ai\/"},"modified":"2026-04-26T17:30:00","modified_gmt":"2026-04-26T17:30:00","slug":"top-medical-journal-warning-against-medical-ai","status":"publish","type":"post","link":"https:\/\/musictechohio.online\/site\/top-medical-journal-warning-against-medical-ai\/","title":{"rendered":"Top Medical Journal Publishes Searing Article Warning Against Medical AI"},"content":{"rendered":"
\n

A recent survey found that millions of Americans<\/a> are asking AI chatbots for medical advice, often instead of consulting human doctors.<\/p>\n

That\u2019s despite researchers continuing to find severe flaws plaguing large language model-based tools that can purportedly offer summaries of medical records<\/a> and dole out health advice based on simple text prompts. For one, hallucinations remain a massive unsolved problem, from AI models generating detailed clinical findings based on images they were never provided<\/a> to falling for fake diseases<\/a> that were invented by researchers in order to trick them.<\/p>\n

In short, it\u2019s no wonder scientists are questioning whether patients, health providers, or health systems should adopt AI at all, especially given the frequently lacking evidence for any real-world benefits. A scathing editorial<\/a> published on Tuesday by the premier medical journal Nature Medicine<\/em> makes the case that \u201cevidence that AI tools create value for patients, providers or health systems remains scarce.\u201d<\/p>\n

\u201cNonetheless, in publications, and in product materials, claims about clinical impact are increasingly more common, even though there is no clear agreement on what level of evidence should be required before such claims are considered credible,\u201d the editorial reads. \u201cThe result is not only scientific uncertainty but also often premature implementation and adoption.\u201d<\/p>\n

The piece therefore calls for the establishment of a \u201cframework for how AI medical technologies should be evaluated, by what metrics and against which benchmarks,\u201d which is \u201curgently needed.\u201d<\/p>\n

AI tools often appear to offer compelling medical advice under perfect experimental conditions, then struggle in the real world. A recent study<\/a> in the journal JAMA Medicine<\/em> found that when provided with more ambiguous symptoms, frontier AI models failed to produce the correct diagnosis upward of 80 percent of the time.<\/p>\n

The topic of AI use in clinical research also remains contentious. While LLMs specialize at summarizing and analyzing data, while answering queries, researchers continue to warn that we\u2019re being blinded by their significant limitations.<\/p>\n

\u201cI think that AI can help speed up many of the processes that are tedious and challenging,\u201d said Harvard Medical School assistant professor of surgery Jamie Robertson in a statement<\/a> last year. \u201cIt can help us come up with code to do data analysis and even suggest scenarios.\u201d<\/p>\n

\u201cBut it\u2019s critical for people who are interacting with AI as part of clinical studies to be knowledgeable about the right and wrong applications, and in the correct context,\u201d she added. <\/p>\n

Researchers warn that over-relying on AI tools could result in sacrificing scientific rigor<\/a>, raising concerns over the proliferation of overgeneralized \u2014 and potentially hallucinated \u2014 data in the medical field.<\/p>\n

In one particularly colorful demonstration, University of Gothenburg medical researcher Almira Osmanovic Thunstr\u00f6m uploaded two clearly fake studies to a preprint server to trick large language models into thinking a made-up skin condition was real<\/a>. It didn\u2019t take long for other peer-reviewed journals to publish (since-retracted) papers that cited these preprints, highlighting glaring questions over their validity.<\/p>\n

\u201cThe next phase of progress will depend not only on better models and new applications but also on clearer expectations for how clinical impact is defined, evaluated and communicated,\u201d the Nature Medicine <\/em>editorial concludes. \u201cWithout a clear connection between claims and evidence, medical AI risks being adopted faster than its real value can be understood.\u201d<\/p>\n

More on AI in medicine:<\/strong> Millions of Americans Are Talking to AI Instead of Going to the Doctor, and It\u2019s Giving Them Horrendously Flawed Medical Advice<\/em><\/a><\/p>\n

The post Top Medical Journal Publishes Searing Article Warning Against Medical AI<\/a> appeared first on Futurism<\/a>.<\/p>\n<\/div>\n

<\/div>","protected":false},"excerpt":{"rendered":"

A recent survey found that millions of Americans are asking AI chatbots for medical advice, often instead of consulting human doctors. That\u2019s despite researchers continuing to find severe flaws plaguing…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[177,3843,3841,3844,3955],"tags":[],"class_list":["post-10534","post","type-post","status-publish","format-standard","hentry","category-artificial-intelligence","category-developments","category-ethics","category-health-medicine","category-medical"],"_links":{"self":[{"href":"https:\/\/musictechohio.online\/site\/wp-json\/wp\/v2\/posts\/10534","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/musictechohio.online\/site\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/musictechohio.online\/site\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/musictechohio.online\/site\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/musictechohio.online\/site\/wp-json\/wp\/v2\/comments?post=10534"}],"version-history":[{"count":0,"href":"https:\/\/musictechohio.online\/site\/wp-json\/wp\/v2\/posts\/10534\/revisions"}],"wp:attachment":[{"href":"https:\/\/musictechohio.online\/site\/wp-json\/wp\/v2\/media?parent=10534"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/musictechohio.online\/site\/wp-json\/wp\/v2\/categories?post=10534"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/musictechohio.online\/site\/wp-json\/wp\/v2\/tags?post=10534"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}