We use cookies to understand how you use our site and to improve your experience. This includes personalizing content and advertising. To learn more, click here. By continuing to use our site, you accept our use of cookies. Cookie Policy.

Features Partner Sites Information LinkXpress hp
Sign In
Advertise with Us
GLOBETECH PUBLISHING LLC

Download Mobile App




New Scoring Systems Increase Accuracy of AI-Generated Radiology Reports

By MedImaging International staff writers
Posted on 07 Aug 2023
Image: Scientists have designed a new way to score accuracy of AI-generated radiology reports (Photo courtesy of Freepik)
Image: Scientists have designed a new way to score accuracy of AI-generated radiology reports (Photo courtesy of Freepik)

Artificial intelligence (AI) tools that efficiently produce detailed narrative reports of CT scans or X-rays can significantly lighten the workload of busy radiologists. These AI reports go beyond simple identification of abnormalities and instead provide complex diagnostic information, detailed descriptions, nuanced findings, and appropriate degrees of uncertainty, similar to how human radiologists describe scan results. While several AI models capable of generating such detailed medical imaging reports have emerged, automated scoring systems meant to assess these tools are proving to be less effective at gauging their performance, according to a new study.

In the study, researchers at Harvard Medical School (Boston, MA, USA) tested various scoring metrics on AI-generated narrative reports and had six human radiologists read these reports. The analysis revealed that automated scoring systems performed poorly compared to human radiologists when it came to evaluating AI-generated reports. These systems misinterpreted and even missed significant clinical errors made by the AI tool. Ensuring the reliability of scoring systems is crucial for AI tools to continue improving and gaining clinicians' trust. However, the metrics tested in the study failed to reliably identify clinical errors in the AI reports, highlighting an urgent need for improvement and the development of high-fidelity scoring systems that accurately monitor tool performance.

In order to create better scoring metrics, the research team designed a new method called RadGraph F1 for evaluating the performance of AI tools generating radiology reports from medical images. Additionally, they created a composite evaluation tool called RadCliQ, which combines multiple metrics to produce a single score that more closely aligns with how a human radiologist would assess an AI model's performance. Using these new scoring tools, the researchers evaluated several state-of-the-art AI models and found a notable gap between their actual scores and the top possible scores.

Going forward, the researchers envision building generalist medical AI models capable of performing various complex tasks, including solving novel problems. Such AI systems could effectively communicate with radiologists and physicians about medical images, assisting in diagnosis and treatment decisions. The team also aims to develop AI assistants that can explain imaging findings directly to patients using everyday language, enhancing patient understanding and engagement. Ultimately, these advancements could revolutionize medical imaging practices, improving efficiency, accuracy, and patient care.

“Accurately evaluating AI systems is the critical first step toward generating radiology reports that are clinically useful and trustworthy,” said study senior author Pranav Rajpurkar, assistant professor of biomedical informatics in the Blavatnik Institute at HMS. “By aligning better with radiologists, our new metrics will accelerate development of AI that integrates seamlessly into the clinical workflow to improve patient care,”

Related Links:
Harvard Medical School 

Pocket Fetal Doppler
CONTEC10C/CL
New
High-Precision QA Tool
DEXA Phantom
Portable X-ray Unit
AJEX140H
X-ray Diagnostic System
FDX Visionary-A

Channels

Ultrasound

view channel
Image: The new implantable device for chronic pain management is small and flexible (Photo courtesy of The Zhou Lab at USC)

Wireless Chronic Pain Management Device to Reduce Need for Painkillers and Surgery

Chronic pain affects millions of people globally, often leading to long-term disability and dependence on opioid medications, which carry significant risks of side effects and addiction.... Read more

Nuclear Medicine

view channel
Image: The diagnostic tool could improve diagnosis and treatment decisions for patients with chronic lung infections (Photo courtesy of SNMMI)

Novel Bacteria-Specific PET Imaging Approach Detects Hard-To-Diagnose Lung Infections

Mycobacteroides abscessus is a rapidly growing mycobacteria that primarily affects immunocompromised patients and those with underlying lung diseases, such as cystic fibrosis or chronic obstructive pulmonary... Read more

Imaging IT

view channel
Image: The new Medical Imaging Suite makes healthcare imaging data more accessible, interoperable and useful (Photo courtesy of Google Cloud)

New Google Cloud Medical Imaging Suite Makes Imaging Healthcare Data More Accessible

Medical imaging is a critical tool used to diagnose patients, and there are billions of medical images scanned globally each year. Imaging data accounts for about 90% of all healthcare data1 and, until... Read more
Copyright © 2000-2025 Globetech Media. All rights reserved.