Guidelines for Purchasing AI Software in Radiology: A Scoping Review and Practical Guide

Purpose: With over 500 FDA approved radiology AI software applications radiologists face the complex challenge of identifying the most suitable AI solution for their practice. This scoping review examines key literature and integrates expert opinion from across the field into a single comprehensive framework and practical guide for AI software purchase and selection.

Methods/Materials: A scoping review literature search was conducted using Ovid MEDLINE, Healthcare Administration Database, Embase, and Scopus databases to identify relevant publications that described choosing, buying, or purchasing AI software for a radiology department. Relevant data was summarized in a diagram.

Results: In total, 20 publications were analyzed to develop a structured set of guidelines for the procurement of AI software, divided into seven major categories: identifying the need for AI software, performing market analysis, identifying software requirements, selecting trial softwares, identifying and evaluating trial endpoints, assessing quality control cycles, and determining final AI software selection. Important consensus points across publications included the importance of understanding primary and secondary use cases, rigorous validation against industry standards, and assessment of the software’s clinical relevance and physician adoption potential. Key practical elements included guidelines for conducting a trial period for selected software, a framework for developing software quality control cycles and guidance for implementation of a weighted scale to help evaluate AI software before selection.

Conclusions: This scoping review establishes a set of guidelines for purchasing AI software in radiology, incorporating a multi-stage evaluation process that begins with defining use cases and concludes with the selection of a single AI application for clinical deployment. This structured approach ensures a comprehensive and balanced evaluation for the procurement of AI software, from inception to implementation in clinical settings.