Establishing Key Research Questions for the Implementation of Artificial Intelligence in Colonoscopy: A Modified Delphi Method

Ahmad et al. (2020) full text summary PDF

For which topic were research priorities identified?

artificial intelligence in colonoscopy

In which location was the research priority setting conducted?


Why was it conducted at all?

It is now widely recognized that the most translationally advanced AI applications in medicine, with the largest number of reported RCTs, belong to gastrointestinal endoscopy. The specialty is now in a prime position to become a leader for implementation research. Despite this, there has been minimal published literature exploring the opportunities and challenges relating to this critical next stage in endoscopy, which is likely to dominate the research agenda for the coming decade. In the field of colonoscopy, a recent review proposed some key principles for AI system development and testing. However, to our knowledge, there has been no prior publication of a formal systematic process to identify research priorities for AI in endoscopy.

What was the objective?

to identify the top implementation research priorities related to the implementation of artificial intelligence in colonoscopy

What was the outcome?

a ranking list of 10 research questions

How long did the research prioritization take?

March 2019 - November 2019

Which methods were used to identify research priorities?


How were the priorities for research identified exactly?

Step 1: Delphi round 1: participants were invited to list an unlimited number of research questions related to the implementation of AI and CAD in colonoscopy, 59 individual research questions were submitted. Step 2: data processing: consolidating responses, resulting in anonymous long-list for review by the steering committee, remaining responses were then categorized into nine broad topics. Step 3: Delphi round 2: participants asked to rate each question, 28 questions moved forward to round 3. Step 4: Delphi round 3: 28 questions from eight themes were presented in rank order, participants were asked to re-rate each question, resulting in final top 10 questions from five themes

Which stakeholders took part?

15 international experts: 12 endoscopists and 3 translational computer scientists/engineers. There were 15 participants in all 3 rounds.

How were stakeholders recruited?

The steering committee identified and invited participants by personal communication to create an international body of experts with experience in translational AI in colonoscopy. The following inclusion criteria were used: current involvement in clinician and engineer/computer scientist collaborative research in the field of AI or computer-aided diagnosis/detection (CAD) in colonoscopy with a specific focus on those with experience across the translation pipeline (i. e. case identification, data acquisition/curation, algorithm development, clinical evaluation, and deployment considerations). Publication history was also considered, with a requirement of at least one peer-reviewed publication in the field of AI/CADin colonoscopy listed on PubMed. In addition, geographic diversity was considered to ensure representation from the major regions involved in AI research and development in endoscopy.

Were stakeholders actively involved or did they just participate?

Stakeholders not only participated but were also actively involved in the research prioritization process: They were part of an advisory board and a steering group. The steering group consisted of 5 members (translational endoscopists and computer scientists). Advisory board members advised on the format and execution of the project. The steering group identified and invited participants.