The A.I. Resident Enters the Colonoscopy Suite

A new systematic review and a first-hand perspective with Pranav Rajpurkar

“That’s an A.I. win”—Tyler Berzin, MD

This week a systematic review of 33 randomized trials of A.I. supported colonoscopy compared with gastroenterologists on their own was published, for a cumulative >27,000 patients.

In real time, the A.I. detects polyps highlighted by the green rectanlges

Across these 33 trials, which represent the largest proportion of randomized trials performed thus far in medical A.I, there was an impressive increased A.I. boost in detection of adenomatous polyps, the ones that are most likely to progress to cancer. As seen in one of the many graphs from the paper, there was a pretty consistent benefit across the trials (nearly all the boxes in the Forest plot are to the right of the vertical line of 1).

Over recent years I have had the privilege of collaborating with and mentoring Pranav Rajpurkar—a young, prolific superstar A.I. researcher. He was recently recognized by Tech Review as one of the leading innovators under age 35. In fact, he’s only 28 and went directly from Stanford after his PhD to join the faculty as an Assistant Professor at Harvard Medical School.

He wrote and I helped edit the following essay about A.I .and GI about Pranav’s experience attending a colonoscopy with Tyler Berzin, a leading gastroenterologist at Beth Israel Deaconess who has worked extensively in A.I.

Firsthand: An AI Resident Enters the Colonoscopy Suite

Pranav Rajpurkar and Eric Topol

In the trenches of the colonoscopy room, A.I.'s entrance may just be the advantage doctors need in the fight against colorectal cancer. Colorectal cancer remains a leading cause of mortality globally that screening colonoscopies aim to combat by detecting precancerous adenomas. However, small polyps are easily overlooked. Artificial intelligence promises to address this by boosting adenoma detection rates.

Randomized controlled trials provide the gold standard for objectively measuring a clinical intervention's impact. Though evidence of AI tools' real-world clinical benefits remains scarce, colonoscopy is leading the way in the number of AI trials across medical fields. In trials comparing AI-assisted and unassisted colonoscopies, the results are clear: AI detects significantly more polyps, especially subtle ones easily missed unaided.

A major new meta-analysis provides the most definitive evidence yet that AI-aided colonoscopy could profoundly reduce missed polyps and prevent future cancer. Analyzing 33 randomized trials encompassing over 27,000 patients, researchers found AI improved detection rates of tiny adenomas by over 39% and advanced adenomas most prone to become malignant by over 50%. The authors conclude AI holds particular promise to enhance accuracy for less experienced endoscopists and younger patients. Improved prep and timing may further optimize AI performance. If integrated appropriately, AI colonoscopy could avert tens of thousands of cancer deaths each year.

Eager to witness AI-aided colonoscopy firsthand, my colleague Dr. Pranav Rajpurkar visited the endoscopy suite at Beth Israel Deaconess Medical Center, where they employ an FDA-approved system to boost adenoma detection. Here, Dr. Rajpurkar recounts his observations:

In the endoscopy unit, I witnessed Dr. Tyler Berzin bring along his new trainee – an AI resident – in the fight against colorectal cancer. No larger than a medical textbook, the AI resident would aid in the critical task of finding polyps in the colon's convoluted passages. Polyps, subtle growths in the colon, can lead to cancer if not detected and removed early. Their detection is pivotal in preventing the 50,000 colorectal cancer deaths the U.S. sees annually.

The AI resident sat quietly on a shelf – at shoulder height – in the instrument tower, nestled among cables, monitors, and cold machinery. With a click, Dr. Berzin switched on the AI. An immediate transformation occurred on the screen in front of him – the once gray corners of the view came alive into a vibrant green, as the AI resident signaled its readiness to join the search.

As Dr. Berzin started descending down the colon, the screen displayed the twisting and turning pathway lined with folds and crevices, all rendered in shades of pinkish-orange. "In textbooks, polyps seem easily spotted," said Dr. Berzin, leaning toward the monitor, scrutinizing it with quiet focus. “In reality, they blend into the tissue, elusive as phantoms." Suddenly, a green box outline flashed on the screen, momentarily highlighting a structure within view: a symbol of a possible discovery. But the physician, steady and assured, quickly dispelled the illusion: "The AI thinks there's a polyp here, but that's just a fold." A moment later, the green box had vanished.

He reflected on the challenges of false positives, expressing concern that some physicians might feel compelled to act on every AI detection, under the misconception that it sees something they cannot. He explained that while false positives were low in the evaluation of this AI system, they only counted when the green box appeared for an extended time.

As Dr. Berzin navigated further, he reached an area marked by a medical tattoo - a blue stain left by a previous colonoscopy to pinpoint a location needing revisiting. Moments later, he exclaimed, "I see that there is a residual polyp in the distance. Let's see whether the AI catches it when I approach."

As he moved closer, the green rectangle box made another appearance on the screen, flickering at first before settling into a steady green state, signaling the AI's detection. Preparing to remove the polyp, Dr. Berzin reflected: "Sometimes it's challenging to determine whether the physician or the AI was the first to spot something. It's not always blindingly obvious when it helps, even though objectively we know it does." Dr. Berzin pointed out the technology's limitation: "It's designed for routine screenings, not follow-ups with tattoos that can confuse it."

Dr. Berzin paused as a small green box flashed near the screen's bottom, leaning in to inspect the tiny protrusion it highlighted. The box remained fixed around an almost imperceptible polyp. "I likely would have missed that without the AI," he said, preparing to remove it.

"That's an AI win," he added.

After finishing, Dr. Berzin pulled out a bulky electronic device – an early AI assistant, now a relic given AI's rapid progress. Yet even then, the kernel of an idea took root - AI as an extra set of eyes for physicians. As I left the procedure room, Dr. Berzin was already preparing for his next patient, his AI resident looking over his shoulder.

Information

Integrating AI into clinics challenges how physicians behave and rely on AI. There’s the risk of automation bias, where physicians may overweight the AI’s predictions compared to their own judgment. But the opposite problem, automation neglect, can be just as dangerous. A recent study with economist colleagues from MIT showed that radiologists did not improve at reading chest X-rays when assisted by AI that was more accurate than two-thirds of them. The assumption that good AI naturally improves physicians is well-meaning but unproven. Harnessing AI’s benefits, while avoiding pitfalls like over-reliance, will require research and training. For now, a balanced approach is key – embracing innovation while upholding standards for evidence.

Hospitals adopting AI need clinical champions - physicians who understand, educate, and integrate the technology. Dr. Berzin has played this champion role for AI in gastroenterology. For nearly a decade, he has led key research on using AI for polyp detection – in the procedure room Dr. Rajpurkar visited, the first U.S. patient underwent an AI-assisted colonoscopy with modern deep learning under Dr. Berzin’s trial leadership. Though Dr. Berzin now routinely uses FDA-approved AI assistance at his endoscopy center in Beth Israel Deaconess Medical Center in Boston, the assistance only covers a fraction of what he encounters and treats while performing procedures.

Though today's tools focus on simple tasks, quick advances make the future seem ripe with possibility, like Generalist Medical AI systems that could more closely resemble doctors in their ability to reason through a wide range of medical tasks, incorporate multiple data modalities, and communicate in natural language. As we envision the future, the concept of the AI resident quietly residing on a shelf alongside human physicians appears tantalizing. Yet, it's imperative to remember that this journey demands not just innovation but rigorous evidence, for in the convergence of technology and medicine, the AI resident could very well hold the keys to our collective medical wisdom.

Thanks for reading, subscribing and sharing Ground Truths!