The IMERSE Lab: A New Era of Autonomous Surgical Robotics
- Lori Preci
- Nov 3
- 8 min read
Inside Johns Hopkins University’s IMERSE Lab, surgical robotics is entering a new frontier. From the early days of robotic assistance in minimally invasive surgery to today’s systems that can recognize anatomy, adapt as tissue shifts, and plan intricate movements with remarkable precision, the field is evolving at an extraordinary pace. At the center is a team of engineers and clinicians rethinking how human skill and machine intelligence can operate in synergy, transforming what surgery renders possible.
From Sci-Fi to Supervised Autonomy
From the development of robots in the early 20th century to the robotic boom of the 1980s, the idea of surgical robots belonged to the realm of science fiction. Yet by 1985, the first steps toward commercialization had begun. Surgeons adapted the PUMA 560, an industrial robotic arm originally designed for factory automation, to guide a needle during CT-guided brain biopsies. In that first successful use, the robot performed a delicate brain tissue sampling with unprecedented steadiness, demonstrating that machines could deliver precision beyond the limits of the human hand.
By the early 2000s, robotic platforms such as Intuitive Surgical’s da Vinci® had entered operating rooms worldwide. Their rise coincided with growing demand for less invasive technologies, hospital competition for technological prestige, and surgeon interest in tools that could reduce fatigue and improve precision. This convergence of clinical, commercial, and technological incentives fueled the rapid adoption of surgical robotics across the U.S. and abroad. Cleared by the FDA in 2000 for general laparoscopic procedures - operations performed through small incisions using slender instruments and a camera - da Vinci transformed possibilities for minimally invasive surgery. The robot’s articulated instruments restored the dexterity lost in traditional laparoscopic tools by mimicking the rotation of a human wrist, while its stereoscopic camera provided a magnified, three-dimensional view of the operative field. Together, these features gave surgeons unprecedented precision in confined anatomical spaces.
Yet for all its advances, da Vinci remains entirely tele-operated: a sophisticated extension of the surgeon's hand. It faithfully executes every console command but lacks the capacity to make independent decisions. Overcoming these limitations defines the next frontier: autonomy.
The IMERSE Lab and the Next Frontier
At Johns Hopkins University, the Intelligent Medical Robotic Systems and Equipment (IMERSE) Lab, led by Dr. Axel Krieger, is bringing robotic autonomy into the operating room. The team is developing robotic systems that interpret complex anatomical environments, plan and adapt their actions as conditions evolve, and move beyond pre-scripted motions toward intelligent, collaborative assistance.
Achieving true autonomy demands close collaboration between engineers, computer scientists, and surgeons. The IMERSE Lab’s work bridges these disciplines, translating advances in machine learning and computer vision into practical surgical systems capable of navigating the variability and unpredictability of living anatomy with reliability, subtleties that still elude even the most advanced simulations.
Importantly, the IMERSE Lab's goal is not to replace surgeons but to amplify their judgment, reduce both the physical and cognitive demands of surgery, and ensure greater consistency in outcomes.
STAR: A First Proof of Concept
Autonomous surgery first drew wide scientific attention through a series of collaborative efforts led by Dr. Axel Krieger and his colleagues at Children's National Hospital, and later continued at Johns Hopkins University. In 2016, Krieger’s team demonstrated an early open-surgery version of the Smart Tissue Autonomous Robot (STAR) - performed through a traditional incision rather than small laparoscopic openings - which successfully carried out autonomous soft-tissue suturing in vivo (in living tissue) on animal models.
Six years later, the refined STAR took on a new mission: laparoscopic intestinal anastomosis - reconnecting two ends of the intestine after removing a diseased segment. This delicate task requires extreme precision, where even minute errors in suture spacing or tension can lead to leakage and serious complications.
Operating laparoscopically intensifies the technical challenge. Surgeons must operate through small incisions using long instruments and a camera, working without direct tactile feedback and within narrow visual and mechanical constraints. Achieving the precision required for secure, tension-free suturing under these conditions challenges even the most experienced hands.
The greatest obstacle, however, lies in the living tissue itself: soft organs bend, stretch, and shift with every breath, demanding sub-millimeter accuracy. To manage this, STAR integrated several innovations: an advanced three-dimensional endoscope to reconstruct the surgical field, near-infrared fluorescent markers to track tissue motion in real time, and machine learning algorithms capable of mapping optimal stitch placement and replanning whenever tissue displacement exceeded roughly three millimeters.
In 2022, STAR successfully performed autonomous intestinal anastomosis in live pigs without human guidance. Evaluations showed that its sutures matched, and in some trials exceeded, those of experienced surgeons in spacing consistency and tissue alignment. The demonstration marked the first laparoscopic autonomous soft-tissue surgery in vivo, building upon the earlier open-surgery version completed in 2016.
Despite its success, STAR remained limited in scope: it was purpose-built for a single procedure and required nearly a decade of iterative development to reach that milestone. The next challenge was to design systems capable of generalizing across operations and adapting flexibly to new surgical contexts with the same level of precision.
The Rise of SRT-H
That challenge - to create systems capable of generalizing across operations and learning to perform different surgical procedures - became the foundation for the next stage of research at the IMERSE Lab. Within the group, Juo-Tung “Justin” Chen, a doctoral researcher in mechanical engineering, helped develop the Surgical Robot Transformer-H (SRT-H), a system designed to teach robots surgical workflows through human demonstration rather than manually programming each motion.
At its core, SRT-H uses a hierarchical decision-making structure. A high-level planner issues task-level directives in plain language, such as “expose Calot’s triangle,” an anatomical landmark near the gallbladder’s ducts, or “apply clip to duct.” Low-level controllers then translate those commands into precise movements of robotic arms and tools using natural language processing (NLP) algorithms that map each instruction to corresponding motor actions. The separation mirrors how people operate: first reasoning through what must be done, then executing it physically.
Before each motion is executed, the system displays the planner’s directive on a surgeon-facing console interface, allowing the operator to review and confirm what the robot intends to do. This visibility provides transparency and allows the supervising surgeons to intervene before the action occurs.
This design makes SRT-H both interpretable and resilient. Surgeons can observe the planner’s language-based instructions and understand what the robot intends to do, offering a degree of transparency uncommon in machine-learning systems. When unexpected situations arise, such as tissue dyed red to simulate bleeding, or an organ shifted into a different orientation, SRT-H relies on stereo cameras and real-time vision sensors that continuously track the operative field. When these sensors detect a deviation beyond a set tolerance, the high-level planner automatically replans the next movement or issues a corrective micro-adjustment, such as “shift gripper slightly right,” allowing the workflow to continue rather than freezing mid-procedure.
SRT-H has already advanced toward the operating room. In published studies, it achieved step-level autonomy - that is, independence across a sequence of surgical steps rather than a single repeated motion - during the clipping and cutting phase of a cholecystectomy (gallbladder removal) on ex vivo (outside-the-body) pig tissue. Across eight trials, the robot completed the sequence without human intervention, achieving 100 percent success. While not yet capable of a full cholecystectomy, this demonstration marked the first time a system achieved autonomous control across multiple surgical subtasks rather than a single isolated skill.
What Makes SRT-H Different
Earlier autonomous prototypes like STAR proved that surgical autonomy was possible but remained limited in flexibility and scope. SRT-H takes a fundamentally different approach, both in how it learns and how it adapts.
Data-Driven Training
Instead of relying on hand-coded rules, the team trained SRT-H on roughly 18,000 surgical demonstrations collected from over 30 pig gallbladders. Operators performed gallbladder dissections using the da Vinci robotic system, recording synchronized video and instrument motion. Clinical collaborators oversaw the process to replicate real surgical conditions, ensuring the data captured reflected the precision and rhythm of a true operating room.
Language-Conditioned Learning
Each demonstration was annotated with one of 17 subtask phrases, such as “grasp gallbladder” or “apply first clip.” By linking actions to language, the robot learns to reason in ways that humans can easily interpret, and, more importantly, correct through feedback expressed in plain terms. This language-conditioned structure not only makes its decisions transparent but also bridges human and machine communication inside the operating room.
Robustness to Variation
To prevent overfitting - when an algorithm performs well on familiar data but fails in new environments, the researchers deliberately perturbed the training data. They altered colors, simulated bleeding, cropped images, and randomized orientations to expose the robot to countless variations. As a result, SRT-H can operate reliably even when confronted with occlusion, distortion, or unexpected visual cues, demonstrating a level of adaptability that earlier systems lacked.
Real-Time Recovery
Whereas STAR paused when conditions changed, SRT-H adapts in real time. It can issue micro-adjustments, such as “move gripper slightly right,” and continue seamlessly through a task. This capacity for real-time correction makes it a far more practical candidate for clinical transition.
Scaling the Vision
For Chen and his colleagues, the gallbladder experiments mark only the first step toward a broader goal: expanding surgical autonomy through shared, data-driven learning. If SRT-H demonstrated that autonomy can extend beyond a single task, the next frontier is scaling that capability across procedures, robots, and institutions.
To achieve this, the IMERSE Lab and its collaborators, including NVIDIA, aim to develop foundation models for surgical robotics. Much like large language models trained on vast collections of text, these systems would learn from multimodal surgical data encompassing videos, robotic motion, imaging, and annotations gathered from diverse clinical environments. By training across such varied datasets, a foundation model could, in principle, generalize surgical knowledge across tasks and platforms, adapting flexibly to new operating conditions.
The long-term ambition is the creation of a shared foundation model capable of enabling robotic systems to perform a wide range of procedures, even in settings without subspecialist expertise. In this vision, surgical autonomy is not a luxury technology reserved for advanced centers but a tool for equitable access, bringing expert-level surgical care to every hospital, anywhere in the world.
Ethical Considerations and the Future Role of the Surgeon
The promise of surgical autonomy brings with it a new set of challenges. Robotic systems may offer surgeons greater precision, consistency, and reduced fatigue, but surgery remains a field where even minor errors can have life-altering consequences - and where public trust, once lost, is difficult to regain.
Chen emphasizes that autonomy in surgery does not mean independence. He envisions a future in which the surgeon becomes a supervisor and decision-maker, guiding the operation while the robot executes repetitive or technically demanding tasks. He likens this to autonomous driving: a car may steer, brake, and park itself, but a human driver still bears responsibility. Likewise, surgical robots are being designed as collaborators - extensions of the surgeon’s skill, not replacements for their judgment.
Realizing this future will require far more than advances in algorithms. Engineers and clinicians will need clear regulatory standards, robust safety mechanisms, and transparent validation frameworks to ensure reliability. Just as importantly, they must cultivate patient trust through openness and ethical design, ensuring that progress in autonomy enhances, not undermines, the human connection at the heart of medicine.
Conclusion
The trajectory of the IMERSE Lab’s work, from STAR’s meticulously engineered sutures to SRT-H’s language-guided learning, captures both the promise and the complexity of surgical autonomy. What once seemed speculative is now unfolding through trial, error, and iteration in real laboratories, where autonomy is being redefined not as independence, but as collaboration.
For now, the surgeon’s hands remain indispensable. Through the work of researchers like Justin Chen, the operating room is steadily evolving into a space of collaboration between human judgment and machine intelligence. The future of surgery may not hinge on choosing between surgeon or robot, but on discovering new ways for the two to think and operate as one.
Comments