From AI that reads images to AI that moves
AI analysis of X-rays is now familiar. This research identifies bone shapes, measures joint space, and classifies disease. But can AI that reads images well also perform surgery? In the operating room, it must move in the physical world as well as interpret images. It needs to know where the patient's body and instruments are and judge how much force to use when pulling tissue. Physical AI refers to AI that turns information from the physical environment into actions. Many questions remain between analyzing a screen and moving safely near a patient's body.
Continuously checking the positions of bones and instruments
The system must keep checking where a bone seen on the preoperative CT is located during surgery and how its position changes when the patient's leg moves. Skin and tissue may hide part of the bone, and hands or instruments may block the view. This is why the prepared images must be matched with the current surgical scene. Finding the bone accurately once does not establish that it will be tracked accurately throughout surgery. The system needs to consider the certainty of its recognition and changes in position while accounting for both the view and movement in the operating room.
Managing force on tissue
An instrument at the same position can affect tissue differently depending on how firmly it pushes or pulls. A surgical assistant adjusts force using both the resistance felt by hand and visible tissue changes. A robot also needs to adjust its actions using tissue condition and contact information. This motivates collecting surgical video, robot movement, force, and tactile information together. We must also study how to judge when the information can be trusted and when movement should stop. Ways for a person to intervene or stop the robot must be designed alongside these functions.
Robots working with the surgical team
An operation involves the surgeon, assisting clinicians, and several devices. The meaning of an action changes with the stage of surgery and what the surgeon intends to do. A robot that holds an instrument and a robot that cuts bone have different roles and risks and must be validated for their respective purposes. CONNECTEVE's surgical-assistance humanoid Zett is being developed to support tasks such as stabilizing or retracting tissue. A demonstration alone does not establish that it can perform an entire clinical operation without human assistance. Actions in a defined setting and safe work across varied patient situations need separate evaluation.
Validation beyond a demonstration
When reading surgical-robotics research, check what the system is designed to assist with, whether the results come from models or experiments, and which performance and safety outcomes have been assessed in actual patients. Expected advantages of a technology under development should be distinguished from effects established clinically. By connecting AI and robotics, we aim to reduce repetitive burdens during surgery and support clinicians' judgment and performance. To become useful for patients, the ability to locate structures, manage force, and work with the surgical team must be validated in turn. I will share this process through papers and explanations.