How augmented reality will make surgery safer

Augmented reality is entering operating theatres across Europe and will soon help save many lives.

Researchers at the EndoCAS Centre at the University of Pisa are developing a surgical navigator Vostars (Video and Optical See-Through Augmented Reality Surgical Systems) that will guide doctors during the delicate stages of an operation.  The apparatus will also provide crucial data in real time. But how exactly does the technology work? And how can surgeons integrate it within their operations?

The introduction of augmented reality pushes surgery into new frontiers.

Vostars will provide a great aid to areas like maxillofacial surgery, says Dr. Giovanni Badiali of the Sant'Orsola hospital of Bologna, Italy: "During surgery, we often have to reposition parts of the skull, to fix malformations, for instance. By mapping visual indicators on the patient, it guarantees symmetry and is an invaluable aid"

This European research project (which is coordinated by the Department of Information Engineering of the University of Pisa), is a hybrid of the two existing AR headsets. The viewer can access crucial data, through the headset, such as incision pressure and the heart rate of the patient.

The project's coordinator, Vincenzo Ferrari, explains: 

"Their main feature is to provide crucial information directly in front of a physician’s eyes. It takes accurate virtual information and overlays it on the surgical area to guarantee incision-making with extreme precision. It can also provide an enlarged view." 

The system captures what the surgeon sees from a head-mounted camera. It then merges the patient’s medical information over reality.

The system captures what the surgeon sees from a head-mounted camera. It then overlays the patient’s medical information over reality. Biomedical engineer Fabrizio Cutolo describes the process behind the technology. 

"Every time we want to align a real scene with a virtual one, we need a system of video cameras to trace the surroundings. We are then better equipped to reconstruct that virtual scene so that it is consistent with the way our eyes perceive it."

The Vostars technology is expected to improve accuracy and reduce surgery times by at least 11%. For this to happen there has to be a strong emphasis on integrating surgical knowledge with the functioning of the viewer. Engineer Marina Carbone explains her role in the project.

"My job is to build a bridge between surgery and the device interface. I work on all the steps to make the system work! Right up to how and when the device will operate during surgery"

The device which is developed by a consortium of universities, research centers, and European companies, will be submitted for three clinical trials in Italy and Germany by late 2019. It is expected to be available on the market by 2022.