A Marker-Less Registration Approach for Mixed Reality–Aided Maxillofacial Surgery: a Pilot Evaluation

Antonio Pepe, Gianpaolo F Trotta, Peter Mohr-Ziak, Christina Gsaxner, Jürgen Wallner, Vitoantonio Bevilacqua, Jan Egger

Research output: Contribution to journalArticlepeer-review


As of common routine in tumor resections, surgeons rely on local examinations of the removed tissues and on the swiftly made microscopy findings of the pathologist, which are based on intraoperatively taken tissue probes. This approach may imply an extended duration of the operation, increased effort for the medical staff, and longer occupancy of the operating room (OR). Mixed reality technologies, and particularly augmented reality, have already been applied in surgical scenarios with positive initial outcomes. Nonetheless, these methods have used manual or marker-based registration. In this work, we design an application for a marker-less registration of PET-CT information for a patient. The algorithm combines facial landmarks extracted from an RGB video stream, and the so-called Spatial-Mapping API provided by the HMD Microsoft HoloLens. The accuracy of the system is compared with a marker-based approach, and the opinions of field specialists have been collected during a demonstration. A survey based on the standard ISO-9241/110 has been designed for this purpose. The measurements show an average positioning error along the three axes of (x, y, z) = (3.3 ± 2.3, − 4.5 ± 2.9, − 9.3 ± 6.1) mm. Compared with the marker-based approach, this shows an increment of the positioning error of approx. 3 mm along two dimensions (x, y), which might be due to the absence of explicit markers. The application has been positively evaluated by the specialists; they have shown interest in continued further work and contributed to the development process with constructive criticism.
Original languageEnglish
Pages (from-to)1008-1018
Number of pages11
JournalJournal of Digital Imaging
Issue number6
Publication statusPublished - 4 Sept 2019


Dive into the research topics of 'A Marker-Less Registration Approach for Mixed Reality–Aided Maxillofacial Surgery: a Pilot Evaluation'. Together they form a unique fingerprint.

Cite this