Markerless Image-to-Face Registration for Untethered Augmented Reality in Head and Neck Surgery

Christina Gsaxner, Antonio Pepe, Dieter Schmalstieg, Jan Egger, Jürgen Wallner

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review

Abstract

In the treatment of head and neck cancer, physicians can benefit from augmented reality in preparing and executing treatment. We present a system allowing a physician wearing an untethered augmented reality headset to see medical visualizations precisely overlaid onto the patient. Our main contribution is a strategy for markerless registration of 3D imaging to the patient’s face. We use a neural network to detect the face using the headset’s depth sensor and register it to computed tomography data. The face registration is seamlessly combined with the headset’s continuous self-localization. We report on registration error and compare our approach to an external, high-precision tracking system.
Original languageEnglish
Title of host publicationInternational Conference on Medical Image Computing and Computer-Assisted Intervention
Pages236-244
Publication statusPublished - 2019

Fingerprint

Dive into the research topics of 'Markerless Image-to-Face Registration for Untethered Augmented Reality in Head and Neck Surgery'. Together they form a unique fingerprint.

Cite this