Activities per year
Abstract
Augmented reality for medical applications allows physicians to obtain an inside view into the patient without surgery. In this context, we present an augmented reality application running on a standard smartphone or tablet computer, providing visualizations of medical image data, overlaid with the patient, in a video see-through fashion. Our system is based on the registration of medical imaging data to the patient using a single 2D photograph of the patient. From this image, a 3D model of the patient’s face is reconstructed using a convolutional neural network, to which a pre-operative CT scan is automatically registered. For efficient processing, this is performed on a server PC. Finally, anatomical and pathological information is sent back to the mobile device and can be displayed, accurately registered with the live patient, on the screen. Hence, our cost-effective, markerless approach needs only a smartphone and a server PC for image processing. We present a qualitative and quantitative evaluation using real patient photos and CT from the clinical routine in facial surgery, reporting overall processing times and registration errors.
Original language | English |
---|---|
Title of host publication | Multimodal Learning for Clinical Decision Support and Clinical Image-Based Procedures |
Subtitle of host publication | 10th International Workshop, ML-CDS 2020, and 9th International Workshop, CLIP 2020, Held in Conjunction with MICCAI 2020, Proceedings |
Editors | Tanveer Syeda-Mahmood, Klaus Drechsler, Hayit Greenspan, Anant Madabhushi, Alexandros Karargyris, Cristina Oyarzun Laura, Stefan Wesarg, Marius George Linguraru, Raj Shekhar, Marius Erdt, Miguel Ángel González Ballester |
Publisher | Springer |
Pages | 64-74 |
Number of pages | 11 |
ISBN (Print) | 9783030609450 |
DOIs | |
Publication status | Published - 1 Jan 2020 |
Event | 2020 Workshop on Clinical Image-Based Procedures: in Conjunction with MICCAI 2020 - Virtual, Virtuell, Peru Duration: 4 Oct 2020 → 8 Oct 2020 https://miccai-clip.org/ |
Publication series
Name | Lecture Notes in Computer Science |
---|---|
Volume | 12445 |
ISSN (Print) | 0302-9743 |
ISSN (Electronic) | 1611-3349 |
Conference
Conference | 2020 Workshop on Clinical Image-Based Procedures |
---|---|
Abbreviated title | CLIP 2020 |
Country/Territory | Peru |
City | Virtuell |
Period | 4/10/20 → 8/10/20 |
Internet address |
Fingerprint
Dive into the research topics of 'Single-Shot Deep Volumetric Regression for Mobile Medical Augmented Reality'. Together they form a unique fingerprint.Activities
- 1 Talk at conference or symposium
-
Single-Shot Deep Volumetric Regression for Mobile Medical Augmented Reality
Christina Schwarz-Gsaxner (Speaker)
4 Oct 2020Activity: Talk or presentation › Talk at conference or symposium › Science to science
Prizes
-
Best Paper Award
Karner, Florian (Recipient), Schwarz-Gsaxner, Christina (Recipient), Pepe, Antonio (Recipient), Li, Jianning (Recipient), Fleck, Philipp (Recipient), Arth, Clemens (Recipient), Wallner, Jürgen (Recipient) & Egger, Jan (Recipient), Oct 2020
Prize: Prizes / Medals / Awards