Abstract
This paper presents the first photometric registration pipeline for
Mixed Reality based on high quality illumination estimation using
convolutional neural networks (CNNs). For easy adaptation and deployment
of the system, we train the CNNs using purely synthetic
images and apply them to real image data. To keep the pipeline accurate
and efficient, we propose to fuse the light estimation results
from multiple CNN instances and show an approach for caching
estimates over time. For optimal performance, we furthermore explore
multiple strategies for the CNN training. Experimental results
show that the proposed method yields highly accurate estimates for
photo-realistic augmentations.
Mixed Reality based on high quality illumination estimation using
convolutional neural networks (CNNs). For easy adaptation and deployment
of the system, we train the CNNs using purely synthetic
images and apply them to real image data. To keep the pipeline accurate
and efficient, we propose to fuse the light estimation results
from multiple CNN instances and show an approach for caching
estimates over time. For optimal performance, we furthermore explore
multiple strategies for the CNN training. Experimental results
show that the proposed method yields highly accurate estimates for
photo-realistic augmentations.
Original language | English |
---|---|
Title of host publication | Proceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2017 |
Publisher | Institute of Electrical and Electronics Engineers |
Pages | 82 - 89 |
Number of pages | 8 |
ISBN (Electronic) | 978-153862943-7 |
DOIs | |
Publication status | Published - 9 Oct 2017 |
Event | 2017 IEEE International Symposium on Mixed and Augmented Reality: ISMAR 2017 - La Cité, Nantes, France Duration: 9 Oct 2017 → 13 Oct 2017 https://ismar2017.sciencesconf.org/ |
Conference
Conference | 2017 IEEE International Symposium on Mixed and Augmented Reality |
---|---|
Abbreviated title | ISMAR 2017 |
Country/Territory | France |
City | Nantes |
Period | 9/10/17 → 13/10/17 |
Internet address |