FWF - 3DDR - Real-time Three-Dimensional Diminished Reality

Real-time Three-Dimensional Diminished Reality (3DDR) develops a novel software framework for real-time diminished reality (DR), a variant of mixed reality (MR) which changes a user's real time perception so that unwanted objects disappear. Our new approach can do this in full 3D and in real time, so that users can move arbitrarily and see through or behind objects. Previous techniques have been limited to the assumption that the region in the image that must be diminished (replaced) is flat, which is often not the case in real-world applications. Our method restores a three-dimensional, non-flat background, and can also fill in the background if no previous observation exists. This type of filling -- a three-dimensional variant of so-called "inpainting" -- must be temporally coherent, so that perspective changes resulting from a moving user position do not destroy the illusion of DR. Our method lends itself to visual inspection tasks, aestethic improvements of mixed reality scenes and telepresence applications with high visual quality.