Good Keyframes to Inpaint

Research output: Contribution to journalArticlepeer-review


Diminished Reality (DR) propagates pixels from a keyframe to subsequent frames for real-time inpainting. Keyframe selection has a significant impact on the inpainting quality, but untrained users struggle to identify good keyframes. Automatic selection is not straightforward either, since no previous work has formalized or verified what determines a good keyframe. We propose a novel metric to select good keyframes to inpaint. We examine the heuristics adopted in existing DR inpainting approaches and derive multiple simple criteria measurable from SLAM. To combine these criteria, we empirically analyze their effect on the quality using a novel representative test dataset. Our results demonstrate that the combined metric selects RGBD keyframes leading to high-quality inpainting results more often than a baseline approach in both color and depth domains. Also, we confirmed that our approach has a better ranking ability of distinguishing good and bad keyframes. Compared to random selections, our metric selects keyframes that would lead to higher-quality and more stably converging inpainting results. We present three DR examples, automatic keyframe selection, user navigation, and marker hiding.
Original languageEnglish
Pages (from-to)1
Number of pages1
JournalIEEE Transactions on Visualization and Computer Graphics
Publication statusPublished - 23 May 2022


  • Augmented Reality
  • Diminished Reality
  • Inpainting
  • Measurement
  • SLAM
  • Diminished reality
  • Image reconstruction
  • keyframe
  • inpainting
  • Simultaneous localization and mapping
  • Three-dimensional displays
  • Streaming media
  • Cameras
  • good keyframes to inpaint
  • Real-time systems

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Good Keyframes to Inpaint'. Together they form a unique fingerprint.

Cite this