Learning Energy Based Inpainting for Optical Flow

Christoph Vogel, Patrick Knöbelreiter, Thomas Pock

Research output: Contribution to conferencePaperpeer-review


Modern optical flow methods are often composed of a cascade of many independent steps or formulated as a black box neural network that is hard to interpret and analyze. In this work we seek for a plain, interpretable, but learnable solution. We propose a novel inpainting based algorithm that approaches the problem in three steps: feature selection and matching, selection of supporting points and energy based inpainting. To facilitate the inference we propose an optimization layer that allows to backpropagate through 10K iterations of a first-order method without any numerical or memory problems. Compared to recent state-of-the-art networks, our modular CNN is very lightweight and competitive with other, more involved, inpainting based methods.
Original languageEnglish
Publication statusPublished - 4 Dec 2018
Event14th Asian Conference on Computer Vision: ACCV 2018 - Perth Western Australia, Perth, Australia
Duration: 4 Dec 20186 Dec 2018


Conference14th Asian Conference on Computer Vision
Abbreviated titleACCV 2018
Internet address


  • Optical Flow
  • Energy Optimization
  • Deep Learning

Cite this