Efficient Multi-Task Learning of Semantic Segmentation and Disparity Estimation

Robert Harb, Patrick Knöbelreiter

Research output: Contribution to conferencePaperpeer-review


We propose a jointly trainable model for semantic
segmentation and disparity map estimation. In this work we
utilize the fact that the two tasks have complementary strength
and weaknesses. Traditional depth prediction algorithms rely on
low-level features and often have problems at large textureless
regions, while for semantic segmentation these regions are
easier to capture. We propose a CNN-based architecture, where
both tasks are tightly interconnected to each other. The model
consists of an encoding stage which computes features for both
tasks, semantic segmentation and disparity estimation. In the
decoding stage we explicitly add the semantic predictions to
the disparity decoding branch and we additionally allow to exchange information in the intermediate feature representations.
Furthermore, we set the focus on efficiency, which we achieve
by the usage of previously introduced ESP building blocks. We
evaluate the model on the commonly used KITTI dataset.
Original languageEnglish
Number of pages152
Publication statusPublished - 9 May 2019
EventOAGM/ARW 2019: ARW & OAGM Workshop 2019 - Steyr, Austria
Duration: 9 May 201910 May 2019


ConferenceOAGM/ARW 2019
OtherAustrian Robotics Workshop and OAGM Workshop 2019


  • Stereo vision
  • Semantic Segmentation
  • Multi-Taks-Learning

Cite this