Casting Geometric Constraints in Semantic Segmentation as Semi-Supervised Learning

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review


We propose a simple yet effective method to learn to segment new indoor scenes from video frames: State-of- the-art methods trained on one dataset, even as large as the SUNRGB-D dataset, can perform poorly when applied to images that are not part of the dataset, because of the dataset bias, a common phenomenon in computer vision. To make semantic segmentation more useful in practice, one can exploit geometric constraints. Our main contribution is to show that these constraints can be cast conveniently as semi-supervised terms, which enforce the fact that the same class should be predicted for the projections of the same 3D location in different images. This is interesting as we can exploit general existing techniques de- veloped for semi-supervised learning to efficiently incorporate the constraints. We show that this approach can efficiently and accurately learn to segment target sequences of ScanNet and our own target sequences using only annotations from SUNRGB-D, and geometric relations between the video frames of target sequences.

Original languageEnglish
Title of host publicationProceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020
Number of pages10
ISBN (Electronic)9781728165530
Publication statusPublished - Mar 2020
Eventwacv2020: WACV 2020 - Snowmass Village, United States
Duration: 1 Mar 20205 Mar 2020


Abbreviated titleWACV 2020
Country/TerritoryUnited States
CitySnowmass Village

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Computer Science Applications

Cite this