General 3D Room Layout from a Single View by Render-and-Compare

Sinisa Stekovic; Shreyas Hampali Shivakumar; Mahdi Rad; Sayan Deb Sarkar; Friedrich Fraundorfer; Vincent Lepetit

doi:10.1007/978-3-030-58517-4_12

General 3D Room Layout from a Single View by Render-and-Compare

Sinisa Stekovic^*, Shreyas Hampali Shivakumar, Mahdi Rad, Sayan Deb Sarkar, Friedrich Fraundorfer, Vincent Lepetit

^*Korrespondierende/r Autor/-in für diese Arbeit

Institut für Maschinelles Sehen und Darstellen (7100)

Publikation: Beitrag in Buch/Bericht/Konferenzband › Beitrag in einem Konferenzband › Begutachtung

Abstract

We present a novel method to reconstruct the 3D layout of a room—walls, floors, ceilings—from a single perspective view in challenging conditions, by contrast with previous single-view methods restricted to cuboid-shaped layouts. This input view can consist of a color image only, but considering a depth map results in a more accurate reconstruction. Our approach is formalized as solving a constrained discrete optimization problem to find the set of 3D polygons that constitute the layout. In order to deal with occlusions between components of the layout, which is a problem ignored by previous works, we introduce an analysis-by-synthesis method to iteratively refine the 3D layout estimate. As no dataset was available to evaluate our method quantitatively, we created one together with several appropriate metrics. Our dataset consists of 293 images from ScanNet, which we annotated with precise 3D layouts. It offers three times more samples than the popular NYUv2 303 benchmark, and a much larger variety of layouts.

Originalsprache	englisch
Titel	Computer Vision – ECCV 2020 - 16th European Conference, Proceedings
Redakteure/-innen	Andrea Vedaldi, Horst Bischof, Thomas Brox, Jan-Michael Frahm
Herausgeber (Verlag)	Springer, Cham
Seiten	187-203
Seitenumfang	17
ISBN (Print)	9783030585167
DOIs	https://doi.org/10.1007/978-3-030-58517-4_12
Publikationsstatus	Veröffentlicht - 23 Aug. 2020
Veranstaltung	16th European Conference on Computer Vision: ECCV 2020 - Virtual, Glasgow, Großbritannien / Vereinigtes Königreich Dauer: 23 Aug. 2020 → 28 Aug. 2020

Publikationsreihe

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Band	12361 LNCS
ISSN (Print)	0302-9743
ISSN (elektronisch)	1611-3349

Konferenz

Konferenz	16th European Conference on Computer Vision
Kurztitel	ECCV 2020
Land/Gebiet	Großbritannien / Vereinigtes Königreich
Ort	Virtual, Glasgow
Zeitraum	23/08/20 → 28/08/20

ASJC Scopus subject areas

Theoretische Informatik
Allgemeine Computerwissenschaft

Zugriff auf Dokument

10.1007/978-3-030-58517-4_12

Andere Dateien und Links

http://www.scopus.com/inward/record.url?scp=85092935640&partnerID=8YFLogxK

Dieses zitieren

Stekovic, S., Hampali Shivakumar, S., Rad, M., Deb Sarkar, S., Fraundorfer, F., & Lepetit, V. (2020). General 3D Room Layout from a Single View by Render-and-Compare. in A. Vedaldi, H. Bischof, T. Brox, & J.-M. Frahm (Hrsg.), Computer Vision – ECCV 2020 - 16th European Conference, Proceedings (S. 187-203). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Band 12361 LNCS). Springer, Cham. https://doi.org/10.1007/978-3-030-58517-4_12

General 3D Room Layout from a Single View by Render-and-Compare. / Stekovic, Sinisa; Hampali Shivakumar, Shreyas; Rad, Mahdi et al.
Computer Vision – ECCV 2020 - 16th European Conference, Proceedings. Hrsg. / Andrea Vedaldi; Horst Bischof; Thomas Brox; Jan-Michael Frahm. Springer, Cham, 2020. S. 187-203 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Band 12361 LNCS).

Publikation: Beitrag in Buch/Bericht/Konferenzband › Beitrag in einem Konferenzband › Begutachtung

Stekovic, S, Hampali Shivakumar, S, Rad, M, Deb Sarkar, S, Fraundorfer, F & Lepetit, V 2020, General 3D Room Layout from a Single View by Render-and-Compare. in A Vedaldi, H Bischof, T Brox & J-M Frahm (Hrsg.), Computer Vision – ECCV 2020 - 16th European Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Bd. 12361 LNCS, Springer, Cham, S. 187-203, 16th European Conference on Computer Vision, Virtual, Glasgow, Großbritannien / Vereinigtes Königreich, 23/08/20. https://doi.org/10.1007/978-3-030-58517-4_12

Stekovic S, Hampali Shivakumar S, Rad M, Deb Sarkar S, Fraundorfer F, Lepetit V. General 3D Room Layout from a Single View by Render-and-Compare. in Vedaldi A, Bischof H, Brox T, Frahm JM, Hrsg., Computer Vision – ECCV 2020 - 16th European Conference, Proceedings. Springer, Cham. 2020. S. 187-203. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-58517-4_12

Stekovic, Sinisa ; Hampali Shivakumar, Shreyas ; Rad, Mahdi et al. / General 3D Room Layout from a Single View by Render-and-Compare. Computer Vision – ECCV 2020 - 16th European Conference, Proceedings. Hrsg. / Andrea Vedaldi ; Horst Bischof ; Thomas Brox ; Jan-Michael Frahm. Springer, Cham, 2020. S. 187-203 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{f35fc132e13e4a7ea84d209991f99611,

title = "General 3D Room Layout from a Single View by Render-and-Compare",

abstract = "We present a novel method to reconstruct the 3D layout of a room—walls, floors, ceilings—from a single perspective view in challenging conditions, by contrast with previous single-view methods restricted to cuboid-shaped layouts. This input view can consist of a color image only, but considering a depth map results in a more accurate reconstruction. Our approach is formalized as solving a constrained discrete optimization problem to find the set of 3D polygons that constitute the layout. In order to deal with occlusions between components of the layout, which is a problem ignored by previous works, we introduce an analysis-by-synthesis method to iteratively refine the 3D layout estimate. As no dataset was available to evaluate our method quantitatively, we created one together with several appropriate metrics. Our dataset consists of 293 images from ScanNet, which we annotated with precise 3D layouts. It offers three times more samples than the popular NYUv2 303 benchmark, and a much larger variety of layouts.",

keywords = "3D geometry, Analysis-by-synthesis, Room layout",

author = "Sinisa Stekovic and {Hampali Shivakumar}, Shreyas and Mahdi Rad and {Deb Sarkar}, Sayan and Friedrich Fraundorfer and Vincent Lepetit",

year = "2020",

month = aug,

day = "23",

doi = "10.1007/978-3-030-58517-4_12",

language = "English",

isbn = "9783030585167",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer, Cham",

pages = "187--203",

editor = "Andrea Vedaldi and Horst Bischof and Thomas Brox and Jan-Michael Frahm",

booktitle = "Computer Vision – ECCV 2020 - 16th European Conference, Proceedings",

note = "16th European Conference on Computer Vision : ECCV 2020, ECCV 2020 ; Conference date: 23-08-2020 Through 28-08-2020",

}

TY - GEN

T1 - General 3D Room Layout from a Single View by Render-and-Compare

AU - Stekovic, Sinisa

AU - Hampali Shivakumar, Shreyas

AU - Rad, Mahdi

AU - Deb Sarkar, Sayan

AU - Fraundorfer, Friedrich

AU - Lepetit, Vincent

PY - 2020/8/23

Y1 - 2020/8/23

N2 - We present a novel method to reconstruct the 3D layout of a room—walls, floors, ceilings—from a single perspective view in challenging conditions, by contrast with previous single-view methods restricted to cuboid-shaped layouts. This input view can consist of a color image only, but considering a depth map results in a more accurate reconstruction. Our approach is formalized as solving a constrained discrete optimization problem to find the set of 3D polygons that constitute the layout. In order to deal with occlusions between components of the layout, which is a problem ignored by previous works, we introduce an analysis-by-synthesis method to iteratively refine the 3D layout estimate. As no dataset was available to evaluate our method quantitatively, we created one together with several appropriate metrics. Our dataset consists of 293 images from ScanNet, which we annotated with precise 3D layouts. It offers three times more samples than the popular NYUv2 303 benchmark, and a much larger variety of layouts.

AB - We present a novel method to reconstruct the 3D layout of a room—walls, floors, ceilings—from a single perspective view in challenging conditions, by contrast with previous single-view methods restricted to cuboid-shaped layouts. This input view can consist of a color image only, but considering a depth map results in a more accurate reconstruction. Our approach is formalized as solving a constrained discrete optimization problem to find the set of 3D polygons that constitute the layout. In order to deal with occlusions between components of the layout, which is a problem ignored by previous works, we introduce an analysis-by-synthesis method to iteratively refine the 3D layout estimate. As no dataset was available to evaluate our method quantitatively, we created one together with several appropriate metrics. Our dataset consists of 293 images from ScanNet, which we annotated with precise 3D layouts. It offers three times more samples than the popular NYUv2 303 benchmark, and a much larger variety of layouts.

KW - 3D geometry

KW - Analysis-by-synthesis

KW - Room layout

UR - http://www.scopus.com/inward/record.url?scp=85092935640&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-58517-4_12

DO - 10.1007/978-3-030-58517-4_12

M3 - Conference paper

SN - 9783030585167

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 187

EP - 203

BT - Computer Vision – ECCV 2020 - 16th European Conference, Proceedings

A2 - Vedaldi, Andrea

A2 - Bischof, Horst

A2 - Brox, Thomas

A2 - Frahm, Jan-Michael

PB - Springer, Cham

T2 - 16th European Conference on Computer Vision

Y2 - 23 August 2020 through 28 August 2020

ER -

General 3D Room Layout from a Single View by Render-and-Compare

Abstract

Publikationsreihe

Konferenz

ASJC Scopus subject areas

Zugriff auf Dokument

Andere Dateien und Links

Fingerprint

Dieses zitieren