HAMMER: Learning Entropy Maps to Create Accurate 3D Models in Multi-View Stereo

Rafael Weilharter*, Friedrich Fraundorfer

*Korrespondierende/r Autor/-in für diese Arbeit

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem KonferenzbandBegutachtung

Abstract

While the majority of recent Multi-View Stereo Networks estimates a depth map per reference image, their performance is then only evaluated on the fused 3D model obtained from all images. This approach makes a lot of sense since ultimately the point cloud is the result we are mostly interested in. On the flip side, it often leads to a burdensome manual search for the right fusion parameters in order to score well on the public benchmarks. In this work, we tackle the aforementioned problem with HAMMER, a Hierarchical And Memory-efficient MVSNet with Entropy-filtered Reconstructions. We propose to learn a filtering mask based on entropy, which, in combination with a simple two-view geometric verification, is sufficient to generate high quality 3D models of any input scene. Distinct from existing works, a tedious manual parameter search for the fusion step is not required. Furthermore, we take several precautions to keep the memory requirements for our method very low in the training as well as in the inference phase. Our method only requires 6 GB of GPU memory during training, while 3.6 GB are enough to process 1920×1024 images during inference. Experiments show that HAMMER ranks amongst the top published methods on the DTU and Tanks and Temples benchmarks in the official metrics, especially when keeping the fusion parameters fixed.

Originalspracheenglisch
TitelProceedings - 2024 IEEE Winter Conference on Applications of Computer Vision, WACV 2024
Herausgeber (Verlag)IEEE
Seiten3454-3463
ISBN (elektronisch)9798350318920
DOIs
PublikationsstatusVeröffentlicht - 3 Jan. 2024
Veranstaltung2024 IEEE/CVF Winter Conference on Applications of Computer Vision: WACV 2024 - Waikoloa, USA / Vereinigte Staaten
Dauer: 4 Jan. 20248 Jan. 2024

Konferenz

Konferenz2024 IEEE/CVF Winter Conference on Applications of Computer Vision
KurztitelWACV 2024
Land/GebietUSA / Vereinigte Staaten
OrtWaikoloa
Zeitraum4/01/248/01/24

ASJC Scopus subject areas

  • Artificial intelligence
  • Angewandte Informatik
  • Maschinelles Sehen und Mustererkennung

Fingerprint

Untersuchen Sie die Forschungsthemen von „HAMMER: Learning Entropy Maps to Create Accurate 3D Models in Multi-View Stereo“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren