FuseSeg: LiDAR Point Cloud Segmentation Fusing Multi-Modal Data

Georg Krispel; Michael Opitz; Georg Waltner; Horst Possegger; Horst Bischof

doi:10.1109/WACV45572.2020.9093584

FuseSeg: LiDAR Point Cloud Segmentation Fusing Multi-Modal Data

Georg Krispel, Michael Opitz, Georg Waltner, Horst Possegger, Horst Bischof

Institute of Computer Graphics and Vision (7100)

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

Abstract

We introduce a simple yet effective fusion method of LiDAR and RGB data to segment LiDAR point clouds. Utilizing the dense native range representation of a LiDAR sensor and the setup calibration, we establish point correspondences between the two input modalities. Subsequently, we are able to warp and fuse the features from one domain into the other. Therefore, we can jointly exploit information from both data sources within one single network. To show the merit of our method, we extend SqueezeSeg, a point cloud segmentation network, with an RGB feature branch and fuse it into the original structure. Our extension called FuseSeg leads to an improvement of up to 18% IoU on the KITTI benchmark. In addition to the improved accuracy, we also achieve real-time performance at 50 fps, five times as fast as the recording speed of the KITTI LiDAR data.

Original language	English
Title of host publication	Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020
Pages	1863-1872
Number of pages	10
ISBN (Electronic)	9781728165530
DOIs	https://doi.org/10.1109/WACV45572.2020.9093584
Publication status	Published - Mar 2020
Event	wacv2020: WACV 2020 - Snowmass Village, United States Duration: 1 Mar 2020 → 5 Mar 2020

Conference

Conference	wacv2020
Abbreviated title	WACV 2020
Country/Territory	United States
City	Snowmass Village
Period	1/03/20 → 5/03/20

ASJC Scopus subject areas

Computer Vision and Pattern Recognition
Computer Science Applications

Access to Document

10.1109/WACV45572.2020.9093584

Cite this

@inproceedings{ffdd6b39b4e14a50b237fbe0abb81955,

title = "FuseSeg: LiDAR Point Cloud Segmentation Fusing Multi-Modal Data",

abstract = "We introduce a simple yet effective fusion method of LiDAR and RGB data to segment LiDAR point clouds. Utilizing the dense native range representation of a LiDAR sensor and the setup calibration, we establish point correspondences between the two input modalities. Subsequently, we are able to warp and fuse the features from one domain into the other. Therefore, we can jointly exploit information from both data sources within one single network. To show the merit of our method, we extend SqueezeSeg, a point cloud segmentation network, with an RGB feature branch and fuse it into the original structure. Our extension called FuseSeg leads to an improvement of up to 18% IoU on the KITTI benchmark. In addition to the improved accuracy, we also achieve real-time performance at 50 fps, five times as fast as the recording speed of the KITTI LiDAR data.",

author = "Georg Krispel and Michael Opitz and Georg Waltner and Horst Possegger and Horst Bischof",

year = "2020",

month = mar,

doi = "10.1109/WACV45572.2020.9093584",

language = "English",

pages = "1863--1872",

booktitle = "Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020",

note = "wacv2020 : WACV 2020, WACV 2020 ; Conference date: 01-03-2020 Through 05-03-2020",

}

TY - GEN

T1 - FuseSeg: LiDAR Point Cloud Segmentation Fusing Multi-Modal Data

AU - Krispel, Georg

AU - Opitz, Michael

AU - Waltner, Georg

AU - Possegger, Horst

AU - Bischof, Horst

PY - 2020/3

Y1 - 2020/3

N2 - We introduce a simple yet effective fusion method of LiDAR and RGB data to segment LiDAR point clouds. Utilizing the dense native range representation of a LiDAR sensor and the setup calibration, we establish point correspondences between the two input modalities. Subsequently, we are able to warp and fuse the features from one domain into the other. Therefore, we can jointly exploit information from both data sources within one single network. To show the merit of our method, we extend SqueezeSeg, a point cloud segmentation network, with an RGB feature branch and fuse it into the original structure. Our extension called FuseSeg leads to an improvement of up to 18% IoU on the KITTI benchmark. In addition to the improved accuracy, we also achieve real-time performance at 50 fps, five times as fast as the recording speed of the KITTI LiDAR data.

AB - We introduce a simple yet effective fusion method of LiDAR and RGB data to segment LiDAR point clouds. Utilizing the dense native range representation of a LiDAR sensor and the setup calibration, we establish point correspondences between the two input modalities. Subsequently, we are able to warp and fuse the features from one domain into the other. Therefore, we can jointly exploit information from both data sources within one single network. To show the merit of our method, we extend SqueezeSeg, a point cloud segmentation network, with an RGB feature branch and fuse it into the original structure. Our extension called FuseSeg leads to an improvement of up to 18% IoU on the KITTI benchmark. In addition to the improved accuracy, we also achieve real-time performance at 50 fps, five times as fast as the recording speed of the KITTI LiDAR data.

UR - http://www.scopus.com/inward/record.url?scp=85085515594&partnerID=8YFLogxK

U2 - 10.1109/WACV45572.2020.9093584

DO - 10.1109/WACV45572.2020.9093584

M3 - Conference paper

SP - 1863

EP - 1872

BT - Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020

T2 - wacv2020

Y2 - 1 March 2020 through 5 March 2020

ER -

FuseSeg: LiDAR Point Cloud Segmentation Fusing Multi-Modal Data

Abstract

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint