Efficient 3D Tracking in Urban Environments with Semantic Segmentation

Martin Hirzer, Clemens Arth, Peter M. Roth, Vincent Lepetit

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review


In this paper, we present a new 3D tracking approach for self-localization in urban environments. In particular, we build on existing tracking approaches (i.e., visual odometry tracking and SLAM), additionally using the information provided by 2.5D maps of the environment. Since this combination is not straightforward, we adopt ideas from semantic segmentation to find a better alignment between the pose estimated by the tracker and the 2.5D model. Specifically, we show that introducing edges as semantic classes is highly beneficial for our task. In this way, we can reduce tracker inaccuracies and prevent drifting, thus increasing the tracker’s stability. We evaluate our approach for two different challenging scenarios, also showing that it is generally applicable in different application domains and that we are not limited to a specific tracking method.
Original languageEnglish
Title of host publicationProceedings of the British Machine Vision Conference (BMVC)
Publication statusPublished - 2017
Event28th British Machine Vision Conference: BMVC 2017 - London, United Kingdom
Duration: 4 Sept 20177 Apr 2018


Conference28th British Machine Vision Conference
Abbreviated titleBMVC 2017
Country/TerritoryUnited Kingdom


Dive into the research topics of 'Efficient 3D Tracking in Urban Environments with Semantic Segmentation'. Together they form a unique fingerprint.

Cite this