Instance segmentation and tracking with cosine embeddings and recurrent hourglass networks

Christian Payer*, Darko Štern, Thomas Neff, Horst Bischof, Martin Urschler

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review

Abstract

Different to semantic segmentation, instance segmentation assigns unique labels to each individual instance of the same class. In this work, we propose a novel recurrent fully convolutional network architecture for tracking such instance segmentations over time. The network architecture incorporates convolutional gated recurrent units (ConvGRU) into a stacked hourglass network to utilize temporal video information. Furthermore, we train the network with a novel embedding loss based on cosine similarities, such that the network predicts unique embeddings for every instance throughout videos. Afterwards, these embeddings are clustered among subsequent video frames to create the final tracked instance segmentations. We evaluate the recurrent hourglass network by segmenting left ventricles in MR videos of the heart, where it outperforms a network that does not incorporate video information. Furthermore, we show applicability of the cosine embedding loss for segmenting leaf instances on still images of plants. Finally, we evaluate the framework for instance segmentation and tracking on six datasets of the ISBI celltracking challenge, where it shows state-of-the-art performance.

Original languageEnglish
Title of host publicationMedical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings
PublisherSpringer Verlag Heidelberg
Pages3-11
Number of pages9
ISBN (Print)9783030009335
DOIs
Publication statusPublished - 16 Sept 2018
Event21st International Conference on Medical Image Computing and Computer Assisted Intervention: MICCAI 2018 - Granada, Spain
Duration: 16 Sept 201820 Sept 2018

Publication series

NameLecture Notes in Computer Science
Volume11071
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference21st International Conference on Medical Image Computing and Computer Assisted Intervention
Country/TerritorySpain
CityGranada
Period16/09/1820/09/18

Keywords

  • Cell
  • Embeddings
  • Instances
  • Recurrent
  • Segmentation
  • Tracking
  • Video

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fields of Expertise

  • Information, Communication & Computing

Cooperations

  • BioTechMed-Graz

Fingerprint

Dive into the research topics of 'Instance segmentation and tracking with cosine embeddings and recurrent hourglass networks'. Together they form a unique fingerprint.

Cite this