Regressing Heatmaps for Multiple Landmark Localization Using CNNs

Christian Payer; Darko Stern; Horst Bischof; Martin Urschler

doi:10.1007/978-3-319-46723-8_27

Regressing Heatmaps for Multiple Landmark Localization Using CNNs

Christian Payer, Darko Stern, Horst Bischof, Martin Urschler

Institute of Computer Graphics and Vision (7100)

Research output: Chapter in Book/Report/Conference proceeding › Chapter › peer-review

Abstract

We explore the applicability of deep convolutional neural networks (CNNs) for multiple landmark localization in medical image data. Exploiting the idea of regressing heatmaps for individual landmark locations, we investigate several fully convolutional 2D and 3D CNN architectures by training them in an end-to-end manner. We further propose a novel SpatialConfiguration-Net architecture that effectively combines accurate local appearance responses with spatial landmark configurations that model anatomical variation. Evaluation of our different architectures on 2D and 3D hand image datasets show that heatmap regression based on CNNs achieves state-of-the-art landmark localization performance, with SpatialConfiguration-Net being robust even in case of limited amounts of training data.

Original language	English
Title of host publication	Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016
Subtitle of host publication	19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II
Editors	Sebastien Ourselin, Leo Joskowicz, Mert R. Sabuncu, Gozde Unal, William Wells
Publisher	Springer International Publishing AG
Pages	230-238
Number of pages	9
Volume	9901
ISBN (Electronic)	978-3-319-46723-8
ISBN (Print)	978-3-319-46722-1
DOIs	https://doi.org/10.1007/978-3-319-46723-8_27
Publication status	Published - 21 Oct 2016
Event	19th International Conference on Medical Image Computing & Computer Assisted Intervention: MICCAI 2016 - Intercontinental Athenaeum, Athens, Greece Duration: 17 Oct 2016 → 21 Oct 2016 http://www.miccai2016.org

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer

Conference

Conference	19th International Conference on Medical Image Computing & Computer Assisted Intervention
Abbreviated title	MICCAI
Country/Territory	Greece
City	Athens
Period	17/10/16 → 21/10/16
Internet address	http://www.miccai2016.org

Fields of Expertise

Information, Communication & Computing

Cooperations

BioTechMed-Graz

Access to Document

10.1007/978-3-319-46723-8_27

FWF - FAME - Fully Automatic MRI-based Age Estimation of Adolescents
Bischof, H. & Urschler, M.
1/07/15 → 31/12/18
Project: Research project
Klinisch-Forensische Bildgebung - Developing methods to standardize computer-assisted medical MRI datas [Original in Deutsch: Klinisch-Forensische Bildgebung] - Standard_MRT
Urschler, M., Bornik, A. & Bischof, H.
1/12/08 → 31/05/15
Project: Research project

Student Travel Award
Payer, Christian (Recipient), 2016
Prize: Prizes / Medals / Awards

Cite this

Payer, C., Stern, D., Bischof, H., & Urschler, M. (2016). Regressing Heatmaps for Multiple Landmark Localization Using CNNs. In S. Ourselin, L. Joskowicz, M. R. Sabuncu, G. Unal, & W. Wells (Eds.), Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II (Vol. 9901, pp. 230-238). (Lecture Notes in Computer Science). Springer International Publishing AG . https://doi.org/10.1007/978-3-319-46723-8_27

Regressing Heatmaps for Multiple Landmark Localization Using CNNs. / Payer, Christian; Stern, Darko; Bischof, Horst et al.
Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II. ed. / Sebastien Ourselin; Leo Joskowicz; Mert R. Sabuncu; Gozde Unal; William Wells. Vol. 9901 Springer International Publishing AG , 2016. p. 230-238 (Lecture Notes in Computer Science).

Research output: Chapter in Book/Report/Conference proceeding › Chapter › peer-review

Payer, C, Stern, D, Bischof, H & Urschler, M 2016, Regressing Heatmaps for Multiple Landmark Localization Using CNNs. in S Ourselin, L Joskowicz, MR Sabuncu, G Unal & W Wells (eds), Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II. vol. 9901, Lecture Notes in Computer Science, Springer International Publishing AG , pp. 230-238, 19th International Conference on Medical Image Computing & Computer Assisted Intervention, Athens, Greece, 17/10/16. https://doi.org/10.1007/978-3-319-46723-8_27

Payer C, Stern D, Bischof H, Urschler M. Regressing Heatmaps for Multiple Landmark Localization Using CNNs. In Ourselin S, Joskowicz L, Sabuncu MR, Unal G, Wells W, editors, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II. Vol. 9901. Springer International Publishing AG . 2016. p. 230-238. (Lecture Notes in Computer Science). doi: 10.1007/978-3-319-46723-8_27

Payer, Christian ; Stern, Darko ; Bischof, Horst et al. / Regressing Heatmaps for Multiple Landmark Localization Using CNNs. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II. editor / Sebastien Ourselin ; Leo Joskowicz ; Mert R. Sabuncu ; Gozde Unal ; William Wells. Vol. 9901 Springer International Publishing AG , 2016. pp. 230-238 (Lecture Notes in Computer Science).

@inbook{6bc99be5c4004c8b90832556bf865de2,

title = "Regressing Heatmaps for Multiple Landmark Localization Using CNNs",

abstract = "We explore the applicability of deep convolutional neural networks (CNNs) for multiple landmark localization in medical image data. Exploiting the idea of regressing heatmaps for individual landmark locations, we investigate several fully convolutional 2D and 3D CNN architectures by training them in an end-to-end manner. We further propose a novel SpatialConfiguration-Net architecture that effectively combines accurate local appearance responses with spatial landmark configurations that model anatomical variation. Evaluation of our different architectures on 2D and 3D hand image datasets show that heatmap regression based on CNNs achieves state-of-the-art landmark localization performance, with SpatialConfiguration-Net being robust even in case of limited amounts of training data.",

author = "Christian Payer and Darko Stern and Horst Bischof and Martin Urschler",

year = "2016",

month = oct,

day = "21",

doi = "10.1007/978-3-319-46723-8_27",

language = "English",

isbn = "978-3-319-46722-1",

volume = "9901",

series = "Lecture Notes in Computer Science",

publisher = "Springer International Publishing AG ",

pages = "230--238",

editor = "Sebastien Ourselin and Leo Joskowicz and Sabuncu, {Mert R.} and Gozde Unal and William Wells",

booktitle = "Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016",

address = "Switzerland",

note = "19th International Conference on Medical Image Computing & Computer Assisted Intervention : MICCAI 2016, MICCAI ; Conference date: 17-10-2016 Through 21-10-2016",

url = "http://www.miccai2016.org",

}

TY - CHAP

T1 - Regressing Heatmaps for Multiple Landmark Localization Using CNNs

AU - Payer, Christian

AU - Stern, Darko

AU - Bischof, Horst

AU - Urschler, Martin

PY - 2016/10/21

Y1 - 2016/10/21

N2 - We explore the applicability of deep convolutional neural networks (CNNs) for multiple landmark localization in medical image data. Exploiting the idea of regressing heatmaps for individual landmark locations, we investigate several fully convolutional 2D and 3D CNN architectures by training them in an end-to-end manner. We further propose a novel SpatialConfiguration-Net architecture that effectively combines accurate local appearance responses with spatial landmark configurations that model anatomical variation. Evaluation of our different architectures on 2D and 3D hand image datasets show that heatmap regression based on CNNs achieves state-of-the-art landmark localization performance, with SpatialConfiguration-Net being robust even in case of limited amounts of training data.

AB - We explore the applicability of deep convolutional neural networks (CNNs) for multiple landmark localization in medical image data. Exploiting the idea of regressing heatmaps for individual landmark locations, we investigate several fully convolutional 2D and 3D CNN architectures by training them in an end-to-end manner. We further propose a novel SpatialConfiguration-Net architecture that effectively combines accurate local appearance responses with spatial landmark configurations that model anatomical variation. Evaluation of our different architectures on 2D and 3D hand image datasets show that heatmap regression based on CNNs achieves state-of-the-art landmark localization performance, with SpatialConfiguration-Net being robust even in case of limited amounts of training data.

U2 - 10.1007/978-3-319-46723-8_27

DO - 10.1007/978-3-319-46723-8_27

M3 - Chapter

SN - 978-3-319-46722-1

VL - 9901

T3 - Lecture Notes in Computer Science

SP - 230

EP - 238

BT - Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016

A2 - Ourselin, Sebastien

A2 - Joskowicz, Leo

A2 - Sabuncu, Mert R.

A2 - Unal, Gozde

A2 - Wells, William

PB - Springer International Publishing AG

T2 - 19th International Conference on Medical Image Computing & Computer Assisted Intervention

Y2 - 17 October 2016 through 21 October 2016

ER -

Regressing Heatmaps for Multiple Landmark Localization Using CNNs

Abstract

Publication series

Conference

Fields of Expertise

Cooperations

Access to Document

Projects

FWF - FAME - Fully Automatic MRI-based Age Estimation of Adolescents

Klinisch-Forensische Bildgebung - Developing methods to standardize computer-assisted medical MRI datas [Original in Deutsch: Klinisch-Forensische Bildgebung] - Standard_MRT

Prizes

Student Travel Award

Cite this