TPDNet: A Tiny Pupil Detection Neural Network for Embedded Machine Learning Processor Arm Ethos-U55

Gernot Fiala; Zhenyu Ye; Christian Steger

TPDNet: A Tiny Pupil Detection Neural Network for Embedded Machine Learning Processor Arm Ethos-U55

Gernot Fiala, Zhenyu Ye, Christian Steger

Institut für Technische Informatik (4480)

Publikation: Beitrag in Buch/Bericht/Konferenzband › Beitrag in einem Konferenzband › Begutachtung

Abstract

Abstract. Augmented reality and virtual reality (AR/VR) systems contain several different sensors including images sensors for gesture recognition, head pose tracking and pupil/eye tracking. The data of all these sensors must be processed by a host processor in real-time. For future AR/VR systems, new sensing technologies are required to fulfill the demands in power consumption and performance. Currently pupil detection is performed with images on resolutions around 300x300 pixels
and above. Therefore, deep neural networks (DNN) need host platforms, which are capable to compute the DNNs with such input resolutions to process them in real-time. In this work, the image resolution for pupil detection is optimized to a resolution of 100x100 pixels. A tiny pupil detection neural network is introduced, which can be processed with the ARM Cortex-M55 and the Embedded Machine Learning (ML) processor Arm Ethos-U55 with a performance of 189 frames per second (FPS)
with high detection rates. This allows to reduce the power consumption of the communication between image sensor and host for future AR/VR devices.

Originalsprache	englisch
Titel	Intelligent Systems and Applications - IntelliSys 2023
Untertitel	Proceedings of 2023 Intelligent Systems Conference (IntelliSys)
Herausgeber (Verlag)	Springer Nature Switzerland AG
Publikationsstatus	Angenommen/In Druck - 17 Mai 2023
Veranstaltung	Intelligent Systems Conference 2023: IntelliSys 2023 - Van der Valk Hotel Amsterdam - Amstel, Amsterdam, Niederlande Dauer: 7 Sept. 2023 → 8 Sept. 2023 https://saiconference.com/Conferences/IntelliSys2023

Publikationsreihe

Name	Lecture Notes in Networks and Systems
Herausgeber (Verlag)	Springer

Konferenz

Konferenz	Intelligent Systems Conference 2023
Kurztitel	IntelliSys 2023
Land/Gebiet	Niederlande
Ort	Amsterdam
Zeitraum	7/09/23 → 8/09/23
Internetadresse	https://saiconference.com/Conferences/IntelliSys2023

ASJC Scopus subject areas

Artificial intelligence
Hardware und Architektur

1 Konferenz oder Fachtagung (Teilnahme an/Organisation von)
1 Vortrag bei Konferenz oder Fachtagung

TPDNet: A Tiny Pupil Detection Neural Network for Embedded Machine Learning Processor Arm Ethos-U55
Gernot Fiala (Redner/in)
7 Sept. 2023 → 8 Sept. 2023
Aktivität: Vortrag oder Präsentation › Vortrag bei Konferenz oder Fachtagung › Science to science
Intelligent Systems Conference 2023
Gernot Fiala (Teilnehmer/-in)
7 Sept. 2023 → 8 Sept. 2023
Aktivität: Teilnahme an / Organisation von › Konferenz oder Fachtagung (Teilnahme an/Organisation von)

Dieses zitieren

TPDNet: A Tiny Pupil Detection Neural Network for Embedded Machine Learning Processor Arm Ethos-U55. / Fiala, Gernot; Ye, Zhenyu; Steger, Christian.
Intelligent Systems and Applications - IntelliSys 2023: Proceedings of 2023 Intelligent Systems Conference (IntelliSys). Springer Nature Switzerland AG, 2023. (Lecture Notes in Networks and Systems).

Publikation: Beitrag in Buch/Bericht/Konferenzband › Beitrag in einem Konferenzband › Begutachtung

Fiala, G, Ye, Z & Steger, C 2023, TPDNet: A Tiny Pupil Detection Neural Network for Embedded Machine Learning Processor Arm Ethos-U55. in Intelligent Systems and Applications - IntelliSys 2023: Proceedings of 2023 Intelligent Systems Conference (IntelliSys). Lecture Notes in Networks and Systems, Springer Nature Switzerland AG, Intelligent Systems Conference 2023, Amsterdam, Niederlande, 7/09/23.

@inproceedings{68952c48c52c492aa4e1a79fd77e581d,

title = "TPDNet: A Tiny Pupil Detection Neural Network for Embedded Machine Learning Processor Arm Ethos-U55",

abstract = "Abstract. Augmented reality and virtual reality (AR/VR) systems contain several different sensors including images sensors for gesture recognition, head pose tracking and pupil/eye tracking. The data of all these sensors must be processed by a host processor in real-time. For future AR/VR systems, new sensing technologies are required to fulfill the demands in power consumption and performance. Currently pupil detection is performed with images on resolutions around 300x300 pixelsand above. Therefore, deep neural networks (DNN) need host platforms, which are capable to compute the DNNs with such input resolutions to process them in real-time. In this work, the image resolution for pupil detection is optimized to a resolution of 100x100 pixels. A tiny pupil detection neural network is introduced, which can be processed with the ARM Cortex-M55 and the Embedded Machine Learning (ML) processor Arm Ethos-U55 with a performance of 189 frames per second (FPS)with high detection rates. This allows to reduce the power consumption of the communication between image sensor and host for future AR/VR devices.",

keywords = "Arm Ethos-U55, Arm M55, augmented reality, deep neural networks, machine learning, neural network processor, pupil detection, virtual reality",

author = "Gernot Fiala and Zhenyu Ye and Christian Steger",

year = "2023",

month = may,

day = "17",

language = "English",

series = "Lecture Notes in Networks and Systems",

publisher = "Springer Nature Switzerland AG",

booktitle = "Intelligent Systems and Applications - IntelliSys 2023",

address = "Switzerland",

note = "Intelligent Systems Conference 2023 : IntelliSys 2023, IntelliSys 2023 ; Conference date: 07-09-2023 Through 08-09-2023",

url = "https://saiconference.com/Conferences/IntelliSys2023",

}

TY - GEN

T1 - TPDNet: A Tiny Pupil Detection Neural Network for Embedded Machine Learning Processor Arm Ethos-U55

AU - Fiala, Gernot

AU - Ye, Zhenyu

AU - Steger, Christian

PY - 2023/5/17

Y1 - 2023/5/17

N2 - Abstract. Augmented reality and virtual reality (AR/VR) systems contain several different sensors including images sensors for gesture recognition, head pose tracking and pupil/eye tracking. The data of all these sensors must be processed by a host processor in real-time. For future AR/VR systems, new sensing technologies are required to fulfill the demands in power consumption and performance. Currently pupil detection is performed with images on resolutions around 300x300 pixelsand above. Therefore, deep neural networks (DNN) need host platforms, which are capable to compute the DNNs with such input resolutions to process them in real-time. In this work, the image resolution for pupil detection is optimized to a resolution of 100x100 pixels. A tiny pupil detection neural network is introduced, which can be processed with the ARM Cortex-M55 and the Embedded Machine Learning (ML) processor Arm Ethos-U55 with a performance of 189 frames per second (FPS)with high detection rates. This allows to reduce the power consumption of the communication between image sensor and host for future AR/VR devices.

AB - Abstract. Augmented reality and virtual reality (AR/VR) systems contain several different sensors including images sensors for gesture recognition, head pose tracking and pupil/eye tracking. The data of all these sensors must be processed by a host processor in real-time. For future AR/VR systems, new sensing technologies are required to fulfill the demands in power consumption and performance. Currently pupil detection is performed with images on resolutions around 300x300 pixelsand above. Therefore, deep neural networks (DNN) need host platforms, which are capable to compute the DNNs with such input resolutions to process them in real-time. In this work, the image resolution for pupil detection is optimized to a resolution of 100x100 pixels. A tiny pupil detection neural network is introduced, which can be processed with the ARM Cortex-M55 and the Embedded Machine Learning (ML) processor Arm Ethos-U55 with a performance of 189 frames per second (FPS)with high detection rates. This allows to reduce the power consumption of the communication between image sensor and host for future AR/VR devices.

KW - Arm Ethos-U55

KW - Arm M55

KW - augmented reality

KW - deep neural networks

KW - machine learning

KW - neural network processor

KW - pupil detection

KW - virtual reality

M3 - Conference paper

T3 - Lecture Notes in Networks and Systems

BT - Intelligent Systems and Applications - IntelliSys 2023

PB - Springer Nature Switzerland AG

T2 - Intelligent Systems Conference 2023

Y2 - 7 September 2023 through 8 September 2023

ER -

TPDNet: A Tiny Pupil Detection Neural Network for Embedded Machine Learning Processor Arm Ethos-U55

Abstract

Publikationsreihe

Konferenz

ASJC Scopus subject areas

Fingerprint

Aktivitäten

TPDNet: A Tiny Pupil Detection Neural Network for Embedded Machine Learning Processor Arm Ethos-U55

Intelligent Systems Conference 2023

Dieses zitieren