Activities per year
Abstract
Abstract. Augmented reality and virtual reality (AR/VR) systems contain several different sensors including images sensors for gesture recognition, head pose tracking and pupil/eye tracking. The data of all these sensors must be processed by a host processor in real-time. For future AR/VR systems, new sensing technologies are required to fulfill the demands in power consumption and performance. Currently pupil detection is performed with images on resolutions around 300x300 pixels
and above. Therefore, deep neural networks (DNN) need host platforms, which are capable to compute the DNNs with such input resolutions to process them in real-time. In this work, the image resolution for pupil detection is optimized to a resolution of 100x100 pixels. A tiny pupil detection neural network is introduced, which can be processed with the ARM Cortex-M55 and the Embedded Machine Learning (ML) processor Arm Ethos-U55 with a performance of 189 frames per second (FPS)
with high detection rates. This allows to reduce the power consumption of the communication between image sensor and host for future AR/VR devices.
and above. Therefore, deep neural networks (DNN) need host platforms, which are capable to compute the DNNs with such input resolutions to process them in real-time. In this work, the image resolution for pupil detection is optimized to a resolution of 100x100 pixels. A tiny pupil detection neural network is introduced, which can be processed with the ARM Cortex-M55 and the Embedded Machine Learning (ML) processor Arm Ethos-U55 with a performance of 189 frames per second (FPS)
with high detection rates. This allows to reduce the power consumption of the communication between image sensor and host for future AR/VR devices.
| Original language | English |
|---|---|
| Title of host publication | Intelligent Systems and Applications - IntelliSys 2023 |
| Subtitle of host publication | Proceedings of 2023 Intelligent Systems Conference (IntelliSys) |
| Publisher | Springer Nature Switzerland AG |
| Publication status | Accepted/In press - 17 May 2023 |
| Event | Intelligent Systems Conference 2023: IntelliSys 2023 - Van der Valk Hotel Amsterdam - Amstel, Amsterdam, Netherlands Duration: 7 Sept 2023 → 8 Sept 2023 https://saiconference.com/Conferences/IntelliSys2023 |
Publication series
| Name | Lecture Notes in Networks and Systems |
|---|---|
| Publisher | Springer |
Conference
| Conference | Intelligent Systems Conference 2023 |
|---|---|
| Abbreviated title | IntelliSys 2023 |
| Country/Territory | Netherlands |
| City | Amsterdam |
| Period | 7/09/23 → 8/09/23 |
| Internet address |
Keywords
- Arm Ethos-U55
- Arm M55
- augmented reality
- deep neural networks
- machine learning
- neural network processor
- pupil detection
- virtual reality
ASJC Scopus subject areas
- Artificial Intelligence
- Hardware and Architecture
Fingerprint
Dive into the research topics of 'TPDNet: A Tiny Pupil Detection Neural Network for Embedded Machine Learning Processor Arm Ethos-U55'. Together they form a unique fingerprint.Activities
-
TPDNet: A Tiny Pupil Detection Neural Network for Embedded Machine Learning Processor Arm Ethos-U55
Gernot Fiala (Speaker)
7 Sept 2023 → 8 Sept 2023Activity: Talk or presentation › Talk at conference or symposium › Science to science
-
Intelligent Systems Conference 2023
Gernot Fiala (Participant)
7 Sept 2023 → 8 Sept 2023Activity: Participation in or organisation of › Conference or symposium (Participation in/Organisation of)