Projekte pro Jahr
Abstract
Many industrial sectors rely on well-trained employees that are able to operate complex machinery. In this work, we demon- strate an immersive assistance system powered by Artificial Intelli- gence (AI) that supports users in performing complex tasks in in- dustrial environments. Our system leverages a Virtual Reality (VR) environment that resembles a juice mixer setup. This digital twin of a physical setup simulates complex industrial machinery used to mix preparations or liquids (e.g., similar to the pharmaceutical indus- try) and includes various containers, sensors, pumps, and flow con- trollers. This setup demonstrates our system’s capabilities in a con- trolled environment while acting as a proof-of-concept for broader industrial applications. The core components of our multimodal AI assistant are a large language model and a speech-to-text model that process a video and audio recording of an expert performing the task in a VR environment. The video and speech input extracted from the expert’s video enables it to provide step-by-step guidance to support users in executing complex tasks. This demonstration showcases the potential of our AI-powered assistant to reduce cognitive load, in- crease productivity, and enhance safety in industrial environments.
Originalsprache | englisch |
---|---|
Titel | 27TH EUROPEAN CONFERENCE ON ARTIFICIAL INTELLIGENCE |
Publikationsstatus | Veröffentlicht - 20 Okt. 2024 |
ASJC Scopus subject areas
- Artificial intelligence
- Information systems
Projekte
- 1 Laufend
-
DDIA - Datengesteuerte immersive Analytik in der digitalen Industrie
Müller-Putz, G. (Teilnehmer (Co-Investigator)), Schreck, T. (Teilnehmer (Co-Investigator)), Lindstaedt, S. (Teilnehmer (Co-Investigator)), Pock, T. (Teilnehmer (Co-Investigator)), Veas, E. E. (Teilnehmer (Co-Investigator)), Pammer-Schindler, V. (Teilnehmer (Co-Investigator)) & Kowald, D. (Teilnehmer (Co-Investigator))
1/01/22 → 31/12/26
Projekt: Forschungsprojekt