In-Situ Virtual Reality Tour using a real-time 360° Video Stream

Simon Fussi

Research output: ThesisBachelor's Thesis

Abstract

With our project, we developed a system capable of live streaming immersive video footage from a simple handheld 360◦ camera to a modern VR Headset. The chosen hardware focuses on providing a lightweight, small-footprint solution allowing us to mount the camera on an RC-Car, thus providing a first-person view of the cockpit. Additionally, the steering servo and cruise control of the RC-Car are controllable via an ESP32 listening to MQTT messages of the VR Headset’s controller input. The implemented software uses highly efficient-hardware accelerated encoding and decoding solutions featuring H.264 video compression standard for rendering. The live stream from the camera is delivered via RTSP(Real Time Streaming Protocol) to the end user’s Oculus Quest 2 VR Headset. The client decodes the video and stitches the equirectangular projected footage into a sphere that fully encloses the Viewpoint in the virtual world. Thus, via simple head movement, the user can look around in his new virtual world represented by the live video footage of the camera.
Original languageEnglish
Awarding Institution
  • Institute of Computer Graphics and Vision (7100)
Supervisors/Advisors
  • Arth, Clemens, Supervisor
  • Fleck, Philipp, Supervisor
Award date9 May 2023
Publication statusSubmitted - 9 May 2023

Keywords

  • Virtual reality (VR)
  • 360°VR
  • MQTT
  • real-time
  • RTMP
  • RTSP
  • video streaming

Cite this