Observation-based pedestrian scenario extraction for virtual testing

An overall reduction of pedestrian-vehicle collisions is expected with the market penetration of Advanced Driver Assistant System (ADAS) and autonomous driving (AD) functions. The performance of ADAS is commonly evaluated through virtual scenario-based testing. Hence, scenario catalogs that represent realistic pedestrian-vehicle interactions are needed.
This study shows an approach to automatically extract pedestrian-vehicle scenarios at a selected road intersection, which was observed with a dual-lens stationary observation system. A deep learning-based visual perception pipeline was implemented to reconstruct road user trajectories via state-of-the-art object detection, visual multi-object tracking and object re-identification models. These models were trained and fine-tuned on a manually annotated, diverse dataset, randomly sampled from recordings over multiple weeks. All models were evaluated using common performance metrics. Additionally, localization precision of reconstructed trajectories was assessed using georeferenced ground truth measurements conducted at the intersection. The visual perception pipeline was applied on selected video data and extracted trajectories converted according to the openSCENARIO standard, including a virtual representation of the selected road intersection. The compiled scenarios were further simulated with the openPASS framework.
The results show that pedestrians and vehicles were tracked with high accuracy (Multiple Object Tracking Accuracy > 83.2%) and trajectories were reconstructed with a mean deviation of 0.9 m for pedestrians and vehicle paths with a deviation of 0.68 (SD 0.5) m. The observation system allowed both the obtaining of typical pathways and also speed profiles. An exemplary reconstructed scenario was successfully resimulated in the openPASS framework.
The described approach is promising and can be used to create new scenario catalogs for scenario-based assessments in line with the openSCENARIO standard. Furthermore, the viewpoint of the observation system allows the reconstruction of pedestrian attributes including poses, age or gender, which, alongside an analysis of the recorded pathways and speed profiles with respect to influencing factors, is a focus of ongoing research.