Scenario Based Testing of Automated Driving Systems: A Literature Survey

Market introduction of automated driving features several motivations including road safety, driving comfort, energy efficiency and totally new transport systems. However, many challenges are blocking it, including performance of the perception system, safety validation, legal and ethical issues, Human-Machine interaction and others. Especially the safety validation of SAE Level 3-5 systems in complex environments with respect to road and weather conditions call for totally new approaches and processes.
Scenario based methods for testing and validation of automated driving systems (ADS) in virtual test environments are gaining importance and becoming an essential component for verification and validation processes of ADS. The high system complexity and costs of real testing lead to an exponential increase of test efforts for real world testing. Using scenario and simulation based approaches this effort can be efficiently reduced with respect to costs and time. Research has shown that it is necessary to drive and test billions of kilometers to ensure safety of ADS which would not be rational considering the time and cost effort for real testing. The biggest challenges are the selection of a suitable simulation framework and the selection of relevant scenarios for the system under test. The literature reports different strategies and approaches for generating relevant scenarios for testing of ADS. All of them have their advantages and disadvantages related to the used environment, vehicle, traffic models and integration complexity.
This paper presents a survey through different approaches and methods for scenario generation and evaluation for testing and validation of ADS. It reviews a total number of 86 different papers, most of them published recently. It proposes a terminology and classification scheme of the different methods for scenario generation but also for the related assessment criteria. The reader should get a thorough state of the art overview on scenario based verification and validation approaches of ADS.