While there was great progress regarding the technology and its implementation for vehicles equipped with automated driving systems (ADS), the challenge of how to prove their safety as a necessary precondition of a market launch remains unsolved. A promising potential solution are scenario-based development and test approaches; however, there is no commonly accepted way of how to systematically reveal the set of relevant scenarios to be tested to sufficiently capture the real-world traffic dynamics.
In this work, the so-called adaptive replay-to-sim approach (ARTS) is presented, which aims in particular to reveal unknown, unsafe scenarios in the development and testing phase of ADS. This approach is characterized by combining the individual advantages of real-world road traffic data and an agent-based (traffic) simulation.
As illustrated by Figure 1, the idea is to split the execution of a test case in two modes, namely a replay-to-sim and an agent-based simulation mode. Within the first mode the ADS-equipped vehicle substitutes a vehicle from the real recorded scenario, e.g., recorded by drones or extracted from a stochastic traffic simulation. An observer is tracking the (dis)similarity d between the original trajectory of the substituted vehicle and the ADS. In case the dissimilarity metric exceeds a defined threshold d_thres, describing a significant behavioral difference between the human-driven vehicle and the ADS, an agent-based mode is activated (trigger point T_tp). From now on, the behavior and thus the resulting trajectories of the involved traffic participants surrounding the ADS are determined by the underlying agent models.
In addition to conceptual details, the proof-of-concept implementation and evaluation of the approach using CarMaker as well as the InD dataset, containing naturalistic urban road user trajectories, is showcased exemplarily. Compared to related approaches, within which the behavior of a vehicle operated by a human driver in real road traffic is compared with the virtual behavior of a passive ADS, the ARTS approach offers various advantages. A major advantage of ARTS is the elimination of the ADS dependency, since the reference data are available independent of the specific ADS (parametrization) and thus, for example, a relative comparison of different ADS is possible. Another benefit is the possibility to evaluate the long-term evolution of the specific scenario. However, challenges potentially arise from the absence of perception sensor information in the isolated ARTS approach and thus validated sensor models as well as agent models are required depending on the system under test. In addition, there are further concepts and approaches that ARTS would enable: An example is the substitution of various ego-vehicles of the data set with ADS-equipped vehicles for evaluating future mixed-traffic scenarios.
