Designing a HIL system to validate autonomous driving functions
Interview with Christoph Funda and Paul Schmidl, Zukunft Mobility GmbH
We sat down with Christoph Funda and Paul Schmidl from Zukunft Mobility GmbH, a competence center of ZF Friedrichshafen AG operating in the domains of driver assistance, integrated vehicle safety and autonomous driving, to talk about the development of a HIL system to validate autonomous driving functions. During TECH WEEKS 2020, they presented the setup of a HIL prototype for closed-loop simulation of an autonomous shuttle bus using a large sensor data set.
The ZF group launched a project for autonomous shuttle busses. What exactly is it about?
Funda: In this project, we want to explore the development of electric shuttles up to SAE level 4 to sustainably reduce the strain on inner cities with a driverless, electric and flexible mobility offer. We need to revolutionize mobility as we know it by making it clean, efficient, comfortable and affordable.
The concept is going to be tested in two German cities, Mannheim and Friedrichshafen, until the end of 2023. In Mannheim, the focus is on mixed road use in the city, whereas in Friedrichshafen, tests focus on interurban transportation.
Could you please describe your tasks in this project?
Funda: Our team for test system development designs the hardware-in-the-loop (HIL) prototype systems based on the National Instruments (NI) technology. The systems aim at validating ZF control units in the context of environment sensor systems and autonomous driving. These prototype systems are then reproduced. Afterwards, the control units and their software are validated with regression tests in HIL farms in six-to-eight-week software cycles to be able to give permission for product release.
We use a sensor data set generated by our colleagues at ZF Friedrichshafen in this process. It is based on the sensor data set from IPG Automotive. What makes it special is that even with a huge number of sensors – about 21 of them – it delivers very high data rates of over 20 Gbps. The data set thus enables a very application-oriented sensor configuration for a highly autonomous vehicle that can prospectively meet the requirements of SAE level 4/5.
Which special aspects did you have to take into account with camera, radar and lidar models?
Funda: The camera, radar and lidar models have to meet a variety of quality criteria, requirements and boundary conditions. They were derived from the characteristics of real sensors and the simulation tool chain – a project of considerable extend. Besides, the sensor models had to be adapted to the necessary interfaces. The models were therefore designed to enable closed-loop simulation.
Which advantages can you derive from using the same models as for software-in-the-loop (SIL)?
Funda: We strongly benefit from the ability to reproduce and compare the test results in the V-model. Furthermore, we are able to reduce development times and to shift from comparatively expensive HIL tests to less expensive software-in-the-loop tests. They are much easier to scale and therefore less expensive to perform.
How can one imagine the HIL setup you are using: Do you rely on established technology or are you breaking new ground here?
Funda: We are doing both: we relied on proven NI technology as a basis and then cooperated closely with National Instruments and IPG Automotive to carry out suitable optimizations and develop new possibilities. In addition, we are using new operating systems such as Linux RT as well as hardware and software prototypes like the 40G Ethernet card to realize the required data rate.
Were you able to guarantee real-time capability on your HIL system despite the high number of sensors?
Schmidl: Through extensive analyses in SIL simulation, we could already achieve a positive real-time capability assessment of the sensor models in the HIL simulation. Using the SIL simulation as a starting point, we were able to carry out CPU and GPU performance tests with single sensors and subsequently with multiple sensors. The results were then used to derive corresponding hardware requirements for the necessary, high-performance simulation PC.
Funda: However, a general guarantee for real-time capability cannot be given. It is highly dependent on the number of objects in the simulated environment – for CPU-intensive sensor models such as object sensors – as well as on the resolution of sensors. The number of received reflections also play a major role for GPU-intensive sensors such as RSI sensors for lidar and radar. This means that the scenarios are decisive here.
For the concept mentioned by Mr. Schmidl, in which conclusions on the real-time capability of the HIL simulation can be drawn from measurements of the SIL simulation, in very close exchange with IPG Automotive. With this approach, you could at least save the time on the HIL that leads to overruns and real-time violations. The scenarios could then be simplified to a point that they can be simulated in real-time on the HIL if necessary.
How did you proceed in order to fast-track the simulation of the mentioned computationally expensive sensor models?
Schmidl: Given that the employed camera and lidar models have an particularly high resolution, they could not be computed in only a single GPU instance. The corresponding CPU processes of the GPU instances were not able to process the point clouds and camera frames fast enough – higher CPU clock frequencies improve the behavior.
By splitting a sensor model into a defined number of sub-models, we were able to ensure that the CPU processes of the GPU instances process the amount of data in less than one millisecond which also guarantees the real-time capability of the system. Up to a certain maximum value, the simulation can be accelerated with the described splitting process.
You held a workshop jointly with National Instruments and IPG Automotive on this topic. What were the goals of the workshop?
Funda: The workshop was geared towards building a suitable architecture of the closed-loop simulation on the HIL system and subsequently to test it directly in small packages in a proof of concept.
We carried out performance measurements to determine the delay of the camera data for example. These were most critical because they have the biggest data rates. The data have to be transmitted from the graphics card over the network interface controller to the HIL which in turn results in additional latencies and bottlenecks on the CPU. If necessary in future, we could switch to other technologies such as Direct memory access for example.
Finally, we elaborated a schedule to test the mentioned architecture and the whole simulation on a beta version of CarMaker in the short term as we switched from Windows/Pharlab to Linux-RT with the NI HIL systems. The transition was necessary since we could not apply the full sensor set in real time with Pharlab.
For us, the next step is to conduct measurement data-based and simulative studies to determine if the quality requirements for the system are met.
What are those studies going to look like?
Funda: Currently, we are mainly validating our HIL systems with measurements from Six Sigma methods. To reinject measurement data into the open-loop simulation, we are also breaking new ground and additionally setting up a discrete event simulation with the IT 7 chair of the FAU Erlangen-Nurnberg. This way, we would like to make reliable statements on the real-time capability of our streaming chain.
For this, the obtained measurement data for the extended input modeling are processed in such a way that we gain at least empirical distributions. However, theoretical distributions are optimal to be able to generate outliers with the simulation as well. We also use the method during development to be able to design the sizes of buffers and queues.
Our goal is to be able to guarantee real-time capability and the correct timing in the given tolerances at the interfaces of the HIL to the system under test.
Thank you for taking the time and for this insightful interview.