FPGA based driver assistance systems
Camera-based driver assistance systems must be able to process video images in real time. SCS has implemented various image processing algorithms such as SGM-Stereo, optical flow and Stixel on an FPGA. These are now in series use and make the road safer.
-
Problem
Camera-based driver assistance systems must be able to process video images in real time. With newly developed algorithms, there is often a need to accelerate the runtime.
-
Solution SCS
The customer's algorithm was available as a PC-Program. SCS ported its functionality to an FPGA platform, which executes the algorithm in real time.
-
Added value
The customer is now able to run the algorithm in the test vehicle in real time and test it in real traffic.
Project insights
The development of driver assistance systems, such as a lane departure or traffic jam assistant, involves solving a wide variety of problems. For example, the system must be taught how to transform image points to known objects (roadway, markings, vehicles, pedestrians, etc.). The algorithms required for this are implemented as PC programmes by our client's experts for research.
Since at the time of the project the performance of modern processors and graphics cards was not sufficient to process the video image of an automotive camera in real time with the algorithm, the technical expert was not able to test the functionality of his algorithm in the moving test vehicle. Several such algorithms were implemented by SCS engineers in an FPGA and integrated on our hardware platform suitable for this purpose. The resulting system allows real-time processing of the video images. In each case, the feasibility was first examined in a feasibility study in close cooperation with the customer's technical experts and the implementation costs were determined.
SCS then carried out the conversion to FPGA. With the implementation on FPGA, the customer's technical experts are able to execute their algorithms in the test vehicle in real time. They can combine them into a complete system and test it in real traffic. Project insight using the Stixel algorithm as an example: Processing video images on a pixel basis requires the algorithm to handle considerable amounts of data. The Stixel algorithm helps to reduce the amount of data: it combines the pixels of an image column into "columns", so-called stixels. In doing so, it forms stixels from vertical surfaces (in vehicle rear fronts, curbs, ...), flat surfaces (roadway, pavement, ...) and background.
In addition to the data reduction, an initial, rough grouping of the pixels into sub-objects is achieved. This algorithm is mathematically complex and correspondingly computationally intensive. As part of a feasibility study, SCS succeeded in mapping the algorithm onto a streaming architecture for a low-cost FPGA. It was then implemented on the SCS FPGA box and integrated with other algorithms. Two automotive cameras can now be connected directly. The result, a Stixel image, is available via network connection for display or further processing on a PC.
The FPGA box from SCS was one of the subsystems of the Mercedes-Benz S-Class INTELLIGENT DRIVE, which was the first car in the world to drive a 100km overland route completely autonomously in a pioneering achievement. The selected route leads from Mannheim through villages and small towns to Pforzheim and has historical significance: because exactly 125 years earlier, Bertha Benz demonstrated the suitability of the patented Benz motor carriage on the same route. The modern S-Class successfully mastered numerous difficult traffic situations along the way.
For an insight, see the text and pictures on the following website:
Pioneering achievement: Autonomous long-distance drive in rural and: Mercedes-Benz S-Class INTELLIGENT DRIVE drives autonomously in the tracks of Bertha Benz