B1 Robust Sensor Fusion und Feature Extraction

Project Leaders

Prof. Dr. Andreas Kolb, Prof. Dr. Christoph Ruland

Initial Situation

Systems with application in civil security generally require robust methods of feature extraction, sensor data fusion and/or spatio-temporal data accumulation. In many cases new imaging sensor modalities, such as depth- multispectral- or THz data are being processed with classic methods of 2D-imaging techniques. But, because of the very different characteristics, this often leads to instable results. The application of 2D-standard features, such as SIFT or SURF for feature extraction of depth data is limited, which is why intensity data of  ToF-cameras are regularly deployed. So far, multimodal 2D/3D features have been researched application-specifically, e.g. for face recognition.

Aims and Work Program

It is the aim of this part of the project, to research multi-platform aspects of real-time methods for feature extraction, sensor data fusion and for spatial/temporal accumulation of sensor data streams, using mainly multimodal data as the basis. In the area of sensor-data fusion the focus is mainly on methods of continual fusion of not synchronized sensor-data. The spatial/temporal accumulation is focused on the processing of 3D- and multispectral data, taking 3D and THz data into consideration, if necessary.

PMD camera and RGB Camera combination	Fusion of 160x120 Pixel PMD Camera with RGB