Radar Signal Processing and Sensor Fusion with Thermal Infrared for Firefighting

The accurate perception and correct interpretation of the surroundings is highly important in firefighting. Smoke is a severe obstruction for the firefighters, particularly during interior operations. Currently, firefighters use thermal infrared (TIR) cameras to see through the smoke. However, the TIR images are difficult to interpret due to the restricted depth understanding and TIR image artifacts. In particular, mirror reflections of firefighters occur on flat surfaces in the TIR image, which can be mistaken for a person to be rescued. Millimeter-wave (mmW) radar possesses great potential to overcome the current restrictions of a TIR only system. Similar to TIR radiation, mmW radar can penetrate smoke. The complementary information of the radar can improve the perception of depth and enable a classification of dangerous scenarios. This work focuses on the sensor design and algorithms for a firefighting system of a TIR camera and a mmW radar. The scope of this work covers i) an improved radar design, signal processing and direction-of-arrival (DOA) estimation for firefighting as well as ii) the sensor fusion of the radar and TIR data for depth augmentation and the classification of mirror reflections of persons.The combination of radar and TIR camera enhances the perception in firefighting by an overlay of radar depth information as well as target classification. Wavelength-diverse multiple-input multiple-output (WLD-MIMO) radar shows further potential to improve the DOA estimation with a low-cost sensor. This work is not only a proof-of-concept for a firefighting TIR camera, but a toolbox for modern radar perception systems in general.