OCCLUDED OBJECT DETECTION WITH INFRARED CAMERA ON MOBILE ROBOT FOR SEARCH AND RESCUE Hansel Kane (a*), Maria Evita (b), Mitra Djamal (c)
a) Instrumentation Physics, Institut Teknologi Bandung
Jalan Ganesha 10, Bandung 40132, Indonesia
liveishappy11[at]gmail.com
b) Instrumentation Physics, Institut Teknologi Bandung
Jalan Ganesha 10, Bandung 40132, Indonesia
c) Instrumentation Physics, Institut Teknologi Bandung
Jalan Ganesha 10, Bandung 40132, Indonesia
Abstract
Disaster effects tend to be more severe in developing countries. Improving the disaster response system is the key to prevent casualties. Therefore, a system that could hasten search and rescue processes is utterly necessary. Mobile robot is a really suitable candidate for such task. Mobile robot in this research has a thermal camera equipped as thermal cameras tends to be unaffected by lighting variation, thus providing more robust detection. The models used MobileNetv2 architecture to detect human in lying position. Models were trained using infrared images containing human in lying position representing a casualty. Three variation of training data that result in three different models (Mnol, Msebagian, Msatu) with different number of dataset, were used to observe the effect of partially occluded augmentation training data. Furthermore, four category of tests were carried out to observe the performance of the models: unoccluded, partially occluded, fully occluded human and human in construction area. The result of unoccluded test were 0.74, 0.9, 0.9- for partially occluded test were 0.63, 0.87, 0.9- for fully occluded test were 0.74, 0.94, 0.97 in terms of mAP for Mnol, Msebagian, and Msatu respectively. Moreover, loss of the three models converged to 2/7 and didn^t differ significantly from one another, with the difference only around 3%. The result of human detection in construction area were 0.48, 0.55, 0.67 for Mnol, Msebagian, and Msatu respectively. Modifying training dataset by adding random-generated occluded images improves model^s performance in general as shown by Msatu that yields the higest mAP in all category. Furthermore, temperature measurement is also carried out yielding 32 to 34 degree Celcius for human skin detected by Lepton 3.5. A live-stream is performed to test the speed of MobileNetv2. Deployed on Raspberry Pi 3b+, MobileNetv2 could run in around 2 fps. All three models successfully detect a lying person in unoccluded, partially occluded,
Keywords: deep learning, mobile robot, passive infrared, MobileNetv2, occluded
Topic: Instrumentation, Acoustics, and Signal Processing
