US army boffins use AI to spot faces in the dark

Neural network recreates real images from thermal cameras

US army researchers have developed a convolutional neural network and a range of algorithms to recognise faces in the dark.

"This technology enables matching between thermal face images and existing biometric face databases or watch lists that only contain visible face imagery,” explained Benjamin Riggan on Monday, co-author of the study and an electronics engineer at the US army laboratory.

"The technology provides a way for humans to visually compare visible and thermal facial imagery through thermal-to-visible face synthesis."

The thermal images are processed and passed to a convolutional neural network to extract facial features using landmarks that mark the corners of the eyes, nose and lips to determine its overall shape. Next, a non-linear regression model maps these features into a corresponding visible representation.

The system, dubbed “multi-region synthesis” is trained with a loss function so that the error between the thermal images and the visible ones is minimized, creating an accurate portrayal of what someone’s face looks like despite only glimpsing it in the dark.

But to be effective, however, it requires the US army to match the recreated image to a face that is previously known in a database such as a watch list or criminal records in order to identify the target.

"When using thermal cameras to capture facial imagery, the main challenge is that the captured thermal image must be matched against a watch list or gallery that only contains conventional visible imagery from known persons of interest," Riggan said.

"Therefore, the problem becomes what is referred to as cross-spectrum, or heterogeneous, face recognition. In this case, facial probe imagery acquired in one modality is matched against a gallery database acquired using a different imaging modality."

Fake views

It’s similar to how general adversarial networks work, the researchers explained in a research paper that was presented at the IEEE Winter Conference Applications of Computer Vision at Lake Tahoe, Nevada, last month. But instead of trying to training a generator network to reproduce fake images good enough to trick discriminator network, it is instead matched to a ground truth in an external dataset.

Preliminary results show that the accuracy is varied. For polarimetric thermal images, a common thermal imaging technique, the multi-region synthesis model achieved 85.43 per cent accuracy, and conventional thermal imaging was 82.49 per cent. Although the results are promising, integrating these algorithms with current thermal imaging technologies is difficult, the paper said.

“The top matches from recent cross-spectrum face recognition methods are not easily verifiable by human analysts due to the large differences in visual appearance of facial imagery collected in the thermal and visible bands of the electromagnetic spectrum.

“Moreover, a significant amount of time and resources have been invested in developing existing visible face recognition technology, which are less effective on cross-spectrum face recognition tasks. Thus, there is a need to alleviate the difficulty of matching thermal and visible spectrum imagery.” ®

PS: Speaking of image recognition, a biz called Terrogence claims to have a terrorist face database of "thousands of suspects harvested from such online sources as YouTube, Facebook and open and closed forums all over the globe," as spotted by infosec journo Tom Fox-Brewster. The outfit is now owned by Israeli vendor Verint, whose customers are intelligence and government agencies from the NSA to the US Navy.