AI augments security in large facilities by tracking multiple people at once

Toshiba has developed AI technology that uses video feeds from multiple surveillance cameras to track the movements of people in shopping malls, train stations, arenas and other large facilities. Although it receives feeds from cameras in locations throughout the facility, the technology realizes high precision with low computation load, allowing routes taken by multiple people to be tracked at the same time. The technology will be reported at the 20th Meeting on Image Recognition and Understanding (MIRU2017), a major Japanese conference on image recognition, on August 8.

The success of millions of surveillance cameras installed worldwide in contributing to crime prevention and identifying criminals, along with the need for enhanced security to monitor potential terrorist threats, is driving demand for advanced crime-prevention solutions based on image-recognition technology. Most attention is being directed to technologies for facial recognition and individual detection that analyzes human attributes and behaviors, such as age and gender, using camera footage.

Up until now, systems that detect human attributes and behaviors have been based on the field-of-view of a single camera. However, in large public facilities, the preferred solution is the ability to identify and track individuals in videos from multiple cameras at locations throughout the site. As the way a person is captured in a video differs from camera to camera, precisely identifying the same individual in many videos is difficult, and successfully identifying multiple people simultaneously is a feat that imposes a huge computational load, as the number of potential combinations is enormous.

Toshiba's AI technology achieves high precision with low computational load. It tracks multiple people captured on numerous cameras, using three essential capabilities to do so.

(1) Robust feature extraction

By increasing luminance and color (multi-channeling), basic information for feature extraction, the features of individuals are extracted without being affected by differences in settings between cameras. Robust feature extraction not influenced by changing poses or similar traits shared by people is secured by dividing each video into several blocks (multilayer block division) and analyzing color distribution in each block (introduction of histogram feature quantities).

(2) Simultaneous feature extraction of multiple people to significantly reduce computational loads

Feature extraction is done with high-precision, even when people overlap in videos. This is achieved by tracking each person in the video feeds from every camera, and simultaneously extracting feature quantities from every image. Feature extraction is 2.3 times faster than when done frame-by-frame, as the computational load is reduced.

(3) Identifying a person in different videos through similarities between videos

In operation, the multiple cameras are clustered to form a single system, and the operational constraint that any individual person can only appear only once in any video at any given point in time is applied. This allows simultaneous similarity extraction of all the videos and selection of the best combination of features for recognizing the same person in different videos over time. The system identifies any particular individual 1,300 times faster than the conventional approach.

Toshiba evaluated the technology using the "CUHK03" a public image database, and found much higher precision than with current technologies. The computational load required to recognize the same person in feeds from multiple cameras was greatly reduced, making it possible to infer the movements of multiple people in close to real-time.

Toshiba aims to build the technology into its communication AI "RECAIUS" by mid-2018.

This new technology makes it much easier to track the paths and current positions in large facilities of particular individual, from lost children to suspicious individuals. Toshiba is also investigating application of the system to provide statistical analysis of attributes that will allow, for example, identification of where large numbers of people gather within facilities.

Provided by Toshiba Corporation