Subito  
 
 
EU Flag
FP7-SEC-2007-2.3-01 Grant No. 218004
 
Project news | Workshop | Public documents | Final Demonstration | Final Results
   

Object Classification

Having robustly detected and tracked objects of interest within the scene the SUBITO system design attempts to perform object classification. This was achieved in two ways:

Back to Algorithm Development Main Page Back to Algorithm Development Main Page

Facial Recognition

With facial recognition, the overall performance depends heavily on the quality of the input image, in particular the image resolution of the facial area within the image, typically measured in the number of pixels of inter eye distance (IED), contrast or colour depth within the facial area, sharpness of the objects within the image and pose of the facial image.

The system architecture design work showed that standard CCTV would not be sufficient to give high enough resolution for facial resolution algorithms to be used in the SUBITO scenarios and thus the use of a Narrow Field Of View (NFoV) PTZ camera to collect high resolution image was defined.

Image resolution is one of the most important limiting factors with standard video-surveillance cameras. In most cases, low resolution prevents object detection at a distance exceeding 15 meters, and prevents face identification in nearly all situations.

The work carried out in the SUBITO project has shown that face recognition can be successfully achieved only in cases of moderate poses (< 25° from head on) with inter-eye distance larger than 20 pixels. Consequently, cameras dedicated to face recognition should be located in a way that can capture channelled person flows (e. g. stairs, passages, entrance and exit points, etc.)

Low IED values normally result in a significant degradation of the face recognition accuracy which would be unacceptable for a SUBITO system. Thus the project developed a concept that would improve recognition performance even in the presence of such variations of IED.

The basic concept implemented was to incrementally improve the facial representation of a person by adding in further sample images of the same individual into the search image or "probe" of the surveillance data.


Facial Detection using single probe image

Facial Detection Using Single Probe Image

For example, starting with a single image of a person (probe) after an unattended baggage event was declared, the system will extract faces from the video streams of the remaining cameras and sort them according to decreasing similarity of the person to be searched for, as illustrated in Figure 2 10. In the case of high similarity values the system will (automatically or with the support of an operator) add corresponding images to the set of probe images. After this, the search may be repeated or continued with the enlarged set of probe images leading to possible further hits, Figure 2 11.


Facial Detection using multiple probe images

Facial Detection Using Multiple Probe Images

In this way the system can iteratively improve the representation of a person covering more and more variations of that individual and thus increasing the probability of recognition at a different camera location and thus to track it across a camera network.

Another key development to come from the work carried out in the SUBITO project was the improvement of the accuracy of the face recognition in the context of the scenario where a person is tracked / recognised across a camera network. In this scenario, the representation of a person differs significantly from camera to camera due to different head poses, varying illumination, camera characteristics, facial resolutions etc.

Back to Top Back to Top

Back to Algorithm Development Main Page Back to Algorithm Development Main Page

Object Description By Use Of PTZ Cameras

The second area of object classification to be considered was object description by use of PTZ cameras. In this case the work carried out in the project looked at the capabilities of a PTZ camera network to provide both stationary object detection and faces associated with long term tracking.

While a PTZ guard tour, see Additional Cameras Supporting Study, can provide high resolution images of any location in the surveyed scene; tracking can only be performed effectively and globally using fixed wide angle cameras which provide a constant view of the entire scene. Therefore the work looked at extracting higher resolution images of objects tracked in the wide angle camera network.

The SUBITO tracking module provides tracks for individuals and baggage in the form of a set of 3D trajectories expressed in ground plane coordinates. Given the full camera network calibration defined during the project, these trajectories were intersected with the guard tour positions to extract higher resolution image patches and thus provided additional images of individuals faces and pieces of baggage, see Figure below.


Image Patch Extraction from PTZ Cameras

Image Patch Extraction from PTZ Cameras

Left: detected tracks in fixed camera. Middle: corresponding boxes in PTZ camera. Right: Extracted patch sample.

This concept was extended further, by using two PTZ cameras simultaneously performing the guard tour strategy and each running the single view stationary object detection, a multi-camera/view object matching algorithm was developed. This algorithm extracts the full potential of the epipolar geometry that can be built from using a pair of PTZ cameras allowing the algorithm to deal with objects which are partially or fully occluded in one of the two views, as illustrated below. Objects that have been identified by the combination of the two cameras can be used to extract additional information such as 3D location, height and volume of stationary objects.


Image Patch Extraction from PTZ Cameras

Image Patch Extraction from PTZ Cameras

Left and right images are mosaics built from 8 guard tour positions. Images here represented are spherically rectified so that matching objects have corresponding heights. The capture is not synchronized: objects may be present in one camera but not visible in the other one.

The developed algorithm was compared to other state-of-the-art methods using several sequences acquired from a guard tour over a 40 x 40 metres area, and was found to provide improved performance over the state-of-the-art methods.

Back to Top Back to Top

Back to Algorithm Development Main Page Back to Algorithm Development Main Page

BoxCurveSX

Related links

BoxCurveDX
     

 

BoxCurveSX

Final Results Contents

BoxCurveDX
 

Executive Summary


System Definition


System Architecture


Algorithm Development


Integration & Demonstration


Socio-economic Impact