Imaging Systems Gain a Brain

Source: Photonics Spectra (04/15/04)

The use of video surveillance as a security tool historically has been plagued by multiple limitations, including cross-platform incompatibility, inadequate nighttime operability, and the human fatigue and error factors.

" Software developed by researchers at the University of Rochester in New York gives surveillance cameras a rudimentary brain. According to Randal Nelson, associate professor of computer science at the university and developer of the software, the idea is to get intelligent machines to do the observing and reacting in on-going tasks, removing the need for continuous monitoring by a human.

The research has been licensed to PL E-Communications, also of Rochester, which developed software solutions dubbed AVT234, that provides users with advanced automatic detection and object discovery to continuously interpret video feeds. It works with any sensing device, is compatible with multiple computer platforms and can be used in conjunction with Internet Protocol cameras that have all applications software embedded, eliminating the need for computer integration.

The software can even be implemented for nighttime surveillance. Visual motion is a particularly effective cue for recognition tasks at night, when detecting motion is not affected by illumination and shadowing. Surveillance automation at night does not require complete knowledge of the scene being surveyed, so low-level autodetection works well. Although initially developed for use in military, law enforcement, commercial facilities and border surveillance applications, the software also can be used in nonsurveillance applications, such as factory automation, non-invasive medical diagnostics and MRI.

Advantages of the software in video surveillance include its ability to automatically distinguish multiple object and motion types in one frame so that it more accurately identifies changes in the scene and dramatically reduces the disctraction caused by false alarms.

The way it works

The software performs automatic detection of changes in still and streaming images using object discovery by recovering and explaining the temporal structure found in each pixel's stream of observations. Although the idea superficially resembles the use of segmentation algorithms that assume local homogenity of object characteristics -- such as color, texture and optical flow vectors -- only temporal information is used to perform pixel clustering.

The software has frame-rate requirements of 1 to 5 Hz, which is far lower than comparable solutions that need miniumum frame rates of 15 to 30 Hz. The lower rate makes detection quicker and less sensitive to distracting motion, while maintaining the ability to discover entire objects, even in cases where thare are partially occluded. Because the software is continuously normalizing the scene, it also recovers complex occlusion relationships, including those in which different objects share the same space, texture and color. It automatically discards problematic sequences rather than generating false alarms, making prioritization of alarms more efficient. And because spatial information is not used in the clustering steps, the technique is very flexible. Change detection and object discovery can be done on a variety of time scales, from rapidly moving object to stationary people and vehicles, for intervals from a few seconds to several hours. It also can be used with any type of sensor data, including multispectral data fusion.

This technology enables onboard fusion/processing of multiple sensor data types to prioritize targets and other changes that are of interest to the user. A critical element of the process model sets priorities for users to look at surveillance scenes in order of importance, flagging urgent scenes for tracking and comparison by military, state and local law enforcement and other key stakeholders. This same technique can be used to prioritize event management for factory automation, non-invasive medical diagnostics and MRI applications.