Design and Implementation of Embedded Video Surveillance Hardware

Abstract: The demands on video surveillance systems are rapidly increasing regarding parameters such as frame rate and resolution. Furthermore, with an ever increasing number of video streams, an automated process for extracting relevant information is required. Due to the large amount of input data and the computational complexity of the algorithms, software implementations are not sufficient to sustain real-time performance for reasonable resolutions. In this thesis an automated digital surveillance system running on an embedded platform in real-time is presented. Algorithms that are well suited for hardware implementation with streamlined dataflow are chosen and dedicated hardware accelerators have been developed. The presented hardware platform has been developed with the goal of presenting a proof of concept for the surveillance system and to identify computational and memory bottlenecks. Furthermore, when proposing modifications to the original algorithms extensive simulations are needed, especially if long-term effects in the video sequences can be envisioned. Utilizing a reconfigurable platform based on a field programmable gate array (FPGA) reduces the simulation and development time considerably. The main bottleneck of the proposed system, as well as for most image processing algorithms, is the high memory requirements. Therefore, simplifications have been proposed both on the algorithmic and the implementation levels. Furthermore, elevating the point of view from block to system level made it possible to perform cross boundary optimization and to explore other system architectures. On a block level, a low complexity morphological datapath unit, a contour tracing label unit that utilize Green's formula to extract object features, and color feature extraction on streaming data are presented. From a system level perspective the following were identified: unique labels are not required; changing color space from RGB to YCbCr both improved the segmentation result and allowed a simple face detector; a stall-free morphological unit will allow operations directly on the input data without buffering. It is also shown how retiming of the system can lead to substantial reduction of memory requirements. On an architectural level, replacing the morphology and labeling block with an image projection unit was investigated. To accommodate more advanced functionality such as face recognition and object classification advanced features have to be used. In this thesis one such feature detector, MSER, has been implemented.