Home our services projects myAudience-Count FPGA
01 September 2012  |   myAudience-Count FPGA

As the first full scale project in the area of FPGA programming it was decided to start with porting of PC-based myAudience-Count SW product on Lattice HDR-60 Video Camera Development Kit powered by ECP3-70 FPGA chip.

The resulting FPGA-based myAudience-Count solution is a low cost, low power People Traffic Counter product that shares the same infrastructure and operational model as original PC version.

It provides browser-based configuration and counting system calibration UI featured by live camera preview, as well as remote backend server connectivity for centralized data aggregation and metrics calculation. The entire image processing and video analytics pipeline is implemented as a number of programmable logic modules. Along with that, servicing business applications are executed by LatticeMico32 (LM32) soft processor running uClinux OS on the same single FPGA chip. Resulting FPGA power consumption is as low as 0.33W.

Driving by necessity to minimize the number of used FPGA logical units and consumed power, Rhonda has developed highly-optimized video analytics pipeline capable to perform all major operations required for image capturing and subsequent processing in such algorithms as optical flow estimation and background modeling. Additional steps were performed to enhance image quality both for human eye and machine vision – in particular, specific support of HDR sensor capability was efficiently introduced with minimal resource allocation.

The quality of resulting people counting routine is around 90%, which is the same as for PC-based SW version.

The business logic part running on LM32 soft processor is operated under uClinux OS with 3.1 kernel. This OS flavor on new kernel was originally introduced by Milkymist open source project team and then improved by Rhonda Software in terms of kernel compatibility of device drivers for Ethernet MAC, Image Sensor, Timer, UART, LED, SPI Flash and Watchdog. Additional rework was made to incorporate newest U-Boot version with added support for Ethernet MAC and UART, kernel configuration via Linux Device Tree as well as to port and adapt several necessary user applications and libraries such as Curl, OpenSSL, CyaSSL, c-ares, thttpd, Mini-XML, Watchdog and dmalloc.

To achieve production quality for the People Counter, both hardware pipeline and business logic parts were put through careful testing, resulting in further stability and performance improvements.

Same as PC-based version, myAudience-Count FPGA provides calibration and configuration web-UI allowing to set up device using web browser. To support this, thttpd web server was ported to uClinux and optimized for AJAX requests. Since calibration stage also requires live video streaming preview (to place and markup counting line and to validate counting quality) an open source JPEG encoder from opencores.org was ported to LatticeECP3 and adapted to the Wishbone bus. Dedicated JPEG-encoder driver was developed for uClinux to establish simple but high-performance interface with software.

As a web-connected device (connection is mostly used for license checking and remote data storage) myAudience-Count FPGA requires corresponding SW modules suitable for resource constrained HW platform. Thus, a number of small embedded-optimized libraries was selected and adapted to ported uClinux OS for secure and reliable remote back-end access. The list of selected libraries includes CyaSSL (secure TCP/IP/HTTP), c-ares (fast domain name resolution) and Mini-XML (data packets parsing and composition).

Resulting solution incorporates production quality implementation residing on a single 70K LUT ECP3 FPGA chip and utilizing part of HDR-60 on-board peripherals (camera sensor, Ethernet port, DDR2) for both video analytics and web connection.

myAudience-Count demo bitstream for HDR-60 Video Camera Development Kit can be downloaded from http://www.computer-vision-software.com/files/videos/count/HDR60_People_Counter.zip

 

Do not hesitate to contact us for any question regarding our services. We will be glad to respond in detail.

...