Design and Implementation of FPGA-Based Ensemble Neural Networks for Real-Time Applications

Abstract

The demand for deploying neural networks (NNs), particularly Convolutional Neural Networks (CNNs), in real-time, resource-constrained environments is rapidly increasing. However, deploying CNNs on IoT devices, robotics, and autonomous systems presents significant challenges due to their high computational and memory requirements. This thesis presents the design and implementation of a general, modular, and reconfigurable FPGA-based framework for ensemble CNN inference. The system integrates multiple lightweight CNNs using voting and weighted averaging ensemble strategies to improve predictive accuracy and robustness. Hardware-specific optimizations, including L1 unstructured pruning, fixed-point quantization (to 16-bit and 24-bit formats), and pipelined module design are employed to reduce resource usage and enhance performance. The architecture is implemented in Verilog and synthesized using the Xilinx Vivado Design Suite, targeting energy-efficient real-time inference on embedded FPGAs. Evaluation is conducted on two digit recognition tasks: the culturally significant Geez (Amharic) Handwritten Digit Dataset and the widely benchmarked MNIST dataset. The proposed system achieves ensemble inference accuracies of 93.53% on Geez and 99.11% on MNIST, with per-image inference latency as low as 31 μs and throughput exceeding 32,000 frames per second at 50 MHz. Dynamic power consumption is constrained to as low as 0.414 W for a single model and 1.267 W for a three-model ensemble, making the design highly efficient for edge deployment. Benchmarking against CPU and GPU implementations as well as other FPGA accelerators shows superior energy efficiency and real time performance. The results demonstrate that the proposed FPGA framework effectively balances accuracy, latency, and resource efficiency, offering a scalable solution for deploying ensemble neural networks in constrained environments.