Hardware/Software Codesign for Convolutional Neural Networks Exploiting Dynamic Partial Reconfiguration on PYNQ

Convolutional Neural Networks (CNN), next to Recurrent Neural Networks (RNNs), are the most important subgroup of Deep Neural Networks (DNNs) due to their recent success in industry, especially regarding image processing. Hardware platforms, like Field Programmable Gate Arrays (FPGAs) and Application Specific Integrated Circuits (ASICs), turned out to be a viable alternative to Graphical Processing Units (GPUs) for DNN implementations, especially according to applications with strict power and performance constraints. Moreover, FPGAs supporting Dynamic Partial Reconfiguration can be reconfigured in the field multiple times. In this paper we present a toolflow designing layer specific hardware blocks and reconfiguring them during runtime. During reconfiguration parts of the inference path are executed in software. Starting at the training of CNNs with the help of the Caffe framework, we provide a parameter parser. Furthermore, layer reimplementation templates in C++ help to easily design layers in software and hardware with the help of High-Level Synthesis according to layer specific characteristics. We further demonstrate the possibilities of our hardware/software codesign approach and hardware reconfiguration on the Deep Landmark Detection application. The whole project is available on Github.