Early-Exit Forecasting of Deep Neural Networks on Energy-Harvesting Edge Devices

Ambient energy harvesting (EH) allows to extend the operational lifetime of battery-constrained wireless sensor nodes (WSNs) deployed in remote environments. The deployment of deep neural networks (DNNs) on EH-WSNs is of great interest for a wide range of applications, including environmental monitoring and wildlife tracking. However, sustaining continuous DNN inference on such systems is infeasible under harsh energy constraints imposed by limited battery capacity and varying amounts of energy harvested. Early-exit deep neural networks (EE-DNNs) allow to dynamically adapt the energy requirements of a DNN inference by integrating multiple exit points within the DNN architecture, enabling shorter inference paths for samples that can be classified with high confidence. This paper presents a novel policy that leverages a secondary small neural network called forecaster, which is trained to predict the most efficient exit point to be used by the EE-DNN for a given sample. This reduces the computational overhead of evaluating multiple early exits sequentially, as is the case with a default confidence-based policy. Empirical evaluation on two vision datasets demonstrates that our policy achieves a up to 23% higher accuracy compared to a default entropy-based policy. We also show that using a forecaster results in a higher accuracy under energy constraints than related work, while ensuring energy-neutral operation of the system.