Poster: Resource-Efficient Deep Subnetworks for Dynamic Resource Constraints on IoT Devices

Deep models running on edge and mobile devices typically encounter dynamic system states due to changes in available resources, fluctuating energy levels and multiple competing real-time tasks. State-of-the-art machine learning pipelines produce resource-agnostic models that cannot dynamically adjust their resource demand at runtime. We present Resource-Efficient Deep Subnetworks (REDS), deep networks that can adapt their size and inference speed at runtime by using structured sparsity to allow for further optimizations on typical embedded platforms. We extend the TFMicro framework to support REDS and present preliminary evaluation on Arduino Nano 33 BLE Sense showing linear speedups and negligible overhead at a price of minor loss in model’s test set accuracy.