Sensor-Guided Adaptive Machine Learning on Resource-Constrained Devices

In recent years, the deployment of deep learning models has ex- tended beyond typical cloud environments to resource-constrained devices such as edge devices and smartphones. This shift is driven by their success in learning and detecting patterns in data. How- ever, deep models are often excessively large and lack robustness to minor input transformations. To solve the challenge, deep learning models are often trained with data augmentation, which requires an even larger model to accommodate the additional knowledge. In this paper, we study ways to mitigate these problems by lever- aging additional sensing modalities to a) adapt the input data and b) adapt the model for typical transformations. We show that both approaches increase the accuracy of deep learning models by up to 6.21% and 7.57% respectively, while using roughly the same number of parameters or even less at inference time. We furthermore study how well these approaches can handle noisy sensor readings.