Mutation Testing for Artificial Neural Networks: An Empirical Evaluation

Description

Testing AI-based systems and especially when they rely on machine learning is considered a challenging task. In this paper, we contribute to this challenge considering testing neural networks utilizing mutation testing. A former paper focused on applying mutation testing to the configuration of neural networks leading to the conclusion that mutation testing can be effectively used. In this paper, we discuss a substantially extended empirical evaluation where we considered different test data and the source code of neural network implementations. In particular, we discuss whether a mutated neural network can be distinguished from the original one after learning, only considering a test evaluation. Unfortunately, this is rarely the case leading to a low mutation score. As a consequence, we see that the testing method, which works well at the configuration level of a neural network, is not sufficient to test neural network libraries requiring substantially more testing effort for assuring quality.

Period	11 Dec 2020
Event title	20th IEEE International Conference on Software Quality, Reliability, and Security, QRS 2020
Event type	Conference
Location	Virtual, Macau, ChinaShow on map