Bridging the Gap Between Models in RL: Test Models vs. Neural Networks

Testing and verification of reinforcement learning policies are becoming ever more important. One of the open questions for testing such policies is how to determine test adequacy. Neuron activation has been proposed both as a metric for determining test adequacy, as well as for steering the test-case generation. However, recent studies have shown that increasing neuron coverage is not necessarily beneficial and might even be harmful. In this paper, we add an additional take on the evaluation of neuron coverage as a metric. We present different approaches to selecting test cases based on a Markov decision process, which is generated via model learning. We evaluate and compare the efficiency as well as the neuron activation achieved by each of the test suites. The approach is demonstrated on an RL agent playing Super Mario Bros. The results show that an intelligent selection of test cases leads to higher failure detection by the test cases, but does not imply high neuron coverage.