Synthesizing Adaptive Test Strategies from Temporal Logic Specifications

Constructing good test cases is difficult and timeconsuming,
especially if the system under test is still under
development and its exact behavior is not yet fixed. We propose
a new approach to compute test cases for reactive systems from
a given temporal logic specification. The tests are guaranteed
to reveal certain simple bugs (like occasional bit-flips) in every
realization of the specification and for every behavior of the
uncontrollable part of the system’s environment. We aim at
unveiling faults for the lowest of four fault occurrence frequencies
possible (ranging from a single occurrence to persistence). Based
on well-established hypotheses from fault-based testing, we argue
that such tests are also sensitive for more complex bugs. Since
the specification may not define the system behavior completely,
we use reactive synthesis algorithms (with partial information) to
compute adaptive test strategies that react to behavior at runtime.
We work out the underlying theory and present first experiments
demonstrating that our approach can be applied to industrial
specifications and that the resulting strategies are capable of
detecting bugs that are hard to detect with random testing.