Abstract
Mutation analysis is a method for evaluating the effectiveness of a test suiteby seeding faults artificially and measuringthe fraction of seeded faults detected by the test suite. The major limitation of mutation analysis is its lengthy execution time because it involves generating, compiling andrunning large numbers of mutated programs, called mutants. Our tool MuVM achieves a significant runtime improvement by performinghigher order mutation analysis using four techniques, metamutation, mutation on virtual machine, higher order split-stream execution, and online adaptation technique. In order to obtain the same behavior as mutating the source code directly, metamutation preserves the mutation location informationwhich may potentially be lost during bitcode compilation and optimization. Mutation on a virtual machine reduces the compilation and testing costby compiling a program once and invoking a process once. Higher order split-stream execution also reducesthe testing cost by executing common parts of the mutants togetherand splitting the execution at a seeded fault. Online adaptation technique reduces the number of generated mutantsby omitting infeasible mutants. Our comparative experiments indicate that our tool is significantly superior to an existing tool, an existing technique (mutation schema generation), and no-split-stream executionin higher order mutation.