Part 1: Coverage Introduction
This component requires the development of a comprehensive tool to analyse a provided test suite for a piece of software. The analysis must cover two key white-box testing metrics: statement coverage and branch coverage. The aim is to assess the efficacy and thoroughness of the test suite in detecting faults and ensuring robustness in the software.
Objectives
• Implement a tool that takes a series of given test inputs and runs them on a program.
• Report the statement coverage and branch coverage for the program when run using the series
of test inputs.
Requirements
1. Statement Coverage
Objective: Determine the percentage of executable statements in the software that are executed by the test cases in the test suite.
2. Branch Coverage
Objective: Identify and report the number of branches through the program’s control flow graph that
are covered by the test suite.
Input Specifications
Your program should take 2 command-line arguments:
1. The path to a Python script
2. The path to a directory containing a set of input (.in) files
It should be called using the following command:
python coverage.py
Output Specifications
Your program should produce output indicating:
1 Statement Coverage: The count of statements executed during testing.
2 Branch Coverage: The count of intra-procedural paths executed during
For example:
1 Statement Coverage: 150
2 Branch Coverage: 19
Code Help
Part 2: Fuzzing with Mutated Inputs Introduction
In this part of the assignment, you will develop a fuzzer designed to automate the generation and mutation of test inputs to maximise the branch coverage of a test suite. The primary goal is to expand the test coverage by identifying and adding inputs that expose new branches in the software under test.
Objectives
• Develop a fuzzer capable of generating and mutating test inputs.
• Implement a method to measure the increase in branch coverage.
• Automate the process of enhancing the test suite with inputs that increase branch coverage.
Requirements
This task requires you to take a program along with a series of inputs and mutate the inputs to achieve a minimum branch coverage (note that in Part I we ask for statement coverage and branch coverage). You must automatically improve the test suite by adding mutated inputs that increase the branch coverage.
Implementation Specifications
• Use the fuzzer to apply mutations to the initial set of inputs.
• For each mutated input, execute the test suite to determine if the mutation results in increased
branch coverage.
• If an input increases branch coverage (by reaching new conditions not previously tested), add
it to a ’population’ of effective test inputs.
• Continuethisprocessuntilnofurtherincreaseinbranchcoverageisobserved,aimingtoachieve
the largest possible branch coverage.
• Write the final set of test inputs that collectively provide the highest branch coverage observed
to a file.
Input Specifications
Your program should take 2 command-line arguments: 1. The path to a Python script
2. The path to a single text (.in) file
It should be called using the following command:
python mutation_fuzzer.py
The text file will contain a set of inputs, each on a new line. For example:
浙大学霸代写 加微信 cstutorcs
Output Specifications
Your program should write back to the provided input (.in) file with exactly the same number of input strings as was provided initially.
For example:
Part 3: Grammar-Based Fuzzing Introduction
Grammar-based fuzzing is a commonly used method to test programs that consume structured inputs, particularly input parsers.
Objectives
• Implement a grammar-based fuzzer to generate structured inputs for testing.
• Explore various grammar structures to hit or exceed a branch coverage threshold specified.
Requirements
This task requires implementing a grammar-based fuzzer capable of generating structured inputs based on a specified grammar. The goal is to hit or exceed a branch coverage threshold by gener- ating a test suite that effectively tests the target program.
Implementation Specifications
• Develop algorithms to interpret grammar specifications and generate inputs accordingly.
• Explore different paths and options within the grammar to maximise the branch coverage.
• Test the generated inputs on the target program to assess its branch coverage.
• Implement mechanisms to adjust the generation process to hit or exceed the input and code
coverage threshold.
Input Specifications
Your program should take 3 command-line arguments:
1. The path to a Python script
2. ThepathtoasinglePython(.py)scriptcontainingthegrammarspecificationsusingthesyntax taught in the lectures and tutorials; the grammar will be stored as the variable ’grammar’
3. The number of strings your program should generate for the test suite
It should be called using the following command:
python grammar_fuzzer.py
Programming Help