joszamama/fandango-fuzzer repository overview

⁠FANDANGO: Evolving Language-Based Testing

FANDANGO is a language-based fuzzer that leverages formal input specifications (grammars) combined with constraints to generate diverse sets of valid inputs for programs under test. Unlike traditional symbolic constraint solvers, FANDANGO uses a search-based approach to systematically evolve a population of inputs through syntactically valid mutations until semantic input constraints are satisfied.

⁠Table of Contents

• Introduction⁠
• Features⁠
• Documentation⁠
• Evaluation⁠
• Contributing⁠
• License⁠

⁠Introduction

Modern language-based test generators often rely on symbolic constraint solvers to satisfy both syntactic and semantic input constraints. While precise, this approach can be slow and restricts the expressiveness of constraints due to the limitations of solver languages.

FANDANGO introduces a search-based alternative, using genetic algorithms to evolve inputs until they meet the specified constraints. This approach not only enhances efficiency—being one to three orders of magnitude faster in our experiments compared to leading tools like ISLa⁠—but also allows for the use of the full Python language and libraries in defining constraints.

With FANDANGO, testers gain unprecedented flexibility in shaping test inputs and can state arbitrary goals for test generation. For example:

⁠Features

• Grammar-Based Input Generation: Define formal grammars to specify the syntactic structure of inputs.
• Constraint Satisfaction: Use arbitrary Python code to define semantic constraints over grammar elements.
• Genetic Algorithms: Employ a search-based approach to evolve inputs, improving efficiency over symbolic solvers.
• Flexible Constraint Language: Leverage the full power of Python and its libraries in constraints.
• Performance: Achieve faster input generation without sacrificing precision.

⁠Documentation

For the complete FANDANGO documentation, including tutorials, references, and advanced usage guides, visit the FANDANGO docs⁠.

Here, you'll find the following sections:

⁠Fuzzing with Fandango

• Fuzzing with Fandango⁠

⁠About Fandango

• About Fandango⁠

⁠Fandango Tutorial

• Fandango Tutorial⁠
• Installing Fandango⁠
• A First Fandango Spec⁠
• Invoking Fandango⁠
• Fuzzing with Fandango⁠
• Some Fuzzing Strategies⁠
• Shaping Inputs with Constraints⁠
• The Fandango Shell⁠
• Data Generators and Fakers⁠
• Complex Input Structures⁠
• Accessing Input Elements⁠
• Case Study: ISO 8601 Date + Time⁠
• Generating Binary Inputs⁠
• Bits and Bit Fields⁠
• Case Study: The GIF Format⁠
• Statistical Distributions⁠
• Coverage-Guided Fuzzing⁠
• Hatching Specs⁠

⁠Fandango Reference

• Fandango Reference⁠
• Installing Fandango⁠
• Fandango Standard Library⁠
• Fandango Spec Locations⁠

⁠Evaluation

FANDANGO has been submitted to ISSTA 2025. FANDANGO has been evaluated against ISLa⁠, a state-of-the-art language-based fuzzer. The results show that FANDANGO is faster and more scalable than ISLa, while maintaining the same level of precision.

To reproduce the evaluation results from ISLa, please refer to their replication package⁠, published in FSE 2022. To reproduce the evaluation results from FANDANGO, please download a development copy of the repository from the official GitHub Repository⁠, execute: (from the root directory)

cd src/evaluation/evaluation &&
python run_evaluation.py

This script will execute FANDANGO on 5 subjects (CSV, reST, ScriptSizeC, TAR and XML). Each subject will be run for an hour, followed up by a computation on each grammar coverage (This process can take a while). The results will be printed in the terminal. Our evaluation showcases FANDANGO's search-based approach as a viable alternative to symbolic solvers, offering the following advantages:

• Speed: Faster by one to three orders of magnitude compared to symbolic solvers.
• Precision: Maintains precision in satisfying constraints.
• Scalability: Efficiently handles large grammars and complex constraints.

⁠Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository.
2. Create a new branch: git checkout -b feature-name.
3. Commit your changes: git commit -am 'Add new feature'.
4. Push to the branch: git push origin feature-name.
5. Submit a pull request.

Please ensure all tests pass and adhere to the coding style guidelines.

⁠License

This project is licensed under the European Union Public Licence V. 1.2. See the LICENSE⁠ file for details.