Language-Agnostic Generation of Compilable Test Programs
Kreutzer P, Kraus S, Philippsen M (2020)
Publication Language: English
Publication Status: Published
Publication Type: Conference contribution, Conference Contribution
Future Publication Type: Conference contribution
Publication year: 2020
Publisher: IEEE Xplore
Edited Volumes: Proceedings of the International Conference on Software Testing, Verification and Validation (ICST 2020)
Pages Range: 39-50
Conference Proceedings Title: Proceedings of the International Conference on Software Testing, Verification and Validation (ICST 2020)
Event location: Porto
URI: https://i2git.cs.fau.de/i2public/publications/-/raw/master/ICST20.pdf
DOI: 10.1109/ICST46399.2020.00015
Abstract
Testing is an integral part of the development of compilers and other language processors. To automatically create large sets of test programs, random program generators, or fuzzers, have emerged. Unfortunately, existing approaches are either language-specific (and thus require a rewrite for each language) or may generate programs that violate rules of the respective programming language (which limits their usefulness).
This work introduces *Smith, a language-agnostic framework for the generation of valid, compilable test programs. It takes as input an abstract attribute grammar that specifies the syntactic and semantic rules of a programming language. It then creates test programs that satisfy all these rules. By aggressively pruning the search space and keeping the construction as local as possible, *Smith can generate huge, complex test programs in short time.
We present four case studies covering four real-world programming languages (C, Lua, SQL, and SMT-LIB 2) to show that *Smith is both efficient and effective, while being flexible enough to support programming languages that differ considerably. We found bugs in all four case studies. For example, *Smith detected 165 different crashes in older versions of GCC and LLVM.
*Smith and the language grammars are available online.
Authors with CRIS profile
Patrick Kreutzer
Lehrstuhl für Informatik 2 (Programmiersysteme)
Stefan Kraus
Lehrstuhl für Informatik 2 (Programmiersysteme)
Michael Philippsen
Lehrstuhl für Informatik 2 (Programmiersysteme)
Related research project(s)
How to cite
APA:
Kreutzer, P., Kraus, S., & Philippsen, M. (2020). Language-Agnostic Generation of Compilable Test Programs. In Proceedings of the International Conference on Software Testing, Verification and Validation (ICST 2020) (pp. 39-50). Porto, PT: IEEE Xplore.
MLA:
Kreutzer, Patrick, Stefan Kraus, and Michael Philippsen. "Language-Agnostic Generation of Compilable Test Programs." Proceedings of the International Conference on Software Testing, Verification and Validation (ICST 2020), Porto IEEE Xplore, 2020. 39-50.
BibTeX: Download