Language-Agnostic Generation of Compilable Test Programs

Kreutzer P, Kraus S, Philippsen M (2020)


Publication Language: English

Publication Status: Published

Publication Type: Conference contribution, Conference Contribution

Future Publication Type: Conference contribution

Publication year: 2020

Publisher: IEEE Xplore

Edited Volumes: Proceedings of the International Conference on Software Testing, Verification and Validation (ICST 2020)

Pages Range: 39-50

Conference Proceedings Title: Proceedings of the International Conference on Software Testing, Verification and Validation (ICST 2020)

Event location: Porto PT

URI: https://i2git.cs.fau.de/i2public/publications/-/raw/master/ICST20.pdf

DOI: 10.1109/ICST46399.2020.00015

Abstract

Testing is an integral part of the development of compilers and other language processors. To automatically create large sets of test programs, random program generators, or fuzzers, have emerged. Unfortunately, existing approaches are either language-specific (and thus require a rewrite for each language) or may generate programs that violate rules of the respective programming language (which limits their usefulness).
This work introduces *Smith, a language-agnostic framework for the generation of valid, compilable test programs. It takes as input an abstract attribute grammar that specifies the syntactic and semantic rules of a programming language. It then creates test programs that satisfy all these rules. By aggressively pruning the search space and keeping the construction as local as possible, *Smith can generate huge, complex test programs in short time.
We present four case studies covering four real-world programming languages (C, Lua, SQL, and SMT-LIB 2) to show that *Smith is both efficient and effective, while being flexible enough to support programming languages that differ considerably. We found bugs in all four case studies. For example, *Smith detected 165 different crashes in older versions of GCC and LLVM.
*Smith and the language grammars are available online.

Authors with CRIS profile

Related research project(s)

How to cite

APA:

Kreutzer, P., Kraus, S., & Philippsen, M. (2020). Language-Agnostic Generation of Compilable Test Programs. In Proceedings of the International Conference on Software Testing, Verification and Validation (ICST 2020) (pp. 39-50). Porto, PT: IEEE Xplore.

MLA:

Kreutzer, Patrick, Stefan Kraus, and Michael Philippsen. "Language-Agnostic Generation of Compilable Test Programs." Proceedings of the International Conference on Software Testing, Verification and Validation (ICST 2020), Porto IEEE Xplore, 2020. 39-50.

BibTeX: Download