Skip to content

pramttl/tstl

 
 

Repository files navigation

TSTL: the Template Scripting Testing Language

TSTL is a "little" language that makes it easy to test software. This implementation targets Python. There is also a (beta) Java version, at https://github.com/flipturnapps/TSTL-Java.

TSTL produces a simple, universal interface for test generators to use -- it essentially turns a definition of valid tests into a graph with transitions, backtrack, replay, automatic reduction, and code coverage support.

For more details on TSTL, see the NASA Formal Methods (NFM) and International Symposium on Software Testing and Analsysis (ISSTA) 2015 papers at http://www.cs.cmu.edu/~agroce/nfm15.pdf and http://www.cs.cmu.edu/~agroce/issta15.pdf.

The published papers use an early version of TSTL syntax, which marks pools and TSTL constructs with % signs. "Modern" TSTL uses <> by default, though if for some reason you need <> in your code (and to prepare for a future C++ version) this can be turned off and only % supported.

Note that documentation below is preliminary. A better guide to usage might be to examine the examples directory and see real TSTL test harnesses. The generators directory includes some simple but useful testing tools for finding bugs in systems defined in TSTL. Only the random tester is extensively tested and supports all TSTL features well at this point. Reading generators/randomtester.py provides considerable guidance in how to (efficiently) use TSTL in a generic testing tool, with TSTL providing an interface to the underlying application/library to be tested.

Installation

git clone https://github.com/agroce/tstl.git
cd tstl
python setup.py install
# Might need to do a sudo on the last step if not using virtualenv

The documentation below is preliminary, generated by some early adapters of TSTL, and may not perfectly match the current version. Use at your own risk. Code in examples directories should be up to date.

Using TSTL

The simplest usage is to go to a directory with a .tstl file, and type:

tstl mytstlfile.tstl
python <tstl-root>/generators/randomtester.py

The first line compiles mytstlfile.tstl and produces sut.py, which the random tester will automatically import and test. Both tstl and the random tester take a variety of command line options, which --help will show.

Example

The easiest way to understand TSTL may be to examine examples/AVL/avlnew.tstl (https://github.com/agroce/tstl/blob/master/examples/AVL/avlnew.tstl), which is a simple example file in the latest language format (easier to read).

Developer Info

There are no developer docs yet, which will hopefully change in the future. The best shakedown test for tstl is to compile and run (using generators/randomtester.py) the AVL example. Removing any call to the balancing function in the avl.py code should cause TSTL to produce a failing test case.

About

Template Scripting Testing Language tool

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%