This project aims to provide tools for constructing graphical user interfaces (GUIs) that are easily testable. A testable GUI is one in which a user session can be recorded in a log file, after which the log file can be instrumented and replayed to ensure that the GUI is performing correctly. The goal is not to test the widget toolkit itself, but to ensure that the program using the toolkit is interacting correctly with the GUI. For example, the tests could check that when a button is pressed the correct routine is executed, or that when the state of an object changes, that change is reflected in the GUI.
The OOF materials modeling project at NIST uses a GUI constructed from the PyGTK toolkit. OOF has a computational back end that interacts via scriptable commands with the GUI. The back end has a text based regression suite, but changes in the back end or the GUI often resulted in errors in the GUI that could not be caught by the regression tests. gtklogger was written to enable GUI tests for OOF. It is designed to be non-invasive: although the programmer has to replace many PyGTK function calls with gtklogger functions, gtklogger imposes virtually no computational overhead during normal operation of a program, is invisible to the end-user, and does not limit the operation of PyGTK.
gtklogger is similar in some ways to PyUseCase, although it has independent origins.
Github is both a way of distributing gtklogger and (we hope) a way of encouraging equivalent tools to be developed for other GUI toolkits.
The philosophy applied to PyGTK in gtklogger can be applied to any GUI toolkit. Because a logger can still be used even if not all of the toolkits components are wrapped, it lends itself well to incremental collaborative development. We hope that this site will encourage the development of loggers for a variety of GUI toolkits.
A guilogger is a module that wraps a GUI toolkit and enables a program using the toolkit to do two main things: recording a user session log file, and replaying the session while running tests. The tests can be inserted into the log file in the form of assert statements. If the program successfully plays the log file, it has then passed the GUI Test Case.
A guilogger module needs to provide the following 5 features:
-
It must have methods for assigning names to GUI widgets, so that the widgets can be identified in log files. In PyGTK, UI components are nested in containers, and the gtklogger identifier for each widget is a path constructed from the names of all of its containers.
-
The guilogger module needs to ensure that it is informed of all actions performed by each logged widget. In gtklogger, this is done by replacing calls to widget.connect(....) with gtklogger.connect(widget, ...), but it could be done differently, by object inheritance for example.
-
For each type of widget, the module needs to provide code that records the widgets actions in the log file.
-
The guilogger needs to provide a way of reading the log file and recreating the widgets' actions.
-
Because timing can be important, the guilogger must provide a way of inserting checkpoints into the log file, so that it does not try to recreate widget actions before the program is ready.
In general, using the guilogger will require a program to be modified in some minor ways. The modifications required by gtklogger (and we hope by all guiloggers) do not impose any run time performance cost in normal use, and only a small cost when recording or replaying a log.
This repository contains the gtklogger code and a demonstration of how it is used.
gtklogger was developed by the OOF team at the National Institute of Standards and technology (NIST).
We encourage you to contribute to that project, since it is providing a different way of designing your GUI tests suite.
gtklogger is free for anybody coding a PyGTK app. Please use it and let us know what you think about it. If you want to add features or fix issues that you encountere, please contact us.
We also encourage the implementation of this concept for other GUI toolkits. We are thinking of: Qt, Glui, Juce, gladexml, jqueryUI, capuccino, etc...
Install PyGTK version 2.6 or later, but not version 3.
To install gtklogger, go inside the folder and type: python setup.py install. If you do not have root privileges add '--user' to install it locally: python setup.py install --user.
To test gtklogger, please take a look at the examples folder. There is a gtkloggerdemo project in it. It is a simple GTK UI app to manage the creation of some geometric shapes. Please look carefully how it integrates gtklogger to enable recording and replaying.
To simply run the app type: python gtkloggerdemo.py.
To record a session in 'log.py', do: python gtkloggerdemo.py --record=log.py. From there you can instrument the log file with some tests. The log.py file provided in the repository contains some examples of tests.
To replay the log file (instrumented or not) type: python gtkloggerdemo.py --replay=log.py.
NOTE: When you start a recording, there is another GUI that appears along with your app. It is the guilogger UI for GTK. It allows you to add comments and view the events in the log file as they are recorded.