greg
The Greg framework is designed to be used in two ways - as an embedded
system from within any application which is linked with the Guile library,
or stand-alone using the command-line greg
driver script.
For both of these methods of usage the test cases are written the same
way and the expected output is the same.
While Greg may produce more verbose output in response to various settings, the basic output of a test run is a series of lines describing the success/failure state of each testcase encountered, followed by a summary of all test cases.
In `normal' mode, only unexpected results are displayed, but in `verbose' output mode, results for all results are displayed.
greg
flags the outcome of each test case with a line consisting of
one of the following codes followed by a colon, a space, and then the
testcase description.
PASS
UPASS
FAIL
XFAIL
UNSUPPORTED
instead.
UNRESOLVED
UNTESTED
PASS
or FAIL
. You can also use this outcome in dummy
"tests" that note explicitly the absence of a real test case
for a particular property.
UNSUPPORTED
A Greg test run expects to find files and directories in a certain layout (modeled on that used by DejaGNU).
The test source directory (normally your current directory) is expected to contain one or more tool directories. Each tool directory should contain one or more test scripts. In fact any file in a tool directory which has a .scm extension is assumed to be a Guile test script.
When a normal Greg test run is done, Greg goes through each tool directory in turn and loads each test script in turn.
You may set the Guile variable greg-tools or use the --tools ... command-line option to specify a list of tools directories to use rather than assuming that all subdirectories are tool directories. If you do this, the tools are tested in the order in which they appear in the list rather than the default order (ASCII sorted by name).
You may set the Guile variable greg-files or use the --files ... command-line option to specify a list of file names to use rather than assuming that all .scm files in each tool directory are test scripts. If you do this, the files are loaded in the order in which they appear in the list.
As a (minor) complication to this simple layout, Greg permits the use of begin.grg and end.grg scripts in both the main source directory and in each tool directory. These scripts permit you to add any initialisation and cleanup code you want. Typically (for non-embedded testing) you would use a begin.grg script to start the application to be tested.
If begin.grg exists in the main source directory, it will be loaded before any tools are tested.
If end.grg exists in the main source directory, it will be loaded after all the tools are tested.
If begin.grg exists in a tool directory, it will be loaded before any test scripts in that directory are loaded.
If end.grg exists in a tool directory, it will be loaded after all the test scripts in that directory are loaded.
Greg is designed primarily for embedded usage - any application that uses Guile as it's scripting language should be able to use Greg to test itself.
To this end - Greg provides a Guile module containing definitions of various procedures (used to run tests) and variables (used to modify the behavior of a test run).
Before trying to use any part of Greg, You need to load the Greg module with (use-modules (ice-9 greg))
The main procedure to run Greg tests is (greg-test-run). You can use this to run tests in much the same way as the greg script is used to run tests from the command-line. The behavior of this procedure is modified by setting the following top-level variables -
.scm
extension in a tool subdirectory are loaded. If the list
contains names, the named scripts are loaded in the order in which they occur
in the list rather than the default order (ASCII sorted by name)..scm
extension - it will be added automatically by Greg.
.
).
.
).
.
).
.
) are assumed to be valid tool
directories. If the list contains names, the named subdirectories are used
in the order in which they occur in the list.
greg
is the executable test driver for Greg. This is a Guile script
that you can use to run tests from the command line.
The command-line options that you can pass to greg
are listed below.
greg
returns an exit code of 1
if any test
has an unexpected result; otherwise (if all tests pass or fail as
expected) it returns 0
as the exit code.
This is the full set of command line options that greg
recognizes.
greg --tool tool ... [ --debug ] [ --file script ... ] [ --help ] [ --objdir path ] [ --outdir path ] [ --posix ] [ --srcdir path ] [ -v | --verbose ] [ -V | --version ]
--tool tool ...
greg
command
line runs tests from the gcc
subdirectory.
The order in which the tools are tested will be the same as the order in
which the tool names occur on the command line.
--file script ...
greg
runs all tests for the tool, but you can
restrict it to particular testsuites by giving the names of the .scm
guile
scripts that control them. You do not need to supply the
.scm file extension - it will be assumed.
testsuite may not include path information; use plain filenames.
The order in which the test scripts are run will be the same as the order in
which the script names occur on the command line.
--debug
expect
internal debugging output. Debugging output
is displayed as part of the greg
output, and logged to a file
called `tests.dbg'. The extra debugging output does not
normally appear on standard output.
--help
greg
options, then exits
(even if you also specify other options).
--objdir path
make
.
--outdir path
greg
. This option affects only the
log and the debug files `tool.log' and
`tool.dbg'.
--srcdir path
greg
looks in this directory for any subdirectory whose name
matches any toolname (specified with --tool).
--verbose
-v
greg
shows only the output of tests that produce unexpected
results; that is, tests with status FAIL (unexpected failure),
UPASS (unexpected success), or ERROR (a severe error in the test
case itself).
Specifying --verbose to see output for tests with status PASS
(success, as expected) and XFAIL (failure, as expected). It also
causes a more detailed summary to be displayed.--version
-V
Go to the first, previous, next, last section, table of contents.