- Events
-
Events
- yath uses Test2 events directly, serialized as json.
- No Test2 -> TAP -> Test2 conversion (lossless!)
- Any/All meta-data and debug info (file, line, etc) is preserved and available
- stderr/stdout are converted to events
- TAP parsing still occurs for tests that opt-out, or just print TAP
- Config
-
Project config
2 config files are supported, .yath.rc and .yath.user.rc.
.yath.rc is intended to be a project config that specifies plugins, preloads, and default arguments. This SHOULD be commited to the repository.
.yath.user.rc is intended to be local overrides to .yath.rc, it SHOULD NOT be commited to the repository.
Both files have the same layout:
- Timeouts
-
Event Timeout
If a test produces no output at all for 60 seconds (configurable) the
test will be killed and reported as a failure.
You can configure the timeout either at the command line, or provide
test-specific timeouts via directives written in the test itself:
Post-Exit Timeout
If a test has exited, but the output looks incomplete, yath will wait for more output, and will timeout after 15 seconds of silence.
This is also configurable:
- Preload
-
Preload
- Yath runs tests by forking a 'runner' process
- You can preload modules before tests are run
- This can save a LOT of time
- You can disable preload and/or forking at the command line
- You can disable preload/fork on a per-test basis with directives
- You can also write custom preloads with helpful features (covered later)
This has a profound impact on the performance of the Moose test suite:
- Plugins
-
Plugins
Plugins are a way to control many aspects of yath.
- Add custom CLI arguments
- Custom logic before and after init
- Custom logic after a run completes
- Custom test file searching
- Modify data about each test before running it
- Directives
-
- Overview
-
In-Test Directives
Directives are special comments at the top of test files that tell the harness the test needs special treatment.
There are 2 forms for directives, the first type is used to set key/value attributes:
The other style is for toggling behaviors on or off:
- Categories
-
Categories
This is the default category, implies nothing special.
Tests with this set cannot run concurrently with any other
tests that set this.
This test cannot run with any other tests, it must be run
non-concurrently.
This test takes a long time to complete. In a concurrent run
this will be run as soon as possible.
Currently an alias to 'long' but may not always be.
- Config
-
Configuration
Set the stage to run in (only useful with custom preloads).
Set the event timeout in seconds.
Set the post-exit timeout in seconds.
Set a custom key/value pair.
Do not run this test with preloads, start a clean process to run
it. Currently this is identical to NO-FORK but that may not always
be the case.
Do not run tests in a forked process, start a new process instead (fork+exec).
Wait forever, do not timeout.
Do not use the Test2 Stream formatter, use TAP instead.
- TempDir
-
Temporary Directories
- Yath has a single temp dir under which all temp data is kept
- When requested, yath will put the temp dir into shm
- Every test that yath runs has a temporary directory (under the main one) where output and meta-data are stored.
- Each test has the $TMPDIR env var set to a test-specific temp dir under the tests dir.
- This makes cleaning temp files easy, everything is nested under the main temp dir.
- Persist
-
Persistant mode
Persist mode lets you launch yath as a daemon that waits for you to run tests.
This mode is useful if you have a lot of preloads.
Yath will watch for changes and re-load when loaded modules are modified.
When a module does change it is added to a blacklist so it will not need to reload while you continue to edit it.
There are commands to start, stop, reload, and watch the persistant instance
If a persistant instance is found it will be used automatically unless another command is specified.