SYNOPSIS
SWAT is Simple Web Application Test ( Tool )
$ swat examples/google/ google.ru
/home/vagrant/.swat/reports/google.ru/00.t ..
# start swat for google.ru/
# try num 2
ok 1 - successful response from GET google.ru/
# data file: /home/vagrant/.swat/reports/google.ru/content.GET.txt
ok 2 - GET / returns 200 OK
ok 3 - GET / returns Google
1..3
ok
All tests successful.
Files=1, Tests=3, 12 wallclock secs ( 0.00 usr 0.00 sys + 0.02 cusr 0.00 csys = 0.02 CPU)
Result: PASS
WHY
I know there are a lot of tests tool and frameworks, but let me briefly
tell *why* I created swat. As devops I update a dozens of web
application weekly, sometimes I just have *no time* sitting and wait
while dev guys or QA team ensure that deploy is fine and nothing breaks
on the road. So I need a tool to run smoke tests against web
applications. Not tool only, but the way to create such a tests from the
scratch in way easy and fast enough.
So this how I came up with the idea of swat.
Key features
SWAT:
* is very pragmatic tool designed for job to be done in a fast and
simple way
* has simple and yet flexible DSL with low price mastering ( see my
tutorial )
* produces TAP output
* leverages famous perl prove
and curl utilities
* has minimal dependency tree and probably will run out of the box on
most linux environments, provided that one has perl/bash/find/curl
by hand ( which is true for most cases )
Tutorial
Install swat
stable release
sudo cpan install swat
developer release
# developer release might be untested and unstable
sudo cpanm --mirror-only --mirror https://stratopan.com/melezhik/swat-release/master swat
Once swat is installed you have swat command line tool to run swat
tests, but before do this you need to create them.
Create tests
mkdir my-app/ # create a project root directory to contain tests
# define http URIs application should response to
mkdir -p my-app/hello # GET /hello
mkdir -p my-app/hello/world # GET /hello/world
# define the content to return by URIs
echo 200 OK >> my-app/hello/get.txt
echo 200 OK >> my-app/hello/world/get.txt
echo 'This is hello' >> my-app/hello/get.txt
echo 'This is hello world' >> my-app/hello/world/get.txt
Run tests
swat ./my-app http://127.0.0.1
DSL
Swat DSL consists of 2 parts. Routes and Swat Data.
Routes
Routes are http resources a tested web application should have.
Swat utilize file system to get know about routes. Let we have a
following project layout:
example/my-app/
example/my-app/hello/
example/my-app/hello/get.txt
example/my-app/hello/world/get.txt
When you give swat a run
swat example/my-app 127.0.0.1
It will find all the *directories with get.txt or post.txt files inside*
and "create" routes:
GET hello/
GET hello/world
When you are done with routes you need to set swat data.
Swat data
Swat data is DSL to describe/generate validation checks you apply to
content returned from web application.
Swat data is stored in swat data files, named get.txt or post.txt.
The validation process looks like:
* Swat recursively find files named get.txt or post.txt in the project
root directory to get swat data.
* Swat parse swat data file and *execute* entries found. At the end of
this process swat creates a *final check list* with "Check
Expressions".
* For every route swat makes http requests to web application and
store content into text file
* Every line of text file is validated by every item in a *final check
list*
*Objects* found in test data file are called *swat entries*. There are
*3 basic type* of swat entries:
* Check Expressions
* Comments
* Perl Expressions and Generators
Check Expressions
This is most usable type of entries you may define at swat data file.
*It's just a string should be returned* when swat request a given URI.
Here are examples:
200 OK
Hello World
Hello World
Using regexps
Regexps are check expressions with the usage of instead of plain strings checks. Everything started with
"regexp:" marker would be treated as perl regular expression.
# this is example of regexp check
regexp: App Version Number: (\d+\.\d+\.\d+)
Comments
Comments entries are lines started with "#" symbol, swat will ignore
comments when parse swat data file. Here are examples.
# this http status is expected
200 OK
Hello World # this string should be in the response
Hello World # and it should be proper html code
Perl Expressions
Perl expressions are just a pieces of perl code to *get evaled* by swat
when parsing test data files.
Everything started with "code:" marker would be treated by swat as perl
code to execute. There are a *lot of possibilities*! Please follow
Test::More documentation to get more info about useful function you may
call here.
code: skip('next test is skipped',1) # skip next check forever
HELLO WORLD
code: skip('next test is skipped',1) unless $ENV{'debug'} == 1 # conditionally skip this check
HELLO SWAT
Generators
Swat entries generators is the way to *create new swat entries on the
fly*. Technically speaking it's just a perl code which should return an
array reference: Generators are very close to perl expressions (
generators code is also get evaled ) with maijor difference:
Value returned from generator's code should be array reference. The
array is passed back to swat parser so it can create new swat entries
from it.
Generators entries start with ":generator" marker. Here is example:
# Place this in swat data file
generator: [ qw{ foo bar baz } ]
This generator will generate 3 swat entries:
foo
bar
baz
As you can guess an array returned by generator should contain *perl
strings* representing swat entries, here is another example: with
generator producing still 3 swat entities 'foo', 'bar', 'baz' :
# Place this in swat date file
generator: my %d = { 'foo' => 'foo value', 'bar' => 'bar value' }; [ map { ( "# $_", "$data{$_}" ) } keys %d ]
This generator will generate 3 swat entities:
# foo
foo value
# bar
bar value
There is no limit for you! Use any code you want with only requirement -
it should return array reference. What about to validate web application
content with sqlite database entries?
# Place this in swat data file
generator: \
use DBI; \
my $dbh = DBI->connect("dbi:SQLite:dbname=t/data/test.db","",""); \
my $sth = $dbh->prepare("SELECT name from users"); \
$sth->execute(); \
my $results = $sth->fetchall_arrayref; \
[ map { $_->[0] } @${results} ]
As an example take a loot at examples/swat-generators-sqlite3 project
Multiline expressions
Sometimes code looks more readable when you split it on separate chunks.
When swat parser meets "\" symbols it postpone entry execution and add
next line to buffer. This is repeated till no "\" found on next. Finally
swat execute *"accumulated"* swat entity.
Here are some exmaples:
# Place this in swat data file
generator: \
my %d = { \
'foo' => 'foo value', \
'bar' => 'bar value', \
'baz' => 'baz value' \
}; \
[ \
map { ( "# $_", "$data{$_}" ) } keys %d \
] \
# Place this in swat data file
generator: [ \
map { \
uc($_) \
} qw( foo bar baz ) \
]
code: \
if $ENV{'debug'} == 1 { # conditionally skip this check \
skip('next test is skipped',1) \
}
HELLO SWAT
Multiline expressions are only allowable for perl expressions and
generators
Generators and Perl Expressions Scope
Swat uses *perl string eval* when process generators and perl
expressions code, be aware of this. Follow
to get more on this.
PERL5LIB
Swat adds $project_root_directory/lib to PERL5LIB , so this is
convenient convenient to place here custom perl modules:
example/my-app/lib/Foo/Bar/Baz.pm
As an example take a loot at examples/swat-generators-with-lib/ project
Anatomy of swat
Once swat runs it goes through some steps to get job done. Here is
description of such a steps executed in orders
Run iterator over swat data files
Swat iterator look for all files named get.txt or post.txt under project
root directory. Actually this is simple bash find loop.
Parse swat data file
For every swat data file find by iterator parsing process starts. Swat
parse data file line by line, at the end of such a process *a list of
Test::More asserts* is generated. Finally asserts list and other input
parameters are serialized as Test::More test scenario written into into
proper *.t file.
Give it a run by prove
Once swat finish parsing all the swat data files there is a whole bunch
of *.t files kept under a designated temporary directory, thus every
swat route maps into Test::More test file with the list of asserts. Now
all is ready for prove run. Internally `prove -r ` command is issued to
run tests and generate TAP report. That is it.
Below is example hwo this looks like
project structure
vagrant@Debian-jessie-amd64-netboot:~/projects/swat$ tree examples/anatomy/
examples/anatomy/
|----FOO
|-----|----BARs
| |---- post.txt
|--- FOOs
|--- get.txt
3 directories, 2 files
swat data files
# /FOOs
FOO
FOO2
generator: | %w{ FOO3 FOO4 }|
# /FOO/BARs
BAR
BAR2
generator: | %w{ BAR3 BAR4 }|
code: skip('skip next 2 tests',2);
BAR5
BAR6
BAR7
Test::More Asserts list
# /FOOs/0.t
SKIP {
ok($status, "successful response from GET $host/FOOs")
ok($status, "GET /FOOs returns FOO")
ok($status, "GET /FOOs returns FOO2")
ok($status, "GET /FOOs returns FOO3")
ok($status, "GET /FOOs returns FOO4")
}
# /FOO/BARs0.t
SKIP {
ok($status, "successful response from POST $host/FOO/BARs")
ok($status, "POST /FOO/BARs returns BAR")
ok($status, "POST /FOO/BARs returns BAR")
ok($status, "POST /FOO/BARs returns BAR3")
ok($status, "POST /FOO/BARs returns BAR4")
skip('skip next 2 tests',2);
ok($status, "POST /FOO/BARs returns BAR5")
ok($status, "POST /FOO/BARs returns BAR6")
ok($status, "POST /FOO/BARs returns BAR7")
}
Hooks
Hooks are files containing any perl code to be `required` into the
beginning of every swat test. There are 2 types of hooks:
* project based hooks
File located at "$project_root_directory/hook.pm". Project based
hooks are applied for every route in project and could be used for
*project initialization*. For example one could define generators
here:
# place this in hook.pm file:
sub list1 { | %w{ foo bar baz } | }
sub list2 { | %w{ red green blue } | }
# now we could use it in swat data file
generator: list()
generator: list2()
* route based hooks
File located at "$project_root_directory/$route_directory/hook.pm".
Routes based hook are route specific hooks and could be used for
*route initialization*. For example one could define route specific
generators here:
# place this in hook.pm file:
# notices that we could tell GET from POST http methods here:
sub list1 {
my $list;
if ($method eq 'GET') {
$list = | %w{ GET_foo GET_bar GET_baz } |
}elsif($method eq 'POST'){
$list = | %w{ POST_foo POST_bar POST_baz } |
}else{
die "method $method is not supported"
}
$list;
}
# now we could use it in swat data file
generator: list()
Post requests
Name swat data file as post.txt to make http POST requests.
echo 200 OK >> my-app/hello/post.txt
echo 200 OK >> my-app/hello/world/post.txt
You may use curl_params setting ( follow "Swat Settings" section for
details ) to define post data, there are some examples:
* "-d" - Post data sending by html form submit.
# Place this in swat.ini file or sets as env variable:
curl_params='-d name=daniel -d skill=lousy'
* "--data-binary" - Post data sending as is.
# Place this in swat.ini file or sets as env variable:
curl_params=`echo -E "--data-binary '{\"name\":\"alex\",\"last_name\":\"melezhik\"}'"`
curl_params="${curl_params} -H 'Content-Type: application/json'"
Dynamic routes
There are possibilities to create a undetermined routes using ":path"
placeholders. Let say we have application confirming GET /foo/:whatever
requests where :whatevery is arbitrary sting like: GET /foo/one or
/foo/two or /foo/baz. Using dynamic routes we could write an swat test
for it.
First let's create difinition for `whatever` path in swat.ini file. This
is as simple as create bash variable with a randome sting value:
# Place this in swat.ini file
export whatever=`cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 5 | head -n 1`
Now we should inform swat to use bash varibale $whatever when generating
request for /foo/whatever
$ mkdir foo/:whatever
And finaly drop some check expressions for it:
$ echo 'generator [ $ENV{"whatever"} ]' > foo/:whatever/get.txt
Of cousre there are as many dynamic parts in http requests as you need:
# Place this in swat.ini file
export whatever=`cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 5 | head -n 1`
export whenever=`date +%s`
$ mkdir -p foo/:whatever/:whenever
$ echo 'generator [ $ENV{"whatever"}, $ENV{"whenever"} ]' > foo/:whatever/:whenever/get.txt
Swat Settings
Swat comes with settings defined in two contexts:
* Environmental Variables
* swat.ini files
Environmental Variables
Defining a proper environment variables will provide swat settings.
* "debug" - set to 1 if you want to see some debug information in
output, default value is 0
* "debug_bytes" - number of bytes of http response to be dumped out
when debug is on. default value is 500
* "ignore_http_err" - ignore http errors, if this parameters is off
(set to 1) returned *error http codes* will not result in test
fails, useful when one need to test something with response differ
from 2**,3** http codes. Default value is 0
* "try_num" - number of http requests attempts before give it up (
useless for resources with slow response ), default value is 2
* "curl_params" - additional curl parameters being add to http
requests, default value is "", follow curl documentation for variety
of values for this
* "curl_connec_timeout" - follow curl documentation
* "curl_max_time" - follow curl documentation
* "port" - http port of tested host, default value is 80
Swat.ini files
Swat checks files named "swat.ini" in the following directories
* ~/swat.ini
* $project_root_directory/swat.ini
* $route_directory/swat.ini
Here are examples of locations of swat.ini files:
~/swat.ini # home directory swat.ini file
my-app/swat.ini # project_root directory swat.ini file
my-app/hello/get.txt
my-app/hello/swat.ini # route directory swat.ini file ( route hello )
my-app/hello/world/get.txt
my-app/hello/world/swat.ini # route directory swat.ini file ( route hello/world )
Once file exists at any location swat simply bash sources it to apply
settings.
Thus swat.ini file should be bash file with swat variables definitions.
Here is example:
# the content of swat.ini file:
curl_params="-H 'Content-Type: text/html'"
debug=1
Settings priority table
Here is the list of settings/contexts in priority ascending order:
| context | location | priority level |
| ------------------------|------------------------ | --------------- |
| swat.ini file | ~/swat.ini | 1 |
| environmental variables | --- | 2 |
| swat.ini file | project root directory | 3 |
| swat.ini file | route directory | 4 |
Swat processes settings *in order*. For every route found swat:
* Clear all settings
* Apply settings from environmental variables ( if any given )
* Apply settings from swat.ini file in home directory ( if any given )
* Apply settings from swat.ini file in project root directory ( if any
given )
* And finally apply settings from swat.ini file in route directory (
if any given )
TAP
Swat produces output in TAP format , that
means you may use your favorite tap parsers to bring result to another
test / reporting systems, follow TAP documentation to get more on this.
Here is example for converting swat tests into JUNIT format
swat $project_root $host --formatter TAP::Formatter::JUnit
See also "Prove settings" section.
Command line tool
Swat is shipped as cpan package, once it's installed ( see "Install
swat" section ) you have a command line tool called swat, this is usage
info on it:
swat
* host - is base url for web application you run tests against, you
also have to define swat routes, see DSL section.
* project_dir - is a project root directory
* swat_package - the name of swat package, see "Swat Packages" section
Prove settings
Swat utilize prove utility to run
tests, so all the swat options *are passed as is to prove utility*.
Follow prove utility
documentation for variety of values you may set here. Default value for
prove options is "-v". Here is another examples:
* "-q -s" - run tests in random and quite mode
Swat Packages
Swat packages is portable archives of swat tests. It's easy to create
your own swat packages and share with other.
This is mini how-to on creating swat packages:
Create swat package
Let's imagine you've got ready swat tests you want to distribute. First
of all you need to create _tar.gz archive_ of swat project root
directory:
tar -zcf $project_root_dir.tar.gz project_root_dir
For example for project ./examples/google
cd ./examples/
tar -zcf google.tar.gz ./google
Upload package to swat repository
Swat repository might be _ANY_ web server. One should upload archive
into server. Let's say we have nginx server. The example below is for
debian:
# install nginx:
sudo apt-get install nginx
# copy swat distributive:
sudo cp google.tar.gz /var/www/html/
# check for archive availability over web server:
curl -s 127.0.0.1/google.tar.gz -D - -o /dev/null | head -n 1
HTTP/1.1 200 OK
You swat repository with nginx swat package uploaded is ready!
Install swat package
Swat comes with utility called swatman to manage swat packages. First of
all one need to setup swat repository:
echo "swat_repo=127.0.0.1" >> ~/swat.ini
Then install package
swatman install google
Run swat tests
Once swat package is installed into your system you may give it a run:
swat google google.ru
Debugging
set "swat_debug" environmental variable to 1
Examples
./examples directory contains examples of swat tests for different
cases. Follow README.md files for details.
Dependencies
Not that many :)
* perl
* curl
* bash
* find
* head
* tar
AUTHOR
Aleksei Melezhik
Swat Project Home Page
https://github.com/melezhik/swat
Thanks
To the authors of ( see list ) without who swat would not appear to
light
* perl
* curl
* TAP
* Test::More
* prove