SYNOPSIS

    See bencher CLI.

DESCRIPTION

    EARLY WORK, LOTS OF UNIMPLEMENTED STUFFS.

    Bencher is a benchmark framework. It helps you:

      * specify what Perl code (functions/module names or coderefs) or
      external commands you want to benchmark

      along with a set of data (function or command-line arguments).

      * run the items

      You can run all the items, only some of them, with some/all
      combinations of arguments, with different module paths/versions,
      different perl paths, and so on.

      * save the result

      * display the result(s) and graph them

      * send the result to a server

SCENARIO

    The core data structure that you need to prepare is the scenario. It is
    a DefHash (i.e. just a regular Perl hash), the two most important keys
    of this hash are: participants and datasets.

    An example scenario (from Bench::Scenario::Example):

     package Bencher::Scenario::Example;
     our $scenario = {
         participants => [
             {fcall_template => q[Text::Wrap::wrap('', '', <text>)]},
         ],
         datasets => [
             { name=>"foobar x100",   args => {text=>"foobar " x 100} },
             { name=>"foobar x1000",  args => {text=>"foobar " x 1000} },
             { name=>"foobar x10000", args => {text=>"foobar " x 10000} },
         ],
     };
     1;

    participants (array) lists Perl code (or external command) that we want
    to benchmark. Instead of just list of coderefs like what Benchmark
    expects, you can use fcall_template instead. It is a string containing
    a function call code. From this value, Bencher can extract the name of
    the module and function used (and can help you load the modules,
    benchmark startup overhead of all involved modules, etc). It can also
    contain variables enclosed in angle brackets, like <text> which will be
    replaced with actual data/value later.

    You can also add name key to a participant so you can refer to it more
    easily later, e.g.:

     participants => [
         {name=>'pp', fcall_template=>'List::MoreUtils::PP::uniq(@{<array>})'},
         {name=>'xs', fcall_template=>'List::MoreUtils::XS::uniq(@{<array>})'},
     ],

    Aside from fcall_template, you can also use code_template (a string
    containing arbitrary code) or code (a subroutine reference, just like
    what you would provide to the Benchmark module).

    Or, if you are benchmarking commands, you specify cmdline (array or
    strings, or strings) instead. An array cmdline will not use shell,
    while the string version will use shell. See
    Bencher::Scenario::Interpreters.

    datasets (array) lists the function inputs (or command-line arguments).
    You can name each dataset too, to be able to refer to it more easily.

    You can instruct the bencher CLI to filter wanted/unwanted modules,
    participants, or datasets before benchmarking.

    Other known scenario properties (keys):

      * name

      From DefHash, scenario name (usually short and one word).

      * summary

      From DefHash, a one-line plaintext summary.

      * description (str)

      From DefHash, longer description in Markdown.

      * on_failure (str, "skip"|"die")

      The default is "die". When set to "skip", will first run the code of
      each item before benchmarking and trap command failure/Perl exception
      and if that happens, will "skip" the item.

      Can be overriden in the CLI with --on-failure option.

SEE ALSO

    bencher

    BenchmarkAnything. There are lot of overlaps of goals between Bencher
    and this project. I hope to reuse or interoperate parts of
    BenchmarkAnything, e.g. storing Bencher results in a BenchmarkAnything
    storage backend, sending Bencher results to a BenchmarkAnything HTTP
    server, and so on.

    Benchmark, Benchmark::Dumb (Dumbbench)

    Bencher::Scenario::* for examples of scenarios.