BatchtoolsParam-class {BiocParallel} | R Documentation

## Enable parallelization on batch systems

### Description

This class is used to parameterize scheduler options on managed high-performance computing clusters using batchtools.

`BatchtoolsParam()`: Construct a BatchtoolsParam-class object.

`batchtoolsWorkers()`: Return the default number of workers for each backend.

`batchtoolsTemplate()`: Return the default template for each backend.

`batchtoolsCluster()`: Return the default cluster.

`batchtoolsRegistryargs()`: Create a list of arguments to be used in batchtools' `makeRegistry`; see `registryargs` argument.

### Usage

    BatchtoolsParam(
        workers = batchtoolsWorkers(cluster),
        cluster = batchtoolsCluster(),
        registryargs = batchtoolsRegistryargs(),
        resources = list(),
        template = batchtoolsTemplate(cluster),
        stop.on.error = TRUE, progressbar = FALSE, RNGseed = NA_integer_,
        timeout = 30L * 24L * 60L * 60L, exportglobals=TRUE,
        log = FALSE, logdir = NA_character_, resultdir=NA_character_,
        jobname = "BPJOB"
    )
    batchtoolsWorkers(cluster = batchtoolsCluster())
    batchtoolsCluster(cluster)
    batchtoolsTemplate(cluster)
    batchtoolsRegistryargs(...)

### Arguments

`workers` | `integer(1)`

Number of workers to divide tasks (e.g., elements in the first argument of `bplapply`) between. On 'multicore' and 'socket' backends, this defaults to `multicoreWorkers()` and `snowWorkers()`. On managed (e.g., slurm, SGE) clusters `workers` has no default, meaning that the number of workers needs to be provided by the user.

`cluster` | `character(1)`

Cluster type being used as the backend by `BatchtoolsParam`. The available options are "socket", "multicore", "interactive", "sge", "slurm", "lsf", "torque" and "openlava". The cluster type if available on the machine registers as the backend. Cluster types which need a `template` are "sge", "slurm", "lsf", "openlava", and "torque". If the template is not given then a default is selected from the `batchtools` package.

`registryargs` | `list()`

Arguments given to the registry created by `BatchtoolsParam` to configure the registry and where it's being stored. The `registryargs` can be specified by the function `batchtoolsRegistryargs()` which takes the arguments `file.dir`, `work.dir`, `packages`, `namespaces`, `source`, `load`, `make.default`. It's useful to configure these option, especially the `file.dir` to a location which is accessible to all the nodes on your job scheduler i.e master and workers. `file.dir` uses a default setting to make a registry in your working directory.

`resources` | `named list()`

Arguments passed to the `resources` argument of `batchtools::submitJobs` during evaluation of `bplapply` and similar functions. These name-value pairs are used for substitution into the template file.

`template` | `character(1)`

Path to a template for the backend in `BatchtoolsParam`. It is possible to check which template is being used by the object using the getter `bpbackend(BatchtoolsParam())`. The template needs to be written specific to each backend. Please check the list of available templates in the `batchtools` package.

`stop.on.error` | `logical(1)`

Stop all jobs as soon as one jobs fails (`stop.on.error == TRUE`) or wait for all jobs to terminate. Default is `TRUE`.

`progressbar` | `logical(1)`

Suppress the progress bar used in BatchtoolsParam and be less verbose. Default is `FALSE`.

`RNGseed` | `integer(1)`

Set an initial seed for the RNG. Default is `NULL` where a random seed is chosen upon initialization.

`timeout` | `list()`

Time (in seconds) allowed for worker to complete a task. If the computation exceeds `timeout` an error is thrown with message 'reached elapsed time limit'.

`exportglobals` | `logical(1)`

Export `base::options()` from manager to workers? Default `TRUE`.

`log` | `logical(1)`

Option given to save the logs which are produced by the jobs. If `log=TRUE` then the `logdir` option must be specified.

`logdir` | `character(1)`

Path to location where logs are stored. The argument `log=TRUE` is required before using the logdir option.

`resultdir` | `logical(1)`

Path where results are stored.

`jobname` | `character(1)`

Job name that is prepended to the output log and result files. Default is "BPJOB".

`...` | name-value pairs

Names and values correspond to arguments from batchtools `makeRegistry`.

### BatchtoolsParam constructor

Return an object with specified values. The object may be saved to disk or reused within a session.

### Methods

The following generics are implemented and perform as documented on the corresponding help page: `bpworkers`, `bpnworkers`, `bpstart`, `bpstop`, `bpisup`, `bpbackend`.

`bplapply` handles arguments `X` of classes derived from `S4Vectors::List` specially, coercing to `list`.

### Author(s)

Nitesh Turaga, mailto:nitesh.turaga@roswellpark.org

### See Also

`getClass("BiocParallelParam")` for additional parameter classes.

`register` for registering parameter classes for use in parallel evaluation.

The batchtools package.

### Examples

    ## Pi approximation
    piApprox = function(n) {
        nums = matrix(runif(2 * n), ncol = 2)
        d = sqrt(nums[, 1]^2 + nums[, 2]^2)
        4 * mean(d <= 1)
    }

    piApprox(1000)

    ## Calculate piApprox 10 times
    param <- BatchtoolsParam()
    result <- bplapply(rep(10e5, 10), piApprox, BPPARAM=param)

    ## Not run:
    ## see vignette for additional explanation
    library(BiocParallel)
    param = BatchtoolsParam(workers=5,
                            cluster="sge",
                            template="script/test-sge-template.tmpl")
    ## Run parallel job
    result = bplapply(rep(10e5, 100), piApprox, BPPARAM=param)

    ## bpmapply
    param = BatchtoolsParam()
    result = bpmapply(fun, x = 1:3, y = 1:3, MoreArgs = list(z = 1),
                       SIMPLIFY = TRUE, BPPARAM = param)

    ## bpvec
    param = BatchtoolsParam(workers=2)
    result = bpvec(1:10, seq_along, BPPARAM=param)

    ## bpvectorize
    param = BatchtoolsParam(workers=2)
    ## this returns a function
    bpseq_along = bpvectorize(seq_along, BPPARAM=param)
    result = bpseq_along(1:10)

    ##bpiterate
    ITER <- function(n=5) {
            i <- 0L
            function() {
                i <<- i + 1L
                if (i > n)
                    return(NULL)
            rep(i, n)
            }
        }

    param <- BatchtoolsParam()
    res <- bpiterate(ITER=ITER(), FUN=function(x,y) sum(x) + y, y=10, BPPARAM=param)

    ## End(Not run)

[Package BiocParallel version 1.16.6 Index]