# Introduction to regionReport

Leonardo Collado-Torres1,2*

1Lieber Institute for Brain Development, Johns Hopkins Medical Campus
2Center for Computational Biology, Johns Hopkins University

*leo.collado@libd.org

#### 20 January 2018

#### Package

regionReport 1.12.2

# 1 Basics

## 1.1 Install regionReport

`R` is an open-source statistical environment which can be easily modified to enhance its functionality via packages. regionReport is a `R` package available via the Bioconductor repository for packages. `R` can be installed on any operating system from CRAN after which you can install regionReport by using the following commands in your `R` session:

    ## try http:// if https:// URLs are not supported
    source("https://bioconductor.org/biocLite.R")
    biocLite("regionReport")

    ## Check that you have a valid Bioconductor installation
    biocValid()

## 1.2 Required knowledge

regionReport is based on many other packages and in particular in those that have implemented the infrastructure needed for dealing with RNA-seq data. That is, packages like GenomicFeatures that allow you to import the data and DESeq2 for generating differential expression results. A regionReport user is not expected to deal with those packages directly.

If you are asking yourself the question “Where do I start using Bioconductor?” you might be interested in this blog post.

## 1.3 Asking for help

As package developers, we try to explain clearly how to use our packages and in which order to use the functions. But `R` and `Bioconductor` have a steep learning curve so it is critical to learn where to ask for help. The blog post quoted above mentions some but we would like to highlight the Bioconductor support site as the main resource for getting help: remember to use the `regionReport` tag and check the older posts. Other alternatives are available such as creating GitHub issues and tweeting. However, please note that if you want to receive help you should adhere to the posting guidelines. It is particularly critical that you provide a small reproducible example and your session information so package developers can track down the source of the error.

## 1.4 Citing regionReport

We hope that regionReport will be useful for your research. Please use the following information to cite the package and the overall approach. Thank you!

    ## Citation info
    citation('regionReport')

    ##
    ## Collado-Torres L, Jaffe AE and Leek JT (2016). "regionReport:
    ## Interactive reports for region-level and feature-level genomic
    ## analyses [version2; referees: 2 approved, 1 approved with
    ## reservations]." _F1000Research_, *4*, pp. 105. doi:
    ## 10.12688/f1000research.6379.2 (URL:
    ## http://doi.org/10.12688/f1000research.6379.2), <URL:
    ## http://f1000research.com/articles/4-105/v2>.
    ##
    ## Collado-Torres L, Nellore A, Frazee AC, Wilks C, Love MI, Langmead
    ## B, Irizarry RA, Leek JT and Jaffe AE (2017). "Flexible expressed
    ## region analysis for RNA-seq with derfinder." _Nucl. Acids Res._.
    ## doi: 10.1093/nar/gkw852 (URL: http://doi.org/10.1093/nar/gkw852),
    ## <URL:
    ## http://nar.oxfordjournals.org/content/early/2016/09/29/nar.gkw852>.
    ##
    ## Collado-Torres L, Jaffe AE and Leek JT (2017). _regionReport:
    ## Generate HTML or PDF reports for a set of genomic regions or
    ## DESeq2/edgeR results_. doi: 10.18129/B9.bioc.regionReport (URL:
    ## http://doi.org/10.18129/B9.bioc.regionReport),
    ## https://github.com/leekgroup/regionReport - R package version
    ## 1.12.2, <URL: http://www.bioconductor.org/packages/regionReport>.
    ##
    ## To see these entries in BibTeX format, use 'print(<citation>,
    ## bibtex=TRUE)', 'toBibtex(.)', or set
    ## 'options(citation.bibtex.max=999)'.

# 2 HTML reports for a set differential region results

regionReport (Collado-Torres, Jaffe, and Leek, 2016) creates HTML or PDF reports for a set of genomic regions such as derfinder (Collado-Torres, Nellore, Frazee, Wilks, et al., 2017) results or for feature-level analyses performed with DESeq2 (Love, Huber, and Anders, 2014) or edgeR (Robinson, McCarthy, and Smyth, 2010; McCarthy, J., Chen, Yunshun, et al., 2012; Chen, Lun, and Smyth, 2014). The HTML reports are styled with rmarkdown (Allaire, Xie, McPherson, Luraschi, et al., 2017) by default but can optionally be styled with knitrBootstrap (Hester, 2017).

This package includes a basic exploration for a general set of genomic regions which can be easily customized to include the appropriate conclusions and/or further exploration of the results. Such a report can be generated using `renderReport()`. regionReport has a separate template for running a basic exploration analysis of derfinder results by using `derfinderReport()`. `derfinderReport()` is specific to single base-level approach derfinder results. A third template is included for exploring DESeq2 or edgeR differential expression results.

All reports are written in R Markdown format and include all the code for making the plots and explorations in the report itself. For all templates, regionReport relies on knitr (Xie, 2014), rmarkdown (Allaire, Xie, McPherson, Luraschi, et al., 2017), `DT` (Xie, 2016) and optionally knitrBootstrap (Hester, 2017) for generating the report. The reports can be either in HTML or PDF format and can be easily customized.

# 3 Using regionReport for DESeq2 results

The plots in regionReport for exploring DESeq2 are powered by ggplot2 (Wickham, 2009) and pheatmap (Kolde, 2015).

## 3.1 Examples

The regionReport supplementary website regionReportSupp has examples of using regionReport with DESeq2 results. In particular, please look at DESeq2.html which has the code for generating some DESeq2 results based on the DESeq2 vignette. Then it uses those results to create HTML and PDF versions of the same report. The resulting reports are available in the following locations:

- HTML version
- PDF version

Note that in both examples we changed the ggplot2 theme to `theme_bw()`. Also, in the PDF version we used the option `device = 'pdf'` instead of the default `device = 'png'` in `DESeq2Report()` since PDF figures are more appropriate for PDF reports: they look better than PNG figures.

If you want to create a similar HTML report as the one linked in this section, simply run `example('DESeq2Report', 'regionReport', ask=FALSE)`. The only difference will be the ggplot2 theme for the plots.

# 4 Using regionReport for edgeR results

regionReport has the `edgeReport()` function that takes as input a `DGEList` object and the results from the differential expression analysis using edgeR. `edgeReport()` internally uses DEFormats to convert the results to DESeq2’s format and then uses `DESeqReport()` to create the final report. The report looks nearly the same whether you performed the differential expression analysis with DESeq2 or edgeR in order to make more homogenous the exploratory data analysis step.

## 4.1 Examples

The regionReport supplementary website regionReportSupp has examples of using regionReport with edgeR results. In particular, please look at edgeR.html which has the code for generating some random data with DEFormats and performing the differential expression analysis with edgeR. Then it uses those results to create HTML and PDF versions of the same report. The resulting reports are available in the following locations:

- HTML version
- PDF version

Note that in both examples we changed the ggplot2 theme to `theme_linedraw()`. Also, in the PDF version we used the option `device = 'pdf'` instead of the default `device = 'png'` in `edgeReport()` since PDF figures are more appropriate for PDF reports: they look better than PNG figures.

If you want to create a similar HTML report as the one linked in this section, simply run `example('edgeReport', 'regionReport', ask=FALSE)`. The only difference will be the ggplot2 theme for the plots and the amount of data simulated with DEFormats.

# 5 Using regionReport for region results

The plots in regionReport for region reports are powered by derfinderPlot (Collado-Torres, Jaffe, and Leek, 2017), ggbio (Yin, Cook, and Lawrence, 2012), and ggplot2 (Wickham, 2009).

## 5.1 Examples

The regionReport supplementary website regionReportSupp has examples of using regionReport with results from DiffBind and derfinder. Included as a vignette, this package also has an example using a small data set derived from bumphunter. These represent different uses of regionReport for results from ChIP-seq, methylation, and RNA-seq data. In particular, the DiffBind example illustrates how to expand a basic report created with `renderReport()`.

## 5.2 General case

For a general use case, you first have to identify a set of genomic regions of interest and store it as a `GRanges` object. In a typical workflow you will have some variables measured for each of the regions, such as p-values and scores. `renderReport()` uses the set of regions and three main arguments:

- `pvalueVars`: this is a character vector (named optionally) with the names of the variables that are bound between 0 and 1, such as p-values. For each of these variables, `renderReport()` explores the distribution by chromosome, the overall distribution, and makes a table with commonly used cutoffs.
- `densityVars`: is another character vector (named optionally) with another set of variables you wish to explore by making density graphs. This is commonly used for scores and other similar numerical variables.
- `significantVar`: is a logical vector separating the regions into by whether they are statistically significant. For example, this information is used to explore the width of all the regions and compare it the significant ones.

Other parameters control the name of the report, where it’ll be located, the transcripts database used to annotate the nearest genes, graphical parameters, etc.

Here is a short example of how to use `renderReport()`. Note that we are using regions produced by derfinder just for convenience sake. You can also run this example by using `example('renderReport', 'regionReport', ask=FALSE)`.

    ## Load derfinder
    library('derfinder')
    regions <- genomeRegions$regions

    ## Assign chr length
    library('GenomicRanges')
    seqlengths(regions) <- c('chr21' = 48129895)

    ## The output will be saved in the 'derfinderReport-example' directory
    dir.create('renderReport-example', showWarnings = FALSE, recursive = TRUE)

    ## Generate the HTML report
    report <- renderReport(regions, 'Example run', pvalueVars = c(
        'Q-values' = 'qvalues', 'P-values' = 'pvalues'), densityVars = c(
        'Area' = 'area', 'Mean coverage' = 'meanCoverage'),
        significantVar = regions$qvalues <= 0.05, nBestRegions = 20,
        outdir = 'renderReport-example')

For derfinder results created via the expressed regions-level approach you can use `renderReport()` to explore the results. If you use DESeq2 to perform the differential expression analysis of the expressed regions you can then use `DESeq2Report()`.

## 5.3 derfinder single base-level case

### 5.3.1 Run derfinder

Prior to using `regionReport::derfinderReport()` you must use derfinder to analyze a specific data set. While there are many ways to do so, we recommend using analyzeChr() with the same prefix argument. Then merging the results with mergeResults(). This is the recommended pipeline for the single base-level approach.

Below, we run derfinder for the example data included in the package. The steps are:

1. Load derfinder
2. Create a directory where we’ll store the results
3. Generate the pre-requisites for the models to use with the example data
4. Generate the statistical models
5. Analyze the example data for chr21
6. Merge the results (only one chr in this case, but in practice there’ll be more)

    ## Load derfinder
    library('derfinder')

    ## The output will be saved in the 'report' directory
    dir.create('report', showWarnings = FALSE, recursive = TRUE)

The following code runs derfinder.

    ## Save the current path
    initialPath <- getwd()
    setwd(file.path(initialPath, 'report'))

    ## Generate output from derfinder

    ## Collapse the coverage information
    collapsedFull <- collapseFullCoverage(list(genomeData$coverage),
    verbose=TRUE)

    ## Calculate library size adjustments
    sampleDepths <- sampleDepth(collapsedFull, probs=c(0.5), nonzero=TRUE,
    verbose=TRUE)

    ## Build the models
    group <- genomeInfo$pop
    adjustvars <- data.frame(genomeInfo$gender)
    models <- makeModels(sampleDepths, testvars=group, adjustvars=adjustvars)

    ## Analyze chromosome 21
    analysis <- analyzeChr(chr='21', coverageInfo=genomeData, models=models,
    cutoffFstat=1, cutoffType='manual', seeds=20140330, groupInfo=group,
    mc.cores=1, writeOutput=TRUE, returnOutput=TRUE)

    ## Save the stats options for later
    optionsStats <- analysis$optionsStats

    ## Change the directory back to the original one
    setwd(initialPath)

For convenience, we have included the derfinder results as part of regionReport. Note that the above functions are routinely checked as part of derfinder.

    ## Copy previous results
    file.copy(system.file(file.path('extdata', 'chr21'), package='derfinder',
    mustWork=TRUE), 'report', recursive=TRUE)

    ## [1] TRUE

Next, proceed to merging the results.

    ## Merge the results from the different chromosomes. In this case, there's
    ## only one: chr21
    mergeResults(chrs = 'chr21', prefix = 'report',
        genomicState = genomicState$fullGenome)

    ## 2018-01-20 20:53:08 mergeResults: Saving options used

    ## 2018-01-20 20:53:08 Loading chromosome chr21

    ## Neither 'cutoffFstatUsed' nor 'optionsStats' were supplied, so the FWER calculation step will be skipped.

    ## 2018-01-20 20:53:08 mergeResults: Saving fullNullSummary

    ## 2018-01-20 20:53:08 mergeResults: Re-calculating the p-values

    ## 2018-01-20 20:53:08 mergeResults: Saving fullRegions

    ## 2018-01-20 20:53:08 mergeResults: assigning genomic states

    ## 2018-01-20 20:53:08 annotateRegions: counting

    ## 2018-01-20 20:53:09 annotateRegions: annotating

    ## 2018-01-20 20:53:09 mergeResults: Saving fullAnnotatedRegions

    ## 2018-01-20 20:53:09 mergeResults: Saving fullFstats

    ## 2018-01-20 20:53:09 mergeResults: Saving fullTime

    ## Load optionsStats
    load(file.path('report', 'chr21', 'optionsStats.Rdata'))

### 5.3.2 Create report

Once the derfinder output has been generated and merged, use derfinderReport() to create the HTML report.

    ## Load derfindeReport
    library('regionReport')

    ## Generate the HTML report
    report <- derfinderReport(prefix='report', browse=FALSE,
        nBestRegions=15, makeBestClusters=TRUE, outdir='html',
        fullCov=list('21'=genomeDataRaw$coverage), optionsStats=optionsStats)

    ## Writing 9 Bibtex entries ... OK
    ## Results written to file 'report/html/basicExploration.bib'
    ## extendedMapSeqlevels: sequence names mapped from NCBI to UCSC for species homo_sapiens
    ## processing file: basicExploration.Rmd
    ## Scale for 'x' is already present. Adding another scale for 'x', which will
    ## replace the existing scale.
    ## Scale for 'x' is already present. Adding another scale for 'x', which will
    ## replace the existing scale.
    ## Scale for 'x' is already present. Adding another scale for 'x', which will
    ## replace the existing scale.
    ## Scale for 'x' is already present. Adding another scale for 'x', which will
    ## replace the existing scale.
    ## Parsing transcripts...
    ## No transcripts found at this region.
    ## Parsing transcripts...
    ## Parsing exons...
    ## Parsing cds...
    ## Parsing utrs...
    ## ------exons...
    ## ------cdss...
    ## ------introns...
    ## ------utr...
    ## aggregating...
    ## Done
    ## Constructing graphics...
    ## output file: basicExploration.knit.md

    ## "C:/PROGRA~2/Pandoc/pandoc" +RTS -K512m -RTS basicExploration.utf8.md --to html --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash+smart --output basicExploration.html --email-obfuscation none --self-contained --standalone --section-divs --table-of-contents --toc-depth 3 --variable toc_float=1 --variable toc_selectors=h1,h2,h3 --variable toc_collapsed=1 --variable toc_smooth_scroll=1 --variable toc_print=1 --template "C:\Users\biocbuild\bbs-3.6-bioc\tmpdir\RtmpK8ganD/BiocStyle/template.html" --no-highlight --variable highlightjs=1 --number-sections --css "C:\Users\biocbuild\bbs-3.6-bioc\R\library\BiocStyle\resources\html\bioconductor2.css" --variable "theme:bootstrap" --include-in-header "C:\Users\biocbuild\bbs-3.6-bioc\tmpdir\RtmpK8ganD\rmarkdown-str11f42c133c02.html" --mathjax --variable "mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" --variable code_folding=hide --variable code_menu=1

    ##
    ## Output created: basicExploration.html

Once the output is generated, you can browse the report from `R` using browseURL() as shown below.

    ## Browse the report
    browseURL(report)

### 5.3.3 Notes

Note that the reports require an active Internet connection to render correctly.

The report is self-explanatory and will change some of the text depending on the input options.

If the report is taking too long to compile (say more than 3 hours), you might want to consider setting nBestCluters to a small number or even set makeBestClusters to `FALSE`.

# 6 Reproducibility

This package was made possible thanks to:

- R (R Core Team, 2017)
- BiocStyle (Oles, Morgan, and Huber, 2018)
- biovizBase (Yin, Lawrence, and Cook, 2018)
- bumphunter (Jaffe, Murakami, Lee, Leek, et al., 2012)
- DEFormats (Oles, 2018)
- derfinder (Collado-Torres, Nellore, Frazee, Wilks, et al., 2017)
- derfinderPlot (Collado-Torres, Jaffe, and Leek, 2017)
- DESeq2 (Love, Huber, and Anders, 2014)
- devtools (Wickham and Chang, 2017)
- DT (Xie, 2016)
- edgeR (Robinson, McCarthy, and Smyth, 2010; McCarthy, J., Chen, Yunshun, et al., 2012; Chen, Lun, and Smyth, 2014)
- GenomeInfoDb (Arora, Morgan, Carlson, and Pagès, 2017)
- GenomicRanges (Lawrence, Huber, Pagès, Aboyoun, et al., 2013)
- ggbio (Yin, Cook, and Lawrence, 2012)
- ggplot2 (Wickham, 2009)
- grid (R Core Team, 2017)
- gridExtra (Auguie, 2017)
- IRanges (Lawrence, Huber, Pagès, Aboyoun, et al., 2013)
- knitcitations (Boettiger, 2017)
- knitr (Xie, 2014)
- knitrBootstrap (Hester, 2017)
- mgcv
- pheatmap (Kolde, 2015)
- RColorBrewer (Neuwirth, 2014)
- rmarkdown (Allaire, Xie, McPherson, Luraschi, et al., 2017)
- TxDb.Hsapiens.UCSC.hg19.knownGene (Carlson and Maintainer, 2015)
- whisker

Code for creating the vignette

    ## Create the vignette
    library('rmarkdown')
    system.time(render('regionReport.Rmd', 'BiocStyle::html_document'))

    ## Extract the R code
    library('knitr')
    knit('regionReport.Rmd', tangle = TRUE)

    ## Clean up
    file.remove('regionReportRef.bib')

    ## [1] TRUE

    unlink('report', recursive = TRUE)

Date the vignette was generated.

    ## [1] "2018-01-20 20:54:10 EST"

Wallclock time spent generating the vignette.

    ## Time difference of 1.068 mins

`R` session information.

    ## Session info ----------------------------------------------------------------------------------------------------------

    ##  setting  value
    ##  version  R version 3.4.3 (2017-11-30)
    ##  system   x86_64, mingw32
    ##  ui       RTerm
    ##  language (EN)
    ##  collate  C
    ##  tz       America/New_York
    ##  date     2018-01-20

    ## Packages --------------------------------------------------------------------------------------------------------------

    ##  package                           * version   date       source
    ##  acepack                             1.4.1     2016-10-29 CRAN (R 3.4.3)
    ##  annotate                            1.56.1    2018-01-20 Bioconductor
    ##  AnnotationDbi                     * 1.40.0    2018-01-20 Bioconductor
    ##  AnnotationFilter                    1.2.0     2018-01-20 Bioconductor
    ##  AnnotationHub                       2.10.1    2018-01-20 Bioconductor
    ##  assertthat                          0.2.0     2017-04-11 CRAN (R 3.4.3)
    ##  backports                           1.1.2     2017-12-13 CRAN (R 3.4.3)
    ##  base                              * 3.4.3     2017-11-30 local
    ##  base64enc                           0.1-3     2015-07-28 CRAN (R 3.4.1)
    ##  bibtex                              0.4.2     2017-06-30 CRAN (R 3.4.3)
    ##  Biobase                           * 2.38.0    2018-01-20 Bioconductor
    ##  BiocGenerics                      * 0.24.0    2018-01-20 Bioconductor
    ##  BiocInstaller                       1.28.0    2018-01-20 Bioconductor
    ##  BiocParallel                        1.12.0    2018-01-20 Bioconductor
    ##  BiocStyle                         * 2.6.1     2018-01-20 Bioconductor
    ##  biomaRt                             2.34.2    2018-01-20 Bioconductor
    ##  Biostrings                          2.46.0    2018-01-20 Bioconductor
    ##  biovizBase                        * 1.26.0    2018-01-21 Bioconductor
    ##  bit                                 1.1-12    2014-04-09 CRAN (R 3.4.1)
    ##  bit64                               0.9-7     2017-05-08 CRAN (R 3.4.1)
    ##  bitops                              1.0-6     2013-08-17 CRAN (R 3.4.1)
    ##  blob                                1.1.0     2017-06-17 CRAN (R 3.4.3)
    ##  bookdown                            0.5       2017-08-20 CRAN (R 3.4.3)
    ##  BSgenome                            1.46.0    2018-01-20 Bioconductor
    ##  bumphunter                        * 1.20.0    2018-01-21 Bioconductor
    ##  checkmate                           1.8.5     2017-10-24 CRAN (R 3.4.3)
    ##  cluster                             2.0.6     2017-03-10 CRAN (R 3.4.3)
    ##  codetools                           0.2-15    2016-10-05 CRAN (R 3.4.3)
    ##  colorspace                          1.3-2     2016-12-14 CRAN (R 3.4.3)
    ##  compiler                            3.4.3     2017-11-30 local
    ##  curl                                3.1       2017-12-12 CRAN (R 3.4.3)
    ##  data.table                          1.10.4-3  2017-10-27 CRAN (R 3.4.3)
    ##  datasets                          * 3.4.3     2017-11-30 local
    ##  DBI                                 0.7       2017-06-18 CRAN (R 3.4.3)
    ##  DEFormats                           1.6.0     2018-01-20 Bioconductor
    ##  DelayedArray                        0.4.1     2018-01-20 Bioconductor
    ##  derfinder                         * 1.12.6    2018-01-21 Bioconductor
    ##  derfinderHelper                     1.12.0    2018-01-20 Bioconductor
    ##  derfinderPlot                     * 1.12.2    2018-01-21 Bioconductor
    ##  DESeq2                              1.18.1    2018-01-20 Bioconductor
    ##  devtools                          * 1.13.4    2017-11-09 CRAN (R 3.4.3)
    ##  dichromat                           2.0-0     2013-01-24 CRAN (R 3.4.1)
    ##  digest                              0.6.14    2018-01-14 CRAN (R 3.4.3)
    ##  doRNG                               1.6.6     2017-04-10 CRAN (R 3.4.3)
    ##  DT                                * 0.2       2016-08-09 CRAN (R 3.4.3)
    ##  edgeR                               3.20.7    2018-01-20 Bioconductor
    ##  ensembldb                           2.2.0     2018-01-21 Bioconductor
    ##  evaluate                            0.10.1    2017-06-24 CRAN (R 3.4.3)
    ##  foreach                           * 1.4.4     2017-12-12 CRAN (R 3.4.3)
    ##  foreign                             0.8-69    2017-06-22 CRAN (R 3.4.3)
    ##  Formula                             1.2-2     2017-07-10 CRAN (R 3.4.1)
    ##  genefilter                          1.60.0    2018-01-20 Bioconductor
    ##  geneplotter                         1.56.0    2018-01-20 Bioconductor
    ##  GenomeInfoDb                      * 1.14.0    2018-01-20 Bioconductor
    ##  GenomeInfoDbData                    1.0.0     2017-12-14 Bioconductor
    ##  GenomicAlignments                   1.14.1    2018-01-20 Bioconductor
    ##  GenomicFeatures                   * 1.30.0    2018-01-20 Bioconductor
    ##  GenomicFiles                        1.14.0    2018-01-21 Bioconductor
    ##  GenomicRanges                     * 1.30.1    2018-01-20 Bioconductor
    ##  GGally                              1.3.2     2017-08-02 CRAN (R 3.4.3)
    ##  ggbio                             * 1.26.0    2018-01-21 Bioconductor
    ##  ggplot2                           * 2.2.1     2016-12-30 CRAN (R 3.4.3)
    ##  graph                               1.56.0    2018-01-20 Bioconductor
    ##  graphics                          * 3.4.3     2017-11-30 local
    ##  grDevices                         * 3.4.3     2017-11-30 local
    ##  grid                              * 3.4.3     2017-11-30 local
    ##  gridExtra                         * 2.3       2017-09-09 CRAN (R 3.4.3)
    ##  gtable                              0.2.0     2016-02-26 CRAN (R 3.4.3)
    ##  highr                               0.6       2016-05-09 CRAN (R 3.4.3)
    ##  Hmisc                               4.1-1     2018-01-03 CRAN (R 3.4.3)
    ##  htmlTable                           1.11.2    2018-01-20 CRAN (R 3.4.3)
    ##  htmltools                           0.3.6     2017-04-28 CRAN (R 3.4.3)
    ##  htmlwidgets                         1.0       2018-01-20 CRAN (R 3.4.3)
    ##  httpuv                              1.3.5     2017-07-04 CRAN (R 3.4.3)
    ##  httr                                1.3.1     2017-08-20 CRAN (R 3.4.3)
    ##  interactiveDisplayBase              1.16.0    2018-01-20 Bioconductor
    ##  IRanges                           * 2.12.0    2018-01-20 Bioconductor
    ##  iterators                         * 1.0.9     2017-12-12 CRAN (R 3.4.3)
    ##  jsonlite                            1.5       2017-06-01 CRAN (R 3.4.3)
    ##  knitcitations                     * 1.0.8     2017-07-04 CRAN (R 3.4.3)
    ##  knitr                             * 1.18      2017-12-27 CRAN (R 3.4.3)
    ##  knitrBootstrap                      1.0.1     2017-07-19 CRAN (R 3.4.3)
    ##  labeling                            0.3       2014-08-23 CRAN (R 3.4.1)
    ##  lattice                             0.20-35   2017-03-25 CRAN (R 3.4.3)
    ##  latticeExtra                        0.6-28    2016-02-09 CRAN (R 3.4.3)
    ##  lazyeval                            0.2.1     2017-10-29 CRAN (R 3.4.3)
    ##  limma                               3.34.5    2018-01-20 Bioconductor
    ##  locfit                            * 1.5-9.1   2013-04-20 CRAN (R 3.4.3)
    ##  lubridate                           1.7.1     2017-11-03 CRAN (R 3.4.3)
    ##  magrittr                            1.5       2014-11-22 CRAN (R 3.4.3)
    ##  markdown                            0.8       2017-04-20 CRAN (R 3.4.3)
    ##  Matrix                              1.2-12    2017-11-20 CRAN (R 3.4.3)
    ##  matrixStats                         0.52.2    2017-04-14 CRAN (R 3.4.3)
    ##  memoise                             1.1.0     2017-04-21 CRAN (R 3.4.3)
    ##  methods                           * 3.4.3     2017-11-30 local
    ##  mgcv                              * 1.8-23    2018-01-15 CRAN (R 3.4.3)
    ##  mime                                0.5       2016-07-07 CRAN (R 3.4.1)
    ##  munsell                             0.4.3     2016-02-13 CRAN (R 3.4.3)
    ##  nlme                              * 3.1-131   2017-02-06 CRAN (R 3.4.3)
    ##  nnet                                7.3-12    2016-02-02 CRAN (R 3.4.3)
    ##  org.Hs.eg.db                      * 3.5.0     2017-12-04 Bioconductor
    ##  OrganismDbi                         1.20.0    2018-01-21 Bioconductor
    ##  parallel                          * 3.4.3     2017-11-30 local
    ##  pillar                              1.1.0     2018-01-14 CRAN (R 3.4.3)
    ##  pkgconfig                           2.0.1     2017-03-21 CRAN (R 3.4.3)
    ##  pkgmaker                            0.22      2014-05-14 CRAN (R 3.4.3)
    ##  plyr                                1.8.4     2016-06-08 CRAN (R 3.4.3)
    ##  prettyunits                         1.0.2     2015-07-13 CRAN (R 3.4.3)
    ##  progress                            1.1.2     2016-12-14 CRAN (R 3.4.3)
    ##  ProtGenerics                        1.10.0    2018-01-20 Bioconductor
    ##  qvalue                              2.10.0    2018-01-20 Bioconductor
    ##  R6                                  2.2.2     2017-06-17 CRAN (R 3.4.3)
    ##  RBGL                                1.54.0    2018-01-20 Bioconductor
    ##  RColorBrewer                      * 1.1-2     2014-12-07 CRAN (R 3.4.1)
    ##  Rcpp                                0.12.15   2018-01-20 CRAN (R 3.4.3)
    ##  RCurl                               1.95-4.10 2018-01-04 CRAN (R 3.4.3)
    ##  RefManageR                          0.14.20   2017-08-17 CRAN (R 3.4.3)
    ##  regionReport                      * 1.12.2    2018-01-21 Bioconductor
    ##  registry                            0.5       2017-12-03 CRAN (R 3.4.3)
    ##  reshape                             0.8.7     2017-08-06 CRAN (R 3.4.3)
    ##  reshape2                            1.4.3     2017-12-11 CRAN (R 3.4.3)
    ##  rlang                               0.1.6     2017-12-21 CRAN (R 3.4.3)
    ##  rmarkdown                           1.8       2017-11-17 CRAN (R 3.4.3)
    ##  RMySQL                              0.10.13   2017-08-14 CRAN (R 3.4.3)
    ##  rngtools                            1.2.4     2014-03-06 CRAN (R 3.4.3)
    ##  rpart                               4.1-12    2018-01-12 CRAN (R 3.4.3)
    ##  rprojroot                           1.3-2     2018-01-03 CRAN (R 3.4.3)
    ##  Rsamtools                           1.30.0    2018-01-20 Bioconductor
    ##  RSQLite                             2.0       2017-06-19 CRAN (R 3.4.1)
    ##  rstudioapi                          0.7       2017-09-07 CRAN (R 3.4.3)
    ##  rtracklayer                         1.38.2    2018-01-20 Bioconductor
    ##  S4Vectors                         * 0.16.0    2018-01-20 Bioconductor
    ##  scales                              0.5.0     2017-08-24 CRAN (R 3.4.3)
    ##  shiny                               1.0.5     2017-08-23 CRAN (R 3.4.3)
    ##  splines                             3.4.3     2017-11-30 local
    ##  stats                             * 3.4.3     2017-11-30 local
    ##  stats4                            * 3.4.3     2017-11-30 local
    ##  stringi                             1.1.6     2017-11-17 CRAN (R 3.4.2)
    ##  stringr                             1.2.0     2017-02-18 CRAN (R 3.4.3)
    ##  SummarizedExperiment                1.8.1     2018-01-20 Bioconductor
    ##  survival                            2.41-3    2017-04-04 CRAN (R 3.4.3)
    ##  tibble                              1.4.1     2017-12-25 CRAN (R 3.4.3)
    ##  tools                               3.4.3     2017-11-30 local
    ##  TxDb.Hsapiens.UCSC.hg19.knownGene * 3.2.2     2017-12-05 Bioconductor
    ##  utils                             * 3.4.3     2017-11-30 local
    ##  VariantAnnotation                   1.24.5    2018-01-21 Bioconductor
    ##  whisker                           * 0.3-2     2013-04-28 CRAN (R 3.4.3)
    ##  withr                               2.1.1     2017-12-19 CRAN (R 3.4.3)
    ##  XML                                 3.98-1.9  2017-06-19 CRAN (R 3.4.1)
    ##  xml2                                1.1.1     2017-01-24 CRAN (R 3.4.3)
    ##  xtable                              1.8-2     2016-02-05 CRAN (R 3.4.3)
    ##  XVector                             0.18.0    2018-01-20 Bioconductor
    ##  yaml                                2.1.16    2017-12-12 CRAN (R 3.4.3)
    ##  zlibbioc                            1.24.0    2018-01-20 Bioconductor

# 7 Bibliography

This vignette was generated using BiocStyle (Oles, Morgan, and Huber, 2018) with knitr (Xie, 2014) and rmarkdown (Allaire, Xie, McPherson, Luraschi, et al., 2017) running behind the scenes.

Citations made with knitcitations (Boettiger, 2017).

[1] J. Allaire, Y. Xie, J. McPherson, J. Luraschi, et al. rmarkdown: Dynamic Documents for R. R package version 1.8. 2017. URL: https://CRAN.R-project.org/package=rmarkdown.

[1] S. Arora, M. Morgan, M. Carlson and H. PagÃƒÂ¨s. GenomeInfoDb: Utilities for manipulating chromosome and other ‘seqname’ identifiers. 2017. DOI: 10.18129/B9.bioc.GenomeInfoDb.

[1] B. Auguie. gridExtra: Miscellaneous Functions for “Grid” Graphics. R package version 2.3. 2017. URL: https://CRAN.R-project.org/package=gridExtra.

[1] C. Boettiger. knitcitations: Citations for ‘Knitr’ Markdown Files. R package version 1.0.8. 2017. URL: https://CRAN.R-project.org/package=knitcitations.

[1] M. Carlson and B. P. Maintainer. TxDb.Hsapiens.UCSC.hg19.knownGene: Annotation package for TxDb object(s). R package version 3.2.2. 2015.

[1] Y. Chen, A. T. L. Lun and G. K. Smyth. “Differential expression analysis of complex RNA-seq experiments using edgeR”. In: Statistical Analysis of Next Generation Sequencing Data. Ed. by S. Datta and D. Nettleton. New York: Springer, 2014, pp. 51-74.

[1] L. Collado-Torres, A. E. Jaffe and J. T. Leek. derfinderPlot: Plotting functions for derfinder. https://github.com/leekgroup/derfinderPlot - R package version 1.12.2. 2017. DOI: 10.18129/B9.bioc.derfinderPlot. URL: http://www.bioconductor.org/packages/derfinderPlot.

[1] J. Hester. knitrBootstrap: ‘knitr’ Bootstrap Framework. R package version 1.0.1. 2017. URL: https://CRAN.R-project.org/package=knitrBootstrap.

[1] R. Kolde. pheatmap: Pretty Heatmaps. R package version 1.0.8. 2015. URL: https://CRAN.R-project.org/package=pheatmap.

[1] McCarthy, D. J., Chen, Yunshun, et al. “Differential expression analysis of multifactor RNA-Seq experiments with respect to biological variation”. In: Nucleic Acids Research 40.10 (2012), pp. 4288-4297.

[1] E. Neuwirth. RColorBrewer: ColorBrewer Palettes. R package version 1.1-2. 2014. URL: https://CRAN.R-project.org/package=RColorBrewer.

[1] A. Oles. DEFormats: Differential gene expression data formats converter. R package version 1.6.0. 2018. URL: https://github.com/aoles/DEFormats.

[1] A. Oles, M. Morgan and W. Huber. BiocStyle: Standard styles for vignettes and other Bioconductor documents. R package version 2.6.1. 2018. URL: https://github.com/Bioconductor/BiocStyle.

[1] R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. Vienna, Austria, 2017. URL: https://www.R-project.org/.

[1] R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. Vienna, Austria, 2017. URL: https://www.R-project.org/.

[1] M. D. Robinson, D. J. McCarthy and G. K. Smyth. “edgeR: a Bioconductor package for differential expression analysis of digital gene expression data”. In: Bioinformatics 26.1 (2010), pp. 139-140.

[1] H. Wickham. ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York, 2009. ISBN: 978-0-387-98140-6. URL: http://ggplot2.org.

[1] H. Wickham and W. Chang. devtools: Tools to Make Developing R Packages Easier. R package version 1.13.4. 2017. URL: https://CRAN.R-project.org/package=devtools.

[1] Y. Xie. DT: A Wrapper of the JavaScript Library ‘DataTables’. R package version 0.2. 2016. URL: https://CRAN.R-project.org/package=DT.

[1] Y. Xie. “knitr: A Comprehensive Tool for Reproducible Research in R”. In: Implementing Reproducible Computational Research. Ed. by V. Stodden, F. Leisch and R. D. Peng. ISBN 978-1466561595. Chapman and Hall/CRC, 2014. URL: http://www.crcpress.com/product/isbn/9781466561595.

[1] T. Yin, D. Cook and M. Lawrence. “ggbio: an R package for extending the grammar of graphics for genomic data”. In: Genome Biology 13.8 (2012), p. R77.

[1] T. Yin, M. Lawrence and D. Cook. biovizBase: Basic graphic utilities for visualization of genomic data. R package version 1.26.0. 2018.