R version: R version 3.5.2 (2018-12-20)
Bioconductor version: 3.8
Package: 1.4.1
Bioconductor has many packages which support analysis of high-throughput sequence data, including RNA sequencing (RNA-seq). The packages which we will use in this workflow include core packages maintained by the Bioconductor core team for importing and processing raw sequencing data and loading gene annotations. We will also use contributed packages for statistical analysis and visualization of sequencing data. Through scheduled releases every 6 months, the Bioconductor project ensures that all the packages within a release will work together in harmony (hence the “conductor” metaphor). The packages used in this workflow are loaded with the library function and can be installed by following the Bioconductor package installation instructions.
A published (but essentially similar) version of this workflow, including reviewer reports and comments is available at F1000Research.
If you have questions about this workflow or any Bioconductor
software, please post these to the
Bioconductor support site.
If the questions concern a specific package, you can tag the post with
the name of the package, or for general questions about the workflow,
tag the post with rnaseqgene
. Note the
posting guide
for crafting an optimal question for the support site.
The data used in this workflow is stored in the airway package that summarizes an RNA-seq experiment wherein airway smooth muscle cells were treated with dexamethasone, a synthetic glucocorticoid steroid with anti-inflammatory effects (Himes et al. 2014). Glucocorticoids are used, for example, by people with asthma to reduce inflammation of the airways. In the experiment, four primary human airway smooth muscle cell lines were treated with 1 micromolar dexamethasone for 18 hours. For each of the four cell lines, we have a treated and an untreated sample. For more description of the experiment see the PubMed entry 24926665 and for raw data see the GEO entry GSE52778.
As input, the count-based statistical methods, such as DESeq2 (Love, Huber, and Anders 2014), edgeR (Robinson, McCarthy, and Smyth 2009), limma with the voom method (Law et al. 2014), DSS (Wu, Wang, and Wu 2013), EBSeq (Leng et al. 2013) and baySeq (Hardcastle and Kelly 2010), expect input data as obtained, e.g., from RNA-seq or another high-throughput sequencing experiment, in the form of a matrix of integer values. The value in the i-th row and the j-th column of the matrix tells how many reads (or fragments, for paired-end RNA-seq) have been assigned to gene i in sample j. Analogously, for other types of assays, the rows of the matrix might correspond e.g., to binding regions (with ChIP-Seq), species of bacteria (with metagenomic datasets), or peptide sequences (with quantitative mass spectrometry).
The values in the matrix should be counts of sequencing reads/fragments. This is important for DESeq2’s statistical model to hold, as only counts allow assessing the measurement precision correctly. It is important to never provide counts that were pre-normalized for sequencing depth/library size, as the statistical model is most powerful when applied to un-normalized counts, and is designed to account for library size differences internally.
Before we demonstrate how to align and then count RNA-seq fragments, we mention that a newer and faster alternative pipeline is to use transcript abundance quantification methods such as Salmon (Patro et al. 2017), Sailfish (Patro, Mount, and Kingsford 2014), kallisto (Bray et al. 2016), or RSEM (Li and Dewey 2011), to estimate abundances without aligning reads, followed by the tximport package for assembling estimated count and offset matrices for use with Bioconductor differential gene expression packages.
A tutorial on how to use the Salmon software for quantifying
transcript abundance can be
found here.
We recommend using the --gcBias
flag
which estimates a correction factor for systematic biases
commonly present in RNA-seq data (Love, Hogenesch, and Irizarry 2016; Patro et al. 2017),
unless you are certain that your data do not contain such bias.
Following the Salmon tutorial, you can use the steps in the
tximport vignette,
which will show how to build a DESeqDataSet. This is our current recommended
pipeline for users, but below we still include steps on aligning reads to the
genome and building count matrices from BAM files.
The advantages of using the transcript abundance quantifiers in conjunction with tximport to produce gene-level count matrices and normalizing offsets, are: this approach corrects for any potential changes in gene length across samples (e.g. from differential isoform usage) (Trapnell et al. 2013); some of these methods are substantially faster and require less memory and disk usage compared to alignment-based methods; and it is possible to avoid discarding those fragments that can align to multiple genes with homologous sequence (Robert and Watson 2015). Note that transcript abundance quantifiers skip the generation of large files which store read alignments, instead producing smaller files which store estimated abundances, counts, and effective lengths per transcript. For more details, see the manuscript describing this approach (Soneson, Love, and Robinson 2015), and the tximport package vignette for software details. The entry point back into this workflow after using a transcript quantifier and tximport would be the section on exploratory data analysis below.
The computational analysis of an RNA-seq experiment begins from the FASTQ files that contain the nucleotide sequence of each read and a quality score at each position. These reads must first be aligned to a reference genome or transcriptome, or the abundances and estimated counts per transcript can be estimated without alignment, as described above. In either case, it is important to know if the sequencing experiment was single-end or paired-end, as the alignment software will require the user to specify both FASTQ files for a paired-end experiment. The output of this alignment step is commonly stored in a file format called SAM/BAM.
A number of software programs exist to align reads to a reference
genome, and we encourage you to check out some of the benchmarking papers that
discuss the advantages and disadvantages of each software, which
include accuracy, sensitivity in aligning reads over splice junctions, speed,
memory required, usability, and many other features.
Here, we used the
STAR read aligner (Dobin et al. 2013)
to align the reads for our current experiment to the Ensembl
release 75 (Flicek et al. 2014) human reference genome. In the following code example, it is assumed that there is a file in the current
directory called files
with each line containing an identifier for
each experiment, and we have all the FASTQ files in a subdirectory
fastq
. If you have downloaded the FASTQ files from the
Sequence Read Archive, the identifiers would be SRA run IDs,
e.g. SRR1039520
. You should have two files for a paired-end
experiment for each ID, fastq/SRR1039520_1.fastq1
and
fastq/SRR1039520_2.fastq
, which give the first and second read for
the paired-end fragments. If you have performed a single-end
experiment, you would only have one file per ID.
We have also created a subdirectory, aligned
,
where STAR will output its alignment files.
for f in `cat files`; do STAR --genomeDir ../STAR/ENSEMBL.homo_sapiens.release-75 \
--readFilesIn fastq/$f\_1.fastq fastq/$f\_2.fastq \
--runThreadN 12 --outFileNamePrefix aligned/$f.; done
SAMtools (Li et al. 2009)
was used to generate BAM files. The -@
flag can be used to allocate
additional threads.
for f in `cat files`; do samtools view -bS aligned/$f.Aligned.out.sam \
-o aligned/$f.bam; done
The BAM files for a number of sequencing runs can then be used to generate count matrices, as described in the following section.
Besides the count matrix that we will use later, the airway package also contains eight files with a small subset of the total number of reads in the experiment. The reads were selected which aligned to a small region of chromosome 1. Here, for demonstration, we chose a subset of reads because the full alignment files are large (a few gigabytes each), and because it takes between 10-30 minutes to count the fragments for each sample. We will use these files to demonstrate how a count matrix can be constructed from BAM files. Afterwards, we will load the full count matrix corresponding to all samples and all data, which is already provided in the same package, and will continue the analysis with that full matrix.
We load the data package with the example data:
library("airway")
The R function system.file can be used to find out where on your
computer the files from a package have been installed. Here we ask for
the full path to the extdata
directory, where R packages store
external data, that is part of the airway package.
indir <- system.file("extdata", package="airway", mustWork=TRUE)
In this directory, we find the eight BAM files (and some other files):
list.files(indir)
## [1] "GSE52778_series_matrix.txt" "Homo_sapiens.GRCh37.75_subset.gtf"
## [3] "SRR1039508_subset.bam" "SRR1039509_subset.bam"
## [5] "SRR1039512_subset.bam" "SRR1039513_subset.bam"
## [7] "SRR1039516_subset.bam" "SRR1039517_subset.bam"
## [9] "SRR1039520_subset.bam" "SRR1039521_subset.bam"
## [11] "SraRunInfo_SRP033351.csv" "sample_table.csv"
Typically, we have a table with detailed information for each of our samples that links samples to the associated FASTQ and BAM files. For your own project, you might create such a comma-separated value (CSV) file using a text editor or spreadsheet software such as Excel.
We load such a CSV file with read.csv:
csvfile <- file.path(indir, "sample_table.csv")
sampleTable <- read.csv(csvfile, row.names = 1)
sampleTable
## SampleName cell dex albut Run avgLength Experiment
## SRR1039508 GSM1275862 N61311 untrt untrt SRR1039508 126 SRX384345
## SRR1039509 GSM1275863 N61311 trt untrt SRR1039509 126 SRX384346
## SRR1039512 GSM1275866 N052611 untrt untrt SRR1039512 126 SRX384349
## SRR1039513 GSM1275867 N052611 trt untrt SRR1039513 87 SRX384350
## SRR1039516 GSM1275870 N080611 untrt untrt SRR1039516 120 SRX384353
## SRR1039517 GSM1275871 N080611 trt untrt SRR1039517 126 SRX384354
## SRR1039520 GSM1275874 N061011 untrt untrt SRR1039520 101 SRX384357
## SRR1039521 GSM1275875 N061011 trt untrt SRR1039521 98 SRX384358
## Sample BioSample
## SRR1039508 SRS508568 SAMN02422669
## SRR1039509 SRS508567 SAMN02422675
## SRR1039512 SRS508571 SAMN02422678
## SRR1039513 SRS508572 SAMN02422670
## SRR1039516 SRS508575 SAMN02422682
## SRR1039517 SRS508576 SAMN02422673
## SRR1039520 SRS508579 SAMN02422683
## SRR1039521 SRS508580 SAMN02422677
Once the reads have been aligned, there are a number of tools that can be used to count the number of reads/fragments that can be assigned to genomic features for each sample. These often take as input SAM/BAM alignment files and a file specifying the genomic features, e.g. a GFF3 or GTF file specifying the gene models.
The following tools can be used generate count matrices: summarizeOverlaps (Lawrence et al. 2013), featureCounts (Liao, Smyth, and Shi 2014), tximport (Soneson, Love, and Robinson 2015), htseq-count (Anders, Pyl, and Huber 2015).
function | package | framework | output | DESeq2 input function |
---|---|---|---|---|
summarizeOverlaps | GenomicAlignments | R/Bioconductor | SummarizedExperiment | DESeqDataSet |
featureCounts | Rsubread | R/Bioconductor | matrix | DESeqDataSetFromMatrix |
tximport | tximport | R/Bioconductor | list of matrices | DESeqDataSetFromTximport |
htseq-count | HTSeq | Python | files | DESeqDataSetFromHTSeq |
We now proceed using summarizeOverlaps.
Using the Run
column in the sample table, we construct the full
paths to the files we want to perform the counting operation on:
filenames <- file.path(indir, paste0(sampleTable$Run, "_subset.bam"))
file.exists(filenames)
## [1] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
We indicate in Bioconductor that these files are BAM files using the
BamFileList function from the Rsamtools package
that provides an R interface to BAM files.
Here we also specify details about how the BAM files should
be treated, e.g., only process 2 million reads at a time. See
?BamFileList
for more information.
library("Rsamtools")
bamfiles <- BamFileList(filenames, yieldSize=2000000)
Note: make sure that the chromosome names of the genomic features
in the annotation you use are consistent with the chromosome names of
the reference used for read alignment. Otherwise, the scripts might
fail to count any reads to features due to the mismatching names.
For example, a common mistake is when the alignment files contain
chromosome names in the style of 1
and the gene annotation in the
style of chr1
, or the other way around. See the seqlevelsStyle
function in the GenomeInfoDb package for solutions.
We can check the chromosome names (here called “seqnames”)
in the alignment files like so:
seqinfo(bamfiles[1])
## Seqinfo object with 84 sequences from an unspecified genome:
## seqnames seqlengths isCircular genome
## 1 249250621 <NA> <NA>
## 10 135534747 <NA> <NA>
## 11 135006516 <NA> <NA>
## 12 133851895 <NA> <NA>
## 13 115169878 <NA> <NA>
## ... ... ... ...
## GL000210.1 27682 <NA> <NA>
## GL000231.1 27386 <NA> <NA>
## GL000229.1 19913 <NA> <NA>
## GL000226.1 15008 <NA> <NA>
## GL000207.1 4262 <NA> <NA>
Next, we need to read in the gene model that will be used for counting reads/fragments. We will read the gene model from an Ensembl GTF file (Flicek et al. 2014), using makeTxDbFromGFF from the GenomicFeatures package. GTF files can be downloaded from Ensembl’s FTP site or other gene model repositories. A TxDb object is a database that can be used to generate a variety of range-based objects, such as exons, transcripts, and genes. We want to make a list of exons grouped by gene for counting read/fragments.
There are other options for constructing a TxDb. For the known genes track from the UCSC Genome Browser (Kent et al. 2002), one can use the pre-built Transcript DataBase: TxDb.Hsapiens.UCSC.hg19.knownGene. If the annotation file is accessible from AnnotationHub (as is the case for the Ensembl genes), a pre-scanned GTF file can be imported using makeTxDbFromGRanges.
Here we will demonstrate loading from a GTF file:
library("GenomicFeatures")
We indicate that none of our sequences (chromosomes) are circular using a 0-length character vector.
gtffile <- file.path(indir,"Homo_sapiens.GRCh37.75_subset.gtf")
txdb <- makeTxDbFromGFF(gtffile, format = "gtf", circ_seqs = character())
txdb
## TxDb object:
## # Db type: TxDb
## # Supporting package: GenomicFeatures
## # Data source: /home/biocbuild/bbs-3.8-bioc/R/library/airway/extdata/Homo_sapiens.GRCh37.75_subset.gtf
## # Organism: NA
## # Taxonomy ID: NA
## # miRBase build ID: NA
## # Genome: NA
## # transcript_nrow: 65
## # exon_nrow: 279
## # cds_nrow: 158
## # Db created by: GenomicFeatures package from Bioconductor
## # Creation time: 2019-01-09 09:06:27 -0500 (Wed, 09 Jan 2019)
## # GenomicFeatures version at creation time: 1.34.1
## # RSQLite version at creation time: 2.1.1
## # DBSCHEMAVERSION: 1.2
The following line produces a GRangesList of all the exons grouped by gene (Lawrence et al. 2013). Each element of the list is a GRanges object of the exons for a gene.
ebg <- exonsBy(txdb, by="gene")
ebg
## GRangesList object of length 20:
## $ENSG00000009724
## GRanges object with 18 ranges and 2 metadata columns:
## seqnames ranges strand | exon_id exon_name
## <Rle> <IRanges> <Rle> | <integer> <character>
## [1] 1 11086580-11087705 - | 98 ENSE00000818830
## [2] 1 11090233-11090307 - | 99 ENSE00000472123
## [3] 1 11090805-11090939 - | 100 ENSE00000743084
## [4] 1 11094885-11094963 - | 101 ENSE00000743085
## [5] 1 11097750-11097868 - | 102 ENSE00003482788
## ... ... ... ... . ... ...
## [14] 1 11106948-11107176 - | 111 ENSE00003467404
## [15] 1 11106948-11107176 - | 112 ENSE00003489217
## [16] 1 11107260-11107280 - | 113 ENSE00001833377
## [17] 1 11107260-11107284 - | 114 ENSE00001472289
## [18] 1 11107260-11107290 - | 115 ENSE00001881401
##
## ...
## <19 more elements>
## -------
## seqinfo: 1 sequence from an unspecified genome; no seqlengths
After these preparations, the actual counting is easy. The function summarizeOverlaps from the GenomicAlignments package will do this. This produces a SummarizedExperiment object that contains a variety of information about the experiment, and will be described in more detail below.
Note: If it is desired to perform counting using multiple cores, one can use
the register and MulticoreParam or SnowParam functions from the
BiocParallel package before the counting call below.
Expect that the summarizeOverlaps
call will take at least 30 minutes
per file for a human RNA-seq file with 30 million aligned reads. By sending
the files to separate cores, one can speed up the entire counting process.
library("GenomicAlignments")
library("BiocParallel")
Here we specify to use one core, not multiple cores. We could have also skipped this line and the counting step would run in serial.
register(SerialParam())
The following call creates the SummarizedExperiment object with counts:
se <- summarizeOverlaps(features=ebg, reads=bamfiles,
mode="Union",
singleEnd=FALSE,
ignore.strand=TRUE,
fragments=TRUE )
We specify a number of arguments besides the features
and the
reads
. The mode
argument describes what kind of read overlaps will
be counted. These modes are shown in Figure 1 of the
Counting reads with summarizeOverlaps vignette for the
GenomicAlignments package.
Note that fragments will be counted only once to each gene, even if
they overlap multiple exons of a gene which may themselves be overlapping.
Setting singleEnd
to FALSE
indicates that the experiment produced paired-end reads, and we want
to count a pair of reads (a fragment) only once toward the count
for a gene.
The fragments
argument can be used when singleEnd=FALSE
to specify if unpaired
reads should be counted (yes if fragments=TRUE
).
In order to produce correct counts, it is important to know if the
RNA-seq experiment was strand-specific or not. This experiment was not
strand-specific so we set ignore.strand
to TRUE
.
However, certain strand-specific protocols could have the reads
align only to the opposite strand of the genes.
The user must check if the experiment was strand-specific and if so,
whether the reads should align to the forward or reverse strand of the genes.
For various counting/quantifying tools, one specifies counting on the
forward or reverse strand in different ways, although this task
is currently easiest with htseq-count, featureCounts, or the
transcript abundance quantifiers mentioned previously.
It is always a good idea to check the column sums of the count matrix
(see below) to make sure these totals match the expected of the number
of reads or fragments aligning to genes. Additionally, one can
visually check the read alignments using a genome visualization tool.
The component parts of a SummarizedExperiment object. The assay
(pink
block) contains the matrix of counts, the rowRanges
(blue block) contains
information about the genomic ranges and the colData
(green block)
contains information about the samples. The highlighted line in each
block represents the first row (note that the first row of colData
lines up with the first column of the assay
).
The SummarizedExperiment container is diagrammed in the Figure above
and discussed in the latest Bioconductor paper (Huber et al. 2015).
In our case we have created a single matrix named “counts” that contains the fragment
counts for each gene and sample, which is stored in assay
. It is
also possible to store multiple matrices, accessed with assays
.
The rowRanges
for our object is the GRangesList we used for
counting (one GRanges of exons for each row of the count matrix).
The component parts of the SummarizedExperiment are accessed with an
R function of the same name: assay
(or assays
), rowRanges
and colData
.
This example code above actually only counted a small subset of fragments
from the original experiment. Nevertheless, we
can still investigate the resulting SummarizedExperiment by looking at
the counts in the assay
slot, the phenotypic data about the samples in
colData
slot (in this case an empty DataFrame), and the data about the
genes in the rowRanges
slot.
se
## class: RangedSummarizedExperiment
## dim: 20 8
## metadata(0):
## assays(1): counts
## rownames(20): ENSG00000009724 ENSG00000116649 ... ENSG00000271794
## ENSG00000271895
## rowData names(0):
## colnames(8): SRR1039508_subset.bam SRR1039509_subset.bam ...
## SRR1039520_subset.bam SRR1039521_subset.bam
## colData names(0):
dim(se)
## [1] 20 8
assayNames(se)
## [1] "counts"
head(assay(se), 3)
## SRR1039508_subset.bam SRR1039509_subset.bam
## ENSG00000009724 38 28
## ENSG00000116649 1004 1255
## ENSG00000120942 218 256
## SRR1039512_subset.bam SRR1039513_subset.bam
## ENSG00000009724 66 24
## ENSG00000116649 1122 1313
## ENSG00000120942 233 252
## SRR1039516_subset.bam SRR1039517_subset.bam
## ENSG00000009724 42 41
## ENSG00000116649 1100 1879
## ENSG00000120942 269 465
## SRR1039520_subset.bam SRR1039521_subset.bam
## ENSG00000009724 47 36
## ENSG00000116649 745 1536
## ENSG00000120942 207 400
colSums(assay(se))
## SRR1039508_subset.bam SRR1039509_subset.bam SRR1039512_subset.bam
## 6478 6501 7699
## SRR1039513_subset.bam SRR1039516_subset.bam SRR1039517_subset.bam
## 6801 8009 10849
## SRR1039520_subset.bam SRR1039521_subset.bam
## 5254 9168
The rowRanges
, when printed, only shows the first GRanges, and tells us
there are 19 more elements:
rowRanges(se)
## GRangesList object of length 20:
## $ENSG00000009724
## GRanges object with 18 ranges and 2 metadata columns:
## seqnames ranges strand | exon_id exon_name
## <Rle> <IRanges> <Rle> | <integer> <character>
## [1] 1 11086580-11087705 - | 98 ENSE00000818830
## [2] 1 11090233-11090307 - | 99 ENSE00000472123
## [3] 1 11090805-11090939 - | 100 ENSE00000743084
## [4] 1 11094885-11094963 - | 101 ENSE00000743085
## [5] 1 11097750-11097868 - | 102 ENSE00003482788
## ... ... ... ... . ... ...
## [14] 1 11106948-11107176 - | 111 ENSE00003467404
## [15] 1 11106948-11107176 - | 112 ENSE00003489217
## [16] 1 11107260-11107280 - | 113 ENSE00001833377
## [17] 1 11107260-11107284 - | 114 ENSE00001472289
## [18] 1 11107260-11107290 - | 115 ENSE00001881401
##
## ...
## <19 more elements>
## -------
## seqinfo: 1 sequence from an unspecified genome; no seqlengths
The rowRanges
also contains metadata about the construction
of the gene model in the metadata
slot. Here we use a helpful R
function, str
, to display the metadata compactly:
str(metadata(rowRanges(se)))
## List of 1
## $ genomeInfo:List of 15
## ..$ Db type : chr "TxDb"
## ..$ Supporting package : chr "GenomicFeatures"
## ..$ Data source : chr "/home/biocbuild/bbs-3.8-bioc/R/library/airway/extdata/Homo_sapiens.GRCh37.75_subset.gtf"
## ..$ Organism : chr NA
## ..$ Taxonomy ID : chr NA
## ..$ miRBase build ID : chr NA
## ..$ Genome : chr NA
## ..$ transcript_nrow : chr "65"
## ..$ exon_nrow : chr "279"
## ..$ cds_nrow : chr "158"
## ..$ Db created by : chr "GenomicFeatures package from Bioconductor"
## ..$ Creation time : chr "2019-01-09 09:06:27 -0500 (Wed, 09 Jan 2019)"
## ..$ GenomicFeatures version at creation time: chr "1.34.1"
## ..$ RSQLite version at creation time : chr "2.1.1"
## ..$ DBSCHEMAVERSION : chr "1.2"
The colData
:
colData(se)
## DataFrame with 8 rows and 0 columns
The colData
slot, so far empty, should contain all the metadata.
Because we used a column of sampleTable
to produce the bamfiles
vector, we know the columns of se
are in the same order as the
rows of sampleTable
. We can assign the sampleTable
as the
colData
of the summarized experiment, by converting
it into a DataFrame and using the assignment function:
colData(se) <- DataFrame(sampleTable)
colData(se)
## DataFrame with 8 rows and 9 columns
## SampleName cell dex albut Run avgLength
## <factor> <factor> <factor> <factor> <factor> <integer>
## SRR1039508 GSM1275862 N61311 untrt untrt SRR1039508 126
## SRR1039509 GSM1275863 N61311 trt untrt SRR1039509 126
## SRR1039512 GSM1275866 N052611 untrt untrt SRR1039512 126
## SRR1039513 GSM1275867 N052611 trt untrt SRR1039513 87
## SRR1039516 GSM1275870 N080611 untrt untrt SRR1039516 120
## SRR1039517 GSM1275871 N080611 trt untrt SRR1039517 126
## SRR1039520 GSM1275874 N061011 untrt untrt SRR1039520 101
## SRR1039521 GSM1275875 N061011 trt untrt SRR1039521 98
## Experiment Sample BioSample
## <factor> <factor> <factor>
## SRR1039508 SRX384345 SRS508568 SAMN02422669
## SRR1039509 SRX384346 SRS508567 SAMN02422675
## SRR1039512 SRX384349 SRS508571 SAMN02422678
## SRR1039513 SRX384350 SRS508572 SAMN02422670
## SRR1039516 SRX384353 SRS508575 SAMN02422682
## SRR1039517 SRX384354 SRS508576 SAMN02422673
## SRR1039520 SRX384357 SRS508579 SAMN02422683
## SRR1039521 SRX384358 SRS508580 SAMN02422677
At this point, we have counted the fragments which overlap the genes in the gene model we specified. This is a branching point where we could use a variety of Bioconductor packages for exploration and differential expression of the count data, including edgeR (Robinson, McCarthy, and Smyth 2009), limma with the voom method (Law et al. 2014), DSS (Wu, Wang, and Wu 2013), EBSeq (Leng et al. 2013) and baySeq (Hardcastle and Kelly 2010). Schurch et al. (2016) compared performance of different statistical methods for RNA-seq using a large number of biological replicates and can help users to decide which tools make sense to use, and how many biological replicates are necessary to obtain a certain sensitivity. We will continue using DESeq2 (Love, Huber, and Anders 2014). The SummarizedExperiment object is all we need to start our analysis. In the following section we will show how to use it to create the data object used by DESeq2.
Bioconductor software packages often define and use a custom class for storing data that makes sure that all the needed data slots are consistently provided and fulfill the requirements. In addition, Bioconductor has general data classes (such as the SummarizedExperiment) that can be used to move data between packages. Additionally, the core Bioconductor classes provide useful functionality: for example, subsetting or reordering the rows or columns of a SummarizedExperiment automatically subsets or reorders the associated rowRanges and colData, which can help to prevent accidental sample swaps that would otherwise lead to spurious results. With SummarizedExperiment this is all taken care of behind the scenes.
In DESeq2, the custom class is called DESeqDataSet. It is built on
top of the SummarizedExperiment class, and it is easy to convert
SummarizedExperiment objects into DESeqDataSet objects, which we
show below. One of the two main differences is that the assay
slot is
instead accessed using the counts accessor function, and the
DESeqDataSet class enforces that the values in this matrix are
non-negative integers.
A second difference is that the DESeqDataSet has an associated
design formula. The experimental design is specified at the
beginning of the analysis, as it will inform many of the DESeq2
functions how to treat the samples in the analysis (one exception is
the size factor estimation, i.e., the adjustment for differing library
sizes, which does not depend on the design formula). The design
formula tells which columns in the sample information table (colData
)
specify the experimental design and how these factors should be used
in the analysis.
The simplest design formula for differential expression would be
~ condition
, where condition
is a column in colData(dds)
that
specifies which of two (or more groups) the samples belong to. For
the airway experiment, we will specify ~ cell + dex
meaning that we want to test for the effect of dexamethasone (dex
)
controlling for the effect of different cell line (cell
). We can see
each of the columns just using the $
directly on the
SummarizedExperiment or DESeqDataSet:
se$cell
## [1] N61311 N61311 N052611 N052611 N080611 N080611 N061011 N061011
## Levels: N052611 N061011 N080611 N61311
se$dex
## [1] untrt trt untrt trt untrt trt untrt trt
## Levels: trt untrt
Note: it is prefered in R that the first level of a factor be the
reference level (e.g. control, or untreated samples), so we can
relevel the dex
factor like so:
library("magrittr")
se$dex %<>% relevel("untrt")
se$dex
## [1] untrt trt untrt trt untrt trt untrt trt
## Levels: untrt trt
%<>%
is the compound assignment pipe-operator from
the magrittr package, the above line of code is a concise way of saying
se$dex <- relevel(se$dex, "untrt")
For running DESeq2 models, you can use R’s formula notation to
express any fixed-effects experimental design.
Note that DESeq2 uses the same formula notation as, for instance, the lm
function of base R. If the research aim is to determine for which
genes the effect of treatment is different across groups, then
interaction terms can be included and tested using a design such as
~ group + treatment + group:treatment
. See the manual page for
?results
for more examples. We will show how to use an interaction
term to test for condition-specific changes over time in a
time course example below.
In the following sections, we will demonstrate the construction of the DESeqDataSet from two starting points:
For a full example of using the HTSeq Python package for read counting, please see the pasilla vignette. For an example of generating the DESeqDataSet from files produced by htseq-count, please see the DESeq2 vignette.
We now use R’s data command to load a prepared
SummarizedExperiment that was generated from the publicly available
sequencing data files associated with Himes et al. (2014),
described above. The steps we used to produce this object were
equivalent to those you worked through in the previous sections,
except that we used all the reads and all the genes. For more details
on the exact steps used to create this object, type
vignette("airway")
into your R session.
data("airway")
se <- airway
Again, we want to specify that untrt
is the reference level for the
dex variable:
se$dex %<>% relevel("untrt")
se$dex
## [1] untrt trt untrt trt untrt trt untrt trt
## Levels: untrt trt
We can quickly check the millions of fragments that uniquely aligned to the genes (the second argument of round tells how many decimal points to keep).
round( colSums(assay(se)) / 1e6, 1 )
## SRR1039508 SRR1039509 SRR1039512 SRR1039513 SRR1039516 SRR1039517 SRR1039520
## 20.6 18.8 25.3 15.2 24.4 30.8 19.1
## SRR1039521
## 21.2
Supposing we have constructed a SummarizedExperiment using
one of the methods described in the previous section, we now need to
make sure that the object contains all the necessary information about
the samples, i.e., a table with metadata on the count matrix’s columns
stored in the colData
slot:
colData(se)
## DataFrame with 8 rows and 9 columns
## SampleName cell dex albut Run avgLength
## <factor> <factor> <factor> <factor> <factor> <integer>
## SRR1039508 GSM1275862 N61311 untrt untrt SRR1039508 126
## SRR1039509 GSM1275863 N61311 trt untrt SRR1039509 126
## SRR1039512 GSM1275866 N052611 untrt untrt SRR1039512 126
## SRR1039513 GSM1275867 N052611 trt untrt SRR1039513 87
## SRR1039516 GSM1275870 N080611 untrt untrt SRR1039516 120
## SRR1039517 GSM1275871 N080611 trt untrt SRR1039517 126
## SRR1039520 GSM1275874 N061011 untrt untrt SRR1039520 101
## SRR1039521 GSM1275875 N061011 trt untrt SRR1039521 98
## Experiment Sample BioSample
## <factor> <factor> <factor>
## SRR1039508 SRX384345 SRS508568 SAMN02422669
## SRR1039509 SRX384346 SRS508567 SAMN02422675
## SRR1039512 SRX384349 SRS508571 SAMN02422678
## SRR1039513 SRX384350 SRS508572 SAMN02422670
## SRR1039516 SRX384353 SRS508575 SAMN02422682
## SRR1039517 SRX384354 SRS508576 SAMN02422673
## SRR1039520 SRX384357 SRS508579 SAMN02422683
## SRR1039521 SRX384358 SRS508580 SAMN02422677
Here we see that this object already contains an informative
colData
slot – because we have already prepared it for you, as
described in the airway vignette.
However, when you work with your own data, you will have to add the
pertinent sample / phenotypic information for the experiment at this stage.
We highly recommend keeping this information in a comma-separated
value (CSV) or tab-separated value (TSV) file, which can be exported
from an Excel spreadsheet, and the assign this to the colData
slot,
making sure that the rows correspond to the columns of the
SummarizedExperiment. We made sure of this correspondence earlier by
specifying the BAM files using a column of the sample table.
Once we have our fully annotated SummarizedExperiment object, we can construct a DESeqDataSet object from it that will then form the starting point of the analysis. We add an appropriate design for the analysis:
library("DESeq2")
dds <- DESeqDataSet(se, design = ~ cell + dex)
In this section, we will show how to build an DESeqDataSet supposing we only have a count matrix and a table of sample information.
Note: if you have prepared a SummarizedExperiment you should skip this section. While the previous section would be used to construct a DESeqDataSet from a SummarizedExperiment, here we first extract the individual object (count matrix and sample info) from the SummarizedExperiment in order to build it back up into a new object – only for demonstration purposes. In practice, the count matrix would either be read in from a file or perhaps generated by an R function like featureCounts from the Rsubread package (Liao, Smyth, and Shi 2014).
The information in a SummarizedExperiment object can be accessed with accessor functions. For example, to see the actual data, i.e., here, the fragment counts, we use the assay function. (The head function restricts the output to the first few lines.)
countdata <- assay(se)
head(countdata, 3)
## SRR1039508 SRR1039509 SRR1039512 SRR1039513 SRR1039516
## ENSG00000000003 679 448 873 408 1138
## ENSG00000000005 0 0 0 0 0
## ENSG00000000419 467 515 621 365 587
## SRR1039517 SRR1039520 SRR1039521
## ENSG00000000003 1047 770 572
## ENSG00000000005 0 0 0
## ENSG00000000419 799 417 508
In this count matrix, each row represents an Ensembl gene, each column a sequenced RNA library, and the values give the raw numbers of fragments that were uniquely assigned to the respective gene in each library. We also have information on each of the samples (the columns of the count matrix). If you’ve counted reads with some other software, it is very important to check that the columns of the count matrix correspond to the rows of the sample information table.
coldata <- colData(se)
We now have all the ingredients to prepare our data object in a form that is suitable for analysis, namely:
countdata
: a table with the fragment countscoldata
: a table with information about the samplesTo now construct the DESeqDataSet object from the matrix of counts and the sample information table, we use:
ddsMat <- DESeqDataSetFromMatrix(countData = countdata,
colData = coldata,
design = ~ cell + dex)
We will continue with the object generated from the SummarizedExperiment section.
There are two separate paths in this workflow; the one we will see first involves transformations of the counts in order to visually explore sample relationships. In the second part, we will go back to the original raw counts for statistical testing. This is critical because the statistical testing methods rely on original count data (not scaled or transformed) for calculating the precision of measurements.
Our count matrix with our DESeqDataSet contains many rows with only zeros, and additionally many rows with only a few fragments total. In order to reduce the size of the object, and to increase the speed of our functions, we can remove the rows that have no or nearly no information about the amount of gene expression. Here we apply the most minimal filtering rule: removing rows of the DESeqDataSet that have no counts, or only a single count across all samples. Additional weighting/filtering to improve power is applied at a later step in the workflow.
nrow(dds)
## [1] 64102
dds <- dds[ rowSums(counts(dds)) > 1, ]
nrow(dds)
## [1] 29391
Many common statistical methods for exploratory analysis of multidimensional data, for example clustering and principal components analysis (PCA), work best for data that generally has the same range of variance at different ranges of the mean values. When the expected amount of variance is approximately the same across different mean values, the data is said to be homoskedastic. For RNA-seq counts, however, the expected variance grows with the mean. For example, if one performs PCA directly on a matrix of counts or normalized counts (e.g. correcting for differences in sequencing depth), the resulting plot typically depends mostly on the genes with highest counts because they show the largest absolute differences between samples. A simple and often used strategy to avoid this is to take the logarithm of the normalized count values plus a pseudocount of 1; however, depending on the choice of pseudocount, now the genes with the very lowest counts will contribute a great deal of noise to the resulting plot, because taking the logarithm of small counts actually inflates their variance. We can quickly show this property of counts with some simulated data (here, Poisson counts with a range of lambda from 0.1 to 100). We plot the standard deviation of each row (genes) against the mean:
lambda <- 10^seq(from = -1, to = 2, length = 1000)
cts <- matrix(rpois(1000*100, lambda), ncol = 100)
library("vsn")
meanSdPlot(cts, ranks = FALSE)
And for logarithm-transformed counts:
log.cts.one <- log2(cts + 1)
meanSdPlot(log.cts.one, ranks = FALSE)
The logarithm with a small pseudocount amplifies differences when the values are close to 0. The low count genes with low signal-to-noise ratio will overly contribute to sample-sample distances and PCA plots.
As a solution, DESeq2 offers two transformations for count data that stabilize the variance across the mean: the variance stabilizing transformation (VST) for negative binomial data with a dispersion-mean trend (Anders and Huber 2010), implemented in the vst function, and the regularized-logarithm transformation or rlog (Love, Huber, and Anders 2014).
For genes with high counts, both the VST and the rlog will give similar result to the ordinary log2 transformation of normalized counts. For genes with lower counts, however, the values are shrunken towards a middle value. The VST or rlog-transformed data then become approximately homoskedastic (more flat trend in the meanSdPlot), and can be used directly for computing distances between samples, making PCA plots, or as input to downstream methods which perform best with homoskedastic data.
Which transformation to choose?
The VST is much faster to compute and is less sensitive to high count
outliers than the rlog. The rlog tends to work well on
small datasets (n < 30), potentially outperforming the VST when there is
a wide range of sequencing depth across samples (an order of magnitude difference).
We therefore recommend the VST for medium-to-large datasets (n > 30).
You can perform both transformations and compare the meanSdPlot
or
PCA plots generated, as described below.
Note that the two transformations offered by DESeq2 are provided for applications other than differential testing. For differential testing we recommend the DESeq function applied to raw counts, as described later in this workflow, which also takes into account the dependence of the variance of counts on the mean value during the dispersion estimation step.
Both vst and rlog return a DESeqTransform object which is based
on the SummarizedExperiment class. The transformed values are no
longer counts, and are stored in the assay slot. The colData that
was attached to dds
is still accessible:
vsd <- vst(dds, blind = FALSE)
head(assay(vsd), 3)
## SRR1039508 SRR1039509 SRR1039512 SRR1039513 SRR1039516
## ENSG00000000003 9.742074 9.430420 9.867627 9.645845 10.183143
## ENSG00000000419 9.333669 9.581707 9.486145 9.523093 9.427283
## ENSG00000000457 8.765274 8.698449 8.651473 8.732426 8.592787
## SRR1039517 SRR1039520 SRR1039521
## ENSG00000000003 9.880416 10.010366 9.639782
## ENSG00000000419 9.574860 9.325999 9.509246
## ENSG00000000457 8.702674 8.761945 8.724101
colData(vsd)
## DataFrame with 8 rows and 10 columns
## SampleName cell dex albut Run avgLength
## <factor> <factor> <factor> <factor> <factor> <integer>
## SRR1039508 GSM1275862 N61311 untrt untrt SRR1039508 126
## SRR1039509 GSM1275863 N61311 trt untrt SRR1039509 126
## SRR1039512 GSM1275866 N052611 untrt untrt SRR1039512 126
## SRR1039513 GSM1275867 N052611 trt untrt SRR1039513 87
## SRR1039516 GSM1275870 N080611 untrt untrt SRR1039516 120
## SRR1039517 GSM1275871 N080611 trt untrt SRR1039517 126
## SRR1039520 GSM1275874 N061011 untrt untrt SRR1039520 101
## SRR1039521 GSM1275875 N061011 trt untrt SRR1039521 98
## Experiment Sample BioSample sizeFactor
## <factor> <factor> <factor> <numeric>
## SRR1039508 SRX384345 SRS508568 SAMN02422669 1.02364764926706
## SRR1039509 SRX384346 SRS508567 SAMN02422675 0.896166721793923
## SRR1039512 SRX384349 SRS508571 SAMN02422678 1.17948612081678
## SRR1039513 SRX384350 SRS508572 SAMN02422670 0.670053829048202
## SRR1039516 SRX384353 SRS508575 SAMN02422682 1.17767135405022
## SRR1039517 SRX384354 SRS508576 SAMN02422673 1.39903646915342
## SRR1039520 SRX384357 SRS508579 SAMN02422683 0.920778683328161
## SRR1039521 SRX384358 SRS508580 SAMN02422677 0.944514089340919
Again, for the rlog:
rld <- rlog(dds, blind = FALSE)
head(assay(rld), 3)
## SRR1039508 SRR1039509 SRR1039512 SRR1039513 SRR1039516
## ENSG00000000003 9.385681 9.052599 9.516877 9.285335 9.839093
## ENSG00000000419 8.869611 9.138274 9.036117 9.075296 8.972125
## ENSG00000000457 7.961373 7.881385 7.824075 7.921493 7.751690
## SRR1039517 SRR1039520 SRR1039521
## ENSG00000000003 9.530313 9.663260 9.277695
## ENSG00000000419 9.131828 8.861529 9.060906
## ENSG00000000457 7.886441 7.957126 7.912125
In the above function calls, we specified blind = FALSE
, which means
that differences between cell lines and treatment (the variables in
the design) will not contribute to the expected variance-mean trend of
the experiment. The experimental design is not used directly in the
transformation, only in estimating the global amount of variability in
the counts. For a fully unsupervised transformation, one can set
blind = TRUE
(which is the default).
To show the effect of the transformation, in the figure below
we plot the first sample
against the second, first simply using the log2 function (after adding
1, to avoid taking the log of zero), and then using the VST and rlog-transformed
values. For the log2 approach, we need to first estimate size factors to
account for sequencing depth, and then specify normalized=TRUE
.
Sequencing depth correction is done automatically for the vst and rlog.
library("dplyr")
library("ggplot2")
dds <- estimateSizeFactors(dds)
df <- bind_rows(
as_data_frame(log2(counts(dds, normalized=TRUE)[, 1:2]+1)) %>%
mutate(transformation = "log2(x + 1)"),
as_data_frame(assay(vsd)[, 1:2]) %>% mutate(transformation = "vst"),
as_data_frame(assay(rld)[, 1:2]) %>% mutate(transformation = "rlog"))
colnames(df)[1:2] <- c("x", "y")
ggplot(df, aes(x = x, y = y)) + geom_hex(bins = 80) +
coord_fixed() + facet_grid( . ~ transformation)
Scatterplot of transformed counts from two samples. Shown are scatterplots using the log2 transform of normalized counts (left), using the VST (middle), and using the rlog (right). While the rlog is on roughly the same scale as the log2 counts, the VST has a upward shift for the smaller values. It is the differences between samples (deviation from y=x in these scatterplots) which will contribute to the distance calculations and the PCA plot.
We can see how genes with low counts (bottom left-hand corner) seem to be excessively variable on the ordinary logarithmic scale, while the VST and rlog compress differences for the low count genes for which the data provide little information about differential expression.
A useful first step in an RNA-seq analysis is often to assess overall similarity between samples: Which samples are similar to each other, which are different? Does this fit to the expectation from the experiment’s design?
We use the R function dist to calculate the Euclidean distance between samples. To ensure we have a roughly equal contribution from all genes, we use it on the VST data. We need to transpose the matrix of values using t, because the dist function expects the different samples to be rows of its argument, and different dimensions (here, genes) to be columns.
sampleDists <- dist(t(assay(vsd)))
sampleDists
## SRR1039508 SRR1039509 SRR1039512 SRR1039513 SRR1039516 SRR1039517
## SRR1039509 44.08478
## SRR1039512 36.82952 51.50926
## SRR1039513 59.23174 43.11327 47.21641
## SRR1039516 41.08424 54.79377 40.18856 59.72262
## SRR1039517 59.08409 47.46042 54.13979 46.36635 44.74738
## SRR1039520 37.27578 54.00938 34.56797 55.44564 42.81024 58.22766
## SRR1039521 59.26716 42.84573 54.13120 35.24746 60.64298 47.80456
## SRR1039520
## SRR1039509
## SRR1039512
## SRR1039513
## SRR1039516
## SRR1039517
## SRR1039520
## SRR1039521 48.24754
We visualize the distances in a heatmap in a figure below, using the function pheatmap from the pheatmap package.
library("pheatmap")
library("RColorBrewer")
In order to plot the sample distance matrix with the rows/columns
arranged by the distances in our distance matrix,
we manually provide sampleDists
to the clustering_distance
argument of the pheatmap function.
Otherwise the pheatmap function would assume that the matrix contains
the data values themselves, and would calculate distances between the
rows/columns of the distance matrix, which is not desired.
We also manually specify a blue color palette using the
colorRampPalette function from the RColorBrewer package.
sampleDistMatrix <- as.matrix( sampleDists )
rownames(sampleDistMatrix) <- paste( vsd$dex, vsd$cell, sep = " - " )
colnames(sampleDistMatrix) <- NULL
colors <- colorRampPalette( rev(brewer.pal(9, "Blues")) )(255)
pheatmap(sampleDistMatrix,
clustering_distance_rows = sampleDists,
clustering_distance_cols = sampleDists,
col = colors)
Heatmap of sample-to-sample distances using the rlog-transformed values.
Note that we have changed the row names of the distance matrix to contain treatment type and patient number instead of sample ID, so that we have all this information in view when looking at the heatmap.
Another option for calculating sample distances is to use the
Poisson Distance (Witten 2011), implemented in the
PoiClaClu package.
This measure of dissimilarity between counts
also takes the inherent variance
structure of counts into consideration when calculating the distances
between samples. The PoissonDistance function takes the original
count matrix (not normalized) with samples as rows instead of columns,
so we need to transpose the counts in dds
.
library("PoiClaClu")
poisd <- PoissonDistance(t(counts(dds)))
We plot the heatmap in a Figure below.
samplePoisDistMatrix <- as.matrix( poisd$dd )
rownames(samplePoisDistMatrix) <- paste( dds$dex, dds$cell, sep=" - " )
colnames(samplePoisDistMatrix) <- NULL
pheatmap(samplePoisDistMatrix,
clustering_distance_rows = poisd$dd,
clustering_distance_cols = poisd$dd,
col = colors)
Heatmap of sample-to-sample distances using the Poisson Distance.
Another way to visualize sample-to-sample distances is a principal components analysis (PCA). In this ordination method, the data points (here, the samples) are projected onto the 2D plane such that they spread out in the two directions that explain most of the differences (figure below). The x-axis is the direction that separates the data points the most. The values of the samples in this direction are written PC1. The y-axis is a direction (it must be orthogonal to the first direction) that separates the data the second most. The values of the samples in this direction are written PC2. The percent of the total variance that is contained in the direction is printed in the axis label. Note that these percentages do not add to 100%, because there are more dimensions that contain the remaining variance (although each of these remaining dimensions will explain less than the two that we see).
plotPCA(vsd, intgroup = c("dex", "cell"))