1.1.1 Download and Installation

To install the latest version of esATAC, you will need to be using the latest version of R. esATAC is part of Bioconductor project starting from Bioc 3.6 built on R 3.4. Please check your current Bioconductor version and R version first. Similar to other bioconductor package, you can download install esATAC and all its dependencies software like this:

source("http://www.bioconductor.org/biocLite.R")
biocLite("esATAC")

NOTE: We recommend to use this package in RStudio. Or you have to install pandoc yourself if you use R terminal

Acquiring more installation detail information for Linux, Windows and macOS, you can visit esATAC Installation Tutorial.

Note: By default, esATAC will build bowtie2 index if there is no bowtie2 index, which may take several hours with single thread. If you want to download or use your own bowtie2 index instead, please see section 1.4

NOTE: Annotation Packages for specific genomes are shown below:

They will not be downloaded and installed during executing biocLite("esATAC") because these package file are very big. They will be downloaded and installed according to the genome parameter when calling the preset pipeline for the first time.

1.1.2 Loading

Just like other R packages, you need to load esATAC like this each time before using the package.

library(esATAC)

1.3.1 For Case-control Analysis

The R scripts below are ready to run. No more edit is needed.

Customize the code commented with “MODIFY” if you need to run on your own data.

Need to be prepared for your own data:

• FASTQ files paths (may be .gz or .bz2 zipped FASTQ files)
  • case:
    • fastqInput1: mate 1 FASTQ file(s)
    • fastqInput2: mate 2 FASTQ file(s)
  • control:
    • fastqInput1: mate 1 FASTQ file(s)
    • fastqInput2: mate 2 FASTQ file(s)
• genome: genome version may be one of these
  • hg19
  • hg38
  • mm9
  • mm10
• refdir:(optional) Directory for installing genome reference and storage for reuse. Default: “./esATAC_pipeline/refdir” will be created if not exist.
• tmpdir:(optional) Directory for intermediate files and results storage. Default: “./esATAC_pipeline/intermediate_results” will be created if not exist.
• threads:(optional) The max threads allowed to be created. Default: 2.

For other genomes, you can built your own pipeline through Customized Pipeline presently. More genome will be supported in the future.

Here, we show an simple runnable example for case-control analysis. Case and control test sample are both paired-end data. Each of them contains two gzipped FASTQ files. The test file paths can are obtained like this: system.file(package="esATAC", "extdata", "chr20_1.1.fq.gz"). They are under folder path “R library path/esATAC/extdata/”.

library(esATAC)

# call pipeline
# all human motif in JASPAR will be processed
conclusion <- 
    atacPipe2(
# MODIFY: Change these paths to your own case files!
# e.g. fastqInput1 = "your/own/data/path.fastq"
        case=list(fastqInput1 = system.file(package="esATAC", "extdata", "chr20_1.1.fq.gz"),
                  fastqInput2 = system.file(package="esATAC", "extdata", "chr20_2.1.fq.gz")), 
# MODIFY: Change these paths to your own control files!
# e.g. fastqInput1 = "your/own/data/path.fastq"
        control=list(fastqInput1 = system.file(package="esATAC", "extdata", "chr20_1.2.fq.bz2"),
                     fastqInput2 = system.file(package="esATAC", "extdata", "chr20_2.2.fq.bz2")),
# MODIFY: Change this path to an permanent path to be used in future!
#       refdir <- "./esATAC_pipeline/refdir",
#       tmpdir = "./esATAC_pipeline/intermediate_results", 
# MODIFY: Set the genome for your data
        genome = "hg19")

Note: By default, esATAC will build bowtie2 index if there is no bowtie2 index under the path set by argument refdir, which may take several hours. If you want to download bowtie2 index instead, please see section 1.4

Note: By default, esATAC will perform footprint analysis for all the TF motif PWM matrix in JASPAR database. This step may take a few hours to 2-days for human genome analysis depends on your hardware. If you only want to analyze a specific motif or your own PWMs, please see section 1.5

Note: For replicates sample, see section 1.8: Gallery for Analysis Result of Real ATAC-seq Dataset

The reference data will be installed in “./esATAC_pipeline/refdir/” and all of temporary data and result will be stored under “./esATAC_pipeline/intermediate_results”. The final brief results like HTML report will be stored under “./esATAC_pipeline/esATAC_results” and “./esATAC_pipeline/esATAC_report”

If you run the scripts above without modification, you are able to obtain default HTML example report.

Real data test

Download raw data from GEO(assertion number GSE88987 - GSM2356780: SRR4435490.sra) and (GSM2356795: SRR4435505.sra)).And then, Use NCBI SRA Toolkit to extract fastq files with command like fastq-dump --split-3 SRR44354XX.sra. Four files will be generated (SRR4435490_1.fastq, SRR4435490_2.fastq, SRR4435505_1.fastq, SRR4435505_2.fastq). Modify and run the scripts above like this example scripts, you will obtain HTML report.

1.3.2 For Single Sample Analysis

The R scripts below are ready to run. No more edit is needed.

Customize the code commented with “MODIFY” if you need to run on your own data.

Need to be prepared for your own data:

• FASTQ files paths (may be .gz or .bz2 zipped FASTQ files)
  • fastqInput1: mate 1 FASTQ file(s)
  • fastqInput2: mate 2 FASTQ file(s)
• genome: may be one of these
  • hg19
  • hg38
  • mm9
  • mm10
• refdir:(optional) Directory for installing genome reference and storage for reuse. Default: “./esATAC_pipeline/refdir” will be created if not exist.
• tmpdir:(optional) Directory for intermediate files and results storage. Default: “./esATAC_pipeline/intermediate_results” will be created if not exist.
• threads:(optional) The max threads allowed to be created. Default: 2.

For other genomes, you can built your own pipeline through Customized Pipeline presently. More genome will be supported in the future.

Here, we show an simple runnable example for single sample analysis. We just use case data in case-control section.

library(esATAC)

# call pipeline
# for overall example(all human motif in JASPAR will be processed)
conclusion <- 
    atacPipe(
# MODIFY: Change these paths to your own case files!
# e.g. fastqInput1 = "your/own/data/path.fastq"
        fastqInput1 = system.file(package="esATAC", "extdata", "chr20_1.1.fq.gz"),
        fastqInput2 = system.file(package="esATAC", "extdata", "chr20_2.1.fq.gz"),
# MODIFY: Change this path to an permanent path to be used in future!
#       refdir <- "./esATAC_pipeline/refdir",
#       tmpdir = "./esATAC_pipeline/intermediate_results", 
# MODIFY: Set the genome for your data
        genome = "hg19")

Note: By default, esATAC will build bowtie2 index if there is no bowtie2 index under the path set by argument refdir, which may take several hours. If you want to download bowtie2 index instead, please see section 1.4

Note: By default, esATAC will perform footprint analysis for all the TF motif PWM matrix in JASPAR database. This step may take a few hours to 2-days for human genome analysis depends on your hardware. If you only want to analyze a specific motif or your own PWMs, please see section 1.5

Note: For replicates sample, see section 1.8: Gallery for Analysis Result of Real ATAC-seq Dataset

The reference data will be installed in “./esATAC_pipeline/refdir/” and all of temporary data and result will be stored under “./esATAC_pipeline/intermediate_results”. The final brief results like HTML report will be stored under “./esATAC_pipeline/esATAC_results”, “./esATAC_pipeline/esATAC_report”

If you run the scripts above without modification, you are able to obtain default HTML Example report

Real data test

Download raw data from GEO (assersion number GSE47753 - GSM1155957(SRR891268.sra)). And then, Use NCBI SRA Toolkit to extract fastq files with command like fastq-dump --split-3 SRR8912XX.sra. Two files will be generated (SRR891268_1.fastq, SRR891268_2.fastq). Modify the scripts above like this example scripts, you will obtain HTML report for GSM1155957 data.

esATAC will download the genome sequence and annotation files, build bowtie2 index, mapping the reads, do the quality control analysis, find peak regions, perform GO analysis and motif enrichment analysis, etc. automatically. Finally, you will get an report file in html format included to the analysis results.

1.5.1 Case-control

### 
library(esATAC)
conclusion2 <- 
    atacPipe2(
        case=list(fastqInput1 = system.file(package="esATAC", "extdata", "chr20_1.1.fq.gz"),
                  fastqInput2 = system.file(package="esATAC", "extdata", "chr20_2.1.fq.gz")), 
        control=list(fastqInput1 = system.file(package="esATAC", "extdata", "chr20_1.2.fq.bz2"),
                     fastqInput2 = system.file(package="esATAC", "extdata", "chr20_2.2.fq.bz2")),
        genome = "hg19",
        motifs = getMotifInfo(motif.file = system.file("extdata", "CustomizedMotif.txt", package="esATAC")))

1.5.2 single sample

library(esATAC)
conclusion <- 
    atacPipe(
        fastqInput1 = system.file(package="esATAC", "extdata", "chr20_1.1.fq.gz"),
        fastqInput2 = system.file(package="esATAC", "extdata", "chr20_2.1.fq.gz"),
        genome = "hg19",
        motifs = getMotifInfo(motif.file = system.file("extdata", "CustomizedMotif.txt", package="esATAC")))

1.6.1 Case-control Sample Conclusion

There are three slots in conclusion list:

• caselist # see Single Sample Conclusion
• ctrllist # see Single Sample Conclusion
• wholesummary # a dataframe of case-control summary table

You can use $ operator to obtain value. For example, conclusion$caselist, conclusion$caselist$atacProcs$renamer, etc.

1.6.2 Single Sample Conclusion

There are four slots in conclusion list:

• filelist # a dataframe of fastq file paths
• wholesummary # a dataframe of summary table
• atacProcs # ATACProc objects list, see Query and Object Operations for object operation
  • unzipAndMerge
  • renamer
  • removeAdapter
  • bowtie2Mapping
  • libComplexQC
  • sam2Bed
  • bedToBigWig
  • tssqc100
  • peakCalling
  • DHSQC
  • blacklistQC
  • fripQC
  • atacQC
• filtstat # a dataframe of reads or peaks filtering summary table

Slot atacProcs is also an list. Similarly, use $ to access them: conclusion$atacProcs$renamer

3.1.1 Loading

Just like other R package, you need to load esATAC like this each time before using the package.

library(esATAC)

If you need to use fseq, we recommend to set max memory size for java (8G, 8000M in the example). Or rJava will use the default parameter for fseq.

options(java.parameters = "-Xmx8000m")

3.1.2 Loading Recommend Package

The BSgenome package, TxDb known gene package and OrgDb annotation package for some functions are required. We recommend to install (use biocLite("packageName")) and load the specific species related packages before using the packages.

library(magrittr)
library(BSgenome.Hsapiens.UCSC.hg19)
library(TxDb.Hsapiens.UCSC.hg19.knownGene)
library(org.Hs.eg.db)
library(R.utils)

Packages for specific genomes are shown below:

3.1.3 Configure Other Parameters

These configurations are also optional. “tmpdir” is the path to save all of the temporary data and the default result storage path. If it is not configured, current work directory will be set as “tmpdir”. “threads” is the maximum threads allowed to be created for data processing. The default value is 1. More thread will consume more memery in some processes.

# we use temp directiory "td" here 
# Change it to your directiory because the intermediate file may be huge
td<-tempdir()
options(atacConf=setConfigure("tmpdir",td))
options(atacConf=setConfigure("threads",8))

3.1.4 Reference Data Installation and Loading

We strongly recommend to install reference data first before using the package although it is optional. “refdir” is the folder that will save all of the reference data. “genome” is the genome name like hg19, hg38, mm10, mm9 and so on. The program will detect the elements that have not been installed and install them. Some resources need to be downloaded from internet. So don’t forget to connect internet during installation. Or the installation will be failed. If all of the reference data was installed, these two lines still need to be called for configuring the reference data path and genome.

options(atacConf=setConfigure("refdir","path/to/refdatafolder"))
options(atacConf=setConfigure("genome","hg19"))

NOTE: The installation will consume several hours for data download and building bowtie2 index depending on computer performance and network bandwidth.

NOTE: The installation is network based. Please keep your network connection. But you don need to worry about disconnect. The program will continue to check finished part and only build unfinish part.

WARNNING: If the reference data is not configured, the related reference argument of functions has to be set manually during using.

3.1.5 Already Have Genome Bowtie2 Index

Build bowtie index may take some time. If you already have bowtie2 index files or you want to download instead of building, you can let esATAC skip the steps by renaming them following the format (genome+suffix) and put them in reference installation path (refdir).

Example: hg19 bowtie2 index files

• hg19.1.bt2
• hg19.2.bt2
• hg19.3.bt2
• hg19.4.bt2
• hg19.rev.1.bt2
• hg19.rev.2.bt2

bowtie2 index download path:

ftp://ftp.ccb.jhu.edu/pub/data/bowtie2_indexes

3.2.1 Configuration Details

There are 4 configurable parameters (“refdir”,“tmpdir”,“threads”,“genome”) in this package. None of them is required because users can also pass them to the function as arguments based on needs. For convenience, by configuring them in advance, these arguments for functions will not need to be set repeatedly. Users can focus more on data flow pipeline implementation rather than dependency. These are the details for the parameters:

“genome”, the genome like “hg19”,“mm10”, etc.

“refdir”, the path for genome reference and annotation data storage.

“tmpdir”, the path for intermediate files and result storage. Default, “./”, current working directory

“threads”, the max number of threads allowed to be created. Default, 1

dir.create("./esATAC_pipeline")
dir.create("./esATAC_pipeline/refdir")
dir.create("./esATAC_pipeline/result")

#configure reference path, result path, the max number of threads, genome
options(atacConf=setConfigure("refdir","esATAC_pipeline/refdir"))
options(atacConf=setConfigure("tmpdir","esATAC_pipeline/result"))
options(atacConf=setConfigure("threads",2))
options(atacConf=setConfigure("genome","hg19"))

In this example, we create folder “./esATAC_pipeline/refdir” for “refdir” and “./esATAC_pipeline/result” for “tmpdir”. Two threads are allowed. Genome reference and annotations will be install under “./esATAC_pipeline/refdir” if they are not complete.

3.2.2 Traverse the Dataflow Subgraph

Pipe operator in R (%>% and %T>%) may help users build pipeline more easily. Parameters are passed seamlessly between upstream and downstream unit.

#raw reads
fastqInput1 = system.file(package="esATAC", "extdata", "chr20_1.1.fq.gz")
fastqInput2 = system.file(package="esATAC", "extdata", "chr20_2.1.fq.gz") 

#pipeline           
atacUnzipAndMerge(fastqInput1 = fastqInput1,fastqInput2 = fastqInput2) %T>%
atacQCReport %>%
atacRenamer %>%
atacRemoveAdapter %>%
atacBowtie2Mapping %T>% 
atacLibComplexQC %>% 
atacSamToBed(maxFragLen = 2000) %T>%
atacBedToBigWig %T>% 
atacFragLenDistr %>%
atacBedUtils(maxFragLen = 100, chrFilterList = NULL) %>%
atacPeakCalling

3.3.1 Preprocess

Users can use %>% to build a pipeline to obtain merged, renamed and adapter removed clean reads fastq file(s) that is ready for mapping.

# Identify adapters
prefix<-system.file(package="esATAC", "extdata", "uzmg")
(reads_1 <-file.path(prefix,"m1",dir(file.path(prefix,"m1"))))
(reads_2 <-file.path(prefix,"m2",dir(file.path(prefix,"m2"))))

reads_merged_1 <- file.path(td,"reads1.fastq")
reads_merged_2 <- file.path(td,"reads2.fastq")
atacproc <- 
atacUnzipAndMerge(fastqInput1 = reads_1,fastqInput2 = reads_2) %>%
atacRenamer %>% atacRemoveAdapter

If you want to modify the parameters of AdapterRemoval, you have to refer to Rbowtie2 package:

library(Rbowtie2)
adapterremoval_usage()

3.3.2 Mapping

If the reference has not been configured, the bowtie2 index should be built first. Then bowtie2 mapping functions could used to map reads to reference genome.

## Building a bowtie2 index
library("Rbowtie2")
refs <- dir(system.file(package="esATAC", "extdata", "bt2","refs"),
full=TRUE)
bowtie2_build(references=refs, bt2Index=file.path(td, "lambda_virus"),
"--threads 4 --quiet",overwrite=TRUE)
## Alignments
reads_1 <- system.file(package="esATAC", "extdata", "bt2", "reads",
"reads_1.fastq")
reads_2 <- system.file(package="esATAC", "extdata", "bt2", "reads",
"reads_2.fastq")
if(file.exists(file.path(td, "lambda_virus.1.bt2"))){
    (bowtie2Mapping(bt2Idx = file.path(td, "lambda_virus"),
       samOutput = file.path(td, "result.sam"),
       fastqInput1=reads_1,fastqInput2=reads_2,threads=3))
    head(readLines(file.path(td, "result.sam")))
}

If you want to modify the parameters of bowtie2, you have to refer to Rbowtie2 package:

library(Rbowtie2)
bowtie2_usage()

3.3.3 Convert SAM File to BED File

The mapping results are stored in a SAM file. SamToBed functions can covert it into BED file. During converting, the operation like sorting, shifting, filtering chromosome and so on can also be set to do in the meantime.

sambzfile <- system.file(package="esATAC", "extdata", "Example.sam.bz2")
samfile <- file.path(td,"Example.sam")
bunzip2(sambzfile,destname=samfile,overwrite=TRUE,remove=FALSE)
samToBed(samInput = samfile) 

3.3.4 Filtering Reads and Calling

Filter the nucleosome free reads(<100bp) for peak calling.

bedbzfile <- system.file(package="esATAC", "extdata", "chr20.50000.bed.bz2")
bedfile <- file.path(td,"chr20.50000.bed")
bunzip2(bedbzfile,destname=bedfile,overwrite=TRUE,remove=FALSE)

bedUtils(bedInput = bedfile,maxFragLen = 100, chrFilterList = NULL) %>%
atacPeakCalling

3.3.5 ATAC-seq Peak Annotation

ATAC-seq peak locate at open chromatin regions. Annotating these peak could find whether they locate at functional regions(such as promoter and enhancer).

Function “atacPeakAnno” and “peakanno” use function “annotatePeak” in package “ChIPseeker” to annotate ATAC-seq peak. for more information about package “ChIPseeker”, please click here[4].

Function “atacPeakAnno” and “peakanno” accept a bed file path as an input, users can change the parameters like “tssRegion”, “TxDb” according to their require. Now, bioconductor offers many species’ annotation database, click here to search more.

The following example is to exhibit how to annotate a UCSC bed file.

## extract example peak file from package "esATAC"
p1bz <- system.file("extdata", "Example_peak1.bed.bz2", package="esATAC")
peak1_path <- as.vector(bunzip2(filename = p1bz,
destname = file.path(getwd(), "Example_peak1.bed"),
ext="bz2", FUN=bzfile, overwrite=TRUE, remove = FALSE))
## run peakanno to annotate peaks
AnnoInfo <- peakanno(peakInput = peak1_path, TxDb = TxDb.Hsapiens.UCSC.hg19.knownGene, annoDb = "org.Hs.eg.db")

The output contains a pie chart in pdf format like below. It reports the percentage of peaks located in different functional regions.

The function also generate a file(with suffix .txt) contains annotation for all peaks. It is converted from dataframe in R, and users could open it with text editor or excel. Below is a part of the output.

3.3.6 Go Analysis

GO analysis is performing enrichment analysis on gene sets. It establishes the relationship between gene sets and functions, and report the most significant function to users.

Function “atacGOAnalysis” and “goanalysis” use function “enrichGO” in package “clusterProfiler” to do GO analysis. for more information about package “clusterProfiler”, please click here[5].

The function need gene Id set as input. User could choose different GO terms(molecular function, biological process and cellular component) according to different input of parameter “ont”.

The following example is to exhibit how to do GO analysis on a gene set.

## extract gene ID
geneId <- as.character(sample(seq(10000), 100))
## do GO analysis
goAna <- goanalysis(gene = geneId, OrgDb = "org.Hs.eg.db", keytype = "ENTREZID", ont = "MF")

The output file(suffix .txt) contains the GO term sorted by p-value, below is a part of the output.

3.3.7 Motif Scan

This function search motif occurrence in the given regions.

Function “atacMotifScan” and “motifscan” use function “matchMotifs” in package “motifmatchr”, for more parameters and usage, click here[6].

Multi-motif is supported, and the output file is named by your input motif list. The input could be PFMatrix, PFMatrixList, PWMatrix, PWMatrixList from TFBSTools.

Using “motifscan” function to search motif in given genome regions, UCSC bed file is recommended.

sample.path <- system.file("extdata", "chr20_sample_peak.bed.bz2", package="esATAC")
sample.path <- as.vector(bunzip2(filename = sample.path,
destname = file.path(getwd(), "chr20_sample_peak.bed"),
ext="bz2", FUN=bzfile, overwrite=TRUE, remove = FALSE))

motif <- readRDS(system.file("extdata", "MotifPFM.rds", package="esATAC"))

motif.data <- motifscan(peak = sample.path, genome = BSgenome.Hsapiens.UCSC.hg19, motifs = motif)

This function reports the exact motif position in the given genome like below(motif: CTCF).

3.3.8 Plot Footprint

The interaction between TF and DNA would leave a “footprint” in motif position, but it is not evident in a single site, so integrated footprint is necessary. In addition, we only consider Tn5 cut site. This function is based on the motif scan.

First, collecting all cut site from the bed file(Note: every line in the bed file is a DNA fragment) and save them.

## extract cut site position from bed file
fra_path <- system.file("extdata", "chr20.50000.bed.bz2", package="esATAC")
frag <- as.vector(bunzip2(filename = fra_path,
destname = file.path(getwd(), "chr20.50000.bed"),
ext="bz2", FUN=bzfile, overwrite=TRUE, remove = FALSE))
cs.data <- extractcutsite(bedInput = frag, prefix = "ATAC")

Next, plot footprint for different motifs.

In the motif scan, we get a variable named “motif.data”, is contains multi-motif information. In order to plot footprint of these motif in a single procedure, we will use the output of function motifscan, here is “motif.data”.

fp <- atacCutSiteCount(atacProcCutSite = cs.data, atacProcMotifScan = motif.data)

The following is CTCF footprint using example data.

Note: we only using a small part of the chromatin 20 as example.