CHiCAGO is a method for detecting statistically significant interaction events in Capture HiC data. This vignette will walk you through a typical CHiCAGO analysis.
A typical Chicago job for two biological replicates of CHi-C data takes 2-3 h wall-clock time (including sample pre-processing from bam files) and uses 50G RAM.
NOTE | A wrapper to perform this type of analysis, called runChicago.R, is provided as part of chicagoTools, which is available from our Bitbucket repository. Refer to the chicagoTools README for more information. |
The statistical foundations of CHiCAGO are presented in a separate paper that is currently available as a preprint (Jonathan Cairns*, Paula Freire-Pritchett* et al. 2015). Briefly, CHiCAGO uses a convolution background model accounting for both ‘Brownian collisions’ between fragments (distance-dependent) and ‘technical noise’. It borrows information across interactions (with appropriate normalisation) to estimate these background components separately on different subsets of data. CHiCAGO then uses a p-value weighting procedure based on the expected true positive rates at different distance ranges (estimated from data), with scores representing soft-thresholded -log weighted p-values. The score threshold of 5 is a suggested stringent score threshold for calling significant interactions.
WARNING | The data set used in this tutorial comes from the package PCHiCdata. This package contains small parts (two chromosomes each) of published Promoter Capture HiC data sets in mouse ESCs (Schoenfelder et al. 2015) and GM12878 cells, derived from human LCLs (Mifsud et al. 2015) - note that both papers used a different interaction-calling algorithm and we are only reusing raw data from them. Do not use any of these sample input data for purposes other than training. |
In this vignette, we use the GM12878 data (Mifsud et al. 2015):
library(Chicago)
library(PCHiCdata)
Before you start, you will need:
countNperBin.py
, countNbaitsPerBin.py
and getProxOE.py
. Refer to the chicagoTools
README file for more details.We recommend that you put all five of these files into the same directory (that we refer to as designDir). An example of a valid design folder, for a two-chromosome sample of the GM12878 data used in this vignette, is provided in the PCHiCdata package, as follows.
dataPath <- system.file("extdata", package="PCHiCdata")
testDesignDir <- file.path(dataPath, "hg19TestDesign")
dir(testDesignDir)
## [1] "h19_chr20and21.baitmap" "h19_chr20and21.nbpb"
## [3] "h19_chr20and21.npb" "h19_chr20and21.poe"
## [5] "h19_chr20and21.rmap"
NOTE | Though we talk about “restriction fragments” throughout, any non-overlapping regions can be defined in .rmap (with a subset of these defined as baits). For example, if one wanted to increase power at the cost of resolution, it is possible to use bins of restriction fragments either throughout, or for some selected regions. However, in the binned fragment framework, we advise setting removeAdjacent to FALSE - see ?setExperiment for details on how to do this. |
bam2chicago.sh
, available as part of chicagoTools
. (To obtain BAM files from raw fastq files, use a Hi-C alignment & QC pipeline such as HiCUP.Example .chinput files are provided in the PCHiCdata package, as follows:
testDataPath <- file.path(dataPath, "GMchinputFiles")
dir(testDataPath)
## [1] "GM_rep1.chinput" "GM_rep2.chinput" "GM_rep3.chinput"
files <- c(
file.path(testDataPath, "GM_rep1.chinput"),
file.path(testDataPath, "GM_rep2.chinput"),
file.path(testDataPath, "GM_rep3.chinput")
)
OPTIONAL: The data set in this vignette requires some additional custom settings, both to ensure that the vignette compiles in a reasonable time and to deal with the artificially reduced size of the data set:
settingsFile <- file.path(system.file("extdata", package="PCHiCdata"),
"sGM12878Settings", "sGM12878.settingsFile")
Omit this step unless you know which settings you wish to alter. If you are using non-human samples, or a very unusual cell type, one set of options that you might want to change is the weighting parameters: see Using different weights.
We run CHiCAGO on the test data as follows. First, we create a blank chicagoData
object, and we tell it where the design files are. For this example, we also provide the optional settings file - in your analysis, you can omit the settingsFile
argument.
library(Chicago)
cd <- setExperiment(designDir = testDesignDir, settingsFile = settingsFile)
The properties of chicagoData
objects are discussed more in The chicagoData object.
Next, we read in the input data files:
cd <- readAndMerge(files=files, cd=cd)
Finally, we run the pipeline with chicagoPipeline()
:
cd <- chicagoPipeline(cd)
chicagoPipeline()
produces a number of plots. You can save these to disk by setting the outprefix
argument in chicagoPipeline()
.
The plots are as follows (an explanation follows):
Here, we describe the expected properties of the diagnostic plots.
Note that the diagnostic plots above are computed on the fly using only a small fraction of the full GM12878 dataset. In real-world, genome-wide datasets, more fragment pools will be visible and thus the trends described below should be more pronounced.
average
bait, as a function of distance, plotted on a log-log scale.Two main output methods are provided. Here, we discuss the first: exporting to disk. However, it is also possible to export to a GenomeInteractions object, as described in Further downstream analysis.
You can export the results to disk, using exportResults()
. (If you use runChicago.R, the files appear in ./<results-folder>/data). By default, the function outputs three different output file formats:
outputDirectory <- tempdir()
exportResults(cd, file.path(outputDirectory, "vignetteOutput"))
## Reading the restriction map file...
## Reading the bait map file...
## Preparing the output table...
## Writing out for seqMonk...
## Writing out interBed...
## Preprocessing for WashU outputs...
## Writing out text file for WashU browser upload...
Each called interaction is assigned a score that represents how strong CHiCAGO believes the interaction is (formally, it is based on -log(adjusted P-value)). Thus, a larger score represents a stronger interaction. In each case, the score threshold of 5 is applied.
Summary of output files:
## bait_chr bait_start bait_end bait_name otherEnd_chr
## 1 20 119103 138049 DEFB126 20
## 2 20 161620 170741 DEFB128 20
## 3 20 233983 239479 DEFB132 20
## 4 20 268639 284501 AL034548.1;C20orf96;ZCCHC3 20
## 5 20 268639 284501 AL034548.1;C20orf96;ZCCHC3 20
## 6 20 305699 309123 RP5-1103G7.4;SOX12 20
## otherEnd_start otherEnd_end otherEnd_name N_reads score
## 1 523682 536237 CSNK2A1 6 6.01
## 2 73978 76092 . 16 5.17
## 3 206075 209203 DEFB129 33 5.36
## 4 293143 304037 . 34 7.27
## 5 304038 305698 . 34 8.78
## 6 293143 304037 . 37 5.49
## V1 V2 V3 V4 V5 V6
## 1 20 119103 138049 DEFB126 6 6.01
## 2 20 523682 536237 CSNK2A1 6 6.01
## 3 20 161620 170741 DEFB128 16 5.17
## 4 20 73978 76092 . 16 5.17
## 5 20 233983 239479 DEFB132 33 5.36
## 6 20 206075 209203 DEFB129 33 5.36
## V1 V2 V3
## 1 chr20,119103,138049 chr20,523682,536237 6.01
## 2 chr20,161620,170741 chr20,73978,76092 5.17
## 3 chr20,233983,239479 chr20,206075,209203 5.36
## 4 chr20,268639,284501 chr20,293143,304037 7.27
## 5 chr20,268639,284501 chr20,304038,305698 8.78
## 6 chr20,305699,309123 chr20,293143,304037 5.49
exportResults()
.For bait-to-bait interactions, the interaction can be tested either way round (i.e. either fragment can be considered the “bait”). In most output formats, both of these tests are preserved. The exception is washU output, where these scores are consolidated by taking the maximum.
NOTE | When comparing interactions detected between multiple replicates, the degree of overlap may appear to be lower than expected. This is due to the undersampled nature of most CHi-C datasets. Sampling error can drive down the sensitivity, particularly for interactions that span large distances and have low read counts. As such, low overlap is not necessarily an indication of a high false discovery rate. Undersampling needs to be taken into consideration when interpreting CHiCAGO results. In particular, we recommend performing comparisons at the score-level rather than at the level of thresholded interaction calls. Potentially, differential analysis algorithms for sequencing data such as DESeq2 (Love, Huber, and Anders 2014) may also be used to formally compare the enrichment at CHiCAGO-detected interactions between conditions at the count level, although power will generally be a limiting factor. Formal methods such as sdef (Blangiardo, Cassese, and Richardson 2010) may provide a more balanced view of the consistency between replicates. Alternatively, additional filtering based on the mean number of reads per detected interaction (e.g. removing calls with N<10 reads) will reduce the impact of undersampling on the observed overlap, but at the cost of decreasing the power to detect longer-range interactions. |
The plotBaits()
function can be used to plot the raw read counts versus linear distance from bait for either specific or random baits, labelling significant interactions in a different colour. By default, 16 random baits are plotted, with interactions within 1 Mb from bait passing the threshold of 5 shown in red and those passing the more lenient threshold of 3 shown in blue.
plottedBaitIDs <- plotBaits(cd, n=6)
peakEnrichment4Features()
tests the hypothesis that other ends in the CHiCAGO output are enriched for genomic features of interest - for example, histone marks associated with enhancers. We find out how many overlaps are expected under the null hypothesis (i.e. that there is no enrichment) by shuffling the other ends around in the genome, while preserving the overall distribution of distances over which interactions span.
You will need additional files to perform this analysis - namely, a .bed file for each feature. We include ChIP-seq data from the ENCODE consortium (The ENCODE Project Consortium 2012), also restricted to chr20 and chr21. (Data accession numbers: Bernstein lab GSM733752, GSM733772, GSM733708, GSM733664, GSM733771, GSM733758)
First, we find the folder that contains the features, and construct a list of the features to use:
featuresFolder <- file.path(dataPath, "GMfeatures")
dir(featuresFolder)
## [1] "featuresGM.txt"
## [2] "spp.wgEncodeBroadHistoneGm12878CtcfStdAln_chr20and21.narrowPeak"
## [3] "wgEncodeBroadHistoneGm12878H3k27acStdAln_chr20and21.narrowPeak"
## [4] "wgEncodeBroadHistoneGm12878H3k27me3StdAln_chr20and21.narrowPeak"
## [5] "wgEncodeBroadHistoneGm12878H3k4me1StdAln_chr20and21.narrowPeak"
## [6] "wgEncodeBroadHistoneGm12878H3k4me3StdAln_chr20and21.narrowPeak"
## [7] "wgEncodeBroadHistoneGm12878H3k9me3StdAln_chr20and21.narrowPeak"
featuresFile <- file.path(featuresFolder, "featuresGM.txt")
featuresTable <- read.delim(featuresFile, header=FALSE, as.is=TRUE)
featuresList <- as.list(featuresTable$V2)
names(featuresList) <- featuresTable$V1
featuresList
## $CTCF
## [1] "spp.wgEncodeBroadHistoneGm12878CtcfStdAln_chr20and21.narrowPeak"
##
## $H3K4me1
## [1] "wgEncodeBroadHistoneGm12878H3k4me1StdAln_chr20and21.narrowPeak"
##
## $H3K4me3
## [1] "wgEncodeBroadHistoneGm12878H3k4me3StdAln_chr20and21.narrowPeak"
##
## $H3k27ac
## [1] "wgEncodeBroadHistoneGm12878H3k27acStdAln_chr20and21.narrowPeak"
##
## $H3K27me3
## [1] "wgEncodeBroadHistoneGm12878H3k27me3StdAln_chr20and21.narrowPeak"
##
## $H3K9me3
## [1] "wgEncodeBroadHistoneGm12878H3k9me3StdAln_chr20and21.narrowPeak"
Next, we feed this information into the peakEnrichment4Features()
function.
As part of the analysis, peakEnrichment4Features()
takes a distance range (by default, the full distance range over which interactions are observed), and divides it into some number of bins. We must select the number of bins; here, we choose that number to ensure that the bin size is approximately 10kb. If the defaults are changed, a different number of bins is more appropriate. See ?peakEnrichment4Features
for more information.
no_bins <- ceiling(max(abs(intData(cd)$distSign), na.rm = TRUE)/1e4)
enrichmentResults <- peakEnrichment4Features(cd, folder=featuresFolder,
list_frag=featuresList, no_bins=no_bins, sample_number=100)
Note the plot produced by this function. For each feature type, the yellow bar represents the number of features that overlap with interaction other ends. The blue bar represents what would be expected by chance, with a 95% confidence interval for the mean number of overlaps plotted. If the yellow bar lies outside of this interval, we reject the null hypothesis, thus concluding that there is enrichment/depletion of that feature.
The information displayed in the plot is also returned in tabular form (OL = Overlap, SI = Significant Interactions, SD = Standard Deviation, CI = Confidence Interval):
enrichmentResults
## OLwithSI MeanOLwithSamples SDOLwithSample LowerCI HigherCI
## CTCF 346 122.39 9.380718 104.00379 140.77621
## H3K4me1 686 273.04 14.066164 245.47032 300.60968
## H3K4me3 374 136.71 11.347847 114.46822 158.95178
## H3k27ac 468 164.32 11.524134 141.73270 186.90730
## H3K27me3 72 71.10 7.293971 56.80382 85.39618
## H3K9me3 226 117.78 10.272627 97.64565 137.91435
We can perform further downstream analysis in R or Bioconductor, using functionality from the GenomicInteractions package. First, we export the significant interactions into a GenomicInteractions object:
library(GenomicInteractions)
## Warning: replacing previous import 'BiocGenerics::Position' by
## 'ggplot2::Position' when loading 'GenomicInteractions'
library(GenomicRanges)
gi <- exportToGI(cd)
From here, we can pass the CHiCAGO results through to other Bioconductor functionality. In the following example, we find out which other ends overlap with the H3K4me1 enhancer mark, using ENCODE data. We use AnnotationHub to fetch a relevant enhancer mark track from the ENCODE project:
library(AnnotationHub)
ah <- AnnotationHub()
hs <- query(ah, c("GRanges", "EncodeDCC", "Homo sapiens", "H3k4me1"))
enhancerTrack <- hs[["AH23254"]]
Next, we use the anchorTwo()
function to extract the other end locations from the GenomicInteractions object (anchorOne()
would give us the bait locations instead). Note that in this particular instance, the seqlevels()
also need to be changed before performing the comparison, adding “chr” to make them match those of the annotation.
otherEnds <- anchorTwo(gi)
otherEnds <- renameSeqlevels(otherEnds, c("chr20","chr21"))
Finally, we look at which other ends overlap the enhancer marks:
findOverlaps(otherEnds, enhancerTrack)
## Hits object with 4564 hits and 0 metadata columns:
## queryHits subjectHits
## <integer> <integer>
## [1] 1 40732
## [2] 1 40735
## [3] 8 40717
## [4] 8 40718
## [5] 9 40721
## ... ... ...
## [4560] 4166 17630
## [4561] 4166 17631
## [4562] 4167 17676
## [4563] 4167 17677
## [4564] 4170 17665
## -------
## queryLength: 4171 / subjectLength: 109612
Further note that the annotation’s genome version should match that of the promoter capture data, namely hg19:
hs["AH23254"]$genome
## [1] "hg19"
In the above workflows, cd is a chicagoData object. It contains three elements:
intData(cd)
is a data.table (note: not a data.frame) that contains information about fragment pairs.settings(cd)
is a list of settings, usually set with the setExperiment() function.params(cd)
is a list of parameters. This list is populated as the pipeline runs, and CHiCAGO estimates them in turn.A closer look at intData(cd)
:
head(intData(cd), 2)
## baitID otherEndID distbin s_j otherEndLen distSign isBait2bait
## 1: 403463 403833 NA 0.2368791 2579 1652804 FALSE
## 2: 403463 403843 NA 0.2368791 6302 1690808 FALSE
## N.1 N.2 N.3 N refBinMean s_i NNb NNboe tlb tblb
## 1: 0 1 0 1 NA 0.9494934 4 4 [0,6] [ 2, 46)
## 2: 0 1 0 1 NA 1.0021336 4 4 (6,13] [ 2, 46)
## Tmean Bmean log.p log.w log.q score
## 1: 0.001485762 0.08970408 -2.443282 1.406451 -3.849733 0
## 2: 0.004021026 0.09243682 -2.389716 1.350609 -3.740324 0
Columns:
?mergeSamples
).?mergeSamples
) or raw count in the case of single-replicate interaction calling.?normaliseBaits
)WARNING: Many functions in CHiCAGO update intData(cd)
by reference, which means that intData(cd)
can change even when you do not explicitly assign to it. To avoid this behaviour, copy the chicagoData object first:
newCd = copyCD(cd)
CHiCAGO uses a p-value weighting procedure to upweight proximal interactions and downweight distal interactions. This procedure has four tuning parameters.
The default values of these tuning parameters were calibrated on calls from seven human Macrophage data sets. Provided that your cell type is not too dissimilar to these calibration data, it should be fine to leave the parameters at their default settings. However, if your data set is from a different species or an unusual cell type, you may wish to recalibrate these parameters using data from cell types similar to yours. You can do this with the fitDistCurve.R script in chicagoTools, which we demonstrate in this section.
First, run all of the samples through chicagoPipeline()
, saving each chicagoData
object in individual .rds files (see saveRDS()
). Alternatively, if you are using the runChicago.R wrapper, .rds files should be generated automatically.
Second, run the fitDistCurve.R script. As an example, if we had three biological replicates of mESC cells, we might run the following script at the Unix command prompt:
Rscript fitDistCurve.R mESC --inputs mESC1.rds,mESC2.rds,mESC3.rds
This script produces the file mESC.settingsFile, which you can read in to modifySettings()
as usual - see the Input files required section.
Additionally, the script produces a plot (in this case, called mESC_mediancurveFit.pdf) that can be used to diagnose unreliable estimates. By default, five coloured lines are shown, each representing a parameter estimate from a subset of the data. An unreliable fit is typically diagnosed when the coloured lines are highly dissimilar to each other, and thus the black median line is not representative of them. (Some dissimilarity may be OK, since the median confers robustness.)
For the user’s convenience, a set of precomputed weights are also provided in the package:
weightsPath <- file.path(system.file("extdata", package="Chicago"),
"weights")
dir(weightsPath)
## [1] "GM12878-2reps.settings" "humanMacrophage-7reps.settings"
## [3] "mESC-2reps.settings"
For example, to use the GM12878 weights, supply the appropriate settings file to setExperiment()
as per the following:
weightSettings <- file.path(weightsPath, "GM12878-2reps.settings")
cd <- setExperiment(designDir = testDesignDir, settingsFile = weightSettings)
sessionInfo()
## R version 3.3.1 (2016-06-21)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 14.04.4 LTS
##
## locale:
## [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
## [3] LC_TIME=en_US.UTF-8 LC_COLLATE=C
## [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
## [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
## [9] LC_ADDRESS=C LC_TELEPHONE=C
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
##
## attached base packages:
## [1] stats4 parallel stats graphics grDevices utils datasets
## [8] methods base
##
## other attached packages:
## [1] BSgenome.Hsapiens.UCSC.hg19_1.4.0 BSgenome_1.40.1
## [3] Biostrings_2.40.2 XVector_0.12.1
## [5] rtracklayer_1.32.2 AnnotationHub_2.4.2
## [7] GenomicInteractions_1.6.2 InteractionSet_1.0.4
## [9] SummarizedExperiment_1.2.3 Biobase_2.32.0
## [11] GenomicRanges_1.24.3 GenomeInfoDb_1.8.7
## [13] IRanges_2.6.1 S4Vectors_0.10.3
## [15] BiocGenerics_0.18.0 PCHiCdata_1.0.2
## [17] Chicago_1.0.4 data.table_1.9.6
## [19] BiocStyle_2.0.3
##
## loaded via a namespace (and not attached):
## [1] httr_1.2.1 splines_3.3.1
## [3] assertthat_0.1 Formula_1.2-1
## [5] shiny_0.14 interactiveDisplayBase_1.10.3
## [7] latticeExtra_0.6-28 Rsamtools_1.24.0
## [9] yaml_2.1.13 RSQLite_1.0.0
## [11] lattice_0.20-34 biovizBase_1.20.0
## [13] chron_2.3-47 digest_0.6.10
## [15] RColorBrewer_1.1-2 colorspace_1.2-6
## [17] htmltools_0.3.5 httpuv_1.3.3
## [19] Matrix_1.2-7.1 plyr_1.8.4
## [21] XML_3.98-1.4 biomaRt_2.28.0
## [23] zlibbioc_1.18.0 xtable_1.8-2
## [25] scales_0.4.0 BiocParallel_1.6.6
## [27] tibble_1.2 ggplot2_2.1.0
## [29] GenomicFeatures_1.24.5 nnet_7.3-12
## [31] Gviz_1.16.5 survival_2.39-5
## [33] magrittr_1.5 mime_0.5
## [35] evaluate_0.9 MASS_7.3-45
## [37] foreign_0.8-67 BiocInstaller_1.22.3
## [39] tools_3.3.1 formatR_1.4
## [41] matrixStats_0.50.2 stringr_1.1.0
## [43] munsell_0.4.3 cluster_2.0.4
## [45] AnnotationDbi_1.34.4 ensembldb_1.4.7
## [47] grid_3.3.1 RCurl_1.95-4.8
## [49] dichromat_2.0-0 Delaporte_3.0.0
## [51] VariantAnnotation_1.18.7 igraph_1.0.1
## [53] bitops_1.0-6 rmarkdown_1.0
## [55] gtable_0.2.0 curl_2.1
## [57] DBI_0.5-1 R6_2.1.3
## [59] GenomicAlignments_1.8.4 gridExtra_2.2.1
## [61] dplyr_0.5.0 knitr_1.14
## [63] Hmisc_3.17-4 stringi_1.1.1
## [65] Rcpp_0.12.7 rpart_4.1-10
## [67] acepack_1.3-3.3
Blangiardo, Marta, Alberto Cassese, and Sylvia Richardson. 2010. “Sdef: An R Package to Synthesize Lists of Significant Features in Related Experiments.” BMC Bioinformatics 11 (1). BioMed Central Ltd: 270.
Jonathan Cairns*, Paula Freire-Pritchett*, Steven W. Wingett, Andrew Dimond, Vincent Plagnol, Daniel Zerbino, Stefan Schoenfelder, Biola-Maria Javierre, Cameron Osborne, Peter Fraser, and Mikhail Spivakov. 2015. “CHiCAGO: Robust Detection of DNA Looping Interactions in Capture Hi-C data.” BioRxiv. Cold Spring Harbor Labs Journals. doi:10.1101/028068.
Love, Michael I, Wolfgang Huber, and Simon Anders. 2014. “Moderated Estimation of Fold Change and Dispersion for RNA-Seq Data with DESeq2.” Genome Biology 15 (12): 550. doi:10.1186/s13059-014-0550-8.
Mifsud, Borbala, Filipe Tavares-Cadete, Alice N Young, Robert Sugar, Stefan Schoenfelder, Lauren Ferreira, Steven W Wingett, et al. 2015. “Mapping Long-Range Promoter Contacts in Human Cells with High-Resolution Capture Hi-c.” Nature Genetics 47 (6). Nature Publishing Group: 598–606.
Schoenfelder, Stefan, Mayra Furlan-Magaril, Borbala Mifsud, Filipe Tavares-Cadete, Robert Sugar, Biola-Maria Javierre, Takashi Nagano, et al. 2015. “The Pluripotent Regulatory Circuitry Connecting Promoters to Their Long-Range Interacting Elements.” Genome Research 25 (4). Cold Spring Harbor Lab: 582–97.
The ENCODE Project Consortium. 2012. “An Integrated Encyclopedia of DNA Elements in the Human Genome.” Nature 489 (7414): 57–74.