AHEnsDbs 1.0.4
Authors: Johannes Rainer
Last modified: 2017-08-31 07:21:30
Compiled: Thu Aug 31 07:22:24 2017
EnsDb databases from AnnotationHubThe AHEnsDbs package provides the metadata for all EnsDb SQLite databases in AnnotationHub. First we load/update the AnnotationHub resource.
library(AnnotationHub)
ah <- AnnotationHub()Next we list all EnsDb entries from AnnotationHub.
query(ah, "EnsDb")## AnnotationHub with 204 records
## # snapshotDate(): 2017-08-28
## # $dataprovider: Ensembl
## # $species: Ailuropoda Melanoleuca, Anas Platyrhynchos, Anolis Carolinens...
## # $rdataclass: EnsDb
## # additional mcols(): taxonomyid, genome, description,
## # coordinate_1_based, maintainer, rdatadateadded, preparerclass,
## # tags, rdatapath, sourceurl, sourcetype
## # retrieve records with, e.g., 'object[["AH53185"]]'
##
## title
## AH53185 | Ensembl 87 EnsDb for Anolis Carolinensis
## AH53186 | Ensembl 87 EnsDb for Ailuropoda Melanoleuca
## AH53187 | Ensembl 87 EnsDb for Astyanax Mexicanus
## AH53188 | Ensembl 87 EnsDb for Anas Platyrhynchos
## AH53189 | Ensembl 87 EnsDb for Bos Taurus
## ... ...
## AH56717 | Ensembl 89 EnsDb for Takifugu Rubripes
## AH56718 | Ensembl 89 EnsDb for Tursiops Truncatus
## AH56719 | Ensembl 89 EnsDb for Vicugna Pacos
## AH56720 | Ensembl 89 EnsDb for Xiphophorus Maculatus
## AH56721 | Ensembl 89 EnsDb for Xenopus TropicalisWe fetch the EnsDb for species Ciona Intestinalis and ensembl release 87.
qr <- query(ah, c("EnsDb", "intestinalis", "87"))
edb <- qr[[1]]## require("ensembldb")## loading from cache '/Users/jo//.AnnotationHub/59931'To get the genes defined for this species we can simply call.
genes(edb)## GRanges object with 17153 ranges and 7 metadata columns:
## seqnames ranges strand | gene_id
## <Rle> <IRanges> <Rle> | <character>
## ENSCING00000017842 1 [ 1636, 1902] + | ENSCING00000017842
## ENSCING00000016550 1 [ 4219, 20394] - | ENSCING00000016550
## ENSCING00000016539 1 [23518, 34155] + | ENSCING00000016539
## ENSCING00000002700 1 [37108, 60412] - | ENSCING00000002700
## ENSCING00000002712 1 [62406, 66903] - | ENSCING00000002712
## ... ... ... ... . ...
## ENSCING00000025225 MT [11275, 11335] + | ENSCING00000025225
## ENSCING00000025226 MT [11337, 12899] + | ENSCING00000025226
## ENSCING00000025227 MT [12910, 12973] + | ENSCING00000025227
## ENSCING00000025228 MT [12990, 14724] + | ENSCING00000025228
## ENSCING00000025229 MT [14725, 14790] + | ENSCING00000025229
## gene_name gene_biotype seq_coord_system
## <character> <character> <character>
## ENSCING00000017842 misc_RNA chromosome
## ENSCING00000016550 protein_coding chromosome
## ENSCING00000016539 protein_coding chromosome
## ENSCING00000002700 protein_coding chromosome
## ENSCING00000002712 protein_coding chromosome
## ... ... ... ...
## ENSCING00000025225 Mt_tRNA chromosome
## ENSCING00000025226 COI protein_coding chromosome
## ENSCING00000025227 Mt_tRNA chromosome
## ENSCING00000025228 NADH5 protein_coding chromosome
## ENSCING00000025229 Mt_tRNA chromosome
## description
## <character>
## ENSCING00000017842 NULL
## ENSCING00000016550 NULL
## ENSCING00000016539 NULL
## ENSCING00000002700 NULL
## ENSCING00000002712 NULL
## ... ...
## ENSCING00000025225 NULL
## ENSCING00000025226 cytochrome oxidase subunit I (mitochondrion) [Source:RefSeq peptide;Acc:NP_758778]
## ENSCING00000025227 NULL
## ENSCING00000025228 NADH dehydrogenase subunit 5 (mitochondrion) [Source:RefSeq peptide;Acc:NP_758779]
## ENSCING00000025229 NULL
## symbol entrezid
## <character> <list>
## ENSCING00000017842 NA
## ENSCING00000016550 NA
## ENSCING00000016539 NA
## ENSCING00000002700 NA
## ENSCING00000002712 100181294
## ... ... ...
## ENSCING00000025225 NA
## ENSCING00000025226 COI 806118
## ENSCING00000025227 NA
## ENSCING00000025228 NADH5 806127
## ENSCING00000025229 NA
## -------
## seqinfo: 763 sequences from KH genomeThis section describes the (semi)-automated way to create ensembldb EnsDb SQLite annotation databases using the Ensembl Perl API and the MySQL database dumps from Ensembl.
EnsDb SQLite databasesTo create the databases we are sequentially downloading and installing the MySQL database dumps from Ensembl locally and query these local databases to extract the annotations that will be inserted into the final SQLite databases. While it would be possible to directly query the Ensembl databases using the Perl API, this has proven to be slower and frequently fails due to access shortages or time outs.
Below we load the ensembldb package and the generate-EnsDbs.R script in its scripts folder.
library(ensembldb)
scr <- system.file("scripts/generate-EnsDbs.R", package = "ensembldb")
source(scr)We next use the createEnsDbForSpecies function. This function queries first the Ensembl ftp server to list all species for a certain Ensembl release. By default it will then process all species, by first downloading the MySQL database dumps, installing it into a local MySQL server and subsequently, using functionality from the ensembldb package and the Ensembl Perl API, exctracting all annotations from it and storing it to a SQLite datababse. The local MySQL database will then by default deleted (setting the parameter dropDb = FALSE does not delete the databases).
Important parameter to set in the function are: + ftp_folder: by default the main Ensembl ftp server will be queried (ftp://ftp.ensembl.org/pub), alternatively, also the Ensemblgenomes database can be queried. To build e.g. EnsDbs for all plants use ftp://ftp.ensemblgenomes.org/pub/release-34/plants/mysql. If this parameter is defined it has to point to the directory in which all MySQL database dumps can be found. + ens_version: the Ensembl version (e.g. 87). + user: the user name for the local MySQL database. + host: the hostname of the local MySQL database (e.g. "localhost"). + pass: the password for the local MySQL database.
Below we create all EnsDb databases for Ensembl release 87. Note that we have to have installed the Ensembl Perl API matching this release.
createEnsDbForSpecies(ens_version = 87, user = "someuser", host = "localhost",
pass = "somepass")All SQLite database, one per species, will be stored in the current working directory.