HCAData 1.0.0
Compiled date: 2019-05-07
Last edited: 2018-11-13
License: MIT + file LICENSE
HCAData
packageThe HCAData package allows a direct access to the dataset generated by the Human Cell Atlas project for further processing in R and Bioconductor. It does so by providing the datasets as SingleCellExperiment objects, i.e. a format which is both efficient and very widely adopted throughout many existing Bioconductor workflows.
The datasets are otherwise available in other formats (also as raw data) at this link: http://preview.data.humancellatlas.org/.
HCAData
The HCAData package can be installed in the conventional way via BiocManager.
if (!requireNamespace("BiocManager", quietly = TRUE))
install.packages("BiocManager")
BiocManager::install("HCAData")
This package makes extensive use of the HDF5Array package to avoid loading the entire data set in memory. Instead, it stores the counts on disk as a HDF5 file, and loads subsets of the data into memory upon request.
HCAData
and the Human Cell Atlas datalibrary("HCAData")
We use the HCAData
function to download the relevant files from Bioconductor’s ExperimentHub web resource.
If no argument is provided, a list of the available datasets is returned, specifying which name to enter as dataset
parameter when calling HCAData
.
HCAData()
The list of relevant files includes the HDF5 file containing the counts, as well as the metadata on the rows (genes) and columns (cells).
The output is a single SingleCellExperiment
object from the SingleCellExperiment package.
Being based on ExperimentHub, the data related to this package can be accessed and queried directly using the package name. Retrieval is then as easy as using their ExperimentHub accession numbers (for the single components of each set), or by using the convenience function provided in this package.
suppressPackageStartupMessages({
library(ExperimentHub)
library(SingleCellExperiment)
})
eh <- ExperimentHub()
query(eh, "HCAData")
#> ExperimentHub with 6 records
#> # snapshotDate(): 2019-04-29
#> # $dataprovider: Human Cell Atlas
#> # $species: Homo sapiens
#> # $rdataclass: character
#> # additional mcols(): taxonomyid, genome, description,
#> # coordinate_1_based, maintainer, rdatadateadded, preparerclass,
#> # tags, rdatapath, sourceurl, sourcetype
#> # retrieve records with, e.g., 'object[["EH2047"]]'
#>
#> title
#> EH2047 | Human Cell Atlas - Census of Immune Cells, Bone marrow, 'dens...
#> EH2048 | Human Cell Atlas - Census of Immune Cells, Bone marrow, sampl...
#> EH2049 | Human Cell Atlas - Census of Immune Cells, Bone marrow, gene ...
#> EH2050 | Human Cell Atlas - Census of Immune Cells, Umbilical cord blo...
#> EH2051 | Human Cell Atlas - Census of Immune Cells, Umbilical cord blo...
#> EH2052 | Human Cell Atlas - Census of Immune Cells, Umbilical cord blo...
# these three are the components to the bone marrow dataset
bonemarrow_h5densematrix <- eh[["EH2047"]]
bonemarrow_coldata <- eh[["EH2048"]]
bonemarrow_rowdata <- eh[["EH2049"]]
# and are put together when calling...
sce_bonemarrow <- HCAData("ica_bone_marrow")
sce_bonemarrow
#> class: SingleCellExperiment
#> dim: 33694 378000
#> metadata(0):
#> assays(1): counts
#> rownames(33694): ENSG00000243485 ENSG00000237613 ... ENSG00000277475
#> ENSG00000268674
#> rowData names(2): ID Symbol
#> colnames(378000): MantonBM1_HiSeq_1-AAACCTGAGCAGGTCA-1
#> MantonBM1_HiSeq_1-AAACCTGCACACTGCG-1 ...
#> MantonBM8_HiSeq_8-TTTGTCATCTGCCAGG-1
#> MantonBM8_HiSeq_8-TTTGTCATCTTGAGAC-1
#> colData names(1): Barcode
#> reducedDimNames(0):
#> spikeNames(0):
# similarly, to access the umbilical cord blood dataset
sce_cordblood <- HCAData("ica_cord_blood")
sce_cordblood
#> class: SingleCellExperiment
#> dim: 33694 384000
#> metadata(0):
#> assays(1): counts
#> rownames(33694): ENSG00000243485 ENSG00000237613 ... ENSG00000277475
#> ENSG00000268674
#> rowData names(2): ID Symbol
#> colnames(384000): MantonCB1_HiSeq_1-AAACCTGAGGAGTTGC-1
#> MantonCB1_HiSeq_1-AAACCTGAGGCATTGG-1 ...
#> MantonCB8_HiSeq_8-TTTGTCATCCAGATCA-1
#> MantonCB8_HiSeq_8-TTTGTCATCGGTCTAA-1
#> colData names(1): Barcode
#> reducedDimNames(0):
#> spikeNames(0):
iSEE
The datasets are provided in the form of a SingleCellExperiment
object.
A natural companion to this data structure is the iSEE package, which can be used for interactive and reproducible data exploration.
Any analysis steps should be performed in advance before calling iSEE
, and since these datasets can be quite big, the operations can be time consuming, and/or require a considerable amount of resources.
For the scope of the vignette, we subset some cells in the bone marrow dataset to reduce the runtime, and apply some of the steps one would ideally follow in analysing droplet based datasets. For more information on how to properly process such datasets, please refer to the amazing set of resources available in the simpleSingleCell workflow package.
In brief: we start loading the required libraries for preprocessing, and taking a subset of the bone marrow dataset
library(scran)
library(scater)
set.seed(42)
sce <- sce_bonemarrow[, sample(seq_len(ncol(sce_bonemarrow)), 1000, replace = FALSE)]
First, we relabel the rows with the gene symbols for easier reading with the uniquifyFeatureNames()
function from scater.
Then we compute some QC metrics, using calculateQCMetrics()
.
rownames(sce) <- uniquifyFeatureNames(rowData(sce)$ID, rowData(sce)$Symbol)
head(rownames(sce))
#> [1] "RP11-34P13.3" "FAM138A" "OR4F5" "RP11-34P13.7"
#> [5] "RP11-34P13.8" "RP11-34P13.14"
counts(sce) <- as.matrix(counts(sce))
sce <- scater::calculateQCMetrics(sce)
We proceed with normalization, performed with the deconvolution method implemented in scran - a pre-clustering step is done in advance, to avoid pooling cells that are very different between each other.
set.seed(42)
clusters <- quickCluster(sce, method="igraph", min.mean=0.1, irlba.args=list(maxit=1000))
table(clusters)
#> clusters
#> 1 2 3 4 5
#> 222 210 129 178 261
sce <- computeSumFactors(sce, min.mean=0.1, cluster=clusters)
summary(sizeFactors(sce))
#> Min. 1st Qu. Median Mean 3rd Qu. Max.
#> 0.000465 0.325936 0.533787 1.000000 0.757043 28.645950
sce <- normalize(sce)
In the following lines of code, we model the mean-variance trend (with technical noise as Poisson), to extract the biological component of the variance for the genes under inspection.
new.trend <- makeTechTrend(x=sce)
fit <- trendVar(sce, use.spikes=FALSE, loess.args=list(span=0.05))
# plot(fit$mean, fit$var, pch=16)
# curve(fit$trend(x), col="dodgerblue", add=TRUE)
# curve(new.trend(x), col="red", add=TRUE)
fit0 <- fit
fit$trend <- new.trend
dec <- decomposeVar(fit=fit)
top.dec <- dec[order(dec$bio, decreasing=TRUE),]
head(top.dec)
#> DataFrame with 6 rows and 6 columns
#> mean total bio tech
#> <numeric> <numeric> <numeric> <numeric>
#> S100A8 1.11493057349193 5.38626448392199 4.29483312079354 1.09143136312844
#> S100A9 1.10437743825487 5.16160825818039 4.07615753701908 1.08545072116131
#> MALAT1 7.3308112726242 4.61018289683281 3.92709581513978 0.683087081693032
#> LYZ 1.15300320649838 4.99483962106588 3.88242885864087 1.11241076242501
#> HBB 1.08439490889895 4.24429550236879 3.17036827050413 1.07392723186466
#> RPS27 5.05339043152688 3.87882156515147 2.66980093676179 1.20902062838968
#> p.value FDR
#> <numeric> <numeric>
#> S100A8 0 0
#> S100A9 0 0
#> MALAT1 0 0
#> LYZ 0 0
#> HBB 0 0
#> RPS27 5.68030787893909e-229 3.54430173468469e-226
plotExpression(sce, features=rownames(top.dec)[1:10])
With the denoisePCA
function from scran, we can compute a reduced dimension view of the data, accounting for the Poisson technical trend.
Once we computed the PCA, we provide that as initialization to runTSNE
, and obtain the t-SNE results, stored in the appropriate slot of our sce
object.
set.seed(42)
sce <- denoisePCA(sce, technical=new.trend, approximate=TRUE)
ncol(reducedDim(sce, "PCA"))
#> [1] 7
plot(attr(reducedDim(sce), "percentVar"), xlab="PC",
ylab="Proportion of variance explained")
abline(v=ncol(reducedDim(sce, "PCA")), lty=2, col="red")
plotPCA(sce, ncomponents=3, colour_by="log10_total_features_by_counts")
set.seed(42)
sce <- runTSNE(sce, use_dimred="PCA", perplexity=30)
plotTSNE(sce, colour_by="log10_total_features_by_counts")
After this, we can compute a shared nearest neighbour graph to identify clusters in our dataset.
Once the cluster memberships are defined, we assign this vector to a corresponding colData
slot, and plot the t-SNE for our subset, coloured accordingly.
snn.gr <- buildSNNGraph(sce, use.dimred="PCA")
clusters <- igraph::cluster_walktrap(snn.gr)
sce$Cluster <- factor(clusters$membership)
table(sce$Cluster)
#>
#> 1 2 3 4 5 6 7 8 9 10
#> 183 171 69 240 69 43 106 35 37 47
plotTSNE(sce, colour_by="Cluster")
Even if we only took a subset of the available full data, we can observe that different clusters are indeed nicely separated. Subsequent steps would then involve identification of marker genes, and more advanced downstream techniques, which are not part of the scope of this vignette. To read more on what can be done, the vignettes of the simpleSingleCell workflow package are an excellent place to start.
iSEE
Once the processing steps above are done, we can call iSEE
with the subsampled SingleCellExperiment
object.
if (require(iSEE)) {
iSEE(sce)
}
You can save the sce
object to a serialized R object with
destination <- "where/to/store/the/processed/data.rds"
saveRDS(sce, file = destination)
The object can be read into a new R session with readRDS(destination)
, provided the HDF5 file remains in its original location (conveniently stored in the default location of ExperimentHub
).
sessionInfo()
#> R version 3.6.0 (2019-04-26)
#> Platform: x86_64-pc-linux-gnu (64-bit)
#> Running under: Ubuntu 18.04.2 LTS
#>
#> Matrix products: default
#> BLAS: /home/biocbuild/bbs-3.9-bioc/R/lib/libRblas.so
#> LAPACK: /home/biocbuild/bbs-3.9-bioc/R/lib/libRlapack.so
#>
#> locale:
#> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
#> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=C
#> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
#> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
#> [9] LC_ADDRESS=C LC_TELEPHONE=C
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
#>
#> attached base packages:
#> [1] parallel stats4 stats graphics grDevices utils datasets
#> [8] methods base
#>
#> other attached packages:
#> [1] scater_1.12.0 ggplot2_3.1.1
#> [3] scran_1.12.0 rhdf5_2.28.0
#> [5] ExperimentHub_1.10.0 AnnotationHub_2.16.0
#> [7] BiocFileCache_1.8.0 dbplyr_1.4.0
#> [9] HCAData_1.0.0 SingleCellExperiment_1.6.0
#> [11] SummarizedExperiment_1.14.0 DelayedArray_0.10.0
#> [13] BiocParallel_1.18.0 matrixStats_0.54.0
#> [15] Biobase_2.44.0 GenomicRanges_1.36.0
#> [17] GenomeInfoDb_1.20.0 IRanges_2.18.0
#> [19] S4Vectors_0.22.0 BiocGenerics_0.30.0
#> [21] BiocStyle_2.12.0
#>
#> loaded via a namespace (and not attached):
#> [1] bitops_1.0-6 bit64_0.9-7
#> [3] httr_1.4.0 dynamicTreeCut_1.63-1
#> [5] tools_3.6.0 R6_2.4.0
#> [7] irlba_2.3.3 HDF5Array_1.12.0
#> [9] vipor_0.4.5 DBI_1.0.0
#> [11] lazyeval_0.2.2 colorspace_1.4-1
#> [13] withr_2.1.2 gridExtra_2.3
#> [15] tidyselect_0.2.5 bit_1.1-14
#> [17] curl_3.3 compiler_3.6.0
#> [19] BiocNeighbors_1.2.0 labeling_0.3
#> [21] bookdown_0.9 scales_1.0.0
#> [23] rappdirs_0.3.1 stringr_1.4.0
#> [25] digest_0.6.18 rmarkdown_1.12
#> [27] XVector_0.24.0 pkgconfig_2.0.2
#> [29] htmltools_0.3.6 limma_3.40.0
#> [31] rlang_0.3.4 RSQLite_2.1.1
#> [33] shiny_1.3.2 DelayedMatrixStats_1.6.0
#> [35] dplyr_0.8.0.1 RCurl_1.95-4.12
#> [37] magrittr_1.5 BiocSingular_1.0.0
#> [39] GenomeInfoDbData_1.2.1 Matrix_1.2-17
#> [41] Rcpp_1.0.1 ggbeeswarm_0.6.0
#> [43] munsell_0.5.0 Rhdf5lib_1.6.0
#> [45] viridis_0.5.1 stringi_1.4.3
#> [47] yaml_2.2.0 edgeR_3.26.0
#> [49] zlibbioc_1.30.0 Rtsne_0.15
#> [51] plyr_1.8.4 grid_3.6.0
#> [53] blob_1.1.1 promises_1.0.1
#> [55] dqrng_0.2.0 crayon_1.3.4
#> [57] lattice_0.20-38 cowplot_0.9.4
#> [59] locfit_1.5-9.1 knitr_1.22
#> [61] pillar_1.3.1 igraph_1.2.4.1
#> [63] reshape2_1.4.3 glue_1.3.1
#> [65] evaluate_0.13 BiocManager_1.30.4
#> [67] httpuv_1.5.1 gtable_0.3.0
#> [69] purrr_0.3.2 assertthat_0.2.1
#> [71] xfun_0.6 rsvd_1.0.0
#> [73] mime_0.6 xtable_1.8-4
#> [75] later_0.8.0 viridisLite_0.3.0
#> [77] tibble_2.1.1 AnnotationDbi_1.46.0
#> [79] beeswarm_0.2.3 memoise_1.1.0
#> [81] statmod_1.4.30 interactiveDisplayBase_1.22.0