1 Introduction

smoothclust is a method for segmentation of spatial domains and spatially-aware clustering in spatial transcriptomics data. The method generates spatial domains with smooth boundaries by smoothing gene expression profiles across neighboring spatial locations, followed by unsupervised clustering. Spatial domains consisting of consistent mixtures of cell types may then be further investigated by applying cell type compositional analyses or differential analyses.

2 Installation

The smoothclust package can be installed from Bioconductor as follows (using R version 4.4 onwards). This is the recommended installation for most users. Additional details are shown on the Bioconductor package landing page.

install.packages("BiocManager")
BiocManager::install("smoothclust")

The latest development version of the package can also be installed from the devel version of Bioconductor or from GitHub.

3 Input data format

Input data can be provided either as a SpatialExperiment object within the Bioconductor framework, or as numeric matrices of expression values and spatial coordinates. See help file (?smoothclust) for details.

In the example workflow below, we assume the input is in SpatialExperiment format.

4 Tutorial

The example workflow in this section demonstrates how to run smoothclust to generate spatial domains with smooth boundaries for a dataset from the 10x Genomics Visium platform.

library(smoothclust)
library(STexampleData)
library(scuttle)
library(scran)
library(scater)
library(ggspavis)

Load dataset from STexampleData package:

# load data
spe <- Visium_humanDLPFC()
## see ?STexampleData and browseVignettes('STexampleData') for documentation
## loading from cache
# keep spots over tissue
spe <- spe[, colData(spe)$in_tissue == 1]

dim(spe)
## [1] 33538  3639
assayNames(spe)
## [1] "counts"

Run smoothclust using default parameter settings, which have been selected to be appropriate for Visium data from human tissue.

The method for smoothing can be specified by providing the method argument, with available options uniform, kernel, and knn. Additional arguments can be used to set parameter values, including bandwidth, k, and truncate, depending on the choice of method. For more details, see the function documentation (?smoothclust) or the paper (in preparation).

# run smoothclust
spe <- smoothclust(spe)

The smoothed expression counts are stored in a new assay named counts_smooth:

# check output object
assayNames(spe)
## [1] "counts"        "counts_smooth"

Calculate log-transformed normalized counts (logcounts) on the smoothed expression counts. Here, the argument assay.type = "counts_smooth" specifies that we want to calculate logcounts using the smoothed counts from smoothclust.

# calculate logcounts
spe <- logNormCounts(spe, assay.type = "counts_smooth")

assayNames(spe)
## [1] "counts"        "counts_smooth" "logcounts"

Run clustering. We use a standard clustering workflow from single-cell data, consisting of k-means clustering on the top principal components (PCs) calculated on the set of top highly variable genes (HVGs) with logcounts as the input.

We use a relatively small number of clusters for demonstration purposes in this example:

# preprocessing steps for clustering

# remove mitochondrial genes
is_mito <- grepl("(^mt-)", rowData(spe)$gene_name, ignore.case = TRUE)
table(is_mito)
## is_mito
## FALSE  TRUE 
## 33525    13
spe <- spe[!is_mito, ]
dim(spe)
## [1] 33525  3639
# select top highly variable genes (HVGs)
dec <- modelGeneVar(spe)
top_hvgs <- getTopHVGs(dec, prop = 0.1)
length(top_hvgs)
## [1] 986
spe <- spe[top_hvgs, ]
dim(spe)
## [1]  986 3639
# dimensionality reduction

# compute PCA on top HVGs
set.seed(123)
spe <- runPCA(spe)
# run k-means clustering
set.seed(123)
k <- 5
clust <- kmeans(reducedDim(spe, "PCA"), centers = k)$cluster
table(clust)
## clust
##    1    2    3    4    5 
##  967 1574  492  272  334
colLabels(spe) <- factor(clust)

Plot clusters / spatial domains generated by the smoothclust workflow:

# color palettes
pal8 <- "libd_layer_colors"
pal36 <- unname(palette.colors(36, "Polychrome 36"))

# plot clusters / spatial domains
plotSpots(spe, annotate = "label", pal = pal8)

Plot manually annotated reference labels, which can be used to evaluate the performance of the clustering in this dataset:

# plot reference labels
plotSpots(spe, annotate = "ground_truth", pal = pal8)

5 Session information

sessionInfo()
## R version 4.4.1 (2024-06-14)
## Platform: x86_64-pc-linux-gnu
## Running under: Ubuntu 24.04.1 LTS
## 
## Matrix products: default
## BLAS:   /home/biocbuild/bbs-3.20-bioc/R/lib/libRblas.so 
## LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.12.0
## 
## locale:
##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
##  [3] LC_TIME=en_GB              LC_COLLATE=C              
##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
## 
## time zone: America/New_York
## tzcode source: system (glibc)
## 
## attached base packages:
## [1] stats4    stats     graphics  grDevices utils     datasets  methods  
## [8] base     
## 
## other attached packages:
##  [1] ggspavis_1.12.0             scater_1.34.0              
##  [3] ggplot2_3.5.1               scran_1.34.0               
##  [5] scuttle_1.16.0              STexampleData_1.13.3       
##  [7] SpatialExperiment_1.16.0    SingleCellExperiment_1.28.0
##  [9] SummarizedExperiment_1.36.0 Biobase_2.66.0             
## [11] GenomicRanges_1.58.0        GenomeInfoDb_1.42.0        
## [13] IRanges_2.40.0              S4Vectors_0.44.0           
## [15] MatrixGenerics_1.18.0       matrixStats_1.4.1          
## [17] ExperimentHub_2.14.0        AnnotationHub_3.14.0       
## [19] BiocFileCache_2.14.0        dbplyr_2.5.0               
## [21] BiocGenerics_0.52.0         smoothclust_1.2.0          
## [23] BiocStyle_2.34.0           
## 
## loaded via a namespace (and not attached):
##   [1] RColorBrewer_1.1-3       jsonlite_1.8.9           wk_0.9.4                
##   [4] magrittr_2.0.3           ggbeeswarm_0.7.2         magick_2.8.5            
##   [7] farver_2.1.2             rmarkdown_2.28           zlibbioc_1.52.0         
##  [10] vctrs_0.6.5              spdep_1.3-6              memoise_2.0.1           
##  [13] tinytex_0.53             htmltools_0.5.8.1        S4Arrays_1.6.0          
##  [16] curl_5.2.3               BiocNeighbors_2.0.0      s2_1.1.7                
##  [19] SparseArray_1.6.0        sass_0.4.9               spData_2.3.3            
##  [22] KernSmooth_2.23-24       bslib_0.8.0              cachem_1.1.0            
##  [25] igraph_2.1.1             mime_0.12                ggside_0.3.1            
##  [28] lifecycle_1.0.4          pkgconfig_2.0.3          rsvd_1.0.5              
##  [31] Matrix_1.7-1             R6_2.5.1                 fastmap_1.2.0           
##  [34] GenomeInfoDbData_1.2.13  digest_0.6.37            colorspace_2.1-1        
##  [37] AnnotationDbi_1.68.0     dqrng_0.4.1              irlba_2.3.5.1           
##  [40] RSQLite_2.3.7            beachmat_2.22.0          labeling_0.4.3          
##  [43] filelock_1.0.3           fansi_1.0.6              httr_1.4.7              
##  [46] abind_1.4-8              compiler_4.4.1           proxy_0.4-27            
##  [49] bit64_4.5.2              withr_3.0.2              BiocParallel_1.40.0     
##  [52] viridis_0.6.5            DBI_1.2.3                highr_0.11              
##  [55] rappdirs_0.3.3           DelayedArray_0.32.0      rjson_0.2.23            
##  [58] classInt_0.4-10          bluster_1.16.0           tools_4.4.1             
##  [61] units_0.8-5              vipor_0.4.7              beeswarm_0.4.0          
##  [64] glue_1.8.0               dbscan_1.2-0             grid_4.4.1              
##  [67] sf_1.0-18                cluster_2.1.6            generics_0.1.3          
##  [70] gtable_0.3.6             class_7.3-22             BiocSingular_1.22.0     
##  [73] ScaledMatrix_1.14.0      metapod_1.14.0           sp_2.1-4                
##  [76] utf8_1.2.4               XVector_0.46.0           ggrepel_0.9.6           
##  [79] BiocVersion_3.20.0       pillar_1.9.0             limma_3.62.0            
##  [82] dplyr_1.1.4              lattice_0.22-6           bit_4.5.0               
##  [85] deldir_2.0-4             tidyselect_1.2.1         locfit_1.5-9.10         
##  [88] Biostrings_2.74.0        knitr_1.48               gridExtra_2.3           
##  [91] bookdown_0.41            edgeR_4.4.0              xfun_0.48               
##  [94] statmod_1.5.0            UCSC.utils_1.2.0         yaml_2.3.10             
##  [97] boot_1.3-31              evaluate_1.0.1           codetools_0.2-20        
## [100] tibble_3.2.1             BiocManager_1.30.25      cli_3.6.3               
## [103] munsell_0.5.1            jquerylib_0.1.4          Rcpp_1.0.13             
## [106] png_0.1-8                parallel_4.4.1           blob_1.2.4              
## [109] sparseMatrixStats_1.18.0 viridisLite_0.4.2        scales_1.3.0            
## [112] e1071_1.7-16             purrr_1.0.2              crayon_1.5.3            
## [115] rlang_1.1.4              KEGGREST_1.46.0