Niche differential gene expression analysis in spatial transcriptomics data identifies context-depen

Kaishu Mason, Nancy Zhang, University of Pennsylvania, Professor of Statistics Anuja Sathe, Stanford University, Postdoctoral Fellow

Single cells influence, and are shaped by, their local spatial niche. High-resolution technologies for in situ profiling of gene expression at the transcriptome scale are rapidly maturing, enabling the detailed interrogation of the spatial distributions of cell types in tissue as well as the elucidation of local signaling patterns between cell types. Towards these goals, methods for the detection of spatially variable genes, which are genes displaying globally differential expression patterns across the tissue region analyzed, have become a standard analysis procedure. Although spatially variable genes are useful for visualization of global patterns in tissue organization, it is unclear whether they reveal local cell-type specific interactions. Also, due to differences in frames of reference, it is unclear how to integrate multiple samples in spatially variable gene detection, and thus, current analyses can only be performed one sample at a time. We explore these limitations and propose a new statistical procedure called niche-differential expression (niche-DE) analysis. Niche-DE identifies cell-type specific niche-associated genes, defined as genes whose single cell expression is significantly up- or down-regulated, as compared to cell-type mean, in the context of specific spatial niches. Although niche-DE is conceptually defined on the single-cell level, we show that niche-DE genes can be recovered from lower resolution spatial transcriptomic (ST) data where each observation is a spot containing a mixture of cell types. We develop effective and interpretable measures for global false discovery control and show that the method is robust to technical artifacts such as bleeding of RNA molecules into adjacent spots/cells. Applied to ST data from liver metastases of colorectal cancer, niche-DE identifies marker genes for tumor-associated versus resident macrophages.