Poster #83: Cristina Martin Linares PhD

A Context-Dependent Foundation Model for Splicing

Cristina P Martin Linares1,2 1Johns Hopkins University, Department of Biomedical Engineering, Baltimore, MD, United States; 2Johns Hopkins University, Center for Computational Biology, Baltimore, MD, United States

Genomic foundation models aim to learn transferable representations from large-scale genomic data, but their ability to generalize to rare or out-of-distribution biology remains uncertain. We investigate this question by benchmarking five pretrained genomic models (AlphaGenome, Caduceus, DNABERT-2, HyenaDNA, and NucleotideTransformer-v3) and three specialized splicing models (OpenSpliceAI, Pangolin, and SpliceTransformer) on a rare, disease-related splicing process. We compared full-model fine-tuning with several PEFT strategies (head-only fine-tuning, LoRA, Houlsby adapters, and LoCon adaptation). Our results show that learned GFM representations do not transfer well to this benchmark under the PEFT strategies evaluated, and that good performance requires full-model fine-tuning with the entire backbone unfrozen. Because current GFMs are trained on a limited set of biological contexts, these findings suggest that better generalization to rare and disease-related tasks may require models conditioned on inputs beyond sequence alone. Existing sequence-to-function models such as AlphaGenome can be represented schematically as ŷ(g) = fθ(x(g)), where x(g) is the genomic sequence surrounding locus g and ŷ(g) is a vector of predictions across a fixed set of output channels. We instead propose a gene expression-conditioned splicing foundation model: y(s,g) ~ pθ(· | x(g), e(s)), with ŷ(s,g) = Eθ[y(s,g) | x(g), e(s)]. Here, e(s) is a gene-expression vector representing the cellular state of sample s, and y(s,g) is a random vector representing splice-site usage across locus g. Thus, cellular context is an input to the model, allowing the same genomic sequence to be conditioned on different observed transcriptomic states to produce sample-specific splicing profiles. As a proof of concept, we adapted the AlphaGenome framework and trained it on thousands of publicly available RNA-seq datasets to demonstrate that our approach can scale and generalize better to out-of-distribution data. More broadly, modular models that each learn a specific biological task may provide a flexible route toward future GFMs with better context-dependent predictions.