Skip to content

scCello

Summary

Yuan, Zhan, Zhang, et al. present scCello, a transcriptome foundation model (TFM) that, unlike prior TFMs treating cells as independent samples, explicitly incorporates the taxonomic relationships between cell types encoded in the Cell Ontology graph (Open Biological and Biomedical Ontology Foundry) during pretraining. Pretrained on 22 million CellxGene cells, scCello combines a standard masked gene expression prediction loss with two novel ontology-informed losses (cell-type coherence and cross-cell-type relational alignment), and reports consistently better generalization to unseen cell types, tissues, and donors than ontology-agnostic TFMs including Geneformer, scGPT, scTab, and UCE.

Method

Three-level pretraining objective, jointly minimized:

  1. Gene level: standard masked token prediction loss, learning gene co-expression patterns (as in Geneformer/scGPT-style TFMs).
  2. Intra-cellular level (cell-type coherence loss): a supervised contrastive loss that pulls same-cell-type representations together and pushes different-cell-type representations apart, using cell-type ontology labels available in CellxGene — more discriminative for representation learning than a simple cross-entropy classification loss.
  3. Inter-cellular level (relational/ontology alignment loss): forces the pairwise structural similarity between cell representations to reflect the pairwise node similarity in the Cell Ontology graph itself, so the model's latent space respects known cell-type lineage relationships, not just per-cell-type clustering.

Performance

  • Zero-shot cell-type clustering: scCello improved AvgBio (average of NMI, ARI, ASW clustering metrics) by 16.1% over baselines on the in-distribution (ID) dataset and 12.1% averaged across six out-of-distribution (OOD) datasets; ontology-agnostic TFMs (Geneformer, scGPT) barely outperformed non-TFM baseline methods, while scCello substantially did.
  • OOD generalization: on unseen tissues, scCello reached AvgBio of 0.6–0.7 where most baselines scored below that range; on unseen donors (the hardest OOD setting, where most methods scored AvgBio 0.45–0.55), scCello led with AvgBio above 0.6 on both evaluated datasets — only UCE outperformed scCello on cell-type clustering specifically, and by a narrow margin (0.03).
  • Fine-tuning: fine-tuned scCello outperformed other fine-tuned TFMs and non-TFM methods on both classification and clustering metrics, with up to 25.9% Macro F1 improvement over the best baseline; notably, even non-fine-tuned scCello surpassed other methods' fine-tuned performance.
  • Novel cell-type classification: by leveraging the Cell Ontology's lineage structure, scCello can classify cells of entirely unseen cell types without further fine-tuning — a task infeasible for supervised methods restricted to training-set labels.
  • Ablations: ablated variants (masked-gene-prediction-only, supervised-classification-only, and both-without-ontology-alignment) all underperformed the full three-loss scCello, confirming each pretraining objective's contribution, particularly the ontology relational-alignment loss.

Availability

Code and pretrained model weights: github.com/DeepGraphLearning/scCello.

See Also

  • Geneformer, scGPT, UCE — ontology-agnostic single-cell foundation models scCello directly benchmarks against and generally outperforms on OOD generalization tasks.

Citations

[1] Yuan, X., Zhan, Z., Zhang, Z., Zhou, M., Zhao, J., Han, B., Li, Y., & Tang, J. (2024). Cell-ontology guided transcriptome foundation model. 38th Conference on Neural Information Processing Systems (NeurIPS 2024).