Skip to content

Vendi Score

Summary

The Vendi Score is a metadata-free metric used to evaluate the diversity of a dataset. Derived from the concept of effective rank in signal processing, it computes diversity by evaluating the eigenvalues of a kernel-based similarity matrix. A higher Vendi Score indicates a larger number of distinct, non-redundant samples in the dataset, representing high intrinsic diversity.

Mathematical Definition

For a collection of $N$ samples ($x_1, \dots, x_N$) and a positive semidefinite similarity function $k$ satisfying $k(x, x) = 1$: 1. Construct the $N \times N$ similarity kernel matrix $K$ where $K_{ij} = k(x_i, x_j)$. 2. Compute the eigenvalues $\lambda_1, \dots, \lambda_N$ of the normalised matrix $K/N$. 3. The Vendi Score ($VS_k$) is the exponential of the Shannon entropy of these eigenvalues:

$$VS_k(x_1, \dots, x_N) = \exp\left(-\sum_i \lambda_i \log \lambda_i\right)$$

When samples are highly similar (redundant), the similarity matrix has few dominant eigenvalues, yielding a low entropy and a low Vendi Score. When samples are highly distinct, the eigenvalues are distributed more evenly, resulting in a higher Vendi Score, with a maximum value of $N$.

Application in Single-Cell Biology

In the pretraining evaluation of Single-Cell Foundation Models by DenAdel et al. (2026), the Vendi Score was used to quantify transcriptomic-specific diversity of downsampled datasets. While metadata-based diversity measures (such as the Shannon Index and Gini-Simpson Index) were used to assess diversity of cell-type labels, the Vendi Score was computed on the highly variable genes of the normalized cell expression matrices to directly evaluate transcriptional diversity. They observed that Geometric Sketching yielded higher Vendi Scores than cell-type re-weighted downsampling, demonstrating its efficacy at capturing transcriptional variation.

Citations

[1] Friedman, D. & Dieng, A. B. The Vendi Score: a diversity evaluation metric for machine learning. Trans. Mach. Learn. Res. (2023). https://openreview.net/forum?id=g97OHbQyk1 [2] DenAdel, A. et al. (2026). Evaluating the role of pretraining dataset size and diversity on single-cell foundation model performance. Nature Methods, 23, 1447–1457. DOI: 10.1038/s41592-026-03120-y. Source paper: s41592-026-03120-y.pdf