Empirical Research Assistance (ERA)

Summary¶

Empirical Research Assistance (ERA) is an artificial intelligence system developed by Google Research and Google DeepMind that automatically generates expert-level scientific software to optimize a measurable quality metric. It combines large language models (LLMs) with tree search to systematically explore a code space and iteratively improve program scores. ERA has demonstrated state-of-the-art performance in complex scientific tasks, including single-cell RNA sequencing (scRNA-seq) batch integration, COVID-19 epidemiology forecasting, and time-series modeling.

Overview¶

ERA treats the creation of empirical software as a search problem over a space of programs to maximize a scorable target metric. Instead of relying on random mutations as in traditional genetic programming, ERA leverages an LLM (such as Gemini 2.5 Flash or Gemini 3.1 Pro) to propose semantic, context-aware code mutations. These mutations are guided by research ideas extracted from scientific literature, academic textbooks, or search engine queries.

Architecture¶

The system consists of the following core components: 1. LLM Mutator: An LLM that takes existing code, error logs, and optional research guidance to generate new code candidates. 2. Execution Sandbox: A secure environment where proposed code is compiled, run, and evaluated against a defined validation dataset. 3. Tree Search Algorithm: A predictor-based Upper Confidence Bound applied to trees (PUCT) algorithm that balances exploitation (refining high-scoring code) and exploration (branching from earlier nodes in the search tree).

Key Applications¶

Bioinformatics (scRNA-seq Batch Integration)¶

In single-cell genomics, removing batch effects while preserving biological variability is a critical challenge. Evaluating on the OpenProblems v2.0.0 batch integration benchmark (as of July 2025), ERA optimized several base methods by injecting paper summaries into the LLM prompt. The resulting system discovered 40 new methods that outperformed the top human-developed methods on the leaderboard. Its top-performing implementation, BBKNN (TS), recombined ComBat-corrected principal component analysis (PCA) embeddings with batch-balanced $k$-nearest neighbors, yielding a 14% improvement over the best published methods.

Epidemiology (COVID-19 Hospitalization Forecasting)¶

ERA was applied retrospectively to the CDC COVID-19 Forecast Hub benchmark (as of May 2025). The system optimized and combined forecasting models, producing the "Google Retrospective" ensemble model that achieved a lower average Weighted Interval Score (WIS) of 26, outperforming the official CDC ensemble's score of 29.

General Time-Series Forecasting¶

Using the GIFT-Eval benchmark (as of May 2025), ERA was used to develop a unified, general-purpose forecasting library from scratch. Guided by the geometric mean of the normalized Mean Absolute Scaled Error (MASE) across 97 datasets, ERA iteratively constructed a model that sequentially isolates and forecasts base levels, linear/polynomial trends, seasonal cycles, datetime/holiday features, and residual corrections, achieving a competitive final MASE of 0.734.

Contributors¶

Google Research — co-developed ERA's program-search and empirical-software workflow.

Citations¶

Aygün, E., Belyaeva, A., Comanici, G., Coram, M., Cui, H., Garrison, J., ... & Brenner, M. P. (2026). An AI system to help scientists write expert-level empirical software. Nature, 654, 909–915. DOI: 10.1038/s41586-026-10658-6. Source paper: s41586-026-10658-6.pdf