Expert Oversight of AI in Bioinformatics
Summary¶
Goh, Polster, Wong & Cvijovic (2026) argue against the "AI will replace bioinformaticians" narrative: AI recognizes statistical patterns but cannot judge biological meaning or verify scientific validity, so its outputs are only as trustworthy as the expert judgement applied to designing the analysis, curating training data, interpreting results, and catching failures. Without that oversight, AI does not democratize science — it democratizes the appearance of science: confident-looking, unvalidated outputs that non-experts cannot evaluate. The paper is a perspective piece, not an empirical study; its claims are argumentative and illustrated with examples rather than tested against new data.
Three Proposed Transitions for Bioinformaticians¶
- Tool user → AI custodian (technical): from consuming AI outputs to being personally responsible for whether they are correct, reproducible, and biologically meaningful. This spans prompt engineering (treated as a citable, version-controlled scientific asset, since benchmarking shows 20–40 percentage-point performance swings across prompt formulations on identical biological tasks), and agentic/orchestrated AI systems — exemplified by the Virtual Lab, where a coordinated team of LLM agents designed nanobody candidates using ESM, AlphaFold-Multimer, and Rosetta — which require architecting agent roles, validation checkpoints, and interpretation of each component model's limitations.
- Pipeline executor → pioneer of expert-dependent discovery (scientific): identifying problems where generic AI is least adequate — multimodal integration across genomics, clinical records, proteomics, and imaging, where the hard part is judging whether identified clusters are biologically real rather than technical artefacts (compounded by a persistent vocabulary gap between technical and clinical communities); and high-complexity applications (protein structure prediction, gene regulation, single-cell trajectory analysis, non-coding variant interpretation) where failure modes are subtle enough that only deep domain knowledge catches a plausible-looking but wrong result.
- Scientific contributor → institutional architect (organisational): closing the "implementation gap" between a method working in a research cohort and reaching clinical deployment (interoperability, clinician training, regulatory alignment); treating bias auditing and subgroup performance stratification as core scientific rigour rather than compliance overhead, citing the documented case of a commercial care-management algorithm that systematically underestimated the health needs of Black patients because healthcare cost was used as a proxy for health need; and engaging with the EU AI Act (in force August 2024, full high-risk medical-device compliance obligations from August 2027) and analogous FDA guidance as the legal codification of scientific principles the paper already argues for.
Illustrative Failure Modes Cited¶
- GWAS pipelines: AI can generate QC/stratification/formatting scaffolding code, but cannot decide appropriate QC thresholds for a specific cohort/platform, distinguish a linkage-disequilibrium artefact from an independent signal, or judge adequacy of population-structure correction — failures with precedent in false-positive associations that required retraction.
- Protein structure prediction: AlphaFold3's scientific value depends on knowing which protein families have unreliable predictions, how to interpret confidence scores in intrinsically disordered regions, and how to use structures to prioritize mutagenesis experiments rather than just generate figures.
- Data quality over architecture: a cited review found that in drug discovery, reported AI-architecture performance gains often disappeared once training sets were curated to remove known data-quality artefacts — arguing that expert-led data curation, not model sophistication, was the primary driver of genuine improvement.
- Explainability: current XAI methods (attention maps, SHAP, feature importance) are described as unstable, implementation-sensitive, and sometimes post-hoc rationalizations rather than genuine explanations — treated as starting points for expert interrogation (biological coherence, robustness under perturbation, external-cohort validation), not as self-sufficient endpoints.