Skip to content

Ret-AAE

Summary

Ret-AAE (Retinal Adversarial Autoencoder) is an unsupervised deep learning framework designed to compress optical coherence tomography (OCT) and color fundus photography (CFP) images into 256-dimensional vector embeddings. By bypassing the limitations of expert-defined anatomical metrics, Ret-AAE enables hypothesis-free, multi-omic exploration of oculo-systemic associations. Ret-AAE-derived latent features have been shown to capture key ophthalmic features and predict future risks of cardiovascular and neurodegenerative diseases.

Overview

Traditional analyses of ophthalmic images rely on predefined structural markers like retinal layer thickness or vessel tortuosity. Ret-AAE presents a latent-space approach that represents complex and abstract structural patterns as continuous vector embeddings. Separate models are trained for CFP and OCT modalities to explore phenotypic and genotypic associations in a manner free from human labeling assumptions. The embeddings have been integrated with genomics, metabolomics, and brain/cardiac imaging to advance the field of Oculomics.

Model Architecture and Loss Functions

The Ret-AAE framework is implemented in PyTorch and incorporates: - Encoder: Employs 2D convolutions, batch normalization, Leaky ReLU, multi-scale residual blocks (using kernel sizes of 3 and 5), and an efficient multi-scale attention module to map inputs (resized to 224x224 pixels) to a 256-dimensional bottleneck. - Decoder: Utilizes transposed 2D convolutions, multi-scale residual blocks, and multi-scale attention modules to reconstruct the original image. - Discriminator: Uses a three-layer feedforward network with Leaky ReLU activations to match the aggregated posterior distribution of latent codes to a multivariate Gaussian prior $N(0, 1)$, ensuring a regularized and smooth latent space.

Modality-Specific Loss Functions

  • OCT-AAE: Regularized using mean absolute error ($L_1$ loss) and adversarial loss, trained for 250 epochs.
  • CFP-AAE: Trained for 400 epochs, integrating $L_1$ loss, adversarial loss, and a perceptual loss calculated via the 4th ReLU layer of a pre-trained VGG16 network. Due to the high contrast and sparsity of vasculature in fundus photographs, the $L_1$ loss is weighted 12.29x higher for pixels corresponding to blood vessels (identified using AutoMorph binary masks) so that vasculature structures contribute 50% of the total reconstruction loss.

Performance and Validation

The quality of reconstructions was assessed using standard metrics (as of 2026-06-16): - CFP Modality: Reached a Structural Similarity Index Metric (SSIM) of 0.89 and a Peak Signal-to-Noise Ratio (PSNR) of 30.65 dB. - OCT Modality: Achieved an SSIM of 0.72 and a PSNR of 29.19 dB. The lower metrics reflect the smoothing of noise (speckles) native to OCT B-scans.

Positive control studies showed Ret-AAE embeddings successfully correlate with 50 conventional ophthalmic traits (visual acuity, refractive error, intraocular pressure) and predict incident ophthalmic disorders such as cataracts, glaucoma, and retinal detachments.

Key Applications and Findings

Using the UK Biobank cohort, Ret-AAE was applied to multi-omic association pipelines: - Cardiovascular Health: Latent features correlate with blood pressure, cardiac cycle functional volumes, arterial stiffness, and predict incident ischemic heart disease, stroke, and heart failure. - Neurodegeneration: Embeddings localizing to the optic nerve head, retinal nerve fiber layer (RNFL), and ellipsoid zone correlate with cerebral gray/white matter volumes and predict future risks of Parkinson's disease, vascular dementia, and Alzheimer's disease. - Genomic and Metabolomic Grounding: Genome-wide association studies (GWAS) of the 512 embeddings (256 per modality) linked latent dimensions to pigmentation genes (TYR, OCA2, DCT, TSPAN10), retinal physiology (RDH5, VSX2), and vascular regulation (PDE3A). MAGMA gene-set analysis highlighted pathways involving the complement system (CFHR), fibroblast growth factor receptor 2 (FGFR2), and alpha-synuclein.

Citations

  • Julian, T. H. et al. (2026). Multi-omic analysis of deep learning-derived phenotypes links ophthalmic imaging to cardiovascular and neurological traits. Nature Cardiovascular Research, 5, 541–554. DOI: 10.1038/s44161-026-00815-5. Source paper: s44161-026-00815-5.pdf