New DNA methylation foundation model CpGPT shows promise for aging and disease prediction, pending peer review

A team of researchers has posted a preprint to the bioRxiv server describing CpGPT, short for Cytosine-phosphate-Guanine Pretrained Transformer, a foundation model built to read DNA methylation data. The paper has not been peer reviewed, and bioRxiv itself cautions that such reports should not be treated as conclusive or used to guide clinical practice or health-related behavior.

The model arrives in a field where most epigenetic aging clocks still rely on simple linear models fed with CpG methylation measurements. Those approaches often overlook the sequence context around each site and the complex interactions between sites. A few predictors, such as AltumAge and DeepMAge, already use deep neural networks, but the authors' starting point is that better models are needed to capture the mechanisms of aging. Transformers have already reshaped other corners of biology, including single-cell transcriptomics through models such as scGPT and Geneformer, and CpGPT applies that approach to methylation.

The training material is a dataset the team calls CpGCorpus. It gathers data from more than 1,502 studies in the Gene Expression Omnibus, totaling over 106,000 human samples. These span a range of tissue types, developmental stages, disease conditions and demographic backgrounds, and come from several Illumina array platforms: the 27k, 450k, EPIC, EPIC+ and EPICv2. Raw data went through the SeSAMe pipeline, while already processed data were used as normalized beta value matrices. The researchers applied quality control and probe harmonization, and split the data into training, validation and test sets with no overlap in samples or studies.

CpGPT draws on three kinds of information at once. The first is sequence context: embeddings of the nucleotides surrounding each CpG site, produced by a pre-trained DNA language model. The second is positional context, handled with a dual scheme combining absolute and relative encodings so the model sees both local and genome-wide location. The third is epigenetic state, meaning each site's methylation beta value, converted into an embedding. The core is Transformer++, a variant modified for training stability and accuracy. Embeddings are grouped by chromosome and stochastically shuffled to prevent positional bias. Separate decoders handle beta value prediction, condition prediction and uncertainty estimation.

Pretraining was unsupervised and multi-task, with tailored loss functions aimed at reconstructing missing data and learning useful representations of whole samples. The authors report that the learned locus embeddings cluster CpG sites by features such as island status and chromatin state, and that sample embeddings cluster by tissue type and cell line. The model also performed zero-shot reference mapping, transferring labels from annotated reference datasets to new datasets without additional training.

On practical tasks, the preprint reports strong imputation of missing methylation values, accurate reconstruction of beta values for missing probes, and improved performance of various existing epigenetic clocks. Its attention mechanism assigns importance scores to individual CpG sites for each sample, and the authors say this highlighted genes relevant to tissue-specific epigenetic regulation. Array conversion was another listed strength. Fine-tuning on mammalian data let the model predict methylation patterns in other species.

For mortality, the team selected CpG sites using intra-class correlation coefficients and z-score thresholds, then trained with a modified Cox proportional hazard loss. Evaluation across multiple cohorts used Cox regression, receiver operating characteristic analyses and survival analyses, adjusted for age. The authors report that the fine-tuned model stratified individuals by biological aging profile and associated significantly with mortality and morbidity, including neurodegenerative disease, cardiovascular problems and physical function measures.

The authors conclude that capturing dependencies among CpG sites lets CpGPT overcome the limits of linear models. Whether those results hold up awaits peer review and independent validation before any clinical use.