Nucleotide Transformer is a set of foundation models pre-trained on DNA sequences, developed by InstaDeep with collaborators. The models range from 50 million up to 2.5 billion parameters and learn from 3,202 human genomes plus 850 genomes from diverse species. They produce context-specific representations of nucleotide sequences, which support accurate predictions of molecular phenotypes even when annotated data is scarce. The paper describing them was published on November 28, 2024.
Nucleotide Transformer is for genomics researchers, bioinformaticians and machine learning scientists who work with DNA sequence data. It helps most when a lab has a small labeled dataset and wants to start from a pre-trained model instead of training from scratch. Because the authors show that fine-tuning is low cost, small academic groups and biotech teams with modest compute can still adapt it. It is a research contribution and reference, so users are expected to be comfortable with model training workflows.
A research group can fine-tune a model to predict regulatory elements such as promoters or enhancers from raw sequence. A clinical genomics team can use the learned attention patterns, which focus on key genomic elements even without supervision, to help prioritize genetic variants. A computational biologist can extract embeddings from the models and feed them into downstream classifiers for tasks where only a few labeled examples exist. Comparative genomics projects can take advantage of the multi-species training data to study sequence patterns across organisms and test how well representations transfer between tasks.
