In these sessions, we move from theory to code. Participants will get hands-on experience adapting LLM architectures for genomic data and exploring their capabilities.
The Goal: Learn how to implement a complete training pipeline to build and train a compact genomic BERT model from scratch.
Learn how to load and normalize a human coding-sequence dataset from Hugging Face.
Learn how to build a custom genomic tokenizer from scratch using character-level or k-mer tokenization approaches.
Learn how to define a lightweight MiniBERT architecture and train it using dynamic masking.
Learn how to save trained weights and tokenizer files to set up fine-tuning and continual learning workflows.
The Goal: Learn how to extract and analyze sequence-level embeddings from genomic data for downstream biological representation tasks.
Learn how to reconstruct the MiniBERT architecture and load pretrained model assets directly from the repository.
Learn how to tokenize Chromosome 22 data to extract 128-dimensional embeddings using mean pooling over the final hidden states.
Learn how to apply PCA to reduce dimensions and visualize biological features like GC content and exon/intron separation.
Learn how to train a downstream gradient boosting probe on the extracted embeddings to evaluate their biological utility.
The Goal: Learn how to apply a specialized genomic language model to evaluate and prioritize mutations in SARS-CoV-2 variant sequences.
Learn how to align Omicron Spike regions against the Wuhan-Hu-1 reference genome to identify nucleotide substitutions.
Learn how to map identified mutations to specific genomic regions and infer synonymous versus non-synonymous amino acid changes.
Learn how to mask mutation sites within their local sequence context to evaluate the model's token probabilities.
Learn how to compute a delta log-likelihood score to establish a model-based prioritization signal for mutation exploration.
Intermediate Python skills are recommended.
GitHub: https://github.com/pabloarozarena/aime2026-T7-genomic-llms