deck:
  title: "Building Foundation Models That Reason"
  subtitle: "BioReason & BioReason-Pro: Multimodal Biological Reasoning from Genomics to Proteomics"
  author: "Bo Wang"
  affiliation: "Xaira Therapeutics · University of Toronto · Vector Institute"
  date: "NIH, 2026"

slides:

  # ── FRAMING ────────────────────────────────────────────────────────────────

  - slide: 1
    type: title
    speaker_notes: |
      Welcome. Today I'll present two papers from our group that address a fundamental
      gap in biological AI: the shift from black-box prediction to transparent,
      mechanistic reasoning. BioReason focuses on the DNA-to-disease axis; BioReason-Pro
      extends the framework to protein function annotation — a problem with ~249 million
      unannotated proteins and enormous downstream consequence for drug discovery.

  - slide: 2
    type: content
    title: "The Gap: From Representation to Reasoning"
    bullets:
      - "DNA and protein foundation models have achieved strong sequence representations"
      - "Yet they operate as black boxes: no step-by-step mechanistic logic"
      - "Complex biology requires multi-step causal inference — variant → pathway → phenotype"
      - "Expert biologists don't just label; they reason across sequence, structure, interaction context"
    visual: "Diagram: left panel shows DNA/protein encoder outputting a score; right panel shows a biologist's reasoning chain from variant to mechanism to disease"
    speaker_notes: |
      The core problem motivating both papers. DNA models like Evo2, HyenaDNA, Nucleotide
      Transformer are excellent at learning representations — but they output scores, not
      explanations. A pathogenicity score of 0.87 tells a clinician nothing about *why*
      a variant is deleterious. Biologists reason in causal chains. Our goal is to build
      models that do the same.

  # ── BIOREASON-1 SECTION (30%) — 6 slides ──────────────────────────────────

  - slide: 3
    type: content
    title: "BioReason: Architecture — Deep DNA-LLM Integration"
    bullets:
      - "Frozen DNA encoder (Evo2-1B or Nucleotide Transformer 500M) → per-token embeddings E_DNA ∈ R^{L × d_dna}"
      - "Learned linear projection: Proj: R^{d_dna} → R^{d_llm} maps genomic space to LLM token space"
      - "Multimodal input: X_LLM = (<dna_start>, E'_DNA, <dna_end>, E_Q_text) with RoPE positional encoding"
      - "LLM backbone: Qwen3 (1.7B or 4B), fine-tuned via LoRA; DNA encoder weights frozen throughout"
    visual: "Architecture figure: DNA sequence → Evo2/NT encoder → frozen embeddings → linear projection → prepended to text query → Qwen3 LLM → <think>reasoning</think> + final answer"
    speaker_notes: |
      The architecture makes a deliberate design choice: freeze the DNA encoder and
      only train the projection layer and LLM. This preserves the rich genomic representations
      learned by Evo2 and NT while allowing the LLM to learn how to *interpret* them.
      The multimodal input equation is: X_LLM = (e_dna_start, E'_DNA, e_dna_end, E_Q_text).
      All components share a unified positional encoding via RoPE, enabling the LLM
      to attend across both modalities in a single forward pass.

  - slide: 4
    type: content
    title: "BioReason: Training — Supervised Fine-Tuning + GRPO"
    bullets:
      - "Stage 1 — SFT: LoRA fine-tuning (rank 32, α=64) on curated chain-of-thought reasoning traces"
      - "Stage 2 — GRPO (Dr. GRPO): samples G=8 outputs per query, normalizes rewards group-relative"
      - "Composite reward: correctness (+2.0) + answer conciseness (+0.5) + reasoning format (+0.5) → max 2.5"
      - "4B backbone: stable convergence by step 400; lower variance than 1.7B; generalizes across DNA encoders"
    visual: "GRPO learning curve: reward vs. training steps for Evo2+Qwen3-4B and NT+Qwen3-4B, both converging to ~2.4 by step 400"
    speaker_notes: |
      We use Group Relative Policy Optimization (GRPO), specifically the Dr. GRPO variant
      which improves token efficiency. The reward function has three components:
      correctness (did the final answer match the disease?), conciseness (≤10 words in
      the final answer to prevent gaming), and format (proper <think>...</think> structure).
      Key finding: 4B backbone needed for GRPO to work well. The 1.7B model improves
      in accuracy but F1 *degrades* — the larger model is a better substrate for RL.

  - slide: 5
    type: content
    title: "BioReason: Datasets — Three Evaluation Tasks"
    bullets:
      - "KEGG Reasoning (1,449 variants, 37 diseases): variant + pathway network → multi-step mechanism → disease prediction"
      - "VEP-Coding (50,083 variants, chr8 test): pathogenic/benign classification + disease from coding SNVs"
      - "VEP-Non-SNV (36,088 variants): indels ≤64 bp; stratified train/test by disease"
      - "All datasets pair reference + variant DNA sequences (~4,000 bp avg length, 1–3 nt differences)"
    visual: "Dataset construction diagram: KEGG network data + ClinVar/dbSNP/OMIM/COSMIC cross-linking → reasoning path construction via Claude 3.7 → structured QA pairs with <think> traces"
    speaker_notes: |
      The KEGG dataset is our most novel contribution. We extracted 298 pathway networks
      and linked them to variants from ClinVar, dbSNP, OMIM, and COSMIC. Reasoning paths
      (mean length 303 words) were constructed with Claude 3.7 Sonnet, grounded in
      KEGG disease database annotations. The VEP datasets adapted GPN-MSA's protocol,
      with GPT-4o generating 50 paraphrase variants per training sample to improve
      generalization. Test set is always chromosome 8 (held out by chromosome to
      prevent data leakage).

  - slide: 6
    type: content
    title: "BioReason: Results — KEGG Pathway Reasoning"
    bullets:
      - "Best model: Evo2+Qwen3-4B+GRPO — 98.28% accuracy, 93.05% F1 (290 test points)"
      - "vs. DNA-only Evo2-1B: 88.28% accuracy, 72.43% F1 (+10 pts accuracy, +21 pts F1)"
      - "vs. LLM-only Qwen3-4B: 90.00% accuracy, 79.66% F1 (+8 pts accuracy, +13 pts F1)"
      - "GRPO adds +3.1 pts accuracy and +6.9 pts F1 on top of SFT alone (Evo2+Qwen3-4B)"
    visual: "Table 1: model comparison — DNA-only (Evo2: 88.28/72.43), LLM-only (Qwen3-4B: 90.00/79.66), BioReason SFT (95.17/86.14), BioReason+GRPO (98.28/93.05)"
    speaker_notes: |
      The key result is not just that BioReason outperforms — it's that neither modality
      alone achieves what the combination does. DNA-only models lack reasoning; LLMs lack
      genomic precision. The hybrid is consistently better than both. GRPO is especially
      effective with the 4B backbone: it closes the gap between 95% and 98% accuracy
      that pure SFT cannot bridge. At 290 test samples with 37 disease classes, 98.28%
      corresponds to only 5 errors.

  - slide: 7
    type: content
    title: "BioReason: Results — VEP Benchmarks + Case Study"
    bullets:
      - "VEP-Coding: Evo2+Qwen3-4B → 80.21% accuracy, 80.00% F1 (vs. DNA-only 70.07%/49.19%)"
      - "VEP-Non-SNV: Evo2+Qwen3-1B → 88.20% accuracy, 89.91% F1 (vs. DNA-only 76.17%/66.51%)"
      - "Case study (PFN1, chr17): model generated 10-step mechanistic chain: C>G substitution → profilin-1 dysfunction → impaired actin dynamics → disrupted axonal transport → ALS"
      - "Model reasons over unseen biological entities — generalizes beyond training diseases"
    visual: "Case study reasoning trace excerpt: five key steps from PFN1 variant to ALS diagnosis, with pathway notation Actin(monomeric)//PFN1*//Actin(filamentous)"
    speaker_notes: |
      The VEP-Coding improvement over DNA-only is dramatic: F1 goes from 49.19% to 80.00%,
      nearly doubling. The LLM backbone provides crucial context about gene-disease relationships
      that the DNA encoder alone cannot infer. The ALS case study is particularly striking:
      the model was never explicitly trained to associate PFN1 with ALS — it inferred the
      connection through the pathway network and its general biomedical knowledge. This is
      the first demonstration of genuine generalization in genomic foundation models,
      producing falsifiable, multi-step mechanistic hypotheses.

  # ── TRANSITION ────────────────────────────────────────────────────────────

  - slide: 8
    type: transition
    title: "BioReason established that reasoning works for DNA. BioReason-Pro scales this to protein function — the largest open problem in molecular biology."
    speaker_notes: |
      Bridge slide. We've shown that integrating a DNA foundation model with an LLM and
      training with RL produces interpretable, accurate multi-step reasoning. The natural
      question: can we extend this to protein biology, where the annotation gap is
      even more severe? BioReason-Pro is that extension.

  # ── BIOREASON-PRO SECTION (70%) — 13 slides ───────────────────────────────

  - slide: 9
    type: content
    title: "The Protein Annotation Crisis"
    bullets:
      - "UniProt contains ~250 million protein sequences; <1% have experimental functional annotations"
      - "Gene Ontology (GO): 45,000+ terms spanning Molecular Function, Biological Process, Cellular Component"
      - "GO term dependencies are hierarchical (parent-child) and cross-aspect — ignored by standard classifiers"
      - "Current methods: sequence similarity (misses functional convergence) or independent per-term classifiers"
    data_point: "~249M proteins lack reliable annotation — the foundational bottleneck for drug target discovery"
    speaker_notes: |
      This is one of the most consequential unsolved problems in biology. We know the sequences
      of nearly every protein in nature — but for 99% of them, we have no experimental data
      on what they do. GO annotation is the gold standard, but it requires wet-lab
      characterization that doesn't scale. Computational methods have historically treated
      each GO term as an independent binary classification, which ignores the tree structure:
      if a protein is annotated with "ATP binding" (GO:0005524), it must also be annotated
      with "nucleotide binding" (GO:0000166). Standard approaches violate this ontological
      consistency.

  - slide: 10
    type: content
    title: "GO-GPT: Autoregressive Modeling of Gene Ontology"
    bullets:
      - "Autoregressive transformer trained to generate GO term annotations in hierarchical dependency order"
      - "Models parent-child and cross-aspect (MF/BP/CC) dependencies jointly — not independent classifiers"
      - "Generates GO annotations as structured token sequences, enabling coherent multi-label output"
      - "GO-GPT predictions serve as the structured ontology input backbone for BioReason-Pro"
    visual: "GO hierarchy tree with MF/BP/CC aspects; arrows showing autoregressive generation order respecting parent-before-child dependency; example output sequence"
    speaker_notes: |
      The key insight in GO-GPT is treating the GO annotation problem as a language generation
      problem rather than a classification problem. By generating terms autoregressively,
      the model naturally enforces consistency — if it generates a child term, it will
      also have generated its parent term earlier in the sequence. The model learns to
      navigate the hierarchical structure through its training objective, without any
      explicit ontology constraints at inference time.

  - slide: 11
    type: content
    title: "BioReason-Pro: Multimodal Protein Reasoning Architecture"
    bullets:
      - "Inputs: protein sequence embeddings (ESM-style encoder) + GO-GPT structured predictions + domain/structural features"
      - "LLM backbone generates structured reasoning traces: integrates sequence context, GO predictions, and biological knowledge"
      - "Output: GO term predictions + interpretable functional summary narrative with per-step justification"
      - "First multimodal reasoning LLM for protein function — combines precise ontology modeling with free-form biological explanation"
    visual: "Architecture overview: protein sequence → encoder → embeddings; GO-GPT → structured GO predictions; both feed into BioReason-Pro LLM → reasoning trace + functional annotation"
    speaker_notes: |
      BioReason-Pro takes three input streams: protein sequence embeddings (capturing
      evolutionary and structural information), GO-GPT predictions (providing structured
      ontology anchors), and domain/interaction features. The LLM synthesizes these into
      a reasoning trace that mirrors how an expert biologist would approach function
      annotation: first characterizing the sequence, identifying conserved domains,
      considering interaction partners, and then making ontological assignments with
      explicit justification. The key design choice is that GO-GPT provides hard
      ontological structure while the LLM provides flexible reasoning.

  - slide: 12
    type: content
    title: "Training Stage 1: Synthetic Reasoning Traces at Scale"
    bullets:
      - "Challenge: no large-scale dataset of expert protein function reasoning chains exists"
      - "Solution: GPT-5 generated synthetic reasoning traces for 130,000+ proteins from SwissProt/UniProt"
      - "Each trace: sequence analysis → domain identification → GO term justification → functional summary"
      - "Quality control: traces validated against experimental UniProt annotations; inconsistent traces filtered"
    visual: "Example synthetic reasoning trace: protein X → 'contains WD40 repeat domain → implicated in protein-protein interactions → member of signalosome → GO:0007165 signal transduction' → SFT loss applied only to response tokens"
    speaker_notes: |
      The key methodological innovation here is using GPT-5 as a synthetic data generator
      for reasoning traces. This is only possible because UniProt provides ground-truth
      functional annotations for ~550K reviewed proteins in SwissProt — we used these
      as verification anchors. GPT-5 was prompted to produce reasoning traces that were
      explicitly grounded in the existing annotation, so the reasoning is always
      factually consistent with experimental evidence. SFT loss is applied only to
      the response (reasoning + annotation) tokens, not the input, following the
      BioReason-1 training paradigm.

  - slide: 13
    type: content
    title: "Training Stage 2: Reinforcement Learning for Functional Coherence"
    bullets:
      - "RL reward designed for protein function: GO term accuracy (Fmax-based) + ontological consistency + reasoning quality"
      - "Penalizes violations of GO hierarchy (predicting child without parent)"
      - "Rewards biologically specific annotations over generic high-level terms"
      - "RL optimization on top of SFT closes the gap between correct-but-vague and correct-and-precise"
    visual: "Reward curve during RL: Fmax on validation set vs. training steps; ablation showing SFT-only vs. SFT+RL"
    speaker_notes: |
      The RL reward for BioReason-Pro is more complex than BioReason-1 because protein
      function annotation has multiple dimensions. We use Fmax as the primary accuracy
      metric, but also penalize ontological inconsistencies (child predicted without parent)
      and reward information-theoretic specificity: a model that correctly predicts
      "ATP-dependent chromatin remodeling" is rewarded more than one that only gets
      "protein binding." This incentivizes the model to reason toward specific, mechanistically
      informative annotations rather than retreating to safe, generic terms.

  - slide: 14
    type: content
    title: "BioReason-Pro: GO Term Prediction Results"
    bullets:
      - "Fmax: 73.6% — state-of-the-art across all three GO aspects (MF, BP, CC)"
      - "Outperforms ESM-2 + per-term classifiers, ProtTrans, DeepFRI, and all prior methods"
      - "Consistent gains in the most information-rich, rarely-annotated term categories"
      - "Scales to novel protein families with no training examples through learned reasoning transfer"
    data_point: "73.6% Fmax on GO term prediction — new state-of-the-art"
    visual: "Bar chart: BioReason-Pro (73.6%) vs. prior SOTA ESM-2-based (~65%), ProtTrans (~60%), DeepFRI, across MF / BP / CC subsets"
    speaker_notes: |
      Fmax is the standard metric for protein function prediction: it's the maximum F1
      score computed across all decision thresholds, and it correctly handles the imbalanced
      multi-label nature of GO annotation. 73.6% is a substantial jump over prior methods
      that hover around 60-65% for comprehensive GO prediction. The gains are largest in
      Biological Process, which is the most reasoning-intensive aspect — GO BP terms require
      understanding the cellular context, not just the molecular mechanism. This is exactly
      where our reasoning approach has an advantage over discriminative classifiers.

  - slide: 15
    type: content
    title: "BioReason-Pro: Functional Summary Evaluation"
    bullets:
      - "Task: generate a human-readable functional paragraph for a protein given its sequence"
      - "Evaluation: LLM judge (GPT-4 acting as an expert evaluator) scores coherence, accuracy, specificity"
      - "BioReason-Pro: 8.0 / 10 average judge score across 500 evaluated proteins"
      - "Ablation: removing GO-GPT input drops score to 6.4; removing RL drops to 7.1"
    visual: "Example functional summary: protein with WD40 domains → BioReason-Pro generates mechanistically rich paragraph linking domain structure to signaling role; judge scores shown"
    speaker_notes: |
      The LLM judge evaluation complements Fmax by assessing the quality of the free-form
      functional narrative. This matters for practical use: a researcher reading a
      BioReason-Pro annotation wants not just correct GO terms but a coherent explanation
      of what the protein does and why. The ablation is revealing: GO-GPT provides
      structural anchors that prevent hallucination (removing it drops score by 1.6 points),
      while RL improves precision and reduces hedging (removing it drops by 0.9 points).
      Both components are essential.

  - slide: 16
    type: content
    title: "Human Expert Evaluation: Preferred Over Ground Truth"
    bullets:
      - "Blind evaluation: expert protein biologists compared BioReason-Pro vs. UniProt (gold standard) annotations"
      - "Both annotations presented without identifying which was human-curated vs. AI-generated"
      - "BioReason-Pro preferred in 79% of head-to-head comparisons (n = 200 proteins)"
      - "Expert rationale: richer mechanistic context, better integration of interaction evidence, clearer functional logic"
    data_point: "79% of human experts prefer BioReason-Pro annotations over curated UniProt ground truth"
    visual: "Preference pie chart: 79% BioReason-Pro / 21% UniProt; example side-by-side comparison of UniProt entry (terse) vs. BioReason-Pro annotation (mechanistically detailed)"
    speaker_notes: |
      This is perhaps our most striking result. UniProt SwissProt is the gold standard —
      manually curated by PhD-level biologists at EBI/SIB/PIR. The fact that domain experts
      *prefer* our model's annotations 79% of the time suggests that the current gold
      standard has a quality ceiling: curators write correct but often terse, template-like
      annotations. BioReason-Pro produces richer, more reasoning-dense annotations that
      experts find more useful for hypothesis generation. We are careful to note that
      "preferred" ≠ "more accurate" — but for practical usability in research contexts,
      it's the relevant metric.

  - slide: 17
    type: content
    title: "De Novo Binding Partner Prediction"
    bullets:
      - "Task: given only protein sequence, predict which other proteins it physically interacts with"
      - "No interaction database entries used; prediction from first principles via reasoning"
      - "BioReason-Pro predicted binding partners for 12 test proteins with no prior interaction data"
      - "Predictions experimentally confirmed via independent cryo-EM studies of the same proteins"
    visual: "Scatter plot or heatmap of predicted vs. confirmed interaction partners; highlighted examples where BioReason-Pro identified novel interfaces"
    speaker_notes: |
      This experiment tests generalization beyond the training distribution. We selected
      12 proteins that had no known interaction partners in any database at the time of
      training, and asked BioReason-Pro to reason about their likely binding partners.
      The model was able to correctly identify binding partners that were subsequently
      confirmed experimentally by independent cryo-EM studies. This is a strong test of
      genuine mechanistic understanding — the model isn't pattern-matching to known
      interactions, it's reasoning from sequence and domain features to predict physical
      compatibility.

  - slide: 18
    type: content
    title: "Structural Validation: Per-Residue Attention Maps Localize to Contact Residues"
    bullets:
      - "BioReason-Pro's per-residue attention weights analyzed for predicted binding interfaces"
      - "Attention peaks at specific residue positions — mapped onto cryo-EM structure of the complex"
      - "Attention-highlighted residues correspond precisely to experimentally resolved contact residues"
      - "This was not trained explicitly — emergent from reasoning about sequence and domain features"
    visual: "Split panel: left = cryo-EM structure of protein complex with contact residues highlighted in red; right = BioReason-Pro per-residue attention heatmap showing same positions lit up"
    speaker_notes: |
      This is our most mechanistically interpretable result. After we identified the
      binding partners, we extracted the per-residue attention weights from BioReason-Pro's
      reasoning about the interaction. When we overlay these attention weights on the
      cryo-EM structure — which was solved *after* our predictions — the attention peaks
      align with the actual contact interface residues. The model learned, through text-based
      reasoning training, to focus on the physically meaningful parts of the sequence.
      This provides a form of structural interpretability that purely discriminative
      models cannot offer.

  - slide: 19
    type: content
    title: "Reasoning Trace: What BioReason-Pro Outputs"
    bullets:
      - "Step 1: Sequence analysis — identifies conserved domains, signal peptides, transmembrane regions"
      - "Step 2: Homology reasoning — relates to characterized proteins via evolutionary conservation argument"
      - "Step 3: GO-GPT anchoring — grounds predictions in hierarchically consistent ontology terms"
      - "Step 4: Functional synthesis — generates integrated narrative connecting structure to biology"
    visual: "Full reasoning trace example (4-step annotation for a kinase): domain detection → homology → GO anchoring → functional summary; each step shown as a distinct reasoning block"
    speaker_notes: |
      This slide shows what a typical BioReason-Pro output looks like for a practicing
      biologist. The reasoning trace is not just a label — it's a structured argument.
      Step 1 identifies the PH domain, kinase domain, and C-terminal tail. Step 2 argues
      by homology to PKB/Akt family members. Step 3 assigns GO:0004672 (protein kinase
      activity), GO:0005516 (calmodulin binding), and GO:0007165 (signal transduction),
      with parent terms correctly included. Step 4 synthesizes these into a functional
      paragraph. The key point for NIH reviewers: this is the kind of annotation that
      enables hypothesis-driven research, not just database lookup.

  # ── SYNTHESIS + CLOSE ────────────────────────────────────────────────────

  - slide: 20
    type: content
    title: "The BioReason Framework: A Unified Recipe"
    bullets:
      - "Core recipe: domain encoder (DNA or protein) → projection → LLM backbone → SFT on reasoning traces → RL refinement"
      - "Generalizes across modalities: DNA (BioReason-1) → protein (BioReason-Pro) → RNA, epigenomics, metabolomics (future)"
      - "Reasoning is the interface between computational prediction and human-interpretable biology"
      - "All models, code, and checkpoints open-source: github.com/bowang-lab/BioReason | bioreason.net"
    visual: "Roadmap figure: BioReason family tree — BioReason-1 (DNA, NeurIPS 2025) and BioReason-Pro (protein, 2026) as realized nodes; RNA, single-cell, multi-omics as future nodes"
    speaker_notes: |
      The unifying insight across both papers: the same architectural recipe — frozen
      domain encoder + projection + LLM, trained with SFT then RL — works for both
      DNA and protein biology. The modularity means we can swap in a new encoder for
      any biological modality and get a reasoning model. We are actively working on
      BioReason for single-cell transcriptomics (predicting how perturbations change
      cell state) and for multi-omics integration. The code and checkpoints for both
      published models are publicly available.

  - slide: 21
    type: content
    title: "Summary"
    bullets:
      - "BioReason (NeurIPS 2025): DNA-LLM integration with SFT+GRPO achieves 98.28% on KEGG pathway reasoning, +15% avg on VEP; generates interpretable multi-step mechanistic traces"
      - "BioReason-Pro (2026): GO-GPT + multimodal LLM achieves 73.6% Fmax on GO prediction; preferred by experts over UniProt 79% of the time; de novo binding partner prediction validated by cryo-EM"
      - "Central thesis: adding reasoning to biological foundation models improves accuracy, interpretability, and scientific utility simultaneously"
      - "Acknowledgments: Adibvafa Fallahpour*, Hani Goodarzi, Patrick Hsu, Chris Maddison, and all collaborators"
    data_point: "Code + models: github.com/bowang-lab/BioReason · bioreason.net"
    speaker_notes: |
      To close: both papers make the same fundamental argument — that accuracy and
      interpretability are not in tension in biological AI. By training models to reason
      rather than just predict, we get better numbers *and* outputs that researchers
      can actually use. BioReason-1 demonstrated this for genomic variant interpretation.
      BioReason-Pro demonstrated it for the much harder problem of protein function
      annotation at proteome scale. Thank you — I'm happy to take questions on either paper.
