GO — Healthy across all four use cases

• Last commit 4d ago
• 12 active contributors
• MIT licensed
• Tests present
• ⚠ Concentrated ownership — top contributor handles 68% of recent commits
• ⚠ No CI workflows detected

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

nanochat is a minimal, single-GPU LLM training harness that trains GPT-2-capability models in ~2 hours on 8XH100 GPUs for ~$48 (or $15 on spot instances). It covers the full LLM pipeline: tokenization, pretraining, supervised finetuning, evaluation on benchmarks (ARC, GSM8K, MMLU, HumanEval), and a ChatGPT-like web UI. The core innovation is compute-optimal scaling via a single --depth hyperparameter dial that auto-calculates all other model dimensions and training recipes. Monolithic package structure: nanochat/ contains core modules (gpt.py=model, engine.py=training loop, dataloader.py=data pipeline, tokenizer.py=vocab, optim.py=optimization). scripts/ has entry points (chat_cli.py, chat_sft.py, chat_eval.py for different stages). tasks/ provides benchmark implementations (arc.py, gsm8k.py, mmlu.py). runs/ contains shell scripts for reproducible experiments (speedrun.sh is the main reference).

ML researchers and engineers who want to rapidly prototype and benchmark small-to-medium LLMs without enterprise infrastructure; people interested in reproducible scaling laws; hobbyists and students training models on budget cloud hardware; contributors optimizing the 'time-to-GPT-2' leaderboard in runs/speedrun.sh.

Actively developed but experimental. Recent commits (Jan-Mar 2026 per leaderboard) show continuous iteration on the speedrun record; 6 leaderboard entries show active optimization. No visible CI/test suite in file structure suggests pre-production quality. Single main contributor (karpathy) with occasional collaborators (@ddudek). Verdict: experimental/research-grade, not production-ready, but rapidly improving.

Single-maintainer repo (karpathy) creates bus factor risk. No visible test infrastructure or CI pipeline in file list means regressions may slip through. Rapidly changing hyperparameters and dataset choices (row 4: NVIDIA ClimbMix swap, row 5-6: 'autoresearch' unexplained changes) suggest API instability. Dependencies not fully listed but likely includes torch, transformers, and custom CUDA kernels (fp8.py, flash_attention.py) which can break on hardware/software version changes.

Active optimization of the pretraining stage to minimize wall-clock time to GPT-2 capability (last entry: 1.65 hours achieved Mar 14 2026). Recent changes include FP8 quantization adoption (row 2), batch size scaling to 1M tokens (row 3), and dataset swaps to ClimbMix (row 4). 'Autoresearch' mode (rows 5-6) suggests automated hyperparameter search is being deployed. Focus is explicitly on the leaderboard speedrun challenge.

git clone https://github.com/karpathy/nanochat.git
cd nanochat
# Check .python-version (likely Python 3.10+)
pip install -e .
# Verify tokenizer and model work:
python -c "from nanochat.gpt import GPT; print(GPT)"

Daily commands:

# Reference speedrun (GPT-2 in 1.65h):
bash runs/speedrun.sh

# Or custom training with --depth dial (e.g., depth 26 ≈ GPT-2):
python scripts/chat_sft.py --depth 26 --batch_size 1000000

# Chat interface:
python scripts/chat_web.py

# Evaluation:
python scripts/chat_eval.py --checkpoint <path>

• nanochat/gpt.py — Core GPT model architecture implementation; defines the transformer backbone that all training/inference pipelines depend on.
• nanochat/engine.py — Central training and inference orchestration engine; handles the main training loop, checkpoint management, and forward/backward passes.
• nanochat/dataloader.py — Data pipeline that feeds batches to the engine; critical for understanding how datasets are loaded and streamed during training.
• scripts/base_train.py — Primary entry point for training jobs; demonstrates the standard workflow for launching a training run with all hyperparameters.
• nanochat/tokenizer.py — Tokenization layer that converts raw text into token sequences; essential for both preprocessing and inference.
• nanochat/common.py — Shared utilities and configuration helpers (e.g., hyperparameter scaling logic); referenced across the entire codebase.
• nanochat/checkpoint_manager.py — Handles model serialization, resumption, and checkpoint versioning; critical for long-running training sessions.

Add a New Benchmark Task

1. Create a new task file in tasks/ directory inheriting from task base class (tasks/customjson.py)
2. Implement the evaluate() method that runs your benchmark and returns metrics (tasks/humaneval.py)
3. Register the task in the core_eval harness by importing and adding to task registry (nanochat/core_eval.py)
4. Run evaluation via: python scripts/chat_eval.py --task <your_task_name> (scripts/chat_eval.py)

Train a Model with Custom Hyperparameters

1. Modify depth, width, and learning rate in the base_train.py script or via CLI args (scripts/base_train.py)
2. Review compute-optimal scaling logic that auto-calculates other hyperparams from depth (nanochat/common.py)
3. Launch training with your dataset; engine handles batching and checkpointing (nanochat/engine.py)
4. Monitor loss and metrics via report.py output or tensorboard logs (nanochat/report.py)

Add a New Optimization Technique

1. Implement new optimizer or learning rate schedule in optim.py (nanochat/optim.py)
2. Optionally add quantization/precision options in fp8.py or flash_attention.py (nanochat/fp8.py)
3. Integrate into engine.py's forward/backward pass or gradient updates (nanochat/engine.py)
4. Test via base_train.py with --optim flag or internal config (scripts/base_train.py)

Extend the Chat Interface

1. Modify the HTML/JavaScript chat UI template (nanochat/ui.html)
2. Extend the Flask/web server in chat_web.py to handle new endpoints (scripts/chat_web.py)
3. Update the inference engine or execution layer if needed for new features (nanochat/execution.py)
4. Restart the web server and test via browser (scripts/chat_web.py)

• PyTorch — Industry-standard deep learning framework with strong GPU support and automatic differentiation.
• Single-GPU design (8XH100 target) — Maximizes accessibility and cost-efficiency; scales complexity via depth parameter rather than data/tensor parallelism.
• Compute-optimal scaling (depth → other hyperparams) — Automates the hyperparameter search space; users dial one knob (depth) and get optimal configuration.
• Flash Attention & FP8 — Reduces wall-clock time and memory; critical for training GPT-2-scale models on budget.
• Web UI (HTML + Flask in chat_web.py) — Familiar ChatGPT-like interface for interactive inference; lowers barrier to using trained models.

• Single GPU node instead of distributed training

  • Why: Simplifies code, reduces debugging complexity, and matches Karpathy's design philosophy of minimal/hackable.
  • Consequence: Limited to models fitting on ~80GB GPU memory; scaling beyond GPT-2 capability requires multi-node (not yet implemented).

• Compute-optimal depth-based scaling (no traditional hyperparameter grid search)

  • Why: Reduces user decision-making and automatically selects learning rates, layer counts, etc. based on compute budget.
  • Consequence: Less flexibility for custom architectures; assumes scaling laws hold across the model size range.

• Minimal dependencies and custom implementations (flash_attention.py, fp8.py)

  • Why: Keeps codebase hackable and reduces external dependency fragility.
  • Consequence: Reinventing optimized operations increases maintenance burden and risk of bugs vs. using well-tested libraries.

• Single-epoch or few-epoch training runs with fixed dataset

  • Why: Simplifies data loading and matches typical LLM training workflows.
  • Consequence: No built-in curriculum learning or data augmentation; assumes clean, high-quality datasets.

• Distributed training across multiple GPUs or nodes (single-node only).
• Real-time chat deployment or production serving (research harness, not production system).
• Multi-modal training (text-only).
• Fine-grained control over every hyperparameter (compute-optimal scaling constrains choices).
• Support for TPU or non-CUDA accelerators.

No CI/automated tests visible; easy to introduce training regressions. The --depth hyperparameter is the only config knob but requires understanding nanochat/common.py's auto-scaling math—wrong depth may cause OOM or training instability. FP8 quantization (fp8.py) is hardware-specific (H100 assumed) and may not work on older GPUs. Dataset paths likely hardcoded or environment-dependent; ClimbMix swap (row 4) suggests external dataset URLs that may become unavailable. 'Autoresearch' mode (rows 5-6) is undocumented—unclear how hyperparameters are being tuned. No Docker setup visible; Python version in .python-version is critical. Leaderboard uses wall-clock time on 8XH100, making reproducibility dependent on exact hardware/cloud provider.

• Compute-Optimal Scaling Laws — nanochat's core insight: automatically derive optimal model width, depth, and training steps from a single --depth dial using empirical scaling laws; critical for understanding why the leaderboard entries vary so much on the same GPU.
• FP8 Quantization — Row 2 speedup (2.91h) uses fp8.py for reduced precision training; essential for understanding the hardware-level optimization gap between baseline and optimized runs.
• FlashAttention — flash_attention.py is a memory-efficient attention kernel crucial for throughput; reduces training time by 30-50% on dense model training—why this file exists separately.
• Byte Pair Encoding (BPE) — tokenizer.py implements BPE to convert raw text to token IDs; the input bottleneck that must be fast for data loading.
• Supervised Fine-Tuning (SFT) — scripts/chat_sft.py adapts pretrained models to chat via SFT on instruction datasets (SmolTalk in tasks/smoltalk.py); how nanochat enables ChatGPT-like behavior.
• DCLM CORE Benchmark — The metric used to rank leaderboard entries; understanding what CORE measures (downstream task performance on FNet downstream tasks) is critical for interpreting leaderboard scores.
• Gradient Checkpointing — Likely used in engine.py to reduce memory during backprop; enables larger batch sizes without OOM, directly impacting speedrun performance.

• karpathy/nanoGPT — Predecessor/inspiration for nanochat; a minimal single-file GPT trainer that nanochat builds upon with production features (finetuning, eval, UI, speedrun optimization).
• meta-llama/llama — Industry-standard LLM codebase; nanochat is a simplified research-grade alternative for single-GPU experimentation with similar architecture but vastly reduced scope.
• huggingface/transformers — Ecosystem dependency providing pretrained models and tokenizers; nanochat likely uses or mirrors tokenization APIs from this library.
• openai/gpt-2 — Reference checkpoint and benchmark target; row 0 of the leaderboard is the original 2019 OpenAI GPT-2 that nanochat aims to replicate 168x faster.
• together-ai/open-instruct — Instruction-tuning pipeline similar to nanochat's SFT stage (scripts/chat_sft.py); useful for understanding how to finetune pretrained models for chat.

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add comprehensive unit tests for nanochat/fp8.py and nanochat/flash_attention.py

These are critical performance optimization modules, but tests/test_attention_fallback.py only covers attention fallback. FP8 quantization and flash attention are hardware-specific optimizations that need regression testing to ensure they work correctly across different GPU types and that numerical precision is maintained. This is especially important since nanochat targets cost-efficient training—broken quantization silently breaks model quality.

• [ ] Create tests/test_fp8.py with tests for quantization/dequantization correctness, gradient flow, and numerical stability
• [ ] Expand tests/test_attention_fallback.py or create tests/test_flash_attention.py with tests for output correctness vs standard attention and shape handling
• [ ] Add test cases for edge cases (very small/large tensors, different dtypes)
• [ ] Reference these modules in the test suite to catch regressions during development

Add missing docstrings and type hints to nanochat/gpt.py and nanochat/engine.py

These are the core model and training loop files, but without comprehensive documentation, new contributors cannot understand the architecture. The repo explicitly claims to be 'minimal/hackable', but minimal code without documentation creates a barrier. Adding type hints also enables better IDE support and catches bugs early. These files are likely the most-read files for anyone trying to understand the system.

• [ ] Add full docstrings (including Args, Returns, Raises) to all public functions in nanochat/gpt.py (GPT model class and forward pass logic)
• [ ] Add full docstrings to key training loop functions in nanochat/engine.py (training step, evaluation, checkpoint management)
• [ ] Add type hints to function signatures throughout both files
• [ ] Document the relationship between depth parameter and other hyperparameters since that's the core 'complexity dial' mentioned in README

Add integration tests for the end-to-end training pipeline in scripts/base_train.py

The repo supports multiple training stages (pretraining, finetuning) via scripts but there are no integration tests validating the full pipeline works. Currently only test_engine.py exists. New contributors or changes to dataloader.py, dataset.py, or checkpoint_manager.py could silently break the training pipeline. A small integration test (e.g., training for 1 epoch on a tiny dataset) would catch regressions before they reach users.

• [ ] Create tests/test_training_pipeline.py with an integration test that runs a minimal training loop using scripts/base_train.py patterns
• [ ] Test the full pipeline: data loading (nanochat/dataloader.py) → training step → checkpoint saving (nanochat/checkpoint_manager.py) → checkpoint loading
• [ ] Add a test for the SFT pipeline (scripts/chat_sft.py) separately since it's a different code path
• [ ] Use a tiny synthetic dataset to keep test runtime under 1 minute on CPU

• Add pytest test suite for nanochat/common.py's auto-scaling functions (verify --depth 24 produces correct widths/lr/batch-size combinations) to prevent regression.
• Document the 'autoresearch' mechanism in rows 5-6 of the leaderboard: add docstrings to nanochat/engine.py explaining how hyperparameters are searched and link to a design doc or GitHub discussion.
• Extend tasks/ with evaluation harness for HellaSwag or TruthfulQA benchmarks (currently only ARC, GSM8K, MMLU, HumanEval); add to scripts/chat_eval.py to broaden leaderboard metrics beyond DCLM CORE.

The nanochat codebase is a research/educational LLM training project with moderate security considerations. Main concerns include potential web UI vulnerabilities (XSS risks), model checkpoint integrity validation, and proper handling of external inputs in training/evaluation scripts. The absence of visible hardcoded secrets is positive. Recommendations focus on input validation, dependency management, and securing model/checkpoint files. For production deployment, comprehensive security audits of web components and data handling are essential.

• Medium · Potential Web UI Security Risks — nanochat/ui.html, scripts/chat_web.py. The presence of nanochat/ui.html and scripts/chat_web.py suggests a web interface for the chat application. Without access to the actual code, there's a risk of XSS vulnerabilities if user input from chat interactions is not properly sanitized before rendering in the HTML UI. Fix: Ensure all user input is properly escaped and sanitized before rendering in the web UI. Use Content Security Policy (CSP) headers. Implement input validation on both client and server side.
• Medium · Missing Dependency Lock File Analysis — pyproject.toml, uv.lock. While uv.lock is present, the pyproject.toml content was not provided. Dependencies for ML frameworks (PyTorch, transformers) and web frameworks could have unpatched vulnerabilities. No version pinning details available for analysis. Fix: Pin all direct and transitive dependencies to specific versions. Regularly audit dependencies using tools like safety or pip-audit. Keep dependencies updated with security patches.
• Medium · Arbitrary Code Execution Risk in Training Scripts — scripts/base_train.py, scripts/chat_rl.py, nanochat/engine.py. Scripts like scripts/base_train.py, scripts/chat_rl.py allow model training and execution. If these scripts accept external configuration or model weights without validation, there's potential for arbitrary code execution through malicious model files or configuration injection. Fix: Validate all external inputs including config files and model checkpoints. Use cryptographic signatures for model weights. Implement strict configuration schema validation. Avoid using eval() or pickle on untrusted data.
• Medium · Checkpoint and Model File Handling — nanochat/checkpoint_manager.py, runs/. The nanochat/checkpoint_manager.py handles model checkpoints. If checkpoints are loaded from untrusted sources or world-readable locations, this could lead to model poisoning or unauthorized access to training data. Fix: Implement file permission checks (restrict to owner only). Validate checkpoint integrity using checksums/signatures. Store checkpoints in secure, access-controlled directories. Document checkpoint security practices.
• Low · Development Files in Repository — dev/ directory. Multiple development notebooks and scripts (dev/estimate_gpt3_core.ipynb, dev/gen_synthetic_data.py) are included in the repository. Jupyter notebooks can contain sensitive information or debug outputs. Fix: Exclude development notebooks from production deployments. Use .gitignore to prevent committing sensitive data in notebooks. Implement pre-commit hooks to scan notebooks for credentials.
• Low · Missing Security Headers in Web UI — scripts/chat_web.py, nanochat/ui.html. If scripts/chat_web.py implements a Flask or FastAPI server, security headers may not be configured. Headers like X-Frame-Options, X-Content-Type-Options, and Strict-Transport-Security should be set. Fix: Add security headers middleware. Implement CORS restrictions. Set X-Frame-Options to 'DENY' or 'SAMEORIGIN'. Enable secure cookie flags (HttpOnly, Secure, SameSite).
• Low · Potential Path Traversal in File Operations — nanochat/checkpoint_manager.py, nanochat/dataloader.py, nanochat/dataset.py. File operations in checkpoint_manager.py, dataloader.py, and dataset.py could be vulnerable to path traversal if user input is used to construct file paths without proper validation. Fix: Use os.path.join() and validate paths are within expected directories. Use pathlib.Path.resolve() and verify the resolved path is within the base directory. Never directly concatenate user input into file paths.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/karpathy/nanochat shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live karpathy/nanochat repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/karpathy/nanochat.

What it runs against: a local clone of karpathy/nanochat — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in karpathy/nanochat | Confirms the artifact applies here, not a fork | | 2 | License is still MIT | Catches relicense before you depend on it | | 3 | Default branch master exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 34 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of karpathy/nanochat. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/karpathy/nanochat.git
#   cd nanochat
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of karpathy/nanochat and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "karpathy/nanochat(\\.git)?\\b" \\
  && ok "origin remote is karpathy/nanochat" \\
  || miss "origin remote is not karpathy/nanochat (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(MIT)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"MIT\"" package.json 2>/dev/null) \\
  && ok "license is MIT" \\
  || miss "license drift — was MIT at generation time"

# 3. Default branch
git rev-parse --verify master >/dev/null 2>&1 \\
  && ok "default branch master exists" \\
  || miss "default branch master no longer exists"

# 4. Critical files exist
test -f "nanochat/gpt.py" \\
  && ok "nanochat/gpt.py" \\
  || miss "missing critical file: nanochat/gpt.py"
test -f "nanochat/engine.py" \\
  && ok "nanochat/engine.py" \\
  || miss "missing critical file: nanochat/engine.py"
test -f "nanochat/dataloader.py" \\
  && ok "nanochat/dataloader.py" \\
  || miss "missing critical file: nanochat/dataloader.py"
test -f "scripts/base_train.py" \\
  && ok "scripts/base_train.py" \\
  || miss "missing critical file: scripts/base_train.py"
test -f "nanochat/tokenizer.py" \\
  && ok "nanochat/tokenizer.py" \\
  || miss "missing critical file: nanochat/tokenizer.py"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 34 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~4d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/karpathy/nanochat"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.