GO — Healthy across the board

• Last commit today
• 49+ active contributors
• Distributed ownership (top contributor 15% of recent commits)
• Apache-2.0 licensed
• CI configured
• Tests present

_{Computed from maintenance signals — commit recency, contributor breadth, bus factor, license, CI, tests, cross-checked against OpenSSF Scorecard}

Transformers is the unified model-definition framework from Hugging Face that provides PyTorch and TensorFlow implementations of state-of-the-art pre-trained models (BERT, GPT-2, T5, ViT, DALL-E, Whisper, etc.) for NLP, computer vision, audio, and multimodal tasks. It abstracts the complexity of loading, fine-tuning, and deploying transformer architectures with a consistent API across modalities and backends. Monorepo structure: src/transformers/ contains modular subdirectories (models/{bert,gpt2,t5,etc.}/modeling_.py for architecture code, models//configuration_.py for hyperparams, models//tokenization_*.py for tokenizers). Supporting utilities in src/transformers/utils/, training loop in src/transformers/trainer.py. Heavy use of factory patterns via AutoModel, AutoTokenizer (src/transformers/models/auto/) to dynamically load correct classes. Tests in tests/ organized by model family.

ML researchers and production engineers building systems with transformer models who need to quickly load checkpoints from Hugging Face Hub, fine-tune on custom data, and deploy inference without writing architecture code from scratch. Contributors include model researchers adding new architectures and practitioners adding quantization/optimization features.

Highly mature and production-ready. The library has extensive CI/CD coverage (.github/workflows/ contains 40+ test workflows), uses CircleCI for distributed testing (.circleci/config.yml), and is actively maintained with constant model additions. The 70MB of Python code and large test surface indicate years of hardening, though the sheer scope (supporting 400+ model architectures) means edge cases still emerge.

Standard open source risks apply.

Active development of multimodal models (indicated by audio/vision subdirectories), quantization support (GPTQ, AWQ integrations), and addition of latest model families (Llama, Mistral, etc.). The .ai/skills/ directory suggests automated code assistance for type-checking and model additions. Benchmarking infrastructure is being actively maintained (benchmark_v2*.yml workflows for A10/MI325 hardware).

git clone https://github.com/huggingface/transformers.git
cd transformers
pip install -e .
pip install torch  # or tensorflow
# Test: python -c "from transformers import AutoModel; model = AutoModel.from_pretrained('bert-base-uncased')"

Daily commands: This is a library, not a runnable service. For development: make style (format code with black/isort via Makefile), python -m pytest tests/models/bert/test_modeling_bert.py (run tests for specific model). To use: from transformers import AutoModel, AutoTokenizer; model = AutoModel.from_pretrained('model-id'); tokenizer = AutoTokenizer.from_pretrained('model-id').

• README.md — Primary entry point documenting the transformers library, its purpose, and core capabilities across text, vision, audio, and multimodal models.
• .github/workflows/pr-ci-caller.yml — Main CI/CD orchestrator for pull requests; defines testing and validation gates all contributions must pass.
• .github/PULL_REQUEST_TEMPLATE.md — Required PR template defining submission guidelines, checklist items, and expectations for all contributors.
• CONTRIBUTING.md — Comprehensive contribution guidelines covering code style, testing, documentation, and commit conventions.
• benchmark/benchmark.py — Core benchmarking infrastructure for performance testing across models and hardware configurations.
• .github/copilot-instructions.md — AI-assisted development guidelines documenting codebase conventions and automated suggestion rules.
• .circleci/config.yml — CircleCI configuration defining secondary CI pipeline, test parallelization, and integration checks.

• Model Definitions (PyTorch torch.nn, TensorFlow keras.layers, JAX numpy) — Provides PyTorch, TensorFlow, and JAX implementations of transformer architectures with identical mathematical behavior across frameworks.
  • Failure mode: Forward pass dimension mismatches cause tensor shape errors; inconsistent framework implementations produce non-deterministic output differences.
• Tokenizer & Feature Extraction (SentencePiece, HuggingFace tokenizers library) — Converts raw text/audio/images to model-compatible tensor representations, handling vocabularies, subword tokenization, and preprocessing normalization.
  • Failure mode: Tokenization mismatches between training and inference produce silent accuracy degradation; vocabulary size mismatch causes index out-of-bounds errors.
• Configuration Management (Python dataclasses, JSON serialization) — Serializes/deserializes model hyperparameters (vocab size, hidden dims, layer count) in standardized JSON format enabling cross-framework compatibility.
  • Failure mode: Config schema evolution breaks backward compatibility with old model checkpoints requiring migration logic.
• CI/CD Pipeline (GitHub Actions YAML, CircleCI config, Python test runners) — Orchestrates testing across 100+ model variants, frameworks, hardware types, and Python versions; gates PRs and manages releases.
  • Failure mode: Flaky hardware tests cause intermittent CI failures; incorrect runner resource allocation exceeds budget limits; dependency conflicts block entire CI runs.
• Benchmarking Infrastructure (Python psutil, gpustat, PostgreSQL) — Measures model inference latency, throughput, memory usage, and training speed; tracks regressions over time via database.
  • Failure mode: Noisy hardware measurements cause false regression alarms; missing baseline comparisons complicate performance attribution to code changes.
• Documentation System (Sphinx, markdown) — Generates API reference, tutorials, model cards, and installation guides from docstrings and markdown files.
  • Failure mode: Outdated docstrings produce misleading API documentation; broken code examples in docs reduce discoverability and usability.

• Hugging Face Hub → Model Definitions — Developers download pretrained weights and configs; transformers library deserializes into PyTorch/TensorFlow modules.
• Raw user input (text/audio/image) → Tokenizer & Feature Extraction — User data preprocessed into normalized token tensors matching training distribution.
• Model Definitions + Tokenized input → Inference (forward pass) — Preprocessed tensors flow through transformer layers producing logits, embeddings, or generation predictions.
• Test results → Benchmarking database — CI/CD pipeline logs performance metrics (latency, throughput, memory) to PostgreSQL for regression tracking.
• Source code + docstrings → Documentation build — Sphinx extracts API signatures and examples; outputs HTML/PDF documentation deployed to website.

Add a new model architecture

1. Create model definition following transformers conventions in the main source directory. (src/transformers/models/{new_model}/modeling_new_model.py (not in provided list, inferred))
2. Add configuration class and tokenizer references. (src/transformers/models/{new_model}/configuration_new_model.py (inferred))
3. Update model registry in init.py to export new model. (src/transformers/__init__.py (inferred))
4. Add integration tests validating forward pass and loading/saving. (tests/models/{new_model}/test_modeling_new_model.py (inferred))
5. Document model in README.md and update model matrix. (README.md)

Add benchmarking for a new hardware target

1. Create hardware-specific benchmark configuration. (benchmark/config/generation.yaml)
2. Define new CI workflow for hardware-specific testing. (.github/workflows/benchmark_v2.yml)
3. Update benchmark database schema if tracking new metrics. (benchmark/utils/init_db.sql)
4. Add hardware runner configuration in CircleCI. (.circleci/config.yml)

Add a new CI validation check

1. Create workflow YAML file in .github/workflows/ directory. (.github/workflows/check_tiny_models.yml)
2. Define trigger conditions (on: pull_request, push, etc.). (.github/workflows/check_tiny_models.yml)
3. Reference workflow in main PR CI caller for orchestration. (.github/workflows/pr-ci-caller.yml)
4. Document check requirements and failure conditions in CONTRIBUTING.md. (CONTRIBUTING.md)

• GitHub Actions + CircleCI — Multi-cloud CI/CD enables testing across diverse hardware (GPUs, TPUs, CPUs) and operating systems without vendor lock-in.
• Python + PyTorch/TensorFlow — Standard ML framework ecosystem with broad community adoption, mature debugging tools, and extensive pre-trained model availability.
• Hugging Face Hub integration — Centralized model versioning, collaborative development, and democratized access to state-of-the-art models without manual hosting.
• Docker containerization — Reproducible build environments isolating dependency conflicts and enabling consistent testing across developer machines and CI servers.

• Monorepo with 600+ files containing models, training, inference, utilities, and CI/CD

  • Why: Single source of truth for model implementations, tokenizers, and configs enables consistent versioning and coordinated releases.
  • Consequence: Slower repository clones, higher maintenance burden for dependency management, and complex CI scheduling to avoid redundant test runs.

• Multi-framework support (PyTorch, TensorFlow, JAX) for same model architectures

  • Why: Maximizes accessibility across user preference and deployment constraints (edge vs. cloud, training vs. inference).
  • Consequence: 3x code duplication for model definitions, manual synchronization burden when fixing bugs, and increased testing matrix complexity.

• Automated benchmarking with historical database tracking

  • Why: Early detection of performance regressions prevents silent degradation and guides optimization priorities.
  • Consequence: Requires persistent database infrastructure, complex metric aggregation logic, and careful interpretation to avoid false positives from noisy hardware.

• Does not provide proprietary commercial model hosting or SaaS inference API.
• Does not handle distributed training orchestration or cluster management; users must integrate external frameworks (Ray, Kubernetes).
• Not a real-time serving system; designed for batch inference and offline model development.
• Does not manage training infrastructure; assumes user has compute resources available locally or on cloud providers.
• Does not provide model fine-tuning as a managed service; users implement custom training loops.

• Avg cyclomatic complexity: ~7 — Monorepo with 600+ files spanning multiple frameworks, hardware targets, and model architectures; CI/CD orchestration adds significant workflow complexity; multi-framework parity requirements increase code duplication.
• Largest file: .github/workflows/pr-ci-caller.yml or .circleci/config.yml (inferred) (2,000 lines)
• Estimated quality issues: ~15 — Framework duplication causes consistency issues; manual synchronization of configs across model variants; complex CI matrix creates

• Duplicated model implementations across frameworks (High) — src/transformers/models/ (inferred directory structure): PyTorch, TensorFlow, and JAX versions of same architecture require synchronized bug fixes and feature updates, leading to implementation drift and inconsistent behavior.
• Circular dependency between model configs and tokenizers (Medium) — Configuration and tokenizer loading in models/ (inferred): Models depend on tokenizers for vocab size, but tokenizers may reference model configs, creating coupling that complicates refactoring.
• Monorepo CI with full test matrix execution on all changes (Medium) — .github/workflows/pr-ci-caller.yml and .circleci/config.yml: Every PR runs tests across 100+ model variants and frameworks regardless of change scope, causing unnecessary compute waste and slow feedback.
• Manual synchronization of breaking changes across documentation and code (Low) — MIGRATION_GUIDE_V5.md and source code: Migration guides require manual updates when APIs change; outdated guides mislead users and increase support burden.

• CI/CD test execution across hardware targets (Infrastructure latency) — Sequential GPU/TPU test scheduling causes 2-4 hour PR feedback loops; hardware resource contention during peak usage increases queue wait times.
• Model weight deserialization on first load (Memory/I/O bottleneck) — Large models (>10GB) require full download from Hub and disk I/O before first inference; no incremental loading or streaming support for memory-constrained devices.
• Benchmarking result aggregation and database queries (Query performance) — Generating regression reports across 100+ model variants requires complex SQL joins; no caching or materialized views for common queries.

(1) Model configs are cached in ~/.cache/huggingface/hub/; offline work requires HF_DATASETS_OFFLINE=1 env var and pre-cached models. (2) PyTorch and TensorFlow can coexist but their model implementations diverge—always specify which backend when testing. (3) The trainer.py assumes gradient accumulation steps, learning rate warmup, and distributed setup complexity; single-GPU development may not catch multi-GPU bugs. (4) Tokenizer outputs vary significantly by model family (token_type_ids, attention_mask shapes)—tests must be model-specific. (5) Many tests are skipped by default (@require_torch, @require_vision) to save CI time; run locally with SLOW=1 pytest tests/ to catch real issues. (6) The monorepo has no hard boundaries—changing utils/ can break 100+ models; requires running broad test suite (use CircleCI for this).

• Attention mechanism (Scaled Dot-Product Attention) — Core mathematical operation in every transformer model in this repo; understanding head dimension, masking, and attention weights is essential for debugging model outputs and implementing variants like multi-query attention
• Tokenization and Special Tokens — Transformers hide tokenization complexity, but mismatch between train/inference tokenizers (BPE vs WordPiece) breaks models; knowing how [CLS], [SEP], [PAD] tokens work is critical for custom data
• Gradient Checkpointing (Activation Checkpointing) — Transformers use this to reduce memory consumption by recomputing activations during backward pass; Trainer enables it with gradient_checkpointing=True, but trades compute for memory—users must understand the tradeoff
• Distributed Data Parallel (DDP) and Distributed Sampler — Trainer abstracts away DDP setup, but bugs in multi-GPU training (e.g., sampling imbalance, synchronization issues) are frequent; knowing torch.nn.parallel.DistributedDataParallel helps debug hang/mismatch issues
• Mixed Precision Training (AMP) — Transformers use torch.cuda.amp to reduce memory and speed up training with FP16; silent overflow/underflow bugs are subtle; amp_dtype and amp_opt_level in TrainingArgs control this behavior
• Positional Encoding (Absolute and Relative) — Different models use different positional encoding schemes (learnable embeddings in BERT, rotational in Llama, relative biases in T5); affects max sequence length and extrapolation behavior
• SafeTensors Format — Transformers default to .safetensors over .bin for checkpoint storage (safer, faster deserialization, language-agnostic); understanding this format matters for checkpoint compatibility and custom loading pipelines

• huggingface/datasets — Companion library for loading, preprocessing, and streaming training data; most Transformers users pair this with Trainer for data pipelines
• huggingface/peft — Parameter-efficient fine-tuning (LoRA, QLoRA) built on top of Transformers; users working with large models almost always layer this on
• huggingface/accelerate — Abstraction for distributed training used internally by Trainer; handles device placement and gradient synchronization across backends
• pytorch/pytorch — Primary backend for Transformers; versions 1.9+ required; major updates often break model implementations
• openai/gpt-2 — Spiritual predecessor; Transformers GPT-2 implementation was reverse-engineered from this reference, establishing the library's initial design

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add comprehensive type checking for model configuration classes

The repo has a .ai/skills/add-or-fix-type-checking/ directory indicating type checking is a priority. Most model config files likely lack complete type hints. This would improve IDE support, catch bugs early, and align with the skill directory's purpose. High-value because configs are used across all 1000+ models.

• [ ] Identify model config files in src/transformers/models/*/config.py lacking complete type hints
• [ ] Add return type annotations and parameter type hints to config classes
• [ ] Run mypy against updated configs to verify completeness
• [ ] Add type checking tests in tests/utils/test_type_checking.py (if it exists, or create it)
• [ ] Reference the SKILL.md in .ai/skills/add-or-fix-type-checking/ for standards

Add missing GitHub Actions workflow for dependency vulnerability scanning

The repo has extensive CI workflows (.github/workflows/) but appears to lack a dedicated security scanning workflow for pinned dependencies (psycopg2, pandas, gpustat). This would catch supply chain risks and CVEs proactively, critical for a widely-used framework.

• [ ] Create .github/workflows/security-dependency-check.yml using GitHub's dependency scanning action or Snyk
• [ ] Configure it to run on: [push to main, pull_request, schedule: weekly]
• [ ] Set up artifact uploads for security reports similar to existing benchmark workflows
• [ ] Update .github/workflows/TROUBLESHOOT.md with debugging info for the new workflow
• [ ] Test against known vulnerable versions in dev dependencies

Create CircleCI to GitHub Actions migration guide and add missing workflow documentation

The repo has both .circleci/config.yml and extensive .github/workflows/, with a .circleci/config.yml.bak backup file suggesting an ongoing migration. .circleci/TROUBLESHOOT.md and .github/workflows/TROUBLESHOOT.md exist but lack cross-references. This documentation gap confuses contributors about which system is authoritative.

• [ ] Document the migration status: which tests remain in CircleCI vs moved to GitHub Actions
• [ ] Create .github/MIGRATION_GUIDE.md explaining how to debug failures in each system
• [ ] Add a matrix table in .github/workflows/TROUBLESHOOT.md showing which workflow runs which test suites
• [ ] Update .circleci/TROUBLESHOOT.md to note it's deprecated/archived (if applicable)
• [ ] Reference this guide in .github/PULL_REQUEST_TEMPLATE.md for CI troubleshooting

• Add type hints to src/transformers/tokenization_utils.py—currently unannotated despite being heavily used. See .ai/skills/add-or-fix-type-checking/ for guidance. This improves IDE support and catches bugs early.
• Write integration tests for AutoModel.from_pretrained() with models that have custom architectures (e.g., MoE or multi-modal variants). Currently tests/models/auto/test_modeling_auto.py only covers standard cases; gaps here hide factory-layer bugs.
• Document the Trainer's grad_accumulation_steps interaction with distributed training in src/transformers/training_args.py—currently has no docstring examples, and users frequently misconfigure. Add a code example showing the math of effective batch size.

The Transformers library has a moderate security posture with several concerning issues. The primary risks involve outdated dependencies (psycopg2, psutil) that lack current security patches, and the inherent RCE risk from loading untrusted model files from the Hub. The incomplete SECURITY.md documentation leaves users without clear guidance on mitigating these risks. The extensive CI/CD infrastructure requires careful access controls and workflow security practices. Immediate actions should focus on: upgrading dependencies, completing security documentation, and implementing stricter defaults for safe model loading formats.

• High · Outdated psycopg2 Dependency with Known Vulnerabilities — dependencies/Package file - psycopg2==2.9.9. psycopg2==2.9.9 is outdated and may contain known security vulnerabilities. The current version (2.9.x) was released in 2023 and does not receive the latest security patches. psycopg2 3.x versions have addressed multiple CVEs including connection string handling and SQL injection vectors. Fix: Upgrade to psycopg2>=3.1.0 or the latest stable version. Review changelog for breaking changes and test thoroughly before deployment.
• High · Pinned Vulnerable psutil Version — dependencies/Package file - psutil==6.0.0. psutil==6.0.0 is a specific pinned version from late 2023. While not critically vulnerable, pinning exact versions without a mechanism for security updates creates maintenance debt. Future vulnerabilities in this version cannot be automatically patched. Fix: Use version pinning with flexibility (e.g., psutil>=6.0.0,<7.0.0) or implement automated dependency update processes for security patches.
• Medium · Remote Code Execution via Model Loading — SECURITY.md - Remote artefacts section. The SECURITY.md acknowledges that the Transformers library can load arbitrary code from the Hugging Face Hub. While safetensors format is recommended, the library still supports loading models in pickle format and other unsafe formats that can execute arbitrary code. Users downloading community-contributed models face RCE risks. Fix: Implement stricter default configurations to block unsafe formats (pickle, PKL) by default. Require explicit opt-in with warnings for loading untrusted code. Validate model signatures or add code provenance tracking.
• Medium · Incomplete Security Policy Documentation — SECURITY.md - Remote artefacts section. The SECURITY.md file appears incomplete (cuts off mid-sentence at 'e.g. [pickle]'). This incomplete documentation may leave users unaware of critical security best practices and attack vectors when using the library. Fix: Complete the SECURITY.md documentation with comprehensive guidance on: safe model loading practices, vulnerability reporting procedures, and security considerations for production use.
• Medium · Dependency Without Version Upper Bounds — dependencies/Package file - pandas>=1.5.0. pandas>=1.5.0 uses a lower-bound-only constraint without an upper bound. This allows installation of incompatible major versions in the future, risking breaking changes or security issues in pandas dependencies. Fix: Use bounded versioning: pandas>=1.5.0,<3.0.0 to ensure compatibility while allowing patch and minor version updates within known-good ranges.
• Low · Multiple CI/CD Workflow Files Without Apparent Signing — .github/workflows/ directory. Numerous GitHub Actions workflow files (.github/workflows/*.yml) are present. Without visible code signing or review procedures, malicious workflow modifications could compromise the build and release pipeline. Fix: Implement workflow file review requirements, code signing for critical workflows, and branch protection rules. Use GitHub's workflow security features and audit all workflow permissions.
• Low · Potential Sensitive Information in CI Configuration — .circleci/config.yml.bak. Files like .circleci/config.yml.bak suggest backup configurations that may contain secrets or sensitive CI/CD settings not properly cleaned. Fix: Scan backup files for secrets using tools like TruffleHog (already configured in trufflehog.yml). Remove or secure backup configuration files. Use CI/CD secret management properly.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Read in the suggested order before editing unfamiliar code. The reading-order list is computed from the actual import graph, not LLM guesses; reading bottom-up materially reduces wrong-edit risk.
3. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
4. Cite source on changes. When proposing an edit, cite the specific path/to/file.ext:Lstart-Lend you're reasoning about, the same way RepoPilot's own RAG cites code in https://repopilot.app/r/huggingface/transformers.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live huggingface/transformers repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/huggingface/transformers.

What it runs against: a local clone of huggingface/transformers — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in huggingface/transformers | Confirms the artifact applies here, not a fork | | 2 | License is still Apache-2.0 | Catches relicense before you depend on it | | 3 | Default branch main exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 30 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of huggingface/transformers. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/huggingface/transformers.git
#   cd transformers
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of huggingface/transformers and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "huggingface/transformers(\\.git)?\\b" \\
  && ok "origin remote is huggingface/transformers" \\
  || miss "origin remote is not huggingface/transformers (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(Apache-2\\.0)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"Apache-2\\.0\"" package.json 2>/dev/null) \\
  && ok "license is Apache-2.0" \\
  || miss "license drift — was Apache-2.0 at generation time"

# 3. Default branch
git rev-parse --verify main >/dev/null 2>&1 \\
  && ok "default branch main exists" \\
  || miss "default branch main no longer exists"

# 4. Critical files exist
test -f "README.md" \\
  && ok "README.md" \\
  || miss "missing critical file: README.md"
test -f ".github/workflows/pr-ci-caller.yml" \\
  && ok ".github/workflows/pr-ci-caller.yml" \\
  || miss "missing critical file: .github/workflows/pr-ci-caller.yml"
test -f ".github/PULL_REQUEST_TEMPLATE.md" \\
  && ok ".github/PULL_REQUEST_TEMPLATE.md" \\
  || miss "missing critical file: .github/PULL_REQUEST_TEMPLATE.md"
test -f "CONTRIBUTING.md" \\
  && ok "CONTRIBUTING.md" \\
  || miss "missing critical file: CONTRIBUTING.md"
test -f "benchmark/benchmark.py" \\
  && ok "benchmark/benchmark.py" \\
  || miss "missing critical file: benchmark/benchmark.py"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 30 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~0d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/huggingface/transformers"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

Computed from the actual import graph (no LLM). Read in this order to learn the codebase from the foundation up — each step builds on the previous ones.

1. benchmark_v2/framework/hardware_metrics.py — Foundation: doesn't import anything internally and is imported by 2 other files. Read first to learn the vocabulary.
2. benchmark_v2/framework/benchmark_config.py — Foundation: imported by 1, no internal dependencies of its own.
3. benchmark_v2/framework/data_classes.py — Built on the foundation; imported by 1 downstream file.
4. benchmark_v2/framework/benchmark_runner.py — Layer 2 — application-level code that wires the lower layers together.

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.