If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/sharkdp/hyperfine shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

GO — Healthy across the board

• Last commit 1w ago
• 27+ active contributors
• Distributed ownership (top contributor 39% of recent commits)
• Apache-2.0 licensed
• CI configured
• Tests present

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live sharkdp/hyperfine repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/sharkdp/hyperfine.

What it runs against: a local clone of sharkdp/hyperfine — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in sharkdp/hyperfine | Confirms the artifact applies here, not a fork | | 2 | License is still Apache-2.0 | Catches relicense before you depend on it | | 3 | Default branch master exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 37 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of sharkdp/hyperfine. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/sharkdp/hyperfine.git
#   cd hyperfine
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of sharkdp/hyperfine and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "sharkdp/hyperfine(\\.git)?\\b" \\
  && ok "origin remote is sharkdp/hyperfine" \\
  || miss "origin remote is not sharkdp/hyperfine (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(Apache-2\\.0)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"Apache-2\\.0\"" package.json 2>/dev/null) \\
  && ok "license is Apache-2.0" \\
  || miss "license drift — was Apache-2.0 at generation time"

# 3. Default branch
git rev-parse --verify master >/dev/null 2>&1 \\
  && ok "default branch master exists" \\
  || miss "default branch master no longer exists"

# 4. Critical files exist
test -f "src/main.rs" \\
  && ok "src/main.rs" \\
  || miss "missing critical file: src/main.rs"
test -f "src/benchmark/executor.rs" \\
  && ok "src/benchmark/executor.rs" \\
  || miss "missing critical file: src/benchmark/executor.rs"
test -f "src/benchmark/scheduler.rs" \\
  && ok "src/benchmark/scheduler.rs" \\
  || miss "missing critical file: src/benchmark/scheduler.rs"
test -f "src/command.rs" \\
  && ok "src/command.rs" \\
  || miss "missing critical file: src/command.rs"
test -f "src/cli.rs" \\
  && ok "src/cli.rs" \\
  || miss "missing critical file: src/cli.rs"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 37 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~7d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/sharkdp/hyperfine"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

Hyperfine is a command-line benchmarking tool written in Rust that measures and compares execution times of arbitrary shell commands with statistical rigor. It automatically determines optimal run counts, detects outliers, performs warmup/cache-clearing, supports parameterized sweeps (varying arguments), and exports results to CSV, JSON, Markdown, and AsciiDoc formats. Modular structure: src/benchmark/ contains executor, scheduler, result aggregation, and relative speed calculation; src/export/ holds format-specific serializers (json.rs, csv.rs, markdown.rs, etc.); src/output/ handles progress bars and warnings; src/parameter/ manages range/step parameters; src/cli.rs and src/options.rs parse arguments via clap; src/command.rs wraps shell execution.

Systems engineers, performance-minded developers, and tool authors (like those building fd or other CLI utilities) who need to rigorously benchmark command execution times and compare alternative implementations without writing custom timing harnesses.

Production-ready and actively maintained. Version 1.20.0 with CI/CD pipeline in .github/workflows/CICD.yml, comprehensive test coverage via assert_cmd and insta snapshot testing, and a clean Rust 1.88+ MSRV. No signs of abandonment; the codebase is well-structured with semantic versioning.

Low risk: minimal dependency footprint (colored, indicatif, statistical, csv, serde, rand, shell-words, thiserror, anyhow), active Dependabot monitoring, and single-maintainer (sharkdp) is mitigated by the tool's singular, well-defined scope. No breaking-change history visible in CHANGELOG.md snippet.

Actively maintained at v1.20.0 with recent focus on statistical accuracy (statistical crate integration), cross-platform Windows support (windows-sys feature flags), and parametrized benchmarks. Snapshot tests in src/export/tests.rs and build.rs integration suggest recent refinement of output formatting.

git clone https://github.com/sharkdp/hyperfine.git
cd hyperfine
cargo build --release
./target/release/hyperfine 'sleep 0.1'

Daily commands:

cargo run --release -- 'command1' 'command2'
# Or for benchmarking itself:
cargo build --release
./target/release/hyperfine --runs 5 --warmup 2 'echo test'

• src/main.rs — Entry point that orchestrates CLI parsing, benchmark execution, and output—all contributors must understand the control flow here
• src/benchmark/executor.rs — Core execution engine that runs individual benchmark commands and captures timing—critical for understanding how benchmarks are actually performed
• src/benchmark/scheduler.rs — Manages the scheduling of warmup runs, cache-clearing, and timing runs across multiple executions—essential for benchmark integrity
• src/command.rs — Command abstraction and shell parsing logic that translates user input into executable operations—fundamental for command handling
• src/cli.rs — CLI argument parsing and options structure—contributors adding features must modify this to expose new functionality
• src/benchmark/benchmark_result.rs — Data structure aggregating timing results and statistical analysis—core for understanding result representation and export
• Cargo.toml — Project manifest and dependency management—required for understanding build requirements and version constraints

Add a New Export Format

1. Create a new module in src/export/ (e.g., src/export/yaml.rs) implementing the Exporter trait from src/export/mod.rs (src/export/yaml.rs)
2. Implement the Exporter trait with the export() method to serialize benchmark results in your format (src/export/yaml.rs)
3. Register the new exporter in the format_exporter() dispatcher function in src/export/mod.rs by adding a new match arm (src/export/mod.rs)
4. Add a new variant to the ExportStyle enum in src/cli.rs to accept the new format from the command line (src/cli.rs)
5. Add corresponding test case to src/export/tests.rs validating the new format's output (src/export/tests.rs)

Add a New Command-Line Option

1. Add a new field to the Options struct in src/options.rs with appropriate documentation and defaults (src/options.rs)
2. Add a new command-line argument to the clap configuration in src/cli.rs with help text and argument name (src/cli.rs)
3. Update the build_options() function in src/main.rs to extract the new CLI argument and populate the Options field (src/main.rs)
4. Thread the option through the relevant execution path (scheduler, executor, or output modules depending on feature purpose) (src/benchmark/scheduler.rs)
5. Add an integration test in tests/integration_tests.rs validating the new option's behavior (tests/integration_tests.rs)

Add a New Statistical Metric or Analysis

1. Create a new function in src/benchmark/benchmark_result.rs or add a new field to compute your metric from timing results (src/benchmark/benchmark_result.rs)
2. Use the statistical crate (already in Cargo.toml) to perform statistical calculations on the TimingResult samples (src/benchmark/benchmark_result.rs)
3. Add the metric field to BenchmarkResult struct and populate it during result aggregation in benchmark_result.rs (src/benchmark/benchmark_result.rs)
4. Display the metric in the human-readable output by updating src/output/format.rs (src/output/format.rs)
5. Export the metric in all relevant export formats by updating the Exporter implementations (json.rs, csv.rs, markdown.rs, etc.) (src/export/json.rs)

Add Platform-Specific Timing Support

1. Create a new timer module in src/timer/ (e.g., src/timer/macos_timer.rs) implementing the Timer trait from src/timer/mod.rs (src/timer/macos_timer.rs)
2. Implement the start() and stop() methods using platform-specific APIs to capture precise execution time (src/timer/macos_timer.rs)
3. Update the select_timer() function in src/timer/mod.rs to detect your platform and instantiate your timer (src/timer/mod.rs)
4. Add cfg-guarded compilation directives (e.g., #[cfg(target_os = "macos")]) to conditionally include your platform code (src/timer/mod.rs)
5. Test on the target platform to ensure timing accuracy is within acceptable bounds (tests/integration_tests.rs)

• Rust + Cargo — Provides memory safety, zero-cost

Windows process handle access: conditional compilation feature windows_process_extensions_main_thread_handle is required for certain thread-aware measurements on Windows; falls back gracefully. Decimal parameter parsing: parametrized benchmarks accept floats but use rust_decimal crate; integer vs. float behavior must match interpolation logic in src/parameter/range_step.rs. Shell parsing: shell-words 1.0 is used to parse quoted command arguments; quoting rules differ from bare shell, so commands with complex escaping may behave unexpectedly. Statistical minimum runs: default is ≥10 runs + ≥3 seconds; very fast commands (µs-scale) will hit the run limit, skewing accuracy. Linux nix dependency: only on Linux; missing libc features on other platforms won't affect compilation but Linux-specific cache clearing won't work elsewhere.

• Tukey Fence Outlier Detection — Hyperfine uses this statistical method in src/outlier_detection.rs to automatically filter anomalous runs caused by system load or cache effects; understanding it prevents misinterpreting spiky results.
• Warmup Runs & Cache Priming — The --warmup and --prepare flags leverage OS disk/CPU caches strategically; key to reproducible benchmarking of I/O-bound code in src/benchmark/scheduler.rs.
• Shell Command Tokenization (shell-words) — Hyperfine uses shell-words 1.0 to parse user-provided command strings without invoking a full shell; prevents injection but requires understanding its quoting rules vs. bash.
• Process Timing via System Calls — On Unix, src/benchmark/executor.rs captures wall-clock time via libc syscalls; on Windows, windows-sys job objects; understanding the platform differences is crucial for porting.
• Statistical Significance Testing — Hyperfine computes mean, median, stddev, and can optionally run Welch's t-test (in scripts/welch_ttest.py) to determine if benchmark differences are meaningful; critical for drawing conclusions.
• Parametrized Benchmarking (Sweeps) — The --parameter-scan feature in src/parameter/range_step.rs varies a command argument across a range; essential for understanding performance scaling (e.g., threads vs. throughput).
• Trait-Based Export Polymorphism — Hyperfine's Exporter trait in src/export/mod.rs allows multiple format handlers (JSON, CSV, Markdown, etc.) to coexist without conditional logic; a Rust idiom worth studying.

• bspeice/criterion.rs — Rust benchmarking library for inline micro-benchmarks; complements hyperfine for code-level profiling vs. CLI-level timing.
• sharkdp/fd — Directory search tool by the same author; serves as a real-world use case and motivation for hyperfine's existence.
• google/benchmark — C++ micro-benchmarking framework; similar statistical rigor and outlier detection approach, different language.
• Linux-tracing/trace-cmd — Kernel-level tracing tool for detailed timing analysis; works orthogonally with hyperfine for drilling into system behavior.
• brendangregg/FlameGraph — Post-processing tool for perf output; pairs well with hyperfine when deep CPU profiling is needed alongside execution time.

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add integration tests for export formats with parametrized benchmarks

The repo has export modules (CSV, JSON, Markdown, AsciiDoc, OrgMode) but tests/integration_tests.rs and tests/execution_order_tests.rs don't have comprehensive coverage for parametrized benchmark exports. The feature is mentioned in README but lacks test validation that all export formats correctly handle the src/parameter/ module outputs.

• [ ] Review src/parameter/mod.rs and src/parameter/range_step.rs to understand parametrized benchmark structure
• [ ] Examine src/export/ modules to identify format-specific parameter handling
• [ ] Add test cases in tests/integration_tests.rs that run parametrized benchmarks with --export-json, --export-csv, and --export-markdown flags
• [ ] Validate that parameter values appear correctly in each export format's output
• [ ] Add a parametrized example input file in tests/ similar to tests/example_input_file.txt

Add statistical validation tests for outlier detection edge cases

src/outlier_detection.rs exists but there are no dedicated unit tests for edge cases (empty results, single run, all values identical, extreme outliers). This is critical for a benchmarking tool's correctness, especially given the statistical analysis features mentioned in the README.

• [ ] Create tests/outlier_detection_tests.rs or add unit tests within src/outlier_detection.rs
• [ ] Test edge cases: empty vector, single value, all identical values, bimodal distribution, extreme outliers
• [ ] Compare results against the statistical crate (already a dependency) to verify correctness
• [ ] Add property-based tests using proptest or quickcheck to validate outlier detection robustness
• [ ] Document expected behavior for edge cases in code comments

Add platform-specific timer validation tests for Windows and Unix

The repo has platform-specific timer implementations (src/timer/unix_timer.rs, src/timer/windows_timer.rs, src/timer/wall_clock_timer.rs) but no integration tests validate timer accuracy or consistency across platforms. This is critical for a benchmarking tool's reliability.

• [ ] Review src/timer/mod.rs and platform-specific implementations to understand the timer interface
• [ ] Create tests/timer_accuracy_tests.rs with platform-conditional tests using #[cfg(...)]
• [ ] Add test that measures known-duration operations (e.g., sleeping for 100ms) and validates timer measurements fall within acceptable range (±5-10%)
• [ ] Test timer consistency: run same command multiple times and verify variance is reasonable
• [ ] Use #[cfg(target_os = "...")] to ensure platform-specific timers are tested on their respective platforms in CI

• Add missing documentation for the outlier detection threshold: src/outlier_detection.rs implements Tukey fence but no README or doc comments explain the multiplier or how to customize it. A first-timer could document the algorithm, explain the parameters, and add an example.
• Extend parameter/range_step.rs with step validation tests: The parametrized benchmark feature lacks edge-case tests (negative steps, zero step, boundary crossing). Add insta snapshot tests to src/parameter/ covering these scenarios using assert_cmd integration tests.
• Create a new JSON export variant (e.g., JSON Lines for streaming): Currently only src/export/json.rs exists. A contributor could add src/export/jsonl.rs for single-object-per-line output, useful for piping to external tools, following the same Exporter trait pattern.

Hyperfine demonstrates reasonably good security practices as a command-line benchmarking tool. The primary security concern is the inherent risk of executing arbitrary shell commands, which is mitigated by proper input handling and the use of the well-maintained shell-words crate. The codebase avoids common vulnerabilities like SQL injection and XSS (not applicable to a CLI tool). Minor issues include a pinned dependency version that could prevent security patch updates and potential path traversal risks in file export functionality. The project maintains good security through proper error handling (thiserror, anyhow), dependency management, and no hardcoded credentials detected. Rust's memory safety characteristics provide strong protection against buffer overflows and memory corruption.

• Medium · Potential Command Injection Risk in Shell Command Execution — src/command.rs, src/benchmark/executor.rs, src/shell-words dependency. The codebase is designed to execute arbitrary shell commands as part of its benchmarking functionality. The file src/command.rs and src/benchmark/executor.rs likely handle command execution. Without proper input validation and sanitization, there could be command injection vulnerabilities if user-supplied input is not properly escaped or if the shell-words crate is used incorrectly. Fix: Ensure all user-supplied command arguments are properly validated and escaped before execution. Review the shell-words crate usage to confirm it's being used correctly for parsing and escaping. Consider using Rust's std::process::Command directly instead of shell invocation where possible, as it avoids shell injection.
• Low · Dependency Version Pinning for indicatif — Cargo.toml - indicatif dependency. The indicatif dependency is pinned to an exact version (=0.17.4) using equality constraint. While this ensures reproducibility, it may prevent receiving important security patches from patch releases. The constraint should allow patch updates. Fix: Change 'indicatif = "=0.17.4"' to 'indicatif = "0.17.4"' (without the equals sign) to allow patch version updates while maintaining compatibility with the specified minor version.
• Low · Direct File I/O Without Validation — src/export/csv.rs, src/export/json.rs, src/export/markdown.rs, src/export/asciidoc.rs. The benchmark results can be exported to various file formats (CSV, JSON, Markdown, AsciiDoc). The src/export modules may write to user-specified paths without adequate path traversal validation, potentially allowing writes outside intended directories. Fix: Implement path validation to ensure that output files are written only to intended directories. Use path canonicalization and checks to prevent directory traversal attacks (e.g., using std::path::Path::canonicalize() and verifying the path is within an expected directory).
• Low · Potential Timing Side-Channel in Benchmark Analysis — src/outlier_detection.rs, src/benchmark/relative_speed.rs. The statistical analysis and outlier detection functionality (src/outlier_detection.rs) could potentially be used to analyze sensitive timing information if benchmarking confidential operations. While not a direct vulnerability in hyperfine itself, users should be aware of timing side-channel risks. Fix: Document timing side-channel risks in the security guidelines for users. Recommend users avoid benchmarking operations involving sensitive cryptographic or security-critical code in untrusted environments.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.