If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/gunnarmorling/1brc shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

GO — Healthy across all four use cases

• 47+ active contributors
• Distributed ownership (top contributor 38% of recent commits)
• Apache-2.0 licensed
• CI configured
• Tests present
• ⚠ Stale — last commit 2y ago

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live gunnarmorling/1brc repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/gunnarmorling/1brc.

What it runs against: a local clone of gunnarmorling/1brc — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in gunnarmorling/1brc | Confirms the artifact applies here, not a fork | | 2 | License is still Apache-2.0 | Catches relicense before you depend on it | | 3 | Default branch main exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 656 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of gunnarmorling/1brc. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/gunnarmorling/1brc.git
#   cd 1brc
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of gunnarmorling/1brc and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "gunnarmorling/1brc(\\.git)?\\b" \\
  && ok "origin remote is gunnarmorling/1brc" \\
  || miss "origin remote is not gunnarmorling/1brc (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(Apache-2\\.0)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"Apache-2\\.0\"" package.json 2>/dev/null) \\
  && ok "license is Apache-2.0" \\
  || miss "license drift — was Apache-2.0 at generation time"

# 3. Default branch
git rev-parse --verify main >/dev/null 2>&1 \\
  && ok "default branch main exists" \\
  || miss "default branch main no longer exists"

# 4. Critical files exist
test -f "README.md" \\
  && ok "README.md" \\
  || miss "missing critical file: README.md"
test -f "pom.xml" \\
  && ok "pom.xml" \\
  || miss "missing critical file: pom.xml"
test -f "ENVIRONMENT.md" \\
  && ok "ENVIRONMENT.md" \\
  || miss "missing critical file: ENVIRONMENT.md"
test -f ".github/workflows/maven.yml" \\
  && ok ".github/workflows/maven.yml" \\
  || miss "missing critical file: .github/workflows/maven.yml"
test -f ".github/pull_request_template.md" \\
  && ok ".github/pull_request_template.md" \\
  || miss "missing critical file: .github/pull_request_template.md"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 656 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~626d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/gunnarmorling/1brc"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

The One Billion Row Challenge (1BRC) is a competitive programming challenge that benchmarks how fast Java implementations can aggregate temperature statistics from a 1-billion-row CSV file containing weather station measurements. Participants optimize their solutions using techniques like parallelization, SIMD, GC tuning, and memory-mapped I/O to compute min/mean/max temperatures per station with minimal latency. Monorepo structure: root contains shared tooling (pom.xml, Maven wrapper in .mvn/), a baseline implementation, and 100+ individual shell scripts (calculate_average_*.sh files) that wrap competing solutions in separate subdirectories. Each submission gets its own executable entry point following the naming convention calculate_average_<ContributorName>.sh. CI runs via GitHub Actions against the maven.yml workflow.

Java performance engineers and competitive programmers who want to push the limits of JVM optimization, learn advanced concurrency patterns (virtual threads, parallel streams), and compete on a standardized benchmark evaluated on identical hardware (Hetzner AX161 8-core setup).

The challenge was active and closed to submissions on Jan 31, 2024; final leaderboards were published Feb 4, 2024. The repo is now in legacy/archive status as a completed challenge artifact. It contains 100+ battle-tested reference implementations with CI pipelines (Maven, GitHub Actions workflows in .github/workflows/maven.yml) but is not maintained for feature development—it's a snapshot of competition results.

Standard open source risks apply.

The challenge is closed as of Feb 1, 2024. No new submissions are accepted. The repo is in finalization/publication phase—results have been published, leaderboards are locked, and the repo now serves as an immutable record of all 100+ submissions with their rankings and performance metrics on standardized hardware.

git clone https://github.com/gunnarmorling/1brc.git
cd 1brc
./mvnw clean verify

Then run a specific submission with: bash calculate_average_baseline.sh < measurements.txt (you must first generate or obtain the 1B-row test file per ENVIRONMENT.md).

Daily commands: Clone, then: ./mvnw clean verify to compile all submissions. Run a single submission: bash calculate_average_baseline.sh < measurements.txt. To regenerate the 1B-row test file, see ENVIRONMENT.md for the data generator command. Each .sh file is a self-contained entry point that invokes the compiled Java code with optimized JVM flags.

• README.md — Defines the One Billion Row Challenge rules, evaluation criteria, and submission guidelines that all contributors must follow
• pom.xml — Maven configuration that specifies Java version, compiler settings, and build profiles used across all submissions
• ENVIRONMENT.md — Documents the evaluation environment specifications, JVM flags, and hardware constraints critical for optimizing submissions
• .github/workflows/maven.yml — CI/CD pipeline that validates submissions are buildable and runnable within the challenge constraints
• .github/pull_request_template.md — Submission template that standardizes how contributors structure their PR descriptions and metadata
• LICENSE.txt — Apache 2.0 license establishing legal terms for all contributions to the challenge

• Shell Script Wrapper (Bash, JVM flags (-Xmx, -XX:+UseG1GC, etc.)) — Entry point that specifies JVM flags, classpath, memory settings, and invokes the Java main class
  • Failure mode: If script has syntax errors or incorrect classpath, submission fails immediately; no output produced
• Java Implementation (CalculateAverage_*) (Java I/O, HashMap/TreeMap, String parsing, optionally Unsafe, Vector API, or memory-mapped files) — Core algorithm that reads measurements.txt line by line, parses city and temperature, and aggregates statistics into min/max/sum/count per city
  • Failure mode: Parsing errors, arithmetic overflow, or incorrect aggregation logic produces wrong temperature values; timeout if algorithm is too slow
• Maven Build System (Maven pom.xml, javac, JDK 21+) — Validates syntax, compiles all submissions, runs optional unit tests, and packages bytecode for execution
  • Failure mode: Compilation errors prevent submission from reaching evaluation stage; CI pipeline exits with non-zero status
• GitHub Actions CI (GitHub Actions workflow (maven.yml), ubuntu-latest runner) — Automated gating that ensures submissions are buildable before manual review and leaderboard evaluation
  • Failure mode: If CI fails, PR is marked as broken and blocked from leaderboard consideration

• measurements.txt (1B row file) → Shell Script Wrapper — File path is hardcoded or passed as argument to ./calculate_average_Username.sh
• Shell Script Wrapper → JVM process — Script spawns JVM with optimized flags and classpath pointing to compiled Java class
• JVM → Java Implementation — JVM classloader loads and executes CalculateAverage_* bytecode from classpath
• Java Implementation → In-memory aggregation (HashMap/TreeMap) — Parser reads each line, extracts city and temperature, updates thread-local or shared hash map counters
• In-memory aggregation → Merge phase — Parallel threads synchronize and combine their per-city statistics (min, max, sum, count)
• Merge phase → TreeMap — Final sorted map keyed by city name ensures deterministic, alphabetically-ordered output
• TreeMap → stdout (formatted string) — Java implementation prints result in format: {city1=min;avg;max, city2=...} to console
• stdout → Evaluation harness — Script captures output; evaluation machine validates correctness and measures end-to-end runtime (wall clock

Submit a New 1BRC Solution

1. Create a new Java class in src/main/java/ directory that reads the 1B row CSV file and computes city temperature statistics (min/max/average) (src/main/java/dev/morling/onebrc/CalculateAverage_YourUsername.java)
2. Create a shell script wrapper that compiles and runs your Java implementation with appropriate JVM flags (calculate_average_YourUsername.sh)
3. Submit a PR with your shell script added to the repository root, following the submission template in .github/pull_request_template.md (.github/pull_request_template.md)
4. Ensure your implementation runs in ≤10 seconds on the evaluation machine and produces correct temperature averages per city in TreeMap-sorted order (ENVIRONMENT.md)

Optimize Java Performance for 1BRC

1. Review ENVIRONMENT.md to understand available heap size, number of CPU cores, and allowed JVM flags (e.g., -XX:+UseG1GC, -XX:+AlwaysPreTouch) (ENVIRONMENT.md)
2. Study the baseline implementation to understand the input format (city;temperature as semicolon-delimited lines) (calculate_average_baseline.sh)
3. Implement optimizations like memory-mapped I/O, custom hash maps, parallel streams, or unsafe memory access in your Java class (src/main/java/dev/morling/onebrc/CalculateAverage_YourUsername.java)
4. Configure Maven compiler settings in pom.xml for target Java version and compiler flags if needed (pom.xml)

• Java (JDK 21+) — Challenge requires Java for compute performance; allows access to advanced JVM features like vector API, virtual threads, and unsafe memory operations
• Maven — Standardized build system ensures reproducible compilation and packaging across 100+ diverse implementations
• GitHub Actions — Automated CI validation gates that submissions compile and execute within time constraints before manual review
• Shell scripts — Lightweight execution wrapper that specifies custom JVM flags, heap size, and GC settings optimized per submission

• Single machine evaluation vs. distributed cloud testing

  • Why: Ensures fair comparison of all submissions under identical hardware/JVM conditions; simplifies leaderboard interpretation
  • Consequence: Challenge is compute-bound on one machine; results may not reflect distributed system performance or cloud deployments

• Strict 10-second runtime limit for new submissions

  • Why: Prevents runaway submissions that would block evaluation queue; focuses on practical, optimizable solutions
  • Consequence: Favors high-effort algorithmic and JVM-tuning submissions; precludes ultra-brute-force approaches

• TreeMap-sorted output requirement

  • Why: Standardizes output format for easy correctness validation and deterministic comparison across submissions
  • Consequence: Adds small overhead to all submissions; rules out unsorted output optimizations

• No framework/library restrictions

  • Why: Encourages creativity and showcases real-world optimization techniques (memory-mapped I/O, unsafe memory, custom collections)
  • Consequence: Submissions vary widely in complexity; harder to extract generalizable best practices for typical use cases

• Distributed or cloud-based challenge evaluation
• Real-time streaming aggregation (one-shot 1B row batch processing only)
• Multi-language support (Java only)
• Handling of incremental/append-only data files
• Submission framework or SDK (each contributor writes from scratch)
• Production deployment or fault tolerance guarantees

Critical: The 1-billion-row test file (measurements.txt, ~13 GB) is not included in the repo—you must generate it yourself using the procedure in ENVIRONMENT.md or download it from the challenge infrastructure. Submissions optimize aggressively with JVM flags (e.g., -XX:+UseG1GC, -XX:+UnlockDiagnosticVMOptions, -XX:PrintFlagsFinal) that may not work on all JDK versions or hardware; test with the exact JDK 21 version and CPU architecture mentioned in ENVIRONMENT.md. The shell scripts use bash-specific features and expect a Unix environment (not Windows natively). Output format is strict: {Station1=min/mean/max, Station2=min/mean/max, ...} with stations sorted alphabetically and values rounded to 1 decimal place—any deviation fails validation.

• Virtual Threads (Project Loom, JEP 444) — Many top-ranked 1BRC submissions use virtual threads (JDK 19+) to parallelize CSV parsing across 1B rows with minimal overhead; understanding this is key to modern Java concurrency.
• Memory-Mapped I/O (java.nio.MappedByteBuffer) — High-performance submissions avoid standard file streams and use MappedByteBuffer to access the 1B-row file as fast off-heap memory, bypassing JVM buffers.
• SIMD Vectorization (Vector API, JEP 460) — Fastest submissions leverage Java's Vector API to process CSV data in SIMD-aligned chunks (parsing multiple fields in parallel); critical for reaching sub-minute runtimes.
• G1GC (Garbage-First Garbage Collector) Tuning — Most winning submissions explicitly tune G1GC flags (-XX:+UseG1GC, -XX:G1HeapRegionSize) to minimize pause time during 1B row aggregation; GC overhead is a bottleneck.
• Custom Hash Tables & Collision Resolution — Aggregating by station name requires fast hash lookups; optimized entries implement custom hash tables (instead of HashMap) with hand-tuned collision chains or linear probing for cache locality.
• Unsafe Memory Access (sun.misc.Unsafe) — Some 1BRC submissions use Unsafe to bypass bounds checks and directly access memory for parsing, trading safety for raw speed on a one-off benchmark.
• Cache-Aware Data Layout (False-Sharing Avoidance) — Multi-threaded submissions must avoid false-sharing when aggregating statistics in parallel; padding or @jdk.internal.vm.annotation.Contended ensures each thread's writes stay in separate CPU cache lines.

• quarkusio/quarkus — Quarkus is a Java framework used in some 1BRC submissions to achieve fast startup and low memory footprint; relevant for understanding how frameworks optimize for performance.
• openjdk/jdk21 — The official OpenJDK 21 repository; understanding the JVM internals (virtual threads, G1GC, escape analysis) is essential to optimize 1BRC implementations.
• TechEmpower/FrameworkBenchmarks — Similar competitive benchmarking framework for web frameworks; shares the same philosophy of standardized hardware evaluation and public leaderboards.
• jvm-profiling-tools/async-profiler — Essential profiling tool used by many 1BRC competitors to identify hotspots and measure improvements from optimizations.
• WireProtocol/mappedbus — Memory-mapped I/O library that some high-performing 1BRC submissions leveraged to efficiently read the massive CSV file.

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Create a submission validation test suite to verify all calculate_average_*.sh scripts conform to the expected interface

With 100+ submission scripts in the repo (calculate_average_*.sh), there's no automated validation that they all accept the correct input format, produce valid output format, or handle edge cases correctly. This would catch broken submissions before they're merged and ensure consistency across all entries. A test harness could verify each script runs successfully, produces JSON output matching the expected schema, and handles the test data correctly.

• [ ] Create src/test/java/dev/morling/SubmissionValidationTest.java to programmatically discover and validate all calculate_average_*.sh scripts
• [ ] Verify each script accepts input file path as first argument
• [ ] Verify output matches the expected JSON format (station names with min/mean/max values)
• [ ] Add Maven failsafe plugin configuration in pom.xml to run these validation tests in verify phase
• [ ] Document in ENVIRONMENT.md the validation requirements for new submissions

Add a benchmark comparison utility to measure relative performance between submissions

The repo has extensive submission scripts but no standardized way for contributors to compare their solution against others or the baseline. This utility would help contributors understand performance characteristics and identify optimization opportunities. A comparison tool would calculate runtime ratios, memory usage, and generate comparison reports.

• [ ] Create src/main/java/dev/morling/demos/BenchmarkComparator.java to run multiple submission scripts and collect metrics
• [ ] Add a Maven profile 'benchmark' in pom.xml to easily run comparisons
• [ ] Create a shell script utility/compare_submissions.sh that accepts submission names and generates a comparison report (runtime, relative speedup vs baseline)
• [ ] Document usage in ENVIRONMENT.md with example: './utility/compare_submissions.sh baseline mysubmission'

Create a submission template generator and checklist to reduce PR review friction

Many submissions are variations on similar approaches but lack consistency in commit messages, script naming, and documentation. Providing a template would standardize submissions and make reviews faster. Currently .github/pull_request_template.md exists but doesn't guide submitters on what to include for a 1BRC submission specifically.

• [ ] Enhance .github/pull_request_template.md with 1BRC-specific sections: optimization techniques used, JVM flags tested, hardware target, expected runtime range
• [ ] Create utility/submission_template.sh that generates a boilerplate calculate_average_USERNAME.sh with proper structure, error handling, and usage comments
• [ ] Create SUBMISSION_GUIDELINES.md documenting: naming conventions, expected performance thresholds (per Jan 12 status: 10sec or less), required comments explaining approach, testing requirements
• [ ] Reference the new guidelines in the PR template

• Add benchmark comparison documentation: Create a markdown file (BENCHMARK_ANALYSIS.md) that categorizes the 100+ submissions by optimization technique (virtual threads, SIMD, GC tuning, custom hash tables) and analyzes which techniques provided the largest speedups; helps newcomers understand what optimizations matter most.
• Write a tutorial for the baseline implementation: Expand README.md with a section that walks through CalculateAverage.java line-by-line, explaining the parsing logic, the hash map aggregation, and sorting—new contributors often struggle to understand how the naive approach works before optimizing.
• Create a Docker environment setup script: Add a docker-setup.sh that builds an image with JDK 21, Maven wrapper, and the data generator pre-configured, eliminating the friction in ENVIRONMENT.md for Windows/macOS developers who cannot easily replicate the exact Linux benchmark environment.

The codebase appears to be a legitimate competitive programming challenge repository with generally low security risk. However, the incomplete pom.xml file prevents a thorough dependency vulnerability assessment. The project uses a non-LTS Java version and outdated JUnit, which could be improved. The main security considerations are ensuring all dependencies are properly vetted, keeping Java and testing frameworks updated, and maintaining code review processes for the numerous shell script submissions. No hardcoded credentials, injection vulnerabilities, or infrastructure misconfigurations were detected based on the available information.

• High · Incomplete Maven pom.xml - Missing Dependency Information — pom.xml. The pom.xml file provided is truncated and incomplete. The dependencies section cuts off mid-declaration, making it impossible to fully assess dependency vulnerabilities. This incomplete configuration suggests the file may not be properly maintained or could hide security issues. Fix: Ensure the pom.xml file is complete and valid. Run 'mvn dependency:check' and use 'mvn dependency:tree' to identify all dependencies and check them against known vulnerability databases (OWASP Dependency-Check, Snyk, etc.).
• Medium · Java 21 Target - Limited Security Patch Window — pom.xml - maven.compiler.release property. The project targets Java 21, which is a non-LTS (Long Term Support) release. Non-LTS Java versions have shorter support windows and may not receive security updates for as long as LTS versions (Java 17, 21 LTS will be 21.0.x only). This could lead to delayed security patch availability. Fix: Consider using Java 21 LTS once available, or stick with Java 17 LTS for production environments that require extended security support.
• Low · Outdated JUnit Version — pom.xml - junit-bom version 5.8.1. The pom.xml specifies JUnit 5.8.1, which is relatively old. While JUnit is primarily a testing framework and not a runtime dependency, using outdated versions may miss bug fixes and improvements. Fix: Update to the latest stable JUnit 5 version (5.9.x or later) to ensure security patches and bug fixes are included.
• Low · No Explicit Security Dependencies — pom.xml. The provided pom.xml excerpt shows no explicit security-related dependencies (e.g., no encryption libraries, security frameworks, or input validation libraries). While this project appears to be a performance challenge and may not require them, it's worth verifying. Fix: Verify that all necessary security dependencies are declared in the complete pom.xml. If handling sensitive operations, ensure appropriate libraries are included.
• Low · Large Number of Unversioned Shell Scripts — calculate_average_*.sh files throughout repository. The repository contains 100+ shell scripts (calculate_average_*.sh) without visible version control or checksums. These scripts could be vectors for code injection or malicious modifications if not properly reviewed. Fix: Implement code review processes for all shell scripts. Use signed commits and branch protection rules. Consider adding checksums or signatures to verify script integrity. Document the purpose of each script.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.