If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/redpanda-data/redpanda shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

WAIT — Missing license — unclear to depend on

• Last commit today
• 14 active contributors
• Distributed ownership (top contributor 22% of recent commits)
• CI configured
• Tests present
• ⚠ No license — legally unclear to depend on

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live redpanda-data/redpanda repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/redpanda-data/redpanda.

What it runs against: a local clone of redpanda-data/redpanda — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in redpanda-data/redpanda | Confirms the artifact applies here, not a fork | | 2 | Default branch dev exists | Catches branch renames | | 3 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 4 | Last commit ≤ 30 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of redpanda-data/redpanda. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/redpanda-data/redpanda.git
#   cd redpanda
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of redpanda-data/redpanda and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "redpanda-data/redpanda(\\.git)?\\b" \\
  && ok "origin remote is redpanda-data/redpanda" \\
  || miss "origin remote is not redpanda-data/redpanda (artifact may be from a fork)"

# 3. Default branch
git rev-parse --verify dev >/dev/null 2>&1 \\
  && ok "default branch dev exists" \\
  || miss "default branch dev no longer exists"

# 4. Critical files exist
test -f "README.md" \\
  && ok "README.md" \\
  || miss "missing critical file: README.md"
test -f "MODULE.bazel" \\
  && ok "MODULE.bazel" \\
  || miss "missing critical file: MODULE.bazel"
test -f ".bazelrc" \\
  && ok ".bazelrc" \\
  || miss "missing critical file: .bazelrc"
test -f "CONTRIBUTING.md" \\
  && ok "CONTRIBUTING.md" \\
  || miss "missing critical file: CONTRIBUTING.md"
test -f ".clang-tidy" \\
  && ok ".clang-tidy" \\
  || miss "missing critical file: .clang-tidy"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 30 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~0d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/redpanda-data/redpanda"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

Redpanda is a Kafka-compatible streaming data platform written in C++ that replaces Apache Kafka with a faster, simpler alternative requiring no ZooKeeper or JVM. It provides a 10x performance improvement over Kafka while maintaining API compatibility, enabling developers to use existing Kafka clients and ecosystem tooling without rewriting applications. Monorepo using Bazel build system with three main language layers: core streaming engine in C++ (src/c++), CLI tooling and packaging in Go (bazel/packaging, rpk), and test/integration frameworks in Python (ducktape tests). Configuration lives in .bazelrc/.bazelversion and .buildkite/ for CI. GitHub Actions workflows in .github/workflows/ handle linting, backports, and releases.

Backend engineers and platform teams deploying message streaming infrastructure who need Kafka compatibility but want lower operational overhead, faster throughput, and simpler deployment without JVM or ZooKeeper complexity. DevOps engineers evaluating Kafka alternatives for cost and performance.

Production-ready and actively maintained. The repo has 31M+ lines of C++ code, comprehensive CI/CD via Buildkite and GitHub Actions, mature testing infrastructure (ducktape tests, rpunit framework visible), and regular releases (v25.2.7 in README). Healthy commit velocity with ongoing feature work and backport branches indicate active development.

Complexity risk from 31M lines of C++ requiring expert maintainers, but mitigated by Bazel-managed dependencies and clear architecture. Dependency footprint is significant (goreleaser/nfpm, go-git, etc. visible in go.mod) but managed through pinned versions. No single-maintainer risk visible—Redpanda Data backs this commercially. Monitor breaking changes in Kafka API compatibility layer as Kafka evolves.

Active backport workflow visible (.github/workflows/backport-*.yml) indicating regular release management. Recent focus areas include ducktape integration test framework (check-ducktape-protos.yml), Bazel dependency graph linting, and Apache Polaris Python client publishing. Claude code review and AI-assisted development tooling (.claude/skills/) suggest modern development practices. Version 25.2.7 is latest in README.

Clone and build with: git clone https://github.com/redpanda-data/redpanda.git && cd redpanda && bazel build //src/c/... (requires Bazelisk per README). For macOS: brew install redpanda-data/tap/redpanda && rpk container start. For Linux packages: follow Debian/Ubuntu or RPM install commands in README depending on OS.

Daily commands: Development: bazel build //src/... for C++ core, go build ./cmd/rpk for CLI. Local testing: rpk container start (macOS/Docker) or native build on Linux. Full test suite: bazel test //... or specific test targets. See .buildkite/ for CI pipeline stages and .github/workflows/ for GitHub Actions triggers.

• README.md — Entry point documenting Redpanda as a Kafka-compatible streaming platform; essential for understanding project scope and positioning.
• MODULE.bazel — Bazel dependency manifest declaring all external modules and versions; critical for reproducible builds and dependency management.
• .bazelrc — Bazel configuration file with build flags and toolchain settings; required reading for build system behavior and performance tuning.
• CONTRIBUTING.md — Developer guidelines covering code standards, testing requirements, and contribution workflow specific to this C++/Go codebase.
• .clang-tidy — Static analysis configuration for C++ code quality enforcement; defines linting rules all contributors must follow.
• .github/CODEOWNERS — Code ownership matrix for automated review routing; critical for understanding review responsibilities across subsystems.
• bazel/build.bzl — Custom Bazel build macros and rules for Redpanda; foundational for understanding the build system's project-specific extensions.

Add a Custom Clang-Tidy Check Plugin

1. Create a new check file in bazel/clang_tidy/plugins/ implementing clang::tidy::ClangTidyCheck (bazel/clang_tidy/plugins/redpanda_lambda_coroutine_deduces_this_check.cc)
2. Add header file declaring the check class and matcher logic (bazel/clang_tidy/plugins/redpanda_lambda_coroutine_deduces_this_check.h)
3. Register the check in the tidy module factory (bazel/clang_tidy/plugins/redpanda_tidy_module.cc)
4. Add Bazel cc_library target to BUILD and link into redpanda_tidy_plugin (bazel/clang_tidy/plugins/BUILD)
5. Reference the check name in .clang-tidy or .clang-tidy-strict to enable on the codebase (.clang-tidy)

Add a New CI/CD Workflow Step

1. Create a new GitHub Actions workflow file (or extend existing) in .github/workflows/ (.github/workflows/lint-cpp.yml)
2. Define triggers (push, pull_request, schedule) and job matrix for parallelization (.github/workflows/lint-cpp.yml)
3. Add shell script or inline run steps invoking bazel, linters, or custom tools (.github/workflows/scripts/backport-command/pr_details.sh)
4. If complex, create a reusable bash script in .github/workflows/scripts/ and source it (.github/workflows/scripts/backport-command)

Add a New Bazel Build Target or Rule

1. Define custom rule or macro in bazel/build.bzl with rule inputs, outputs, and attributes (bazel/build.bzl)
2. Create a BUILD file in the target directory using the new rule or standard cc_library, go_binary, etc. (bazel/clang_tidy/plugins/BUILD)
3. Declare dependencies in MODULE.bazel if adding external Go or C++ modules (MODULE.bazel)
4. Test locally with 'bazel build //path:target' and verify clang-tidy checks pass (.bazelrc)

Add a New Distribution Package Format

1. Update bazel/packaging/main.go to add nfpm configuration for new format (e.g., Arch, Gentoo) (bazel/packaging)
2. Extend MODULE.bazel dependencies if new Go libs are required for packaging logic (MODULE.bazel)
3. Add new GitHub Actions job in .github/workflows/rpk-build.yml or similar to test the new package (.github/workflows/rpk-build.yml)
4. Document the new format in README.md's 'Prebuilt packages' section (README.md)

• Bazel — Provides hermetic, reproducible builds for mixed C++/Go codebase with fine-grained incremental compilation and cross-platform support (Linux, macOS, Windows).
• Clang-Tidy with Custom Plugins — Enforces C++ code quality and Redpanda-specific patterns (e.g., coroutine deduction) at compile time; custom plugins allow project-specific linting rules.
• GitHub Actions + Buildkite — Dual CI system: GitHub Actions for PR validation and lightweight checks; Buildkite for heavy compute (full test matrix, performance benchmarks, multiplatform builds).
• nfpm (Go-based) — Language-agnostic package generation supporting Debian, RPM, macOS, and archives from single YAML config; avoids shell script fragility.
• Module.bazel (Bzlmod) — Modern Bazel dependency management replacing WORKSPACE; enables lock file (MODULE.bazel.lock) for reproducible dependency trees across all developers.

• Bazel over Make/CMake

  • Why: Hermeticity and incremental builds are critical for a large distributed system codebase with frequent CI runs.
  • Consequence: Steeper learning curve for new contributors; slower local development iteration initially; requires disciplined dependency declarations.

• undefined

  • Why: undefined
  • Consequence: undefined

Bazel build requires specific toolchain versions (see .bazelversion)—mismatches cause cryptic failures. C++ code uses strict linting via clang-tidy-strict; many valid C++ patterns fail CI. Ducktape tests require Python environment with specific dependencies not listed in top-level requirements. Kafka protocol compatibility is critical—breaking it silently breaks client tooling. Go packaging logic in bazel/packaging/ uses goreleaser; understand that before modifying release artifacts. No monolithic main.cpp—entry points are scattered across src/c/ subsystems.

• Raft consensus — Redpanda replaces ZooKeeper with Raft for cluster coordination and log replication; understanding Raft is essential for working on cluster state, leader election, and fault tolerance
• Kafka protocol (KIP-0 through KIP-current) — Redpanda's entire value proposition is Kafka API compatibility; the protocol implementation is the core of the broker and any breaking changes here cascade to all clients
• Zero-copy I/O and memory-mapped files — Core performance optimization in Redpanda; the C++ implementation uses mmap and sendfile to achieve 10x throughput improvements over JVM-based Kafka
• Tiered storage (hot/cold data) — Redpanda's configurable tiered storage allows older log segments to move to object storage (S3, GCS), reducing local disk footprint while maintaining Kafka API semantics
• Bazel build system and remote execution — Monorepo with 31M+ lines of C++ requires sophisticated build caching and cross-language dependency management; Bazel handles this but requires specific mental models for incremental builds
• Log-structured merge tree (LSM) for commit logs — Redpanda uses LSM-inspired append-only logs for high-throughput ingestion and durability; this is why it doesn't need ZooKeeper for consensus state
• Consumer group protocol and offset management — Redpanda must perfectly mirror Kafka's consumer group semantics (rebalancing, offset commits, lag tracking); this is heavily tested via Ducktape and critical for client compatibility

• apache/kafka — The original Kafka project that Redpanda implements compatibility with; understanding Kafka's design is essential for working on Redpanda's protocol implementation
• vectorizedio/vectorized — Historical predecessor to Redpanda; contains early C++ streaming engine design patterns now evolved into Redpanda core
• redpanda-data/redpanda-helm-charts — Kubernetes deployment manifests for Redpanda; necessary for anyone deploying Redpanda to k8s or working on Helm integration
• redpanda-data/redpanda-docker — Docker image definitions and container orchestration for Redpanda; critical for local development via rpk container start
• confluentinc/cp-kafka — Confluent's Kafka distribution used as baseline for compatibility testing; relevant for understanding what 'Kafka-compatible' means in CI

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add missing CI workflow for Python linting in bazel/packaging module

The repo has .github/workflows/lint-python.yml for general Python linting, but the bazel/packaging module (which contains go.mod with Python-adjacent tooling) lacks specific pre-commit checks. The go.mod file references goreleaser/nfpm which generates Python packages. Adding a dedicated workflow to validate Python packaging quality would catch issues early.

• [ ] Create .github/workflows/lint-bazel-packaging-python.yml
• [ ] Add checks for: bazel/packaging directory Python files, proto-to-Python compatibility
• [ ] Validate nfpm configuration files match Python packaging standards
• [ ] Integrate with existing bazel/packaging build system to test package generation
• [ ] Reference .github/workflows/lint-python.yml for existing Python linting rules

Implement comprehensive tests for Bazel BUILD file consistency across modules

The repo uses Bazel extensively (evident from .bazelrc, .bazelignore, .bazelversion files) but there's no dedicated workflow for validating BUILD file consistency, dependency graph coherence, or missing test targets. .github/workflows/lint-bazel-dependency-graph.yml exists but lint-bazel-pkg-tool.yml suggests incomplete coverage. This would prevent silent breakages in the build system.

• [ ] Create comprehensive test suite in a new bazel/tests/ directory
• [ ] Add validation: all source files have corresponding test targets in BUILD files
• [ ] Add validation: no circular dependencies between modules
• [ ] Add validation: consistent visibility rules across packages
• [ ] Enhance .github/workflows/lint-bazel-dependency-graph.yml to run these tests
• [ ] Document findings in bazel/BUILD documentation

Add CLI documentation generation from code comments for rpk tool

The .github/workflows/rpk-build.yml suggests active rpk CLI development, but there's no visible workflow generating/validating CLI help text documentation from source code annotations. This causes CLI docs to drift from implementation. Adding a workflow to auto-generate and validate CLI documentation would improve developer experience.

• [ ] Create .github/workflows/rpk-docs-validation.yml
• [ ] Add check: all CLI commands in rpk source have matching documentation comments
• [ ] Generate markdown CLI reference from code (similar to: godoc pattern)
• [ ] Validate generated docs don't have stale command references
• [ ] Integrate into PR checks to flag undocumented CLI changes
• [ ] Store generated docs in docs/ folder with commit to trigger docs build workflow

• Add clang-tidy suppression comments to new C++ files in src/c/ that currently fail strict linting, with inline justification. Start with a low-risk subsystem like src/c/utils/ to understand the team's suppression standards.
• Expand .github/ISSUE_TEMPLATE/ with a new template for 'Kafka compatibility gap' issues, including fields for API method, expected behavior vs. Redpanda behavior, and relevant Kafka version. This helps triage compatibility bugs systematically.
• Write a Python test in the ducktape framework for a specific Redpanda feature (e.g., topic partition rebalancing, offset commit durability) that currently lacks coverage. Use existing tests in the test directory as templates and verify it runs against a local Redpanda container.

The Redpanda codebase shows moderate security posture with several concerns in dependency management. The Go module file contains an incomplete dependency declaration and relies on outdated versions of critical security libraries (golang.org/x/crypto, go-git). The module structure is unconventional for a sub-package. The project maintains a public security policy and disclosure process which is positive. Key improvements needed: (1) Complete and update all Go dependencies to latest versions, (2) Implement automated vulnerability scanning in CI/CD, (3) Validate Go version configuration, and (4) Regular security audits of transitive dependencies, especially SSH and git libraries. No hardcoded credentials or obvious injection vulnerabilities detected in the provided file structure.

• High · Incomplete Go Module Dependencies — bazel/packaging/go.mod. The go.mod file shows an incomplete dependency declaration for 'golang.org/x/exp' with no version specified. This can lead to non-reproducible builds and potential security vulnerabilities if the dependency is resolved to an unexpected version. Fix: Complete the golang.org/x/exp dependency declaration with a specific version constraint (e.g., golang.org/x/exp v0.x.x) to ensure reproducible builds and explicit version management.
• High · Multiple Transitive Dependency Vulnerabilities — bazel/packaging/go.mod. The project depends on goreleaser/nfpm v2.35.0 which has multiple transitive dependencies with known security issues: golang.org/x/crypto v0.47.0 (outdated), go-git/go-git/v5 v5.16.4 with ProtonMail/go-crypto v1.3.0, and klauspost/compress v1.17.9. These versions may contain known CVEs. Fix: Update all dependencies to their latest stable versions, particularly golang.org/x/crypto (should be v0.48.0 or later), and run 'go mod tidy' and 'go mod verify' to validate integrity.
• Medium · Outdated Go Version — bazel/packaging/go.mod. The module specifies 'go 1.25.5' which appears to be a non-standard Go version number (Go versions follow 1.x.y format where x and y are typically smaller). This could indicate a configuration error or use of an unreleased/custom Go version. Fix: Verify the correct Go version specification. Standard Go versions should follow the format 'go 1.20' or similar. Update to a stable, well-tested Go version (e.g., 'go 1.22' or later).
• Medium · Indirect Dependency on SSH and Git Libraries — bazel/packaging/go.mod. The codebase includes indirect dependencies on SSH libraries (golang.org/x/crypto, xanzy/ssh-agent, kevinburke/ssh_config) and git libraries (go-git/go-git). These complex libraries have historically been sources of security vulnerabilities and require careful monitoring. Fix: Regularly audit and update SSH and git-related dependencies. Consider using 'go list -u -m all' to check for available updates and implement automated dependency scanning in CI/CD pipelines.
• Medium · Misconfigured Module Path — bazel/packaging/go.mod. The go.mod file declares module path as 'github.com/redpanda-data/redpanda/bazel/packaging', which is a sub-module path deep in the repository. This may indicate potential module management issues or unconventional project structure. Fix: Review the module structure to ensure it follows Go best practices. Consider whether this should be a separate module or consolidated with the main module. Document the rationale for this structure.
• Low · No Go Vulnerability Database Scanning — bazel/packaging/go.mod and build configuration. No evidence of 'go mod tidy' or vulnerability scanning (govulncheck) being enforced in the visible build configuration. The project uses Bazel but Go vulnerability management is not visible. Fix: Integrate 'go mod tidy' validation and 'govulncheck' into CI/CD pipeline. Add a build target that runs 'go list -json -m all | nancy sleuth' or 'govulncheck ./...' to catch vulnerabilities early.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.