If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/apache/iggy shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

GO — Healthy across the board

• Last commit 1d ago
• 27+ active contributors
• Distributed ownership (top contributor 23% of recent commits)
• Apache-2.0 licensed
• CI configured
• Tests present

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live apache/iggy repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/apache/iggy.

What it runs against: a local clone of apache/iggy — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in apache/iggy | Confirms the artifact applies here, not a fork | | 2 | License is still Apache-2.0 | Catches relicense before you depend on it | | 3 | Default branch master exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 31 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of apache/iggy. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/apache/iggy.git
#   cd iggy
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of apache/iggy and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "apache/iggy(\\.git)?\\b" \\
  && ok "origin remote is apache/iggy" \\
  || miss "origin remote is not apache/iggy (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(Apache-2\\.0)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"Apache-2\\.0\"" package.json 2>/dev/null) \\
  && ok "license is Apache-2.0" \\
  || miss "license drift — was Apache-2.0 at generation time"

# 3. Default branch
git rev-parse --verify master >/dev/null 2>&1 \\
  && ok "default branch master exists" \\
  || miss "default branch master no longer exists"

# 4. Critical files exist
test -f "Cargo.toml" \\
  && ok "Cargo.toml" \\
  || miss "missing critical file: Cargo.toml"
test -f ".github/config/components.yml" \\
  && ok ".github/config/components.yml" \\
  || miss "missing critical file: .github/config/components.yml"
test -f ".github/workflows/pre-merge.yml" \\
  && ok ".github/workflows/pre-merge.yml" \\
  || miss "missing critical file: .github/workflows/pre-merge.yml"
test -f "README.md" \\
  && ok "README.md" \\
  || miss "missing critical file: README.md"
test -f ".github/CODEOWNERS" \\
  && ok ".github/CODEOWNERS" \\
  || miss "missing critical file: .github/CODEOWNERS"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 31 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~1d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/apache/iggy"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

Apache Iggy is a high-performance, distributed message streaming platform written in Rust, designed as a hyper-efficient alternative to Kafka for real-time data pipelines. It provides persistent message broking with multi-language SDKs (Rust, Java, C#, TypeScript, Go, Python) and a gRPC-based protocol, optimized for low-latency, high-throughput streaming workloads with support for partitioned topics, consumer groups, and consensus-based replication. Monorepo structure: core/server contains the main broker engine, core/sdk wraps the client protocol, core/binary_protocol defines gRPC messaging, core/cli is the command-line tool, and SDKs live in language-specific dirs (sdk/rust, sdk/java, etc.). Connectors (core/connectors/sinks/ and core/connectors/sources/) handle data integration with external systems (Postgres, Elasticsearch, MongoDB, S3/Delta). Benchmarking/monitoring in core/bench with a Svelte-based dashboard.

Distributed systems engineers and data platform teams building real-time event streaming architectures who need a Kafka alternative with lower resource consumption, faster startup times, and easier operational complexity; SDK users across six major languages integrating message streaming into microservices and data pipelines.

Actively developed and production-ready: Apache Software Foundation project (ASF) with comprehensive CI/CD across 7+ languages, code coverage tracking via Codecov, and multi-target testing (Rust pre/post-merge, Java Gradle, C# .NET, Go, Node.js, Python via Maturin). Large codebase (~12.4M lines of code) with well-established testing harnesses (BDD Gherkin tests in bdd/rust, integration tests in core/integration).

Low organizational risk due to ASF governance and active development, but consider: (1) Rust is the primary language (9.2M LOC) with complex memory safety patterns—contributor pool smaller than JVM-first projects; (2) Distributed consensus (core/consensus) is mission-critical and deserves deep review before production deployment; (3) Multi-language SDK maintenance (6 languages) can create version skew if not carefully coordinated across release cycles.

Active development across multiple fronts: server-ng (next-generation server), AI MCP integration (core/ai/mcp), multiple sink/source connectors being maintained, and cross-language SDK updates. GitHub Actions workflows visible for all major languages (Rust, Java, C#, Go, Node, Python) indicate frequent testing and releases.

Clone: git clone https://github.com/apache/iggy.git && cd iggy. For Rust core development: cargo build (Cargo.toml at workspace root). For CLI: cargo run --manifest-path core/cli/Cargo.toml. For Docker: docker build -f Dockerfile -t iggy .. Language-specific: Java uses gradle (see .github/actions/java-gradle), C# uses dotnet, Node uses npm (see .github/actions/node-npm), Python uses maturin (see .github/actions/python-maturin).

Daily commands: Broker server: cargo run --manifest-path core/server/Cargo.toml (or use Docker). CLI client: cargo run --manifest-path core/cli/Cargo.toml -- [command] (e.g., producer, consumer). Tests: cargo test in workspace or specific crates. BDD/integration: cargo test --manifest-path bdd/rust/Cargo.toml (uses Gherkin scenarios). Bench dashboard: cd core/bench/dashboard && npm install && npm run dev.

• Cargo.toml — Workspace root configuration defining the monorepo structure with multiple language bindings (Rust, Python, Node.js, Go, Java, C#, C++); essential for understanding the full project scope
• .github/config/components.yml — Defines the modular component architecture and orchestrates multi-language builds; critical for understanding how the 600-file codebase is organized
• .github/workflows/pre-merge.yml — CI/CD pipeline running tests across all 7 language implementations; reveals build, test, and deployment architecture
• README.md — Project overview documenting Iggy as a message streaming platform; necessary context for all contributors to understand core mission
• .github/CODEOWNERS — Defines ownership and review requirements for different language implementations and core components
• CONTRIBUTING.md — Contribution guidelines specific to the polyglot architecture and Apache project requirements
• .asf.yaml — Apache Software Foundation configuration governing licensing, governance, and release processes for this project

Add Support for a New Language Binding

1. Create language-specific directory structure and build files (e.g., sdk/python for Python, sdk/go for Go) (Cargo.toml)
2. Add component definition in the central components configuration (.github/config/components.yml)
3. Create pre-merge workflow action for building and testing the new language binding (.github/actions/{language}/pre-merge/action.yml)
4. Create post-merge workflow action for publishing to language-specific package manager (PyPI, npm, crates.io, etc.) (.github/actions/{language}/post-merge/action.yml)
5. Update the main pre-merge workflow to include the new language binding in the matrix (.github/workflows/pre-merge.yml)
6. Add new language binding to the publish workflow (.github/config/publish.yml)
7. Update CODEOWNERS to assign reviewers for the new language implementation (.github/CODEOWNERS)

Add a New Streaming Feature to the Core Engine

1. Implement feature in the Rust core engine, following existing patterns in Cargo.toml workspace structure (Cargo.toml)
2. Add unit and integration tests for the new feature (.config/nextest.toml)
3. Update language bindings to expose the new feature (Python via Maturin, Node.js, Go, etc.) (.github/actions/python-maturin/pre-merge/action.yml)
4. Ensure all pre-merge CI tests pass across all implementations (.github/workflows/pre-merge.yml)
5. Add feature documentation and update README if it's a major capability (README.md)

Modify or Add Build Configuration

1. Update workspace dependencies and features in the Cargo.toml if it's Rust-related (Cargo.toml)
2. If adding new development tools or tasks, update the Rail configuration (.config/rail.toml)
3. If adding pre-commit hooks or linting rules, update pre-commit configuration (.pre-commit-config.yaml)
4. If updating Docker setup, modify the Dockerfile or Docker build actions (Dockerfile)
5. Run pre-merge workflow to verify all language implementations still build correctly (.github/workflows/pre-merge.yml)

Update Governance or Contribution Guidelines

1. Update the main contribution document to reflect new guidelines (CONTRIBUTING.md)
2. If changing code ownership, update CODEOWNERS file (.github/CODEOWNERS)
3. If adding new issue types or processes, create/modify issue templates (.github/ISSUE_TEMPLATE/config.yml)
4. If changing dependencies or adding new requirements, update DEPENDENCIES.md (DEPENDENCIES.md)
5. For Apache compliance changes, update .asf.yaml governance file (.asf.yaml)

1. Workspace setup: The monorepo uses Cargo workspaces extensively; cargo build from root builds all crates, but feature flags may differ per crate. 2. Protocol versioning: Binary protocol changes must be backward-compatible; verify core/binary_protocol changes against bdd/rust feature files. 3. Consensus complexity: core/consensus requires careful testing; no quick way to develop this without understanding Raft-like algorithms. 4. Language SDK sync: Updates to core/sdk must be manually ported to Java, C#, Go, TypeScript, Python—no code generation; easy to drift. 5. Docker build context: .dockerignore may exclude needed artifacts; check before debugging container builds. 6. Config schema (core/configs_derive): Uses procedural macros; changes require cargo rebuild and may cache stale schemas.

• Message Partitioning — Core concept in Iggy (core/partitions/); understanding how topics shard across partitions and how consumer groups balance load is essential to architecture decisions
• Consumer Groups & Offset Management — Central to Iggy's consumer semantics; knowing how offsets track progress and how groups rebalance is critical for correctness in production use
• Distributed Consensus (Raft-like) — core/consensus implements leader election and log replication; Iggy's fault tolerance depends on this; essential for understanding durability and split-brain handling
• gRPC & Protocol Buffers — Iggy's client-server protocol is gRPC (core/binary_protocol); all SDKs communicate via gRPC stubs; affects performance tuning and SDK generation
• Memory-Mapped I/O & Journal Storage — core/journal likely uses mmap for efficient message persistence; understanding page cache behavior and fsync tradeoffs is critical for durability tuning
• Async Rust & Tokio — Iggy core is async Rust (inferred from typical Rust broker patterns); understanding Future, async/await, and Tokio scheduler helps debug performance and deadlocks
• Connector Pattern & Sinks/Sources — core/connectors/ architecture shows how external systems integrate; knowing source/sink contract (backpressure, retries, batching) is key for extending Iggy

• confluentinc/kafka — Direct predecessor and alternative; Iggy is designed as a lighter, faster replacement for Kafka with similar partitioned topic model
• redpanda-data/redpanda — Another Kafka-compatible streaming broker written in C++ with lower latency; similar market positioning and architecture decisions
• vectordotdev/vector — Ecosystem partner for data collection and routing; Iggy connectors (sinks/sources) often feed into or consume from Vector pipelines
• openobserve/openobserve — Common downstream use case; Iggy users stream logs/metrics to OpenObserve via HTTP/sinks for observability
• apache/pulsar — Alternative Apache streaming platform; shares multi-tenancy and geo-replication goals but different architecture (topic-centric vs partition-centric)

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add missing post-merge CI workflow for C++ Bazel builds

The repo has pre-merge CI for C++ Bazel (.github/actions/cpp-bazel/pre-merge/action.yml) but no corresponding post-merge workflow. Other language stacks (Rust, Go, Java, Node, Python, C#) all have both pre-merge and post-merge actions. This is a gap in the CI pipeline that should publish/cache C++ build artifacts after merges to main.

• [ ] Create .github/actions/cpp-bazel/post-merge/action.yml following the pattern in .github/actions/rust/post-merge/action.yml
• [ ] Add post-merge build step for C++ Bazel targets (likely in core/bench or similar)
• [ ] Reference the new action in the appropriate workflow file in .github/workflows/
• [ ] Test locally with act or similar to ensure Bazel commands execute correctly

Add integration tests for multi-language SDK interoperability

The workspace contains SDKs in multiple languages (Java, Go, Node, Python, C#) alongside core Rust server. The core/integration directory exists but appears underutilized. Adding cross-language integration tests (e.g., Java client → Rust server → Python client) would validate protocol compatibility and catch regressions early.

• [ ] Audit existing tests in core/integration/ to identify gaps between languages
• [ ] Create a new test suite in core/integration/ with scenarios like: publish from Node → consume with Python, consume with Go
• [ ] Update .github/workflows/ to run cross-language integration tests (requires coordinating multiple language CI actions)
• [ ] Document test setup in CONTRIBUTING guide with language-specific server setup steps

Add missing Helm chart validation and linting CI workflow

The repo has .github/actions/utils/setup-helm-tools which suggests Helm chart usage, but there's no dedicated CI workflow for Helm chart validation, linting (via helm lint), or schema validation. Given Apache's Kubernetes deployment targets, a dedicated workflow would catch chart issues before release.

• [ ] Create new GitHub workflow file .github/workflows/helm-lint.yml that runs on PR changes to charts/
• [ ] Add steps using helm lint, kubeval, and chart-testing tools (referenced in setup-helm-tools/action.yml)
• [ ] Add validation for values.schema.json against actual values.yaml in chart templates
• [ ] Document in CONTRIBUTING.md how to validate Helm charts locally before submitting PRs

• Add integration tests for the TypeScript SDK (sdk/typescript) against a live broker instance in core/integration; most language SDKs have cross-language test harnesses but TypeScript is under-represented in the test matrix.
• Document the connector lifecycle and error-handling contract in core/connectors/sdk/src/ with inline code examples; currently no README or doc comments explain how sources/sinks handle backpressure, retries, or failures.
• Write a quick-start guide for building a custom sink connector, either as an .md in core/connectors/sinks/ or as a worked example in examples/; the SDK exists but no template or tutorial exists for new contributors.

Apache Iggy demonstrates moderate security maturity with GitHub Actions-based CI/CD and Dependabot integration already configured. However, the project's complexity across multiple languages and numerous external connectors introduces significant attack surface. Key concerns include: 1) Multi-ecosystem dependency management requiring coordinated security updates, 2) Multiple data connectors (SQL databases, Elasticsearch, etc.) requiring strict input validation and secure credential handling, 3) Custom binary protocol and consensus implementation requiring formal security review, 4) AI/LLM integration (MCP) introducing novel injection vectors. Strengths include established build system with pre-merge checks, ASF licensing compliance, and organized dependency management. Recommended improvements: establish a security policy, implement SBOM generation, conduct security audits of consensus and binary protocol code, and formalize secure coding guidelines for connectors.

• Medium · Multiple Language Ecosystem Dependencies — Workspace structure with multiple language SDKs and connectors. The codebase spans multiple programming languages (Rust, Python, Java, Go, C#, Node.js, C++) with separate dependency management systems. This increases the attack surface and makes security auditing more complex. Each ecosystem has different vulnerability disclosure practices and patch availability timelines. Fix: Implement a centralized dependency scanning tool (e.g., SBOM generation, supply chain security scanning) that covers all language ecosystems. Use tools like Snyk, Dependabot (already configured), or OWASP Dependency-Check for continuous monitoring.
• Medium · Numerous External Connectors and Integrations — core/connectors/sinks/* and core/connectors/sources/*. The codebase includes multiple data connectors (Elasticsearch, MongoDB, PostgreSQL, InfluxDB, Delta, Iceberg, HTTP, Quickwit) and source/sink implementations. Each external integration introduces potential security risks including credential handling, injection attacks, and insecure serialization. Fix: Implement connector-specific security guidelines: 1) Enforce secure credential storage (avoid hardcoding), 2) Validate all inputs from external sources, 3) Use parameterized queries for database connectors, 4) Implement connection pooling with timeouts, 5) Add comprehensive logging without exposing sensitive data.
• Medium · Distributed Consensus Component — core/consensus. The presence of a consensus module (core/consensus) suggests distributed system coordination. Consensus protocols are security-critical and prone to issues like Byzantine attacks, message tampering, and leader election vulnerabilities if not properly implemented. Fix: Ensure cryptographic message signing for inter-node communication, implement message authentication codes, validate consensus state transitions, and conduct formal security audits of the consensus algorithm implementation.
• Low · Binary Protocol Implementation — core/binary_protocol. Custom binary protocol implementation (core/binary_protocol) can be vulnerable to deserialization attacks, buffer overflows, and format string vulnerabilities if parsing is not carefully implemented. Fix: Use well-tested serialization libraries, implement fuzzing tests, enforce size limits on deserialized data, validate protocol version compatibility, and consider using established protocols (protobuf, CBOR) instead of custom implementations.
• Low · MCP (Model Context Protocol) Integration — core/ai/mcp. The presence of MCP integration (core/ai/mcp) suggests AI/LLM integration. This introduces risks around prompt injection, model hijacking, and sensitive data exposure to third-party AI services. Fix: Implement strict input sanitization for AI prompts, audit AI model responses before processing, minimize sensitive data passed to AI services, implement rate limiting, and add comprehensive audit logging for AI integration activities.
• Low · Dynamic Configuration System — core/configs, core/configs_derive. The configs and configs_derive modules suggest dynamic configuration loading. Improper configuration validation could lead to security misconfigurations. Fix: Implement schema validation for all configuration files, use type-safe configuration loading, avoid evaluating configuration strings as code, implement configuration change auditing, and document all security-relevant configuration options.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.