If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/deepseek-ai/3FS shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

GO — Healthy across the board

• Last commit 2d ago
• 38+ active contributors
• Distributed ownership (top contributor 36% of recent commits)
• MIT licensed
• CI configured
• Tests present

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live deepseek-ai/3FS repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/deepseek-ai/3FS.

What it runs against: a local clone of deepseek-ai/3FS — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in deepseek-ai/3FS | Confirms the artifact applies here, not a fork | | 2 | License is still MIT | Catches relicense before you depend on it | | 3 | Default branch main exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 32 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of deepseek-ai/3FS. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/deepseek-ai/3FS.git
#   cd 3FS
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of deepseek-ai/3FS and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "deepseek-ai/3FS(\\.git)?\\b" \\
  && ok "origin remote is deepseek-ai/3FS" \\
  || miss "origin remote is not deepseek-ai/3FS (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(MIT)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"MIT\"" package.json 2>/dev/null) \\
  && ok "license is MIT" \\
  || miss "license drift — was MIT at generation time"

# 3. Default branch
git rev-parse --verify main >/dev/null 2>&1 \\
  && ok "default branch main exists" \\
  || miss "default branch main no longer exists"

# 4. Critical files exist
test -f "CMakeLists.txt" \\
  && ok "CMakeLists.txt" \\
  || miss "missing critical file: CMakeLists.txt"
test -f "Cargo.toml" \\
  && ok "Cargo.toml" \\
  || miss "missing critical file: Cargo.toml"
test -f "src/client/trash_cleaner" \\
  && ok "src/client/trash_cleaner" \\
  || miss "missing critical file: src/client/trash_cleaner"
test -f "src/storage/chunk_engine" \\
  && ok "src/storage/chunk_engine" \\
  || miss "missing critical file: src/storage/chunk_engine"
test -f "README.md" \\
  && ok "README.md" \\
  || miss "missing critical file: README.md"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 32 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~2d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/deepseek-ai/3FS"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

3FS (Fire-Flyer File System) is a high-performance distributed file system optimized for AI training and inference workloads, delivering 6.6+ TiB/s aggregate throughput across 180+ storage nodes using RDMA networks and NVMe SSDs. It implements Chain Replication with Apportioned Queries (CRAQ) for strong consistency and exposes a POSIX-like file interface backed by transactional metadata (FoundationDB-style) to simplify distributed application development. Monorepo structured around distributed system components: src/client (FUSE mount + client agents), src/storage (chunk engine in Rust), src/lib (core APIs including UsrbIo), with separate benchmarking tools (benchmarks/fio_usrbio, benchmarks/storage_bench) and deployment orchestration (deploy/data_placement). CMake + Cargo hybrid build system, with extensive cmake/ infrastructure for sanitizers, code coverage, and compiler flags.

ML/AI infrastructure engineers and platform teams building large-scale training clusters who need disaggregated shared storage with strong consistency guarantees, random-access dataloaders, and high-throughput parallel checkpointing—not data scientists, but the people provisioning their storage layer.

Actively developed and production-grade: Deepseek-AI maintains it with comprehensive CI/CD (GitHub Actions in .github/workflows/), extensive C++ codebases (5.2M LOC), formal design docs (docs/design_notes.md), and deployment guides (deploy/README.md). The Rust components (265K LOC) and mixed-language architecture suggest it's beyond MVP but still evolving for performance and feature completeness.

Standard open source risks apply.

Active development evident from build CI workflows, mixed-language components being actively maintained (Rust 1.85.0 MSRV specified in Cargo.toml), and benchmarking infrastructure. The presence of admin_cli.toml, monitor_collector_main.toml, and launcher configs suggests ongoing work on operations/management tooling, though exact recent commits are not visible in the metadata.

Clone and build with CMake + Cargo: git clone https://github.com/deepseek-ai/3fs.git && cd 3fs && mkdir build && cd build && cmake .. && make -j$(nproc). Then consult deploy/README.md for cluster setup or benchmarks/fio_usrbio/README.md for local performance testing.

Daily commands: For benchmarking: cd benchmarks/fio_usrbio && make && ./hf3fs_usrbio (see benchmarks/fio_usrbio/README.md for tuning). For cluster deployment: follow deploy/README.md which references launcher configs (e.g., configs/storage_main_launcher.toml, configs/meta_main_launcher.toml). For client usage: mount via FUSE using hf3fs_fuse_main with configs/hf3fs_fuse_main.toml.

• CMakeLists.txt — Root build configuration orchestrating the entire C++/Rust project; essential for understanding compilation, dependencies, and cross-platform builds.
• Cargo.toml — Rust workspace root defining three core members (trash_cleaner, chunk_engine, usrbio-sys); mandatory for Rust dependency and module organization.
• src/client/trash_cleaner — Rust client-side component responsible for garbage collection and resource cleanup; critical for understanding distributed cleanup semantics.
• src/storage/chunk_engine — Core storage engine managing chunk-based data organization; load-bearing for understanding the disaggregated SSD storage layer.
• README.md — High-level system design documenting CRAQ consistency, FUSE interface, and disaggregated architecture; required context for all contributors.
• docs/design_notes.md — Detailed architectural decisions including chain replication, transactional metadata via FoundationDB, and performance trade-offs.
• src/lib/rs/hf3fs-usrbio-sys — FFI bindings to user-space block I/O abstractions; critical for understanding low-level RDMA and SSD access patterns.

Add a New Storage Node Component

1. Create a new module in src/storage/ following chunk_engine patterns for managing a specialized storage device type. (src/storage/chunk_engine)
2. Add CMake target and link against usrbio-sys FFI bindings for low-level I/O. (CMakeLists.txt)
3. Add node-specific configuration template under configs/ and corresponding systemd service unit. (configs/storage_main.toml)
4. Update data_placement logic to include the new node in replication chain generation. (deploy/data_placement/src/setup/gen_chain_table.py)

Add a New FUSE File System Operation

1. Extend hf3fs_fuse/fuse.py with a new operation handler (e.g., getattr, read, write, mkdir). (hf3fs_fuse/fuse.py)
2. Implement underlying I/O logic in hf3fs_fuse/io.py to delegate to chunk_engine or metadata service. (hf3fs_fuse/io.py)
3. Add end-to-end test case in the benchmarks or integration test suite. (benchmarks/storage_bench/StorageBench.cc)

Configure a New Deployment Environment

1. Create environment-specific config files under configs/ (e.g., meta_main.toml, storage_main.toml, mgmtd_main.toml). (configs/storage_main.toml)
2. Add or modify systemd service units in deploy/systemd/ to start services on boot. (deploy/systemd/storage_main.service)
3. Run data_placement tool to generate chain topology and node assignment. (deploy/data_placement/src/setup/gen_chain_table.py)
4. Initialize monitoring database schema and update collector configuration. (deploy/sql/3fs-monitor.sql)

Enhance Performance Benchmarking

1. Add new benchmark scenario to StorageBench.cc covering specific workload patterns (sequential, random, mixed). (benchmarks/storage_bench/StorageBench.cc)
2. Run via FIO usrbio plugin (hf3fs_usrbio.cpp) to measure realistic I/O patterns. (benchmarks/fio_usrbio/hf3fs_usrbio.cpp)
3. Document results in docs/images/ and update design notes with performance insights. (docs/design_notes.md)

• C++ with CMake — Low-level performance for storage engine, RDMA, and SSD management; fine-grained control over memory and I/O scheduling.
• Rust (Cargo workspace: chunk_engine, trash_cleaner, usrbio-sys) — Memory safety for distributed client-side logic; FFI bindings for zero-copy user-space I/O; MSRV 1.85.0 ensures stability.
• Python FUSE daemon (hf3fs_fuse) — Rapid prototyping and integration of file system semantics; easy to extend with new operations; POSIX-compatible.
• FoundationDB (transactional KV store) — Strong consistency and ACID guarantees for distributed metadata; enables simple application semantics via CRAQ protocol.
• RDMA network + user-space I/O (usrbio-sys) — Bypasses kernel for ultra-low latency (microseconds) and high throughput; disaggregation across hundreds of nodes.
• Chain Replication with Apportioned Queries (CRAQ) — Linearizable consistency without sacrificing read throughput; tail node serves reads while head coordinates writes.

• Disaggregated architecture (separate compute/storage nodes)

  • Why: Enables massive scale-out: aggregate SSD capacity and network bandwidth from hundreds of nodes; simplifies resource provisioning for variable workloads.
  • Consequence: Introduces network latency; requires careful data placement and RDMA tuning to maintain performance; adds operational complexity.

• Strong consistency via CRAQ

  • Why: undefined
  • Consequence: undefined

1. RDMA networking required: Benchmarks and production assume 200Gbps InfiniBand; standard Ethernet will perform poorly or fail. 2. FoundationDB dependency: Metadata layer assumes a running FoundationDB cluster; no local fallback mode evident in configs. 3. Dual-language expertise: Building and debugging requires both C++ (5.2M LOC) and Rust (265K LOC) proficiency; CMake + Cargo interaction can be opaque. 4. Configuration complexity: Each component (storage_main, meta_main, client_agent, fuse_main) has separate TOML configs; mismatches silently degrade performance. 5. FUSE mount overhead: While the system targets 6.6 TiB/s raw, FUSE mount (src/client/hf3fs_fuse_main) adds latency; benchmark uses direct UsrbIo API instead.

• Chain Replication with Apportioned Queries (CRAQ) — 3FS's consistency model: enables strong write consistency via ordered chain while allowing read load-balancing across replicas; critical to understanding why application code is simple
• Disaggregated Storage Architecture — Core design principle: separates compute and storage into independent failure domains, allowing 180+ storage nodes to serve 500+ client nodes with locality-obliviousness; key to achieving 6.6 TiB/s throughput
• RDMA (Remote Direct Memory Access) — Networking substrate enabling zero-copy, kernel-bypass data movement between storage nodes and clients; 200Gbps InfiniBand assumed in benchmarks; performance bottleneck if unavailable
• FUSE (Filesystem in Userspace) — User-space filesystem implementation (hf3fs_fuse_main) bridges POSIX file interface to 3FS's network protocol; abstracts distributed storage as mounted directory
• Transactional Key-Value Store (FoundationDB-style) — Metadata layer backing hierarchical file structure; transactions ensure atomic directory operations across distributed metadata servers, preventing split-brain
• NVMe Chunk Engine — Rust-based block storage layer (src/storage/chunk_engine) managing 14TiB SSDs per storage node; handles wear leveling, garbage collection, and raw I/O scheduling
• Apportioned Query Load Balancing — CRAQ optimization: reads can be served from any replica in the chain, not just tail; 3FS exploits this to distribute read load across storage nodes without sacrificing consistency

• deepseek-ai/smallpond — Official companion repo: uses 3FS as the storage layer for distributed sort (GraySort) benchmark mentioned in README; demonstrates real AI workload integration
• foundationdb/foundationdb — Upstream metadata store; 3FS's transactional key-value backend for hierarchical directory metadata and strong consistency guarantees
• ceph/ceph — Architectural alternative: Ceph also disaggregates storage but uses object semantics; 3FS differentiates with POSIX files + CRAQ for AI workloads
• alluxio/alluxio — Similar problem space (data layer for ML); Alluxio focuses on in-memory + tiered caching, 3FS on distributed NVMe + RDMA throughput
• wongxingjun/CRAQ — Reference implementation of Chain Replication with Apportioned Queries; 3FS's core consistency protocol

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add integration tests for CRAQ chain replication consensus in src/meta

The README highlights Chain Replication with Apportioned Queries (CRAQ) as a core feature for strong consistency, but there are no visible test files in the meta component directory. This is critical for a distributed system claiming strong consistency guarantees. New contributors can add comprehensive tests validating chain formation, read/write consistency across replicas, and failover scenarios.

• [ ] Examine src/meta source files to understand CRAQ implementation (not fully visible in file structure)
• [ ] Create tests/integration/craq_replication_test.cpp validating chain formation with 3+ nodes
• [ ] Add tests for read consistency when writes are in-flight across the replica chain
• [ ] Add failover tests simulating node crashes and verifying consistency is maintained
• [ ] Integrate new tests into CMakeLists.txt and GitHub Actions workflows (build.yml)

Add Python deployment validation tests for deploy/data_placement module

The deploy/data_placement directory contains Python code for data placement optimization but has only test files in deploy/data_placement/test/. These tests appear minimal and lack CI validation. Data placement is critical for AI workload performance. A contributor should expand test coverage and add a GitHub Actions workflow to validate deployment configuration generation.

• [ ] Review existing tests in deploy/data_placement/test/ (test_model.py, test_setup.py, test_plan.py)
• [ ] Add edge case tests for gen_chain_table.py with invalid topology configurations
• [ ] Add parametrized tests validating data placement across different cluster sizes (3, 10, 100+ nodes)
• [ ] Create .github/workflows/test-deployment-python.yml to run pytest on every PR
• [ ] Add test coverage reporting to the workflow output

Add benchmark harness for FUSE client performance in benchmarks/fio_usrbio with CI tracking

The benchmarks/fio_usrbio directory contains hf3fs_usrbio.cpp and a Makefile but no automated benchmark results tracking or performance regression detection. For a performance-critical distributed filesystem, performance regression CI is essential. A contributor can set up automated benchmark runs with results storage and PR comments on regressions.

• [ ] Extend benchmarks/fio_usrbio/Makefile to generate JSON output from FIO results
• [ ] Create .github/workflows/benchmark-fio.yml that runs benchmarks on each push to main/dev branches
• [ ] Add workflow step to store baseline results in a results/ directory or external storage (GH Pages or artifact storage)
• [ ] Implement performance comparison logic to comment on PRs if latency/throughput regresses >5%
• [ ] Document benchmark setup and interpretation in benchmarks/fio_usrbio/README.md with specific metrics

• Add integration tests for the Rust chunk_engine (src/storage/chunk_engine) covering basic read/write/delete operations against a mock storage backend, mirroring patterns in benchmarks/storage_bench/StorageBench.cc.
• Expand UsrbIo.md API documentation with concrete C++ client code examples showing how to open, read, write, and close files, matching the fio_usrbio.cpp benchmark pattern.
• Create a Python test suite (tests/python/) that validates POSIX file semantics (directory traversal, rename, symlink handling) via the FUSE mount interface, complementing C++ benchmarks.

• Medium · Potential SQL Injection in Monitoring Database — deploy/sql/3fs-monitor.sql and monitoring collector components. The presence of 'deploy/sql/3fs-monitor.sql' suggests direct database schema management. Without visibility into how SQL queries are constructed for monitoring data insertion/retrieval, there's risk of SQL injection if dynamic queries are built using unsanitized input from metrics collection. Fix: Use parameterized queries/prepared statements for all database operations. Implement input validation and sanitization for all monitoring data before database insertion. Review monitor_collector_main source code for SQL construction patterns.
• High · Configuration Files May Contain Sensitive Data — configs/*.toml (all configuration files). Multiple .toml configuration files in 'configs/' directory are committed to the repository. These files likely contain database credentials, authentication tokens, network endpoints, and other sensitive configuration. If any contain plaintext secrets, they are exposed in version control history. Fix: Move sensitive configuration to environment variables or secrets management systems (HashiCorp Vault, AWS Secrets Manager, etc.). Use .gitignore to exclude local config files with secrets. Implement a config template system (e.g., config.example.toml). Scan commit history for exposed credentials.
• Medium · Rust MSRV May Have Known Vulnerabilities — Cargo.toml (rust-version specification) and Cargo.lock. The project specifies rust-version = '1.85.0' as MSRV. While this is a recent version, ensuring all transitive dependencies are audited is critical. The Cargo.lock file should be reviewed for known CVEs in dependencies. Fix: Run 'cargo audit' regularly in CI/CD pipeline to detect known vulnerabilities. Set up automated dependency updates with security scanning. Consider using cargo-deny for additional supply chain security checks.
• High · Potential Privilege Escalation via FUSE Mount — hf3fs/fuse.py, configs/hf3fs_fuse_main*.toml, deploy/systemd/hf3fs_fuse_main.service. The hf3fs_fuse_main component is a FUSE filesystem. FUSE implementations can introduce privilege escalation vulnerabilities if not properly sandboxed. The fuse mount point configuration and access controls may be misconfigured. Fix: Ensure FUSE daemon runs with minimal required privileges (not root if possible). Implement strict access controls on mount points. Use seccomp/AppArmor/SELinux policies to restrict FUSE operations. Validate all FUSE request parameters before processing.
• Medium · Docker Build Artifacts May Expose Source Code — dockerfile/dev.*.dockerfile. Multiple Dockerfiles present (dev.centos9.dockerfile, dev.dockerfile, dev.opencloudos9.dockerfile, dev.ubuntu2004.dockerfile) in the repository. Development Docker images may inadvertently include source code, build artifacts, or credentials. Fix: Use multi-stage Docker builds to separate build and runtime stages. Ensure sensitive files are not copied into runtime images. Use .dockerignore to exclude unnecessary files. Never commit credentials or private keys in Docker images. Scan built images with container security scanners (Trivy, Grype).
• Medium · RDMA Network Access Control Not Visible — Network/protocol implementation (not visible in provided structure). 3FS uses RDMA networks for high-performance data transfer. No visible RDMA access control, authentication, or encryption configuration in the provided file structure. RDMA traffic may be unencrypted or unauthenticated. Fix: Implement mutual TLS/DTLS for RDMA connections where supported. Enforce network segmentation and firewall rules for RDMA traffic. Implement authentication and authorization mechanisms for RDMA endpoint access. Document security model for cluster communication.
• Medium · Python Dependency Management in Deploy Scripts — deploy/data_placement/requirements.txt. The 'deploy/data_placement/' directory contains Python code with a requirements.txt file, but no version pinning is visible. Unpinned dependencies could pull in vulnerable versions. Fix: Pin all Python package versions explicitly using '==' in requirements.txt. Use 'pip-audit' to check for known vulnerabilities. Consider using 'pip-compile' to generate locked dependencies. Review transitive dependencies for security issues.
• **** · undefined — undefined. undefined Fix: undefined

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.