If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/Physical-Intelligence/openpi shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

GO — Healthy across the board

• Last commit 2d ago
• 19 active contributors
• Distributed ownership (top contributor 36% of recent commits)
• Apache-2.0 licensed
• CI configured
• Tests present

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live Physical-Intelligence/openpi repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/Physical-Intelligence/openpi.

What it runs against: a local clone of Physical-Intelligence/openpi — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in Physical-Intelligence/openpi | Confirms the artifact applies here, not a fork | | 2 | License is still Apache-2.0 | Catches relicense before you depend on it | | 3 | Default branch main exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 32 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of Physical-Intelligence/openpi. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/Physical-Intelligence/openpi.git
#   cd openpi
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of Physical-Intelligence/openpi and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "Physical-Intelligence/openpi(\\.git)?\\b" \\
  && ok "origin remote is Physical-Intelligence/openpi" \\
  || miss "origin remote is not Physical-Intelligence/openpi (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(Apache-2\\.0)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"Apache-2\\.0\"" package.json 2>/dev/null) \\
  && ok "license is Apache-2.0" \\
  || miss "license drift — was Apache-2.0 at generation time"

# 3. Default branch
git rev-parse --verify main >/dev/null 2>&1 \\
  && ok "default branch main exists" \\
  || miss "default branch main no longer exists"

# 4. Critical files exist
test -f "src/openpi/models/pi0.py" \\
  && ok "src/openpi/models/pi0.py" \\
  || miss "missing critical file: src/openpi/models/pi0.py"
test -f "src/openpi/models/model.py" \\
  && ok "src/openpi/models/model.py" \\
  || miss "missing critical file: src/openpi/models/model.py"
test -f "packages/openpi-client/src/openpi_client/runtime/runtime.py" \\
  && ok "packages/openpi-client/src/openpi_client/runtime/runtime.py" \\
  || miss "missing critical file: packages/openpi-client/src/openpi_client/runtime/runtime.py"
test -f "scripts/train_pytorch.py" \\
  && ok "scripts/train_pytorch.py" \\
  || miss "missing critical file: scripts/train_pytorch.py"
test -f "src/openpi/models/tokenizer.py" \\
  && ok "src/openpi/models/tokenizer.py" \\
  || miss "missing critical file: src/openpi/models/tokenizer.py"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 32 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~2d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/Physical-Intelligence/openpi"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

openpi is an open-source robotics framework from Physical Intelligence that provides pre-trained vision-language-action (VLA) models—π₀, π₀-FAST, and π₀.₅—capable of understanding images and language to generate robot actions. The models are flow-based or autoregressive transformers trained on 10k+ hours of real robot manipulation data, enabling both zero-shot inference and fine-tuning on custom robot platforms like ALOHA, DROID, and Libero. Monorepo structure: root contains core training/inference code; examples/ (aloha_real, aloha_sim, droid, libero) are self-contained Docker-ready applications with data conversion scripts (convert_aloha_data_to_lerobot.py, convert_droid_data_to_lerobot.py); docs/ covers docker, normalization stats, remote inference; .github/workflows orchestrate CI.

Roboticists and ML engineers building manipulation systems who want to leverage pre-trained foundation models; researchers adapting VLAs to custom robot hardware (ALOHA arms, DROID grippers, simulation environments); teams running inference at > 8GB GPU memory or fine-tuning with 22.5–70GB VRAM.

Actively developed and recently upgraded (Sept 2025 PyTorch support, π₀.₅ release, DROID training pipeline). The project ships with CI/CD (.github/workflows/pre-commit.yml, test.yml), Docker support, and multiple complete examples (aloha_real, aloha_sim, droid, libero). Verdict: production-ready with experimental edge cases—models work out-of-box but authors explicitly state 'may or may not work for you' on unsupported platforms.

Medium-high risk: GPU dependency (A100/H100 for full fine-tuning) creates reproducibility friction; single team (Physical Intelligence) maintains core models; submodule dependencies (.gitmodules) add cloning complexity. Recent rapid iteration (π₀.₅ release, flow-vs-autoregressive architecture changes) suggests API stability is still settling. Ubuntu 22.04 OS lock-in limits portability.

Sept 2025: PyTorch backend support shipped (previously JAX-only); π₀.₅ released with knowledge insulation for improved open-world generalization; improved idle filter added to DROID training pipeline (examples/droid/README_train.md); multi-example inference/training notebooks (examples/inference.ipynb, examples/policy_records.ipynb) actively maintained.

git clone --recurse-submodules https://github.com/Physical-Intelligence/openpi.git && cd openpi && python -m pip install -r requirements.txt (inferred from .python-version and project structure). For real-robot use: cd examples/aloha_real && docker compose up (per examples/aloha_real/compose.yml). For sim: cd examples/aloha_sim && python main.py (after installing examples/aloha_sim/requirements.txt).

Daily commands: Inference (see examples/inference.ipynb): load model checkpoint → forward pass with image + language prompt → get action predictions. Real robot (examples/aloha_real/main.py): docker compose up → env.py spins robot/vision, policy inference loop in main.py. Simulation (examples/aloha_sim/main.py): python main.py with dm_control env.py and saver.py for trajectory logging. Training: python train.py --config <config.yaml> (config structure in examples/droid/README_train.md).

• src/openpi/models/pi0.py — Core π₀ model implementation—the primary vision-language-action model that all training and inference pipelines depend on.
• src/openpi/models/model.py — Base model abstraction layer used by all variants (π₀, π₀-FAST, π₀.₅)—defines the interface for tokenization, forward passes, and checkpoint loading.
• packages/openpi-client/src/openpi_client/runtime/runtime.py — Entry point for production inference—orchestrates agent initialization, environment stepping, and policy execution for deployed models.
• scripts/train_pytorch.py — Main training loop and configuration for fine-tuning models on custom datasets—critical for end-users extending the base models.
• src/openpi/models/tokenizer.py — Action and observation tokenization logic shared across all model variants—essential for understanding data representation.
• examples/aloha_real/main.py — Reference implementation for real-world ALOHA robot deployment—demonstrates the complete pipeline from model loading to real-time control.
• packages/openpi-client/src/openpi_client/base_policy.py — Policy interface abstraction for inference clients—defines how remote and local policies are invoked uniformly.

Add a new model variant (e.g., π₀.₂)

1. Create model class inheriting from Model base class in src/openpi/models/model.py (src/openpi/models/pi0_v2.py)
2. Implement forward(), get_action_tokens(), and checkpoint/loading logic (src/openpi/models/pi0_v2.py)
3. Export the new model in src/openpi/models/__init__.py (src/openpi/models/__init__.py)
4. Add unit tests following the pattern in src/openpi/models/pi0_test.py (src/openpi/models/pi0_v2_test.py)
5. Update scripts/train_pytorch.py to register and load your variant (scripts/train_pytorch.py)

Add support for a new robot platform

1. Create example directory at examples/my_robot/ with main.py for inference loop (examples/my_robot/main.py)
2. Create env.py defining observation/action spaces using gym.Env interface (examples/my_robot/env.py)
3. Optionally add data converter in examples/my_robot/convert_my_robot_data_to_lerobot.py (examples/my_robot/convert_my_robot_data_to_lerobot.py)
4. Add Docker configuration if deployment-critical (see examples/aloha_real/Dockerfile) (examples/my_robot/Dockerfile)
5. Document in examples/my_robot/README.md with setup and usage instructions (examples/my_robot/README.md)

Deploy a model as a remote service

1. Configure model checkpoint path and environment in scripts/docker/serve_policy.Dockerfile (scripts/docker/serve_policy.Dockerfile)
2. Use scripts/serve_policy.py to start the HTTP/WebSocket inference server (scripts/serve_policy.py)
3. Create Docker Compose file for multi-service setup (see scripts/docker/compose.yml) (scripts/docker/compose.yml)
4. In client code, use WebsocketClientPolicy in packages/openpi-client/src/openpi_client/websocket_client_policy.py to connect (packages/openpi-client/src/openpi_client/websocket_client_policy.py)
5. Run inference via packages/openpi-client/src/openpi_client/runtime/runtime.py with the remote policy (packages/openpi-client/src/openpi_client/runtime/runtime.py)

Fine-tune a model on custom data

1. Prepare dataset in LeRobot format or convert using examples like examples/aloha_real/convert_aloha_data_to_lerobot.py (examples/aloha_real/convert_aloha_data_to_lerobot.py)
2. Compute normalization statistics by running scripts/compute_norm_stats.py (scripts/compute_norm_stats.py)
3. Configure training hyperparameters and data path in scripts/train_pytorch.py (or JAX variant in scripts/train.py) (scripts/train_pytorch.py)
4. Run training script with model checkpoint path and dataset directory (scripts/train_pytorch.py)
5. Validate on holdout set, then load checkpoint and deploy via packages/openpi-client/src/openpi_client/runtime/runtime.py (packages/openpi-client/src/openpi_client/runtime/runtime.py)

• PyTorch + JAX dual support — PyTorch for production inference and fine-tuning accessibility; JAX for research flexibility and XLA compilation
• LeRobot dataset format — Standardized,

Submodule initialization mandatory (--recurse-submodules required on clone, see CONTRIBUTING.md). Ubuntu 22.04 only—no macOS/Windows support. NVIDIA GPU required; CPU inference not tested/supported. Model checkpoints externally hosted (not in repo); download URLs likely in README or docs. Flow-matching head enforced for π₀.₅ (per README note on training/inference support). Data normalization stats (docs/norm_stats.md) must be computed per dataset; using wrong stats breaks performance. FSDP multi-GPU training not yet multi-node capable (stated in README).

• Flow Matching for Action Generation — π₀ and π₀.₅ use flow-matching (not diffusion) as the action head; understanding this generative paradigm is essential for modifying inference logic and training objectives.
• Vision-Language-Action (VLA) Models — Core architecture of all models in this repo; VLAs fuse image encoders with language models to condition action prediction on both visual state and text instructions.
• Action Tokenization (FAST) — π₀-FAST uses an autoregressive tokenizer to discretize continuous actions; critical for understanding the performance/latency tradeoff vs. flow-matching.
• Knowledge Insulation — π₀.₅ training technique that improves open-world generalization by preventing task-specific knowledge leakage; relevant for fine-tuning on new domains.
• Fully Sharded Data Parallel (FSDP) — Multi-GPU training strategy used via fsdp_devices config parameter; essential for scaling training beyond single GPU without full model parallelism complexity.
• LeRobot Data Format — Standardized format for robot trajectories (HDF5-based) used by all data conversion scripts; understanding schema is required for custom data integration.
• LoRA (Low-Rank Adaptation) Fine-Tuning — Memory-efficient alternative to full fine-tuning (22.5 GB vs. 70 GB); critical for adapting models on resource-constrained hardware.

• lerobot/lerobot — Core data format standard (LeRobot) used by all data conversion scripts in openpi; provides dataset tools and benchmarks.
• google-deepmind/dm_control — Simulation environment used in examples/aloha_sim/ and examples/libero/; provides MuJoCo-based robot control and observation pipelines.
• droid-dataset/droid — Canonical DROID robot dataset; openpi examples/droid/ is the official open-source training implementation for DROID-based models.
• tonyzhaozh/aloha — Original ALOHA hardware platform; openpi examples/aloha_real/ targets this robot and includes real deployment code.
• railrl/libero — Libero simulation benchmark used in examples/libero/ for task learning evaluation and data generation.

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add comprehensive unit tests for openpi-client package

The openpi-client package in packages/openpi-client/src/openpi_client/ has only one test file (image_tools_test.py) but multiple critical modules lack test coverage: base_policy.py, action_chunk_broker.py, and msgpack_numpy.py. These are core inference components that deserves unit tests, especially given the repo's focus on robotics inference reliability.

• [ ] Create tests/test_base_policy.py covering policy initialization, inference, and error handling
• [ ] Create tests/test_action_chunk_broker.py for action buffering/chunking logic with edge cases
• [ ] Create tests/test_msgpack_numpy.py for serialization/deserialization round-trips with various numpy dtypes
• [ ] Update packages/openpi-client/pyproject.toml to include pytest as a dev dependency
• [ ] Add test execution to .github/workflows/test.yml for the openpi-client package

Add integration tests for Docker examples and compose setups

The repo has 6 Docker-based examples (aloha_real, aloha_sim, libero, droid, ur5, simple_client) with compose.yml files, but no automated verification that these containers build and run correctly. This is critical for maintainability as dependencies drift over time.

• [ ] Create .github/workflows/docker-build.yml to build Docker images for examples/*/Dockerfile on PR/push
• [ ] Add a validation script (e.g., scripts/validate_docker_builds.sh) that attempts to build each example's Dockerfile
• [ ] Verify compose.yml files have valid syntax using docker-compose config in CI
• [ ] Test at least the simple_client example can successfully start and respond to basic requests
• [ ] Document Docker build requirements in docs/docker.md (currently exists but could detail CI approach)

Create type stubs and mypy validation for openpi-client

The openpi-client package lacks type hints in several core modules (base_policy.py, action_chunk_broker.py, image_tools.py), making it harder for downstream users to get IDE autocomplete and catch type errors. This is especially important for a robotics inference library where correctness is critical.

• [ ] Add type hints to packages/openpi-client/src/openpi_client/base_policy.py (input/output types for policies)
• [ ] Add type hints to packages/openpi-client/src/openpi_client/action_chunk_broker.py (buffer management types)
• [ ] Add type hints to packages/openpi-client/src/openpi_client/image_tools.py (image format conversions)
• [ ] Update packages/openpi-client/pyproject.toml to include mypy as a dev dependency with py.typed marker
• [ ] Add mypy check to .github/workflows/pre-commit.yml or create new workflow for type checking

• Add test coverage for examples/droid/compute_droid_nonidle_ranges.py—idle filtering is critical for training but has no visible unit tests in file structure.
• Document action/observation normalization workflow in a standalone guide (e.g., docs/normalization_tutorial.md) with concrete examples from examples/droid/ and examples/aloha_real/, since docs/norm_stats.md exists but lacks step-by-step integration instructions.
• Create a minimal end-to-end example for non-ALOHA/DROID robots (e.g., examples/custom_platform/) that generates dummy LeRobot-format data and runs fine-tuning with LoRA—currently only ALOHA, DROID, and Libero examples exist, blocking adoption for other platforms.

• High · Outdated NumPy Version with Known Vulnerabilities — examples/aloha_real/requirements.txt (and likely other requirements.txt files). The codebase uses numpy==1.24.4, which is outdated and may contain known security vulnerabilities. NumPy 1.24.x has reached end-of-life and security patches are no longer provided. Current secure versions should be 1.26.x or 2.0.x depending on compatibility requirements. Fix: Update numpy to a current stable version (>=1.26.0 or >=2.0.0). Run pip audit to identify specific CVEs and update all dependencies to patched versions.
• High · Outdated Pillow Version with Known Vulnerabilities — examples/aloha_real/requirements.txt. Pillow 10.4.0 is specified but may contain known image processing vulnerabilities. Recent versions (11.0.0+) contain important security patches for image parsing and buffer overflow issues. Fix: Update Pillow to version 11.0.0 or later to receive the latest security patches.
• Medium · OpenCV-Python Pinned to Potentially Vulnerable Version — examples/aloha_real/requirements.txt. opencv-python==4.10.0.84 is used. While relatively recent, this should be regularly audited for vulnerabilities as OpenCV is a complex C++ library with a history of security issues. Fix: Regularly scan for OpenCV CVEs using pip audit or safety check. Consider implementing automated dependency scanning in CI/CD pipeline.
• Medium · Outdated dm-control and Related Dependencies — examples/aloha_real/requirements.txt and examples/aloha_real/requirements.in. dm-control==1.0.23 (2024) and mujoco==3.2.3 are older versions. These complex simulation libraries may have unpatched vulnerabilities, particularly given their use in robotics contexts where security is important. Fix: Check for available updates to dm-control and mujoco. Implement automated dependency checking to identify when newer secure versions are released.
• Medium · Missing Security Headers in Docker Configurations — examples/aloha_real/Dockerfile, examples/aloha_sim/Dockerfile, examples/libero/Dockerfile, scripts/docker/serve_policy.Dockerfile. Multiple Dockerfiles present (examples/aloha_real/Dockerfile, examples/aloha_sim/Dockerfile, etc.) but no visible security hardening measures such as: running as non-root user, using minimal base images, or security scanning. Fix: Implement Docker security best practices: (1) Use specific base image tags instead of 'latest', (2) Add USER directive to run as non-root, (3) Use multi-stage builds to reduce image size, (4) Run 'docker scan' for vulnerabilities, (5) Implement image signing.
• Medium · No Visible Input Validation in Policy and Runtime Code — packages/openpi-client/src/openpi_client/websocket_client_policy.py, packages/openpi-client/src/openpi_client/runtime/. Files like packages/openpi-client/src/openpi_client/websocket_client_policy.py, runtime/agent.py, and runtime/environment.py handle external input (websocket messages, sensor data) without visible input validation or sanitization visible in file structure. Fix: Implement comprehensive input validation for all external inputs: (1) Validate websocket message structure and types, (2) Sanitize and validate sensor data ranges, (3) Implement bounds checking, (4) Use type hints and runtime type checking.
• Medium · Potential Unsafe Deserialization in msgpack_numpy — packages/openpi-client/src/openpi_client/msgpack_numpy.py. Custom msgpack serialization (packages/openpi-client/src/openpi_client/msgpack_numpy.py) may implement unsafe deserialization patterns. Deserializing untrusted msgpack data can lead to arbitrary code execution if not properly constrained. Fix: Review msgpack_numpy.py to ensure: (1) No pickle usage or similar unsafe serializers, (2) Explicit object type whitelisting during deserialization, (3) Size limits on des

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.