If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/realsenseai/librealsense shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

GO — Healthy across the board

• Last commit 1d ago
• 6 active contributors
• Apache-2.0 licensed
• CI configured
• Tests present
• ⚠ Concentrated ownership — top contributor handles 58% of recent commits

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live realsenseai/librealsense repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/realsenseai/librealsense.

What it runs against: a local clone of realsenseai/librealsense — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in realsenseai/librealsense | Confirms the artifact applies here, not a fork | | 2 | License is still Apache-2.0 | Catches relicense before you depend on it | | 3 | Default branch master exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 31 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of realsenseai/librealsense. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/realsenseai/librealsense.git
#   cd librealsense
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of realsenseai/librealsense and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "realsenseai/librealsense(\\.git)?\\b" \\
  && ok "origin remote is realsenseai/librealsense" \\
  || miss "origin remote is not realsenseai/librealsense (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(Apache-2\\.0)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"Apache-2\\.0\"" package.json 2>/dev/null) \\
  && ok "license is Apache-2.0" \\
  || miss "license drift — was Apache-2.0 at generation time"

# 3. Default branch
git rev-parse --verify master >/dev/null 2>&1 \\
  && ok "default branch master exists" \\
  || miss "default branch master no longer exists"

# 4. Critical files exist
test -f "CMakeLists.txt" \\
  && ok "CMakeLists.txt" \\
  || miss "missing critical file: CMakeLists.txt"
test -f "CMake/lrs_options.cmake" \\
  && ok "CMake/lrs_options.cmake" \\
  || miss "missing critical file: CMake/lrs_options.cmake"
test -f "common/device-model.h" \\
  && ok "common/device-model.h" \\
  || miss "missing critical file: common/device-model.h"
test -f "common/calibration-model.cpp" \\
  && ok "common/calibration-model.cpp" \\
  || miss "missing critical file: common/calibration-model.cpp"
test -f ".github/workflows/buildsCI.yaml" \\
  && ok ".github/workflows/buildsCI.yaml" \\
  || miss "missing critical file: .github/workflows/buildsCI.yaml"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 31 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~1d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/realsenseai/librealsense"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

RealSense SDK 2.0 is a cross-platform C++/C library that provides unified access to Intel RealSense depth cameras for 3D image capture, depth streaming, and calibration data retrieval. It abstracts hardware complexity across D400/L500 series cameras and multiple OS platforms (Linux, Windows, macOS, Android) through a consistent API, enabling applications to acquire synchronized color and depth frames with intrinsic/extrinsic camera parameters. Core monolith: src/ contains camera interface abstractions and sensor drivers; src/core/ likely houses main SDK logic; src/proc/ contains post-processing pipelines; wrappers/ provides language bindings (Python via pyrealsense2Config.cmake, C# via realsense2-netConfig.cmake). Visualization demos in examples/, CMake build system with modular external dependency handling (CMake/external_*.cmake), and platform-specific logic in CMake/unix_config.cmake and CMake/windows_config.cmake.

Computer vision engineers, roboticists, and 3D reconstruction developers who need production-grade depth camera access without writing low-level USB/V4L2 drivers. Also supports researchers prototyping SLAM, object detection, and AR/VR applications using RealSense hardware.

Highly mature: this is Intel's official production SDK with active GitHub CI/CD (buildsCI.yaml, ROS2 package workflows), extensive multi-language bindings (Python, C#, Java via pybind11), and beta releases pushed to master. Last commit policy shows ongoing development with over 13M lines of C++ code. Production-ready for enterprise depth sensing applications.

Low risk from maintenance perspective but moderate dependency surface: relies on external libs (libusb, libcurl, FastDDS, glfw3 for visualization) managed via CMake externals, and platform-specific USB drivers. The monolithic 13M+ LOC C++ codebase means non-trivial onboarding; however, strong CI coverage (cppcheck static analysis, memory leak detection in .github/workflows/) mitigates quality risk. Organization migration from IntelRealSense→realsenseai in flight may cause temporary URL/tooling friction.

Active beta release cycle with master branch receiving ongoing updates per new branch policy; ROS2 package integration being maintained; static analysis (cppcheck) and memory leak detection (valgrind) are continuous CI jobs. Codebase shows recent additions in CUDA support (41K lines—likely GPU-accelerated processing), Python bindings expansion, and cross-platform testing infrastructure.

git clone https://github.com/realsenseai/librealsense.git && cd librealsense && mkdir build && cd build && cmake .. && make -j$(nproc) && sudo make install. Requires CMake 3.1+, libusb-dev, and platform USB drivers (udev rules embedded via CMake/embedd_udev_rules.cmake).

Daily commands: Build: cmake -DCMAKE_BUILD_TYPE=Release .. && make && make install. Run Python examples: python3 examples/python/example.py (verify cameras detected with rs-enumerate-devices). C++ examples in examples/ compiled automatically; run build/examples/cpp-config-ui or similar.

• CMakeLists.txt — Root build configuration that orchestrates the entire SDK compilation, dependencies, and platform-specific logic across Windows, Linux, Android, and macOS.
• CMake/lrs_options.cmake — Defines all configurable build options (Python bindings, examples, tests, platform features) that control SDK scope and compilation behavior.
• common/device-model.h — Core device abstraction layer representing RealSense camera devices; essential for understanding device lifecycle and streaming abstractions.
• common/calibration-model.cpp — Implements intrinsic and extrinsic calibration data handling, critical for depth-to-color alignment and 3D reconstruction accuracy.
• .github/workflows/buildsCI.yaml — Continuous integration pipeline validating builds across multiple platforms and configurations; documents supported environments and test coverage.
• CMake/external_libusb.cmake — USB communication layer configuration for device connectivity; foundational for all RealSense hardware interaction.

Add support for a new RealSense device model

1. Create device-specific calibration class in common/ extending calibration-model.h (common/calibration-model.h)
2. Add device detection logic and model enum to device-model.h (common/device-model.h)
3. Implement on-chip calibration routines for the new device (e.g., d500-on-chip-calib.cpp as template) (common/d500-on-chip-calib.cpp)
4. Register new device in CMake configuration (CMake/lrs_options.cmake)
5. Add device-specific build rules and USB descriptor handling (CMake/global_config.cmake)

Add a new image processing filter

1. Define filter configuration struct and API in embedded-filter-model.h (common/embedded-filter-model.h)
2. Implement filter logic in embedded-filter-model.cpp (common/embedded-filter-model.cpp)
3. Integrate filter into device option system via option-model.cpp (common/option-model.cpp)
4. Add visualization support in model-views.cpp if UI preview needed (common/model-views.cpp)

Extend the build to support a new platform

1. Create platform-specific CMake config (e.g., CMake/android_config.cmake as template) (CMake/android_config.cmake)
2. Add platform conditionals and compiler flags to CMake/global_config.cmake (CMake/global_config.cmake)
3. Define platform-specific dependencies in CMake/external_*.cmake files (CMake/external_libusb.cmake)
4. Add platform-specific CI workflow (.github/workflows/*.yaml) (.github/workflows/buildsCI.yaml)
5. Document platform requirements in CONTRIBUTING.md (CONTRIBUTING.md)

Add a new diagnostic or visualization tool

1. Create measurement/metric classes in common/measurement.h or common/graph-model.h (common/measurement.h)
2. Implement data collection logic using device-model APIs (common/device-model.cpp)
3. Add OpenGL rendering routines in opengl3.cpp for visualization (common/opengl3.cpp)
4. Expose tool via CLI argument parser in common/cli.h (common/cli.h)

• CMake — Cross-platform build abstraction enabling seamless compilation on Windows, Linux, macOS, Android, and embedded systems with conditional feature toggles.
• LibUSB — Hardware abstraction for USB communication with RealSense depth camera sensors; enables portable driver-level access without OS-specific USB APIs.
• OpenGL 3+ — GPU-accelerated rendering for real-time depth/color visualization and on-GPU filter processing to reduce CPU bottlenecks.
• Python bindings (pybind11) — Bridges C++ core library to Python for rapid prototyping, ML integration, and ease-of-use for data science workflows.
• DDS (Data Distribution Service) — Middleware for networked camera streaming and multi-device synchronization in distributed robotics/vision systems.

• Calibration data baked into device firmware vs. external calibration files

  • Why: Device-internal calibration (on-chip) ensures consistency and prevents miscalibration; external files add flexibility but risk cache invalidation.
  • Consequence: On-chip calibration requires firmware updates for recalibration, but eliminates user configuration errors.

• Embedded (on-device) filters vs. post-processing in application

  • Why: On-device filters leverage firmware/ASIC compute, reducing CPU/GPU load and USB bandwidth; post-processing allows custom filters but increases latency.
  • Consequence: Limited filter set constrained by device hardware, but throughput and power efficiency optimized for real-time use.

• Synchronous streaming API with blocking reads vs. asynchronous callbacks

  • Why: Blocking API simpler for single-threaded applications; callbacks enable high-frequency event-driven pipelines.
  • Consequence: Device-model exposes both paradigms, but blocking reads risk frame drops under heavy CPU load.

• Intrinsic/extrinsic calibration matrix format (fixed 4x4 homogeneous transforms)

  • Why: Industry-standard format ensures compatibility with OpenCV, ROS, and 3D vision libraries; simplifies depth-to-color alignment.
  • Consequence: Less flexible than per-pixel distortion models, but sufficient for most depth cameras and computationally efficient.

• Does not handle authentication or multi-user device management.
• Does not provide real-time OS

USB device permissions: on Linux, udev rules (embedded via CMake/embedd_udev_rules.cmake) must be installed for non-root camera access—omission causes silent permission failures. Frame callback timing: synchronization between depth/color streams is implicit; timestamp drift can occur without explicit frame alignment (use align() post-processor). CMake configuration: MSVC on Windows requires specific libusb static build flags (CMake/windows_config.cmake); mismatch causes link errors. Python 3 only: wrappers/ targets Python 3.6+; Python 2 bindings are deprecated.

• Depth camera intrinsics and extrinsics — Core to RealSense SDK—intrinsics (focal length, principal point, distortion) transform pixel coordinates to 3D; extrinsics define spatial relationship between RGB and depth sensors. Misunderstanding calibration causes incorrect 3D reconstruction.
• Synchronized frame streaming (color + depth alignment) — RealSense cameras capture color and depth asynchronously; SDK provides frame synchronization and spatial alignment (warp depth to RGB space or vice versa) via align() post-processor. Critical for multi-modal vision apps.
• USB video class (UVC) and isochronous transfers — RealSense cameras communicate via USB UVC protocol with isochronous transfers for real-time video; SDK abstracts this via libusb backend. Understanding UVC bandwidth negotiation helps debug frame drops.
• Post-processing pipelines (depth filters) — Raw sensor depth is noisy; SDK provides temporal, spatial, and morphological filters (src/proc/) applied on-GPU or CPU. Filter chains compound latency—design choice matters for real-time systems.
• Memory-mapped frame buffers and zero-copy callbacks — SDK uses frame queues with callback registration (observer pattern) to avoid double-buffering copies; frames are mutable until callback returns. Incorrect callback timing causes frame corruption.
• Cross-platform USB backend abstraction — Windows (WinUSB), Linux (libusb/V4L2), and macOS (IOKit) have different USB APIs; SDK encapsulates via CMake conditionals and src/backends/. New platform support requires new backend implementation.
• pybind11 type conversion and GIL management — Python bindings in wrappers/ use pybind11 to expose C++ objects; frame callbacks must release Python GIL for non-blocking C++ I/O. Improper GIL handling deadlocks streaming.

• IntelRealSense/librealsense_old — Deprecated v1.x SDK—migration target docs point here for legacy code compatibility patterns
• OpenKinect/libfreenect2 — Alternative open-source depth camera SDK; similar USB/V4L2 abstraction patterns for comparison
• IntelRealSense/librealsense-ros — Official ROS/ROS2 wrapper (referenced in .github/workflows/build-ROS2-package-CI.yaml)—essential for robotics integrations
• IntelRealSense/librealsense-cmake-module — Standalone CMake find_package(realsense2) helper—used by downstream projects to locate SDK without custom detection
• opencv/opencv — Optional downstream dependency for CV pipelines; librealsense frames often fed directly to cv::Mat for processing

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add comprehensive unit tests for CMake configuration modules

The CMake directory contains 30+ configuration files (external_*.cmake, *_config.cmake) that handle critical build dependencies like libusb, cuda, opengl, and fastdds. These files lack dedicated unit tests to verify cross-platform configuration logic, especially for edge cases like missing dependencies or version mismatches. This would catch build configuration regressions early.

• [ ] Create .github/workflows/cmake-validation.yaml to run CMake syntax checks and configuration dry-runs
• [ ] Add CMake test suite in CMake/tests/ directory with test cases for each external dependency (libusb_config, cuda_config, opengl_config, etc.)
• [ ] Test scenarios: missing libraries, version conflicts, platform-specific paths (Windows/Linux/macOS), and optional feature flags
• [ ] Reference: CMake/external_libusb.cmake, CMake/windows_config.cmake, CMake/unix_config.cmake as primary test targets

Add memory leak detection CI workflow with baseline tracking

The repo has a memory-leaks-check directory with test infrastructure (.github/workflows/memory-leaks-check/) but no active GitHub Actions workflow to run these checks on PR submissions. Creating a scheduled workflow with baseline comparison would catch memory regressions before merge, critical for a hardware SDK handling streaming data.

• [ ] Create .github/workflows/memory-leaks-detection.yaml that runs on PRs and scheduled basis (e.g., daily)
• [ ] Integrate existing .github/workflows/memory-leaks-check/mem-leak-test.cpp with valgrind/ASAN tooling
• [ ] Store baseline memory metrics in repo and compare against PR changes, failing if delta exceeds threshold
• [ ] Add artifact uploads for detailed leak reports and integration with pull_request_template.md to guide contributors

Create Python binding tests for pyrealsense2 with cross-platform coverage

The CMake/external_pybind11.cmake and CMake/pyrealsense2Config.cmake.in files indicate Python bindings exist, but there's no dedicated Python CI workflow visible in the workflows directory. Add pytest-based integration tests to verify Python API functionality across Linux/Windows/macOS, ensuring parity with C++ SDK features.

• [ ] Create .github/workflows/python-bindings-test.yaml to test pyrealsense2 on multiple Python versions (3.8, 3.9, 3.10, 3.11)
• [ ] Add tests/python/ directory with pytest suite covering: camera enumeration, depth/color streaming, intrinsic/extrinsic calibration data retrieval
• [ ] Reference common/calibration-model.h and common/device-model.h to ensure Python bindings expose critical device features
• [ ] Include platform matrix (Ubuntu, Windows, macOS) to catch platform-specific pybind11 issues early

• Add missing unit tests for src/proc/colorizer.cpp color mapping edge cases (NaN depth values, out-of-range confidence scores)—look at test/unit/ for Catch2 test patterns.
• Document D415 vs D435 camera capability matrix in doc/camera_matrix.md (resolution, FPS limits, RGB vs IR sensor differences)—requires reading CMake sensor detection logic in src/backends/
• Implement missing sensor enumeration for OAK-D Pro thermal (thermal + stereo fusion)—add sensor_type enum in src/types.h and USB endpoint discovery in src/backends/usb/device-hid.cpp.

The RealSense SDK codebase demonstrates moderate security posture. Key concerns include: (1) External dependency management lacks checksum verification, (2) Memory safety in C++ code requires enhanced runtime protections, (3) Firmware update mechanisms need cryptographic validation, (4) DDS integration security should be explicitly configured. The project has good practices with static analysis (cppcheck) and memory leak checks in CI/CD, but should enhance runtime security validation, implement stricter dependency management, and document security-critical components. No hardcoded secrets or SQL injection risks detected. Recommend focusing on dependency verification, memory safety hardening, and firmware security validation.

• Medium · External Dependencies Management — CMake/external_*.cmake files (especially external_libcurl.cmake, external_libusb.cmake, external_fastdds.cmake). Multiple external dependencies are fetched via CMake (libusb, libcurl, pybind11, fastdds, foonathan_memory). The CMake files use download mechanisms that may not validate checksums or signatures, potentially exposing the build to man-in-the-middle attacks or compromised dependencies. Fix: Implement checksum verification (SHA256) for all downloaded dependencies. Use git submodules or lock files for version pinning. Consider using vcpkg or Conan package managers with verified binary caches.
• Medium · Potential Memory Safety Issues — common/*.cpp, src/ (C++ implementation files). The codebase includes C++ code with memory operations and pointer handling. The memory-leaks-check workflow exists but static analysis via cppcheck may not catch all memory safety issues, particularly in camera interface code handling raw data buffers. Fix: Enable compiler-level protections: AddressSanitizer, MemorySanitizer in CI/CD. Use smart pointers (unique_ptr, shared_ptr) throughout. Consider fuzzing critical data parsing paths. Review camera buffer handling code for buffer overflows.
• Medium · Firmware Update Mechanism — common/fw-update-helper.cpp, common/fw-update-helper.h. The fw-update-helper component suggests firmware updating capabilities. Without evidence of signature verification or secure update mechanisms, this could allow installation of unsigned/malicious firmware. Fix: Implement cryptographic signature verification for all firmware updates. Use secure boot mechanisms. Validate firmware before writing to device. Document the firmware update security model.
• Low · Build Configuration Exposure — .github/workflows/cppcheck_run.log, cppcheck_run.parsed.log. GitHub Actions workflows are stored in the repository. While this is standard, the cppcheck configuration and parsed logs may contain sensitive information about code vulnerabilities. Fix: Exclude detailed logs from repository commits. Store CI/CD artifacts in secure build systems. Use GitHub Secrets for sensitive configurations. Consider using private build logs.
• Low · DDS Integration Security — common/dds-model.cpp, common/dds-model.h, CMake/external_fastdds.cmake. The codebase includes DDS (Data Distribution Service) integration via fastdds. DDS configurations may have security implications if not properly configured, including authentication and encryption between components. Fix: Review DDS configuration for authentication and encryption settings. Implement DDS security policies (access control, encryption). Document DDS security requirements. Test against unauthorized access scenarios.
• Low · Third-party Python Binding — CMake/external_pybind11.cmake. pybind11 is used for Python bindings, which exposes C++ internals to Python. Improper bindings could lead to type confusion or unsafe memory access from Python code. Fix: Validate all pybind11 bindings for type safety. Avoid exposing raw pointers. Use py::capsule for opaque data. Add input validation on Python-facing functions. Test with fuzzing.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.