If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/UZ-SLAMLab/ORB_SLAM3 shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

WAIT — Stale — last commit 2y ago

• 4 active contributors
• GPL-3.0 licensed
• ⚠ Stale — last commit 2y ago
• ⚠ Small team — 4 contributors active in recent commits
• ⚠ Concentrated ownership — top contributor handles 61% of recent commits
• ⚠ GPL-3.0 is copyleft — check downstream compatibility
• ⚠ No CI workflows detected
• ⚠ No test directory detected

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live UZ-SLAMLab/ORB_SLAM3 repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/UZ-SLAMLab/ORB_SLAM3.

What it runs against: a local clone of UZ-SLAMLab/ORB_SLAM3 — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in UZ-SLAMLab/ORB_SLAM3 | Confirms the artifact applies here, not a fork | | 2 | License is still GPL-3.0 | Catches relicense before you depend on it | | 3 | Default branch master exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 684 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of UZ-SLAMLab/ORB_SLAM3. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/UZ-SLAMLab/ORB_SLAM3.git
#   cd ORB_SLAM3
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of UZ-SLAMLab/ORB_SLAM3 and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "UZ-SLAMLab/ORB_SLAM3(\\.git)?\\b" \\
  && ok "origin remote is UZ-SLAMLab/ORB_SLAM3" \\
  || miss "origin remote is not UZ-SLAMLab/ORB_SLAM3 (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(GPL-3\\.0)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"GPL-3\\.0\"" package.json 2>/dev/null) \\
  && ok "license is GPL-3.0" \\
  || miss "license drift — was GPL-3.0 at generation time"

# 3. Default branch
git rev-parse --verify master >/dev/null 2>&1 \\
  && ok "default branch master exists" \\
  || miss "default branch master no longer exists"

# 4. Critical files exist
test -f "CMakeLists.txt" \\
  && ok "CMakeLists.txt" \\
  || miss "missing critical file: CMakeLists.txt"
test -f "Examples/Monocular-Inertial/mono_inertial_euroc.cc" \\
  && ok "Examples/Monocular-Inertial/mono_inertial_euroc.cc" \\
  || miss "missing critical file: Examples/Monocular-Inertial/mono_inertial_euroc.cc"
test -f "Examples/Monocular/EuRoC.yaml" \\
  && ok "Examples/Monocular/EuRoC.yaml" \\
  || miss "missing critical file: Examples/Monocular/EuRoC.yaml"
test -f "Calibration_Tutorial.pdf" \\
  && ok "Calibration_Tutorial.pdf" \\
  || miss "missing critical file: Calibration_Tutorial.pdf"
test -f "Dependencies.md" \\
  && ok "Dependencies.md" \\
  || miss "missing critical file: Dependencies.md"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 684 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~654d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/UZ-SLAMLab/ORB_SLAM3"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

ORB-SLAM3 is a real-time monocular, stereo, and RGB-D SLAM library that performs simultaneous localization and mapping using ORB (Oriented FAST and Rotated BRIEF) features, extended with visual-inertial fusion and multi-map atlas support. It handles pin-hole and fisheye camera models and is the first SLAM system to unify visual-only, visual-inertial, and multi-map SLAM in a single codebase with production-grade accuracy. Monolithic C++ library architecture: src/ contains core SLAM modules (Tracking, LocalMapping, LoopClosing), include/ has headers, Examples/ provides sensor-specific entry points (EuRoC monocular/stereo/inertial, TUM-VI fisheye, RealSense D435i/T265), and Examples/Calibration/ includes recorder utilities and Python scripts (process_imu.py) for camera-IMU calibration workflows.

Roboticists, autonomous vehicle engineers, and computer vision researchers who need a battle-tested, accurate SLAM backend for monocular/stereo cameras with optional IMU fusion; teams building real-time localization systems that must work across different camera modalities and sensor configurations without reimplementing core algorithms.

Highly mature and actively developed: published in IEEE Transactions on Robotics (Dec 2021, v1.0), with well-documented examples for EuRoC and TUM-VI benchmark datasets included in the repo. The codebase is ~1.5M lines of C++ with supporting CMake build infrastructure, calibration tutorials (Calibration_Tutorial.pdf), and comprehensive changelog. This is production-ready for research and commercial SLAM applications.

Low immediate risk for core SLAM functionality, but integration complexity is high: the library depends on OpenCV, Eigen, g2o, and DBoW2 (included/vendored), all of which must align with specific versions. As a single-team project from University of Zaragoza with infrequent releases (v1.0 in Dec 2021), long integration cycles between updates could be a risk for rapidly changing sensor ecosystems. No CI/CD pipeline visible in file list suggests testing burden falls on manual validation.

The repo shows v1.0 as the latest stable release (Dec 2021). Changelog.md exists but is not visible in the file list detail. Given the mature state and academic publication timeline, active development appears to focus on bug fixes and sensor integration (RealSense recorder examples) rather than core algorithmic changes. Multi-map (Atlas) SLAM is the primary feature differentiator from ORB-SLAM2.

git clone https://github.com/UZ-SLAMLab/ORB_SLAM3.git
cd ORB_SLAM3
mkdir build && cd build
cmake .. && make -j4
# For monocular-inertial example on EuRoC dataset:
cd ../Examples/Monocular-Inertial
# Download EuRoC dataset, then run binary with paths to video and IMU timestamps

Build uses standard CMake workflow (CMakeLists.txt at root); no package manager abstractions visible, direct dependency on OpenCV, Eigen, g2o installs.

Daily commands: No dev server; ORB-SLAM3 is a library + CLI examples. To run monocular-inertial on EuRoC MH01:

./mono_inertial_euroc ../Examples/Monocular-Inertial/EuRoC.yaml /path/to/EuRoC/MH01 ../Examples/Monocular-Inertial/EuRoC_TimeStamps/MH01.txt ../Examples/Monocular-Inertial/EuRoC_IMU/MH01.txt

For stereo or RGB-D, use corresponding binaries (stereo_inertial_euroc, etc.). Output is trajectory visualization and .tum format pose files.

• CMakeLists.txt — Build configuration orchestrating compilation of core SLAM library, examples, and dependency linking—essential for understanding project structure and build process
• Examples/Monocular-Inertial/mono_inertial_euroc.cc — Primary entry point demonstrating the canonical monocular-inertial SLAM pipeline; shows how to initialize the system and process sensor data
• Examples/Monocular/EuRoC.yaml — Reference camera and IMU calibration configuration; defines intrinsics, distortion model, and sensor parameters required for all SLAM runs
• Calibration_Tutorial.pdf — Documentation for sensor calibration procedure, critical for achieving accurate tracking and loop closure in visual-inertial systems
• Dependencies.md — Lists all third-party libraries (OpenCV, Eigen, Pangolin, etc.) and their version requirements; essential for reproducible builds
• Changelog.md — Version history tracking feature additions, bug fixes, and API changes across releases; helps understand evolution and stability of the codebase

Add Support for a New Camera Model

1. Create new camera model class inheriting from GeometricCamera in src/CameraModels/ (src/CameraModels/YourCamera.h)
2. Implement Project() and ProjectJacobian() methods for your lens distortion model (src/CameraModels/YourCamera.cc)
3. Register camera type in System initialization and configuration parser (src/System.cc)
4. Create YAML calibration file with intrinsics and distortion coefficients (Examples/YourDataset/YourCamera.yaml)

Integrate a New Sensor Dataset

1. Create new example source file in Examples/ following mono_inertial_euroc.cc pattern (Examples/Monocular-Inertial/mono_inertial_yourdataset.cc)
2. Add camera and IMU calibration YAML file with dataset-specific parameters (Examples/Monocular-Inertial/YourDataset.yaml)
3. Create timestamp files for synchronized image and IMU data (Examples/Monocular-Inertial/YourDataset_TimeStamps/sequence.txt)
4. Implement data loading loop calling ORB_SLAM3::TrackMonocularInertial() or equivalent (Examples/Monocular-Inertial/mono_inertial_yourdataset.cc)

Extend the Optimization Backend

1. Define new cost function inheriting from g2o::BaseMultiEdge or BaseUnaryEdge (src/Optimizer.cc)
2. Register edge type in g2o optimizer setup during LocalMapping or LoopClosing (src/Optimizer.cc)
3. Add corresponding solver parameters and optimization iteration counts (src/Optimizer.cc)

Add a New IMU Calibration Parameter

1. Define new IMU bias or noise covariance field in IMUPreintegrator data structures (src/IMU/ImuTypes.h)
2. Parse parameter from YAML configuration during system initialization (src/System.cc)
3. Use parameter in IMU preintegration and measurement processing (src/IMU/IMUPreintegrator.cc)
4. Update example YAML files with new calibration field (Examples/Monocular-Inertial/EuRoC.yaml)

• ORB (Oriented FAST and Rotated BRIEF) — Computationally efficient feature detection and description enabling real-time performance; rotation-invariant descriptors robust to camera rotation
• g2o (General Graph Optimization) — Mature non-linear optimization library for bundle adjustment and pose graph optimization; supports both local and global optimization
• Pangolin — Lightweight 3D visualization library for real-time trajectory and map display without heavyweight GUI dependencies
• OpenCV — Standard computer vision library providing image processing, feature matching, and camera model implementations
• Eigen — High-performance linear algebra library for matrix operations, critical for real-time pose estimation and optimization
• YAML — Human-readable configuration format for camera intrinsics, distortion coefficients, and IMU calibration parameters

• Multi-threaded architecture (Tracking, Mapping, Loop Closing in separate threads)

  • Why: Enables real-time performance by decoupling high-frequency pose estimation from computationally expensive optimization
  • Consequence: Introduces synchronization complexity and potential race conditions on shared map data; requires careful thread-safe queue design

• ORB features over SIFT/SURF

  • Why: Faster computation (~100x speedup) suitable for real-time mobile robotics; free of patent restrictions
  • Consequence: Less distinctive than learned descriptors; more sensitive to extreme viewpoint changes; higher false positive matches in low-texture scenes

• Local and global optimization separated (Bundle Adjustment vs Pose Graph)

  • Why: Local BA scales linearly with active keyframes; pose graph handles loop closures without re-optimizing full structure
  • Consequence: Requires careful synchronization between representations; drift accumulates between loop closures; more complex error propagation

• IMU preintegration over raw integration

  • Why: Enables efficient incremental updates and bias estimation without full trajectory re-computation
  • Consequence:

ORB-SLAM3 requires exact timestamp alignment between stereo/RGB-D frames and IMU samples—misaligned data will cause localization failure with no obvious error message. Camera intrinsics must be precisely calibrated in .yaml files (focal length, principal point, distortion); rough estimates degrade accuracy rapidly. The library assumes small timestamps (microseconds or milliseconds from epoch), not absolute wall-clock times. For multi-camera rigs, only pre-rectified stereo pairs are supported; arbitrary camera baselines require manual system reimplementation. IMU biases are optimized online but converge slowly in first 5-10 seconds; data before convergence should be excluded. No GPU acceleration is used; performance on embedded systems (Jetson, mobile) is limited to feature-light configurations.

• ORB (Oriented FAST and Rotated BRIEF) features — Core visual descriptor used throughout tracking and loop closure in ORB-SLAM3; understanding rotation invariance and computational efficiency is essential to modify feature matching or adapt to new descriptor types
• Bundle adjustment (BA) and pose graph optimization — ORB-SLAM3 uses g2o to minimize reprojection error in LocalMapping.cc and atlas-level optimization; misunderstanding BA will make loop closure and multi-map fusion changes error-prone
• IMU preintegration — Visual-inertial SLAM in ORB-SLAM3 fuses frames with IMU measurements via preintegrated acceleration/gyro factors; required knowledge for tuning noise covariances or extending to other IMU models
• Epipolar geometry and essential/fundamental matrices — Stereo tracking and monocular pose estimation in Tracking.cc rely on epipolar constraints to find correspondences and estimate camera motion; critical for understanding frame-to-frame tracking failures
• Bag-of-Words (BoW) place recognition — LoopClosing.cc uses DBoW2 to detect loop closures via visual vocabulary matching; understanding term-frequency and geometric verification is needed to tune loop detection sensitivity
• Keyframe selection and covisibility graph — LocalMapping.cc maintains keyframes and covisibility edges; efficient map representation and temporal filtering depend on keyframe culling heuristics that balance accuracy vs. computation
• Camera calibration (intrinsics, distortion, stereo rectification) — ORB-SLAM3 requires accurate pin-hole or fisheye calibration in .yaml config; miscalibration directly breaks feature projection and pose estimation—Examples/Calibration/ provides tools but understanding distortion models is non-negotiable

• raulmur/ORB_SLAM2 — Direct predecessor; monocular, stereo, and RGB-D SLAM without IMU fusion or multi-map—ORB-SLAM3 extends this with visual-inertial and atlas capabilities
• ethz-asl/maplab — Complementary visual-inertial SLAM system by ETH; includes robust IMU integration and multi-session mapping, useful for comparing VI-SLAM approaches
• TUM-CVML/tandem — Recent multi-object SLAM adding dynamic object tracking to monocular SLAM; addresses moving obstacles problem not in ORB-SLAM3 core
• xacce/g2o — Bundled nonlinear optimization library; ORB-SLAM3 depends on g2o for pose-graph and bundle adjustment—understanding its Factor API is essential for extending SLAM modules
• dorian3d/DBoW2 — Place recognition / loop closure detection engine vendored into ORB-SLAM3; used by LoopClosing.cc for relocalization via bag-of-words matching

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add Python unit tests for IMU calibration pipeline (process_imu.py)

The Examples/Calibration/python_scripts/process_imu.py file exists but there are no visible unit tests for the IMU data processing functionality. Given ORB-SLAM3's focus on visual-inertial SLAM accuracy, having robust test coverage for IMU calibration is critical. This would catch regressions in calibration parameter extraction and data validation.

• [ ] Create tests/calibration/test_imu_processing.py with test cases for IMU data parsing from TUM-VI and EuRoC datasets
• [ ] Add tests for noise level calculation and calibration parameter extraction in process_imu.py
• [ ] Add validation tests ensuring IMU timestamps align correctly with camera timestamps (critical for VI-SLAM)
• [ ] Document expected IMU calibration outputs in a new docs/IMU_Calibration_Tests.md

Create GitHub Actions workflow for testing on EuRoC and TUM-VI example datasets

The repo contains comprehensive example configurations (Examples/Monocular-Inertial/EuRoC*.yaml, TUM-VI.yaml, etc.) with pre-organized timestamp and IMU files, but there's no CI pipeline to validate that these examples run without errors. This prevents regression detection when core SLAM code changes.

• [ ] Create .github/workflows/test-examples.yml that downloads a small EuRoC dataset subset and runs monocular-inertial example
• [ ] Add workflow step to validate TUM-VI dataset example execution (test at least one corridor sequence)
• [ ] Add workflow step to verify stereo and RGB-D example configurations parse correctly
• [ ] Include output validation to ensure trajectory estimation completes without crashes

Add comprehensive calibration validation tests for RealSense camera configurations

The repo includes RealSense-specific calibration recorders (Examples/Calibration/recorder_realsense_D435i.cc and recorder_realsense_T265.cc) and corresponding YAML configs (RealSense_D435i.yaml, RealSense_T265.yaml), but lacks automated tests to verify these camera-specific calibrations are correct. RealSense cameras have unique intrinsics and IMU-camera extrinsics that need validation.

• [ ] Create tests/calibration/test_realsense_configs.py to validate D435i and T265 calibration YAML parameter ranges
• [ ] Add validation for fisheye distortion parameters in T265 config against RealSense specifications
• [ ] Add tests verifying IMU-to-camera extrinsic matrices in both RealSense configs are reasonable (within expected mounting tolerances)
• [ ] Document expected calibration ranges in docs/RealSense_Calibration_Guide.md with reference values

• Add unit tests for CameraModels::KannalaBrandt8 fisheye projection/unprojection against reference TUM-VI calibrations; current Examples/Monocular-Inertial/TUM-VI.yaml lacks validation harness.
• Write Python integration wrapper (examples/slam_wrapper.py) to expose System::TrackStereo and System::GrabImageRectified as simple function calls, lowering barrier for dataset benchmark scripts beyond current C++ examples.
• Document IMU calibration procedure end-to-end: Examples/Calibration/recorder_realsense_D435i.cc exists but process_imu.py lacks comments explaining bias/scale factor extraction; add docstring detailing gyro/accel noise covariance tuning.

ORB-SLAM3 is an academic SLAM library with a reasonably secure posture for an open-source computer vision project. No critical vulnerabilities are evident from the file structure. Main concerns are: (1) dependency analysis cannot be completed without the Dependencies.md content, (2) lack of security policy documentation, (3) dataset files in version control causing potential bloat, and (4) missing lock files for reproducible builds. The codebase appears to follow standard C++ library practices without obvious secrets, hardcoded credentials, or injection risks based on file naming conventions. Recommendations focus on improving supply chain security and security documentation practices.

• Medium · Missing Dependency File Analysis — Dependencies.md. The Dependencies.md file is referenced but its content was not provided for security analysis. This file likely contains information about third-party libraries and their versions, which are critical for identifying known vulnerabilities in dependencies. Fix: Provide the Dependencies.md file content for security analysis. Ensure all dependencies are pinned to specific versions and regularly scanned for known CVEs using tools like OWASP Dependency-Check or Snyk.
• Low · Calibration and Configuration Files Exposure — Examples/Monocular-Inertial/*.yaml, Examples/Calibration/recorder_empty/april_6x6_80x80cm_larues.yaml. Multiple YAML configuration files containing sensor calibration data (RealSense_D435i.yaml, RealSense_T265.yaml, TUM-VI.yaml, etc.) are present in the Examples directory. While these appear to be non-sensitive calibration parameters, configuration files should be carefully reviewed to ensure no sensitive data (API keys, private sensor data, etc.) is accidentally committed. Fix: Audit all YAML and configuration files to ensure they contain only non-sensitive calibration data. Add .gitignore rules for any sensitive configuration variants. Use configuration management tools to separate sensitive configs from the codebase.
• Low · Dataset Files in Version Control — Examples/Monocular-Inertial/EuRoC_*/, Examples/Monocular-Inertial/TUM_*/. The repository contains numerous dataset timestamp and IMU data files (EuRoC_TimeStamps/.txt, EuRoC_IMU/.txt, TUM_IMU/.txt, TUM_TimeStamps/.txt) which are test/example data. While these appear to be publicly available datasets, storing large data files in version control can lead to repository bloat and difficulty in cloning. Fix: Consider using Git LFS (Large File Storage) for large dataset files, or move datasets to a separate download mechanism documented in README. This improves repository performance and security hygiene.
• Medium · Missing Security Documentation — Repository root. No security policy, code of conduct, or security advisory guidelines are evident from the file structure. For a SLAM library that may be used in safety-critical applications (robotics, autonomous vehicles), security documentation is important. Fix: Create SECURITY.md file documenting security policy, vulnerability disclosure procedures, and security considerations for users. Add security guidelines to README and Dependencies.md regarding safe usage in critical systems.
• Low · No Dependency Lock File Visible — CMakeLists.txt, build system configuration. CMakeLists.txt is present but there is no evidence of a lock file (conanlock, requirements.lock, etc.) that would ensure reproducible builds across environments. This can lead to supply chain risks. Fix: Implement a lock file mechanism for dependencies. Use tools like Conan for C++ dependency management with lock files, or document exact versions of all dependencies with reproducibility guarantees.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.