If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/sebastianstarke/AI4Animation shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

WAIT — Missing license — unclear to depend on

• Last commit 3w ago
• 6 active contributors
• Distributed ownership (top contributor 33% of recent commits)
• ⚠ No license — legally unclear to depend on
• ⚠ No CI workflows detected
• ⚠ No test directory detected

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live sebastianstarke/AI4Animation repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/sebastianstarke/AI4Animation.

What it runs against: a local clone of sebastianstarke/AI4Animation — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in sebastianstarke/AI4Animation | Confirms the artifact applies here, not a fork | | 2 | Default branch master exists | Catches branch renames | | 3 | 4 critical file paths still exist | Catches refactors that moved load-bearing code | | 4 | Last commit ≤ 52 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of sebastianstarke/AI4Animation. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/sebastianstarke/AI4Animation.git
#   cd AI4Animation
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of sebastianstarke/AI4Animation and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "sebastianstarke/AI4Animation(\\.git)?\\b" \\
  && ok "origin remote is sebastianstarke/AI4Animation" \\
  || miss "origin remote is not sebastianstarke/AI4Animation (artifact may be from a fork)"

# 3. Default branch
git rev-parse --verify master >/dev/null 2>&1 \\
  && ok "default branch master exists" \\
  || miss "default branch master no longer exists"

# 4. Critical files exist
test -f "AI4Animation/Plugins/Libraries/Eigen/Eigen" \\
  && ok "AI4Animation/Plugins/Libraries/Eigen/Eigen" \\
  || miss "missing critical file: AI4Animation/Plugins/Libraries/Eigen/Eigen"
test -f "AI4Animation/Plugins/Command.txt" \\
  && ok "AI4Animation/Plugins/Command.txt" \\
  || miss "missing critical file: AI4Animation/Plugins/Command.txt"
test -f ".gitignore" \\
  && ok ".gitignore" \\
  || miss "missing critical file: .gitignore"
test -f ".gitattributes" \\
  && ok ".gitattributes" \\
  || miss "missing critical file: .gitattributes"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 52 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~22d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/sebastianstarke/AI4Animation"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

AI4Animation is a comprehensive deep learning framework for data-driven character animation and control in Unity, enabling neural network-based character locomotion (biped/quadruped), scene interactions, and physics-based movement synthesis. It bridges motion capture data, PyTorch/TensorFlow training pipelines, and real-time runtime inference to automate character animation without hand-keyframing. Monorepo structure: AI4Animation/ contains Unity C# project; AI4Animation/Plugins/ houses C++ neural network inference engine and Eigen for matrix ops; AI4Animation/Plugins/Libraries/Eigen/ is vendored linear algebra library. Runtime architecture is ECS-style (Entity-Component-System) with motion data pipelines feeding neural network inference modules.

Game developers and animation researchers building intelligent NPCs or player characters; VR/AR avatar engineers needing embodied motion control; motion capture specialists processing mocap data into learnable animation policies; ML researchers exploring reinforcement learning and imitation learning for character control.

Actively developed with a 2026 Python remake (AI4AnimationPy) in progress at facebookresearch/ai4animationpy; original codebase (~18.8M lines C++, ~9.9M lines C#) shows sustained engineering effort. No clear CI/test suite visible in file list, and commit recency unknown from metadata alone. Verdict: production-capable but research-grade—mature enough for professional games/research but assumes deep technical literacy.

Heavy C++ dependency chain (Eigen linear algebra, likely CUDA/cuDNN for GPU inference) with no explicit dependency lock file visible; core C++ plugins must compile per-platform (Windows/Linux). Single maintainer (Sebastian Starke) concentration risk. Python remake is separate repo, signaling potential framework fragmentation. Breaking changes likely as deep learning practices evolve.

Active pivot to Python ecosystem—AI4AnimationPy remake removes Unity dependency for data processing/training while maintaining game-engine architecture (ECS, update loops). New demos show biped/quadruped locomotion, IK solving, motion editor, and mocap import (BVH/FBX/GLB/NPZ). Web demos hosted on Hugging Face Spaces.

git clone https://github.com/sebastianstarke/AI4Animation.git
cd AI4Animation
# For C++ plugins: requires CMake 3.x and C++17 compiler
mkdir build && cd build && cmake .. && cmake --build . --config Release
# For Unity project: open AI4Animation/ folder in Unity 2020.3+ Editor
# For Python remake (recommended): visit https://github.com/facebookresearch/ai4animationpy

Daily commands: No Makefile or package.json in file list (Unity project). For C++ plugins: cd AI4Animation/Plugins && cmake build && make. For Python remake: pip install -e . && python demos/biped_locomotion.py (from AI4AnimationPy repo). Runtime in Unity Editor: load scene and press Play.

• AI4Animation/Plugins/Libraries/Eigen/Eigen — Core linear algebra library header that must be understood for all numerical computations in the neural network training and character animation pipeline.
• AI4Animation/Plugins/Command.txt — Build and compilation commands that define how the C++ plugins are compiled and integrated with Unity, essential for setting up the development environment.
• .gitignore — Defines which build artifacts, compiled binaries, and large model files are excluded from version control—critical for understanding the build output structure.
• .gitattributes — Specifies line ending and binary file handling for cross-platform compatibility between Windows and Unix systems used in animation data processing.

• Eigen Linear Algebra Engine (C++, SIMD (AVX/AVX512)) — Performs all numerical computations: matrix multiplication, SVD decomposition, linear system solving, and constraint projection.
  • Failure mode: Numerical overflow/underflow in extreme joint angles or missing SIMD instruction set support causes fallback to slow scalar ops or NaN outputs.
• C++ Neural Network Plugin (C++, Eigen, Plugin API) — Implements neural network layer computations, model serialization/deserialization, and interfaces between Unity C# and Eigen C++.
  • Failure mode: Memory corruption from buffer overruns, platform-specific binary incompatibility, or out-of-sync model format versions crashes animation system.
• Unity Animation Renderer (C#, Animator, SkinnedMeshRenderer) — Applies neural network-predicted joint transforms to character skeleton and renders animations at target frame rate.
  • Failure mode: Plugin not loaded or returned invalid transforms cause frozen or glitchy character animations.
• Motion Capture Data Pipeline (Python, Eigen (via C++ plugin), NumPy) — Ingests BVH/FBX animation files, extracts features, normalizes for training, and generates labeled datasets.
  • Failure mode: Malformed mocap data or incorrect normalization causes network training divergence and poor runtime synthesis quality.

• Motion Capture Files → Data Processing Pipeline — Raw BVH/FBX files are parsed, skeleton hierarchies extracted, and joint rotations/positions normalized to zero-mean unit-variance features.
• Processed Motion Features → Neural Network Training — Normalized features fed as input/output pairs to deep networks; backpropagation updates weights to minimize motion prediction error.
• Trained Model Weights → C++ Plugin Runtime — Serialized model parameters loaded into memory at Unity startup; used for fast inference queries during gameplay.
• Runtime Character State → Neural Network Inference — Current joint angles, velocities, and control inputs (locomotion direction, speed) encoded as feature vector and fed to network.
• Network Predictions → Constraint Solver — Joint rotations and velocities output from network validated against physical constraints and blended with previous frame for temporal smoothness.
• Constrained Predictions → Unity Animator — Final joint transforms applied to character skeleton via Animator's SetBoneLocalRotation and IK systems, rendered each frame.

Add a New Neural Network Training Pipeline

1. Create new C++ neural network layer using Eigen matrix operations in a new plugin source file (AI4Animation/Plugins/Libraries/Eigen/src/Core/Product.h)
2. Add compilation commands for the new plugin in the build script (AI4Animation/Plugins/Command.txt)
3. Update .gitignore to exclude generated model weights and training checkpoints (.gitignore)

Integrate a New Character Animation Type

1. Create motion processing functions using Eigen SVD and matrix decompositions for feature extraction (AI4Animation/Plugins/Libraries/Eigen/src/SVD/SVD.h)
2. Add matrix-based transformations for skeleton joint computations (AI4Animation/Plugins/Libraries/Eigen/src/Core/Transpose.h)
3. Compile and link new plugin through the build command infrastructure (AI4Animation/Plugins/Command.txt)

Add Numerical Optimization for Motion Synthesis

1. Implement solver using Eigen's linear system solvers for optimization (AI4Animation/Plugins/Libraries/Eigen/src/Core/Solve.h)
2. Add support for solving triangular systems during backpropagation (AI4Animation/Plugins/Libraries/Eigen/src/Core/SolveTriangular.h)
3. Update build commands to include optimization-specific compilation flags (AI4Animation/Plugins/Command.txt)

• Eigen C++ Library — Provides highly optimized SIMD-accelerated linear algebra operations critical for real-time neural network inference and character skeletal transformations without external dependencies.
• C++ Plugins — Delivers native performance necessary for sub-16ms animation frame times required for smooth 60 FPS character control in Unity.
• Unity Integration — Enables visual authoring of animation rigs, scene design, and runtime debugging while leveraging native C++ for compute-intensive operations.

• C++ plugins over pure C# for neural network inference

  • Why: C# runtime would introduce unacceptable latency for real-time character animation at 60 FPS.
  • Consequence: Increased build complexity and platform-specific compilation requirements; developers must understand both C++ and C# layering.

• Eigen library for linear algebra vs. custom implementations

  • Why: Eigen provides production-grade optimizations, SIMD support, and mathematical correctness off-the-shelf.
  • Consequence: Large library footprint and external dependency; must maintain compatibility across Windows/Mac/Linux platforms.

• Data-driven deep learning approach vs. hand-crafted animation blending

  • Why: Neural networks enable more natural, generalized motion synthesis across diverse character types and interaction scenarios.
  • Consequence: Requires extensive training datasets and offline preprocessing; higher upfront computational cost before runtime deployment.

• This repository does not provide real-time network multiplayer synchronization—animations are local to each client.
• Does not implement full physics simulation; physics constraints are learned implicitly through neural network training data.
• Not a generic machine learning framework; specifically optimized for character animation and locomotion tasks.
• Does not handle procedural level generation or gameplay logic—focused exclusively on character control and motion synthesis.

• Eigen Template Instantiation in Headers (Medium) — AI4Animation/Plugins/Libraries/Eigen/src/Core/*.h: Heavy reliance on C++ template metaprogramming in header files can lead to massive compilation times and difficult-to-debug error messages; contributes to bloated object files.
• No Apparent Error Handling in Plugin Interface (High) — AI4Animation/Plugins/Command.txt: Build commands and plugin architecture lack visible error checking or validation; invalid neural network model files or corrupted weights could silently produce garbage outputs.
• Platform-Specific Binary Artifacts Not Versioned (Medium) — .gitignore: Compiled plugins (.dll/.so/.dylib) excluded from git means developers must rebuild locally; risk of subtle platform-specific bugs going undetected until runtime.

• AI4Animation/Plugins/Libraries/Eigen/src/Core/Product.h (undefined) — undefined

Platform-specific C++ compilation: Plugins must compile for target OS (Windows/Linux/macOS); Visual Studio or GCC with C++17 required. GPU inference: CUDA/cuDNN may be required for real-time performance; no CPU fallback documented. Unity version lock: Project likely targets Unity 2020.3+; older versions may have incompatible C# features. Eigen header-only: All Eigen instantiations happen at compile time; template errors are verbose. No pre-built binaries visible: Must compile plugins from source per platform.

• Physics-Based Character Control via Neural Networks — Core thesis of AI4Animation: replacing hand-authored animation state machines with learned policies trained on mocap; defines how characters move in real time.
• Inverse Kinematics (IK) Solving — Used to map high-level motion targets (e.g., 'reach for object') to joint angles; IK demos in AI4AnimationPy show this core capability.
• Motion Capture Data Processing & Feature Engineering — Raw mocap streams (BVH/FBX) must be normalized, featurized, and phase-labeled before training; pipeline shown in AI4AnimationPy demos.
• Entity-Component-System (ECS) Architecture — Runtime uses ECS pattern (entities = characters, components = motion controllers/IK solvers) for decoupled animation updates; core AI4AnimationPy design.
• Phase-Functioned Neural Networks — Specialized NN architecture (phase as latent variable) enables smooth gait transitions and phase-aware motion synthesis; underpins locomotion controllers.
• ONNX Model Export & Runtime Inference — PyTorch/TensorFlow models exported as ONNX for efficient C++ runtime inference in Unity; bridges training and real-time execution.
• Eigen C++ Linear Algebra Library — Vendored library used for all neural network matrix operations in C++ plugins; critical for performance-critical forward passes on CPU/GPU.

• facebookresearch/ai4animationpy — Official Python remake (2026) removing Unity dependency; same author, recommended path forward for new projects.
• google/motion-imitation — Parallel framework for learning locomotion policies via RL; uses PyBullet + TensorFlow, complementary approach to AI4Animation's imitation learning.
• deepmind/dm_control — DeepMind's continuous control suite for physics simulation; often paired with AI4Animation for RL-based policy learning.
• Unity-Technologies/Barracuda — Unity's native neural network inference library; lightweight alternative to custom C++ plugins for runtime animation inference.
• CMU-Perceptron/openpose — Motion capture preprocessing tool; widely used upstream of AI4Animation pipelines for human pose estimation and data capture.

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Create build documentation and CI workflow for Eigen library compilation

The repo includes a full Eigen library (AI4Animation/Plugins/Libraries/Eigen/) with CMakeLists.txt but no documentation on how to build or compile these dependencies. There's a Command.txt file suggesting build steps exist but aren't formalized. A new contributor could create a GitHub Actions workflow to automatically build and test Eigen dependencies, plus add BUILD.md documenting the compilation process for Windows/Linux/Mac targets.

• [ ] Create .github/workflows/eigen-build.yml to compile Eigen on multiple platforms
• [ ] Document build steps in AI4Animation/Plugins/README.md covering CMake configuration
• [ ] Add pre-built artifact caching to speed up subsequent builds
• [ ] Reference AI4Animation/Plugins/Command.txt in the documentation to clarify existing build commands

Add unit tests for Core neural network modules in AI4Animation/Assets

The repository is a deep learning framework for character animation but the file structure shows no visible test suite. Given the complexity of neural network inference and character control, adding unit tests for core modules (animation blending, motion prediction, control systems) would significantly improve maintainability and catch regressions. This is critical for a framework that others depend on.

• [ ] Explore AI4Animation/Assets structure to identify core neural network and control modules (likely C# scripts)
• [ ] Create Tests/ directory with test harness setup for Unity testing framework
• [ ] Add tests for motion prediction, IK solving, and animation blending logic
• [ ] Document test coverage in a TESTING.md file with examples of how to run tests locally

Create migration guide and examples from AI4Animation (C#/Unity) to AI4AnimationPy

The README clearly indicates a Python remake exists at facebookresearch/ai4animationpy (2026 update), but there's no bridging documentation for users of the original repo. A new contributor could create a comprehensive migration guide showing how to port character controllers, dataset loaders, and model architectures from the C#/Unity version to Python, lowering the barrier for adoption of the newer project.

• [ ] Create MIGRATION_TO_PYTHON.md documenting API mapping between C# modules and Python equivalents
• [ ] Add example scripts in /examples showing before/after code for common tasks (loading motion data, training networks, runtime inference)
• [ ] Document differences in dependency handling (Eigen/C++ in original vs NumPy/PyTorch in Python version)
• [ ] Link to specific Plugins/Libraries/Eigen usage patterns and their Python numpy/scipy alternatives

• Add unit tests for C++ neural inference plugins in AI4Animation/Plugins/ (currently no test directory visible; would enable safe refactoring of matrix operations).
• Document the C# → C++ interop layer for neural network calls with examples (P/Invoke or IL2CPP calling conventions for different platforms).
• Create a minimal runnable example showing biped locomotion in Vanilla Unity without proprietary assets; currently requires setup of motion capture data pipelines not visible in repo.

The AI4Animation repository appears to be a research/educational project with a reasonable baseline security posture. No critical vulnerabilities were identified from the visible file structure. However, there are gaps in dependency management visibility, security documentation, and input validation practices that should be addressed. The primary concerns are: (1) lack of dependency tracking and CVE monitoring for bundled third-party libraries like Eigen, (2) absence of security policies and disclosure guidelines, and (3) limited visibility into input handling and validation in the codebase. For a production deployment, especially in interactive/VR environments, stronger security hardening and documentation would be recommended.

• Medium · Third-party Library Dependencies Not Validated — AI4Animation/Plugins/Libraries/Eigen/. The repository includes Eigen library (linear algebra library) as a bundled dependency in AI4Animation/Plugins/Libraries/Eigen. Without a dependency manifest file (package.json, requirements.txt, etc.) provided, it is unclear if this library and its transitive dependencies are being kept up-to-date with security patches. The CMakeLists.txt indicates C++ build configuration, but no version pinning or security audit trail is visible. Fix: Maintain a DEPENDENCIES.md or use a package manager with lock files (vcpkg, conan) to track and version dependencies. Regularly audit Eigen and other libraries for CVEs using tools like OWASP Dependency-Check or Snyk.
• Low · Lack of Security Documentation — Repository root. No SECURITY.md file is present in the repository. There is no clear security policy, vulnerability disclosure process, or guidance for reporting security issues. The README does not mention security considerations for the AI/ML models being trained or deployed. Fix: Add a SECURITY.md file with vulnerability disclosure guidelines. Document security considerations for model training, data handling, and runtime deployment in Unity environments.
• Low · Potential Build Artifacts in Repository — .gitignore. The .gitignore file is present but its contents are not provided. Without visibility into what is being ignored, there is a risk that compiled binaries, build artifacts, or cached files with sensitive information could be committed. Fix: Ensure .gitignore properly excludes: compiled binaries (.dll, .so, .o), build directories (bin/, build/), IDE files (.vs/, .idea/), temporary files, and any API keys or credentials. Use a comprehensive C++/Unity .gitignore template.
• Low · No Input Validation Patterns Visible — AI4Animation/ (entire project structure). Based on the file structure, this is a machine learning and animation framework. Without access to actual implementation code, it is unclear if there are proper validations for model inputs, animation data loading, or user-provided parameters that could lead to unexpected behavior or crashes. Fix: Implement strict input validation for all model inputs, animation data files, and user-configurable parameters. Use type checking and bounds validation to prevent injection or malformed data attacks.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.