If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/AlexsJones/llmfit shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

GO — Healthy across the board

• Last commit 1d ago
• 21+ active contributors
• Distributed ownership (top contributor 46% of recent commits)
• MIT licensed
• CI configured
• Tests present

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live AlexsJones/llmfit repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/AlexsJones/llmfit.

What it runs against: a local clone of AlexsJones/llmfit — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in AlexsJones/llmfit | Confirms the artifact applies here, not a fork | | 2 | License is still MIT | Catches relicense before you depend on it | | 3 | Default branch main exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 31 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of AlexsJones/llmfit. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/AlexsJones/llmfit.git
#   cd llmfit
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of AlexsJones/llmfit and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "AlexsJones/llmfit(\\.git)?\\b" \\
  && ok "origin remote is AlexsJones/llmfit" \\
  || miss "origin remote is not AlexsJones/llmfit (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(MIT)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"MIT\"" package.json 2>/dev/null) \\
  && ok "license is MIT" \\
  || miss "license drift — was MIT at generation time"

# 3. Default branch
git rev-parse --verify main >/dev/null 2>&1 \\
  && ok "default branch main exists" \\
  || miss "default branch main no longer exists"

# 4. Critical files exist
test -f "llmfit-core/src/lib.rs" \\
  && ok "llmfit-core/src/lib.rs" \\
  || miss "missing critical file: llmfit-core/src/lib.rs"
test -f "llmfit-core/src/fit.rs" \\
  && ok "llmfit-core/src/fit.rs" \\
  || miss "missing critical file: llmfit-core/src/fit.rs"
test -f "llmfit-core/src/hardware.rs" \\
  && ok "llmfit-core/src/hardware.rs" \\
  || miss "missing critical file: llmfit-core/src/hardware.rs"
test -f "llmfit-core/src/models.rs" \\
  && ok "llmfit-core/src/models.rs" \\
  || miss "missing critical file: llmfit-core/src/models.rs"
test -f "llmfit-tui/src/main.rs" \\
  && ok "llmfit-tui/src/main.rs" \\
  || miss "missing critical file: llmfit-tui/src/main.rs"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 31 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~1d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/AlexsJones/llmfit"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

llmfit is a terminal tool that detects your hardware (RAM, CPU, GPU) and recommends which LLM models will actually run well on your machine. It scores hundreds of models from multiple providers (Ollama, llama.cpp, MLX, Docker, LM Studio) across quality, speed, fit, and context dimensions, then surfaces real-world performance benchmarks via community data from localmaxxing.com. Ships as an interactive TUI by default with CLI fallback, supporting multi-GPU setups, MoE architectures, and dynamic quantization. Workspace monorepo: llmfit-core/src/ contains the engine (lib.rs, hardware.rs, models.rs, providers.rs, benchmarks.rs, fit.rs); llmfit-tui wraps it with terminal UI; llmfit-desktop builds a native app. Core logic lives in llmfit-core/src/{fit.rs, plan.rs, quality.rs} for scoring; data/ holds model manifests and baseline benchmarks as JSON/YAML.

ML engineers and hobbyists running local LLMs who need to right-size model choices to specific hardware without trial-and-error. Users with RTX cards, Apple Silicon, consumer GPUs, or heterogeneous setups who want empirical tok/s and VRAM data before downloading multi-GB models.

Actively developed with a v0.9.22 release tag (see Cargo.toml), comprehensive CI pipeline (.github/workflows with ci.yml, docker.yml, release workflows), and structured benchmarking infrastructure (llmfit-core/data with baselines.json and docker_models.json). Appears production-ready for local LLM discovery; community benchmarks feature is recently added.

Single maintainer (AlexsJones) visible in repo name; monorepo spans three workspace members (llmfit-core, llmfit-tui, llmfit-desktop) with significant Rust complexity (~1.2M LOC). No visible open issue backlog or recent security advisory workflow, but desktop build (Cargo.toml in llmfit-desktop) adds platform-specific risk. Heavy reliance on external model registries (hf_models.json, docker_models.json) that require regular curation.

Recent additions: community benchmarks feature (visible in README as 'New: Community Benchmarks'), download manager (D), advanced configuration (A), hardware simulation (S). Desktop and TUI binaries are active release targets (release-desktop.yml, release-please.yml in workflows). Nix flake support added (flake.nix, flake.lock).

git clone https://github.com/AlexsJones/llmfit.git
cd llmfit
cargo build --release
./target/release/llmfit

Or install from crates.io: cargo install llmfit. For desktop app, run cargo build -p llmfit-desktop --release.

Daily commands: TUI (default): cargo run -p llmfit-core or cargo run -p llmfit-tui. CLI mode: check --help flag for arguments. Desktop: cargo run -p llmfit-desktop. Makefile present (Makefile at root) may contain shortcuts — check make help or make run.

• llmfit-core/src/lib.rs — Entry point for core library; exports public API and orchestrates hardware detection, model matching, and benchmarking.
• llmfit-core/src/fit.rs — Core algorithm that matches models to hardware constraints; must be understood before adding new filtering logic.
• llmfit-core/src/hardware.rs — Hardware capability detection; critical for accurate model-to-device matching and performance predictions.
• llmfit-core/src/models.rs — Model metadata parsing and management; defines how model specs are loaded and exposed to the CLI.
• llmfit-tui/src/main.rs — CLI entry point; orchestrates TUI initialization, event loop, and integration with core library.
• llmfit-core/data/benchmarks.yaml — Benchmark baseline data that calibrates model runtime predictions; critical for accuracy.
• Cargo.toml — Workspace configuration; defines members, shared version, and dependency resolution strategy.

Add support for a new hardware device or constraint

1. Define detection logic for the new hardware type (GPU, NPU, etc.) (llmfit-core/src/hardware.rs)
2. Add baseline performance metrics for the new device to calibrate predictions (llmfit-core/data/baselines.json)
3. Update the fit algorithm to score models against the new constraint (llmfit-core/src/fit.rs)
4. Test detection and scoring with an integration test (llmfit-tui/tests/cli_smoke.rs)

Add a new model provider (beyond Hugging Face)

1. Define provider metadata and model fetching logic (llmfit-core/src/providers.rs)
2. Create model registry JSON in the data directory following the hf_models.json schema (llmfit-core/data/docker_models.json)
3. Update the models parser to load the new provider's models (llmfit-core/src/models.rs)
4. Expose the new provider through the public API in lib.rs (llmfit-core/src/lib.rs)

Add a new UI control or filter option

1. Define the filter parameter struct and validation logic (llmfit-tui/src/filter_config.rs)
2. Integrate the filter into the core matching algorithm (llmfit-core/src/fit.rs)
3. Add TUI component for the new control (input field, toggle, etc.) (llmfit-tui/src/tui_ui.rs)
4. For web UI, add React component to FilterBar (llmfit-web/src/components/FilterBar.jsx)

Improve model runtime prediction accuracy

1. Collect real-world benchmark data from users (llmfit-tui/src/download_history.rs)
2. Add new benchmark scenarios to the test suite (llmfit-core/src/benchmarks.rs)
3. Refine prediction algorithm with improved baselines (llmfit-core/data/benchmarks.yaml)
4. Update scoring logic to incorporate new data points (llmfit-core/src/quality.rs)

• Rust — Memory-safe systems programming; minimal overhead for hardware detection and algorithm execution; cross-platform CLI distribution
• Tauri (Desktop) & React (Web) — Leverage existing frontend ecosystem for reach; Tauri bundles Rust backend with native-looking desktop UI
• JSON/YAML data files — Versioned, human-readable model registry and benchmarks; decoupled from code; supports community contributions
• Workspace Cargo project — Shared core logic (llmfit-core) reused by CLI, Desktop, and Web; unified dependency management

• Model data stored as static JSON/YAML files rather than a database

  • Why: Simplicity, offline availability, version control; enables community PRs for model registry updates
  • Consequence: Requires rebuild/redistribution to add new models; does not support dynamic model queries or real-time updates

• Benchmarks pre-computed and cached locally rather than queried on-demand

  • Why: Instant predictions without external API calls; works offline; predictable latency
  • Consequence: Baseline data can become stale; users with novel hardware configurations may see inaccurate estimates until benchmarks are re-run

• Single-pass matching algorithm (no iterative refinement)

  • Why: Fast response (sub-second); simple to reason about and extend
  • Consequence: Cannot adapt predictions mid-session if user feedback contradicts baseline; no A/B testing of ranking strategies

• TUI as primary CLI interface over simple argument-based tool

  • Why: Rich interactivity, real-time filtering, better UX for exploration
  • Consequence: Harder to compose with shell pipes/scripting; larger compiled binary

• Does not fine-tune or train models; purely a discovery and prediction tool
• Does not handle model download/execution; focuses on matching and ranking only
• Does not support real-time hardware monitoring (static snapshot at launch)
• Not a replacement for official model benchmarks; uses public baselines as approximations
• Does not authenticate users or manage permissions; assumes single-user or trusted environment
• Not a model marketplace; does not price, license, or provision models

Data freshness: hf_models.json and docker_models.json in llmfit-core/data/ are static manifests requiring manual curation — outdated entries will mislead users. Community benchmark sync: localmaxxing.com integration (README mentions it) is external; network failures or API changes will break the b feature. GPU detection: hardware.rs likely has platform-specific code paths (Linux NVIDIA vs macOS vs Windows); untested GPU configurations may silently fall back to CPU-only estimates. Quantization assumptions: quality.rs and fit.rs likely assume standard GGUF or specific quantization formats; custom or future formats will break scoring. Nix flake: flake.nix present but not core to development; Nix users may hit stale lock (flake.lock) issues.

• Model Quantization & GGUF Format — fit.rs and quality.rs scoring depend on understanding quantization levels (Q4, Q5, Q8 etc.) to predict VRAM usage and inference speed; misunderstanding quantization breaks all hardware-model matching
• Time-to-First-Token (TTFT) vs Throughput (tok/s) — benchmarks.rs and quality.rs distinguish between latency and sustained throughput; llmfit recommends models differently for chat (TTFT-sensitive) vs batch (throughput-sensitive) workloads
• Mixture-of-Experts (MoE) Routing — README mentions 'MoE architectures' support; MoE models like Mixtral conditionally activate parameters, requiring fit.rs to estimate active vs total parameter VRAM differently than dense models
• Multi-GPU Memory Management & Tensor Parallelism — README states 'multi-GPU setups' support; hardware.rs must detect and aggregate VRAM across GPUs, and fit.rs must account for whether a model fits in single-GPU or requires sharding
• Provider Abstraction & Runtime Polymorphism — providers.rs abstracts 5+ runtime backends (Ollama, llama.cpp, MLX, Docker, LM Studio); adding a new provider or supporting version-specific API changes requires understanding the trait-based design
• Hardware Profiling & System Introspection — hardware.rs must reliably detect GPU type, VRAM, CPU architecture, and RAM across Linux/macOS/Windows; incorrect detection ruins all downstream recommendations
• Score Aggregation & Pareto Frontier — fit.rs likely ranks models across 4 dimensions (quality, speed, fit, context); the Advanced Configuration (A) lets users tune weights, implying multi-objective optimization and possibly Pareto frontier selection

• lm-sys/llm — Rust-based LLM inference library; could replace or complement provider.rs abstractions for local model serving
• ggerganov/llama.cpp — C++ inference engine that llmfit-core/src/providers.rs likely wraps; community updates there drive model compatibility changes
• ollama/ollama — Primary runtime provider supported by llmfit; users will install Ollama to actually run recommended models
• AlexsJones/llmserve — Sibling project mentioned in README; TUI for serving local LLM models — complementary to llmfit's discovery role
• mlc-ai/mlc-llm — Alternative local inference framework (MLC support visible in providers.rs); competes with llama.cpp for backend choice

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add integration tests for llmfit-core hardware detection across platforms

The repo has llmfit-core/src/hardware.rs but no corresponding test file or CI workflow that validates hardware detection on multiple platforms (Linux, macOS, Windows). The CI pipeline (.github/workflows/ci.yml) likely doesn't test hardware-specific logic comprehensively. This is critical since the core value proposition is 'find what runs on your hardware'—untested hardware detection is a liability.

• [ ] Create llmfit-core/src/hardware.rs.tests or llmfit-core/tests/hardware_integration_test.rs
• [ ] Add tests for CPU detection (cores, architecture), memory availability, and GPU detection (if applicable)
• [ ] Update .github/workflows/ci.yml to run hardware tests on ubuntu-latest, macos-latest, and windows-latest matrices
• [ ] Document expected behavior in tests for common hardware scenarios (e.g., 4-core CPU, 8GB RAM)

Add Python bindings CI workflow and packaging for llmfit-python

The workspace includes llmfit-python directory with a Makefile but there's no dedicated GitHub Actions workflow for building, testing, or publishing Python wheels. The main CI likely only tests Rust artifacts. Python users need automated validation that the Python package builds correctly and passes tests across Python versions (3.8+).

• [ ] Create .github/workflows/python-ci.yml that runs on pushes to Python-related paths
• [ ] Configure matrix testing for Python 3.8, 3.9, 3.10, 3.11, 3.12 on ubuntu-latest
• [ ] Add steps to build wheels using maturin or equivalent, run llmfit-python tests via the Makefile
• [ ] Add optional step to publish wheels to PyPI on release tags (reference existing release.yml patterns)

Add comprehensive API documentation tests for API.md endpoints

The repo has API.md documenting endpoints but no validation that examples in the documentation are accurate or tested. The llmfit-core/src/lib.rs likely exports the public API, but there's no CI job ensuring documentation examples compile/run. This leads to stale docs and user frustration.

• [ ] Create llmfit-core/tests/api_doc_examples.rs that extracts and validates code examples from API.md
• [ ] Add a CI step in .github/workflows/ci.yml that runs cargo test --test api_doc_examples
• [ ] Ensure all function signatures and example code blocks in API.md are validated against the actual exported API in llmfit-core/src/lib.rs
• [ ] Document the pattern in CONTRIBUTING.md so future API changes require doc example updates

• Add test coverage for llmfit-core/src/quality.rs scoring logic: currently no visible test files in llmfit-core/src/; write unit tests for quality score calculations (VRAM fitting, TTFT estimation, tok/s projection) to catch regressions as weights change in Advanced Configuration.
• Extend llmfit-core/data/benchmarks.yaml with community-contributed benchmark schemas: the feature is new (README mentions 'New: Community Benchmarks'); document the YAML schema for submitting real-world measurements and add validation in src/benchmarks.rs to parse and de-duplicate submissions.
• Add provider detection tests for edge cases: llmfit-core/src/providers.rs abstracts Ollama, llama.cpp, MLX, Docker, and LM Studio; write integration tests that verify each provider's version detection and fallback behavior when a provider is installed but misconfigured.

The codebase demonstrates moderate security practices with a multi-stage Docker build and non-root user execution. However, there is a critical issue with the incomplete USER directive that prevents proper privilege separation. Additional concerns include lack of container health checks, missing build-time secret handling, and no binary integrity verification. The Rust workspace structure itself appears sound with no obvious hardcoded credentials detected in the file structure. Recommend prioritizing the Docker configuration fix and implementing security best practices for container runtime security.

• High · Incomplete Docker User Configuration — Dockerfile, line 25. The Dockerfile creates a non-root user 'llmfit' but the USER directive is incomplete/truncated ('USER llmf' instead of 'USER llmfit'). This may cause the container to run as root or fail, creating a privilege escalation risk. Fix: Correct the USER directive to 'USER llmfit' to ensure the container runs with proper non-root privileges.
• Medium · Missing HEALTHCHECK in Dockerfile — Dockerfile. The Dockerfile does not include a HEALTHCHECK instruction for the runtime stage. This makes it difficult to monitor container health and can lead to deployments of unhealthy containers. Fix: Add a HEALTHCHECK instruction to verify the binary is functional, e.g., 'HEALTHCHECK CMD llmfit --version || exit 1'
• Medium · Hardcoded Binary Path and Ownership — Dockerfile, lines 19-21. The binary is copied to a hardcoded path '/usr/local/bin/llmfit' with explicit ownership to user 'llmfit'. If the binary gets overwritten or permissions changed, there could be security implications. No integrity verification (checksums) is performed. Fix: Consider adding checksum verification after COPY and implement immutable file permissions. Use '--chown' flag in COPY directive for clarity.
• Medium · Build-stage Secrets Exposure Risk — Dockerfile, lines 13-15. The builder stage copies the entire workspace including potential sensitive files (config files, private keys, etc.). While cleaned in this case, cargo build could still access sensitive environment variables or files during compilation. Fix: Use BuildKit secrets with '--mount=type=secret' for sensitive build-time data. Consider using .dockerignore to explicitly exclude sensitive files.
• Low · Missing APT Cache Cleanup in Builder — Dockerfile, lines 9-11. While apt cache is cleaned in the runtime stage, the builder stage doesn't clean up after installing build dependencies, creating unnecessary layer bloat (minor concern in multi-stage builds but still a best practice). Fix: Add '&& rm -rf /var/lib/apt/lists/*' after apt-get install in the builder stage for consistency and minimal image layers.
• Low · Missing WORKDIR Cleanup — Dockerfile, line 13. The builder stage leaves build artifacts in /build. While they don't persist in the final image due to multi-stage builds, explicit cleanup is a good practice. Fix: Document that multi-stage builds handle cleanup, or add explicit cleanup steps for clarity.
• Low · Deprecated Debian Base Image — Dockerfile, line 17. While debian:bookworm-slim is relatively recent, Debian-based images have a larger attack surface compared to distroless or alpine alternatives. Fix: Consider using 'debian:bookworm-slim' with security updates regularly, or evaluate 'distroless' images for reduced attack surface.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.