Onboarding: AUTOMATIC1111/stable-diffusion-webui

Generated by RepoPilot · 2026-05-05 · Source

Verdict

WAIT — Mixed signals — read the receipts

• Last commit 2mo ago
• 5 active contributors
• AGPL-3.0 licensed
• CI configured
• Tests present
• ⚠ Small team — 5 top contributors
• ⚠ Concentrated ownership — top contributor handles 62% of commits
• ⚠ AGPL-3.0 is copyleft — check downstream compatibility

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

TL;DR

A Gradio-based Python web UI for running Stable Diffusion image generation models locally. It wraps the core diffusion pipeline with a full-featured browser interface supporting txt2img, img2img, inpainting, outpainting, LoRA/Textual Inversion, upscaling (ESRGAN/LDSR/SwinIR), and X/Y/Z parameter plotting — solving the problem of needing a usable GUI without writing any code. Monolithic Python app with a strong extension system: core logic lives in modules/ (not shown but implied by the scale), built-in extensions under extensions-builtin/ (Lora/, LDSR/) follow a scripts/ + preload.py pattern, and the frontend is a mix of Gradio-generated HTML plus custom JavaScript in a JS-heavy static layer. The Lora extension alone spans 12+ files covering network types (network_hada.py, network_lokr.py, network_oft.py, etc.).

Who it's for

AI artists, researchers, and hobbyists who want to run Stable Diffusion locally with GPU acceleration and need fine-grained control over sampling, prompt weighting, LoRA networks, and post-processing — without writing Python code. Extension developers who build on top of the extension API also contribute regularly.

Maturity & risk

Extremely mature and widely deployed — it has one of the largest user bases of any open-source AI art tool, a structured CI pipeline under .github/workflows/ (run_tests.yaml, on_pull_request.yaml), a detailed CHANGELOG.md, and CITATION.cff indicating academic usage. It is actively maintained and effectively production-ready for local inference use cases.

Single primary maintainer (AUTOMATIC1111) creates bus-factor risk, though community contributions are heavy. The project depends on the volatile PyTorch/diffusers/xformers ecosystem where upstream breaking changes (e.g. new SD model architectures like SD3 requiring configs/sd3-inference.yaml) frequently require rapid patches. VRAM requirements, CUDA version pinning, and model weight licensing are non-trivial operational concerns.

Active areas of work

Active work visible includes SD3 support (configs/sd3-inference.yaml present), OFT (Orthogonal Fine-Tuning) network type added at extensions-builtin/Lora/network_oft.py, and ongoing linting enforcement via .eslintrc.js and .pylintrc. The .git-blame-ignore-revs file indicates recent large-scale reformatting commits.

Get running

git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git cd stable-diffusion-webui

On Linux/macOS:

bash webui.sh

On Windows:

webui-user.bat

First run auto-installs Python deps via pip into a venv and downloads required model files.

Daily commands:

Linux/macOS

bash webui.sh

Windows

webui-user.bat

With custom flags (e.g. lower VRAM):

Edit webui-user.bat and set COMMANDLINE_ARGS=--medvram

JS linting only:

npm run lint

Map of the codebase

• extensions-builtin/Lora/networks.py — Core LoRA network loading and patching logic; any contributor working with model weight modifications or extra network support must understand this file.
• extensions-builtin/Lora/lora_patches.py — Implements monkey-patching of the diffusion model's forward pass to inject LoRA weights at inference time.
• extensions-builtin/Lora/network.py — Defines the base Network and NetworkModule abstractions that all LoRA variant types (LoCon, LoHa, etc.) inherit from.
• extensions-builtin/Lora/lora_script.py — Entry-point script that registers LoRA as a Gradio UI extension and wires up the extra-networks UI callbacks.
• extensions-builtin/Lora/extra_networks_lora.py — Implements the ExtraNetworkLora class that hooks into the webui's extra-networks subsystem for LoRA activation from prompts.
• extensions-builtin/hypertile/hypertile.py — Core hypertile implementation that patches UNet attention layers for tiled image generation; critical for memory-efficient large image generation.
• extensions-builtin/soft-inpainting/scripts/soft_inpainting.py — Implements soft-inpainting blend scheduling; central to understanding how the inpainting pipeline deviates from vanilla img2img.

How to make changes

Add a new LoRA decomposition algorithm

1. Create a new module file following the NetworkModule base class interface; implement create_module(), finalize_updown(), and calc_updown() methods. (extensions-builtin/Lora/network_lora.py)
2. Register the new module type in the module-type dispatch table so networks.py can instantiate it by name from the checkpoint keys. (extensions-builtin/Lora/networks.py)
3. If the algorithm requires helper math (e.g. Tucker decomposition), add utilities here. (extensions-builtin/Lora/lyco_helpers.py)

Add a new model-based upscaler

1. Add model architecture class (nn.Module) or import it from an existing library. (extensions-builtin/LDSR/ldsr_model_arch.py)
2. Create a new script file that subclasses the webui's Upscaler base class, implements load_model() and do_upscale(), and calls script_callbacks to register the upscaler. (extensions-builtin/SwinIR/scripts/swinir_model.py)
3. Add a preload.py to declare any model download paths or command-line arguments needed before the UI starts. (extensions-builtin/SwinIR/preload.py)

Add a new image post-processing step for training data

1. Create a new script file subclassing ScriptPostprocessing; implement process() and ui() methods following the existing postprocessing script conventions. (extensions-builtin/postprocessing-for-training/scripts/postprocessing_focal_crop.py)
2. Place the file in the same directory so the extension loader auto-discovers it as a postprocessing script. (extensions-builtin/postprocessing-for-training/scripts/postprocessing_autosized_crop.py)

Add a new built-in UI extension with settings

1. Create the extension directory and a scripts/ subdirectory; write a script that uses script_callbacks.on_ui_settings() to register new options. (extensions-builtin/canvas-zoom-and-pan/scripts/hotkey_config.py)
2. Add any required JavaScript for client-side behaviour in a javascript/ subdirectory — the webui auto-injects all .js files from extension directories. (extensions-builtin/canvas-zoom-and-pan/javascript/zoom.js)
3. Add a style.css for any custom styling; it will also be auto-injected. (extensions-builtin/canvas-zoom-and-pan/style.css)

Traps & gotchas

1. The launch scripts (webui.sh / webui-user.bat) auto-install into a local venv — do NOT pre-install deps manually or version conflicts will occur. 2) CUDA version must match your PyTorch build; mismatches produce cryptic 'CUDA error' failures. 3) The --xformers flag requires a separately compiled xformers wheel matching your exact PyTorch+CUDA version. 4) Model checkpoints must be placed in models/Stable-diffusion/ — the UI will silently show no models if the path is wrong. 5) Extensions loaded from extensions-builtin/ and user extensions/ can conflict; load order matters and is not guaranteed.

Architecture

Concepts to learn

• LoRA (Low-Rank Adaptation) — The entire extensions-builtin/Lora/ subsystem implements LoRA and its variants; understanding rank decomposition (W = W0 + BA) is essential to reading network_lora.py and lora_patches.py.
• HyperDreambooth / Hadamard Product Adapters (HaDa) — network_hada.py implements this LyCORIS variant where weight updates use Hadamard (element-wise) products of two low-rank matrices, a non-obvious generalization of standard LoRA.
• Orthogonal Fine-Tuning (OFT) — network_oft.py implements OFT, which constrains weight updates to the orthogonal group to preserve hyperspherical energy — a 2023 research technique just added to the codebase.
• Classifier-Free Guidance (CFG) — The cfg_scale slider exposed in the UI controls this core inference technique; understanding it explains why negative prompts work and why cfg > 15 causes artifacts.
• Latent Diffusion Models — SD operates in a compressed latent space via a VAE, not pixel space — this is why VRAM usage is tractable and why the LDSR extension (extensions-builtin/LDSR/) works in that same latent domain.
• Textual Inversion (Embedding) — The embeddings/ directory and UI support for TI embeddings lets users add new concepts to the model by learning new token vectors — no weight changes required, but the token injection mechanism in the prompt parser is non-obvious.
• Kronecker Product Decomposition (LoKr) — network_lokr.py implements LoKr, a LyCORIS variant using Kronecker products for more parameter-efficient fine-tuning than standard LoRA — requires understanding Kronecker products to debug weight loading issues.

Related repos

• comfyanonymous/ComfyUI — Node-graph-based alternative UI for Stable Diffusion targeting power users who prefer visual pipeline composition over form-based UI.
• invoke-ai/InvokeAI — Another full-featured Stable Diffusion web UI with a different UX philosophy and stronger focus on professional/studio workflows.
• KohyaSS/kohya_ss — The primary companion tool for training the LoRA/LyCORIS weights that this repo loads and applies via extensions-builtin/Lora/.
• huggingface/diffusers — Upstream library providing many of the diffusion pipeline primitives and model architectures that this webui wraps and extends.
• CompVis/stable-diffusion — Original Stable Diffusion codebase that this project forked and heavily extended to add the web UI layer.

PR ideas

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Split extensions-builtin/Lora/networks.py into focused modules

The file extensions-builtin/Lora/networks.py is a large monolithic file that handles LoRA network loading, patching, and application logic. Given the repo already has separate files like network.py, network_lora.py, network_hada.py, network_lokr.py etc. for individual network types, networks.py likely contains mixed responsibilities (registry, loading, hooks, weight application). Splitting it into networks_loader.py, networks_hooks.py, and networks_registry.py would improve maintainability and make it easier for contributors to work on specific subsystems without merge conflicts.

• [ ] Read extensions-builtin/Lora/networks.py in full and categorize functions by responsibility (loading, hook registration, weight patching, UI callbacks)
• [ ] Create extensions-builtin/Lora/networks_loader.py for model loading and file discovery logic
• [ ] Create extensions-builtin/Lora/networks_hooks.py for forward-hook and model-patch registration (referencing existing lora_patches.py for context)
• [ ] Update extensions-builtin/Lora/scripts/lora_script.py and extensions-builtin/Lora/extra_networks_lora.py imports to reference the new modules
• [ ] Verify no circular imports are introduced between network.py, networks.py, and lora_patches.py
• [ ] Run existing test suite (referenced in .github/workflows/run_tests.yaml) to confirm no regressions

Add unit tests for LoRA network module variants (network_hada.py, network_lokr.py, network_oft.py)

The repo has a CI workflow at .github/workflows/run_tests.yaml but the extensions-builtin/Lora directory contains multiple network algorithm implementations (HaDa, LoKr, OFT, GLoRA, IA3, full, norm) with no visible corresponding test files in the file listing. These weight-decomposition algorithms are mathematically distinct and bugs in them silently produce wrong images. Adding focused unit tests with small synthetic weight tensors would catch regressions during refactors and give contributors confidence when adding new network types.

• [ ] Create tests/test_lora_networks.py (or tests/extensions/test_lora_networks.py matching the existing test layout inferred from run_tests.yaml)
• [ ] Write a test for extensions-builtin/Lora/network_hada.py: construct a minimal HaDa weight dict and assert the reconstructed weight tensor matches a hand-computed expected value
• [ ] Write a test for extensions-builtin/Lora/network_lokr.py: verify Kronecker product decomposition and reconstruction round-trips correctly
• [ ] Write a test for extensions-builtin/Lora/network_oft.py: verify the orthogonal fine-tuning matrix stays orthogonal after apply_weights
• [ ] Write a test for extensions-builtin/Lora/lyco_helpers.py utility functions (matrix factorization helpers) in isolation
• [ ] Ensure tests are discovered by the existing pytest config referenced in .github/workflows/run_tests.yaml

Add a dedicated GitHub Actions workflow for JavaScript/ESLint with per-PR annotations

The repo has an eslint config (.eslintrc.js, .eslintignore) and npm scripts ('lint', 'fix' in package.json), and a zoom.js and other JS files in extensions-builtin. However, the existing workflows (on_pull_request.yaml, run_tests.yaml, warns_merge_master.yml) appear focused on Python. There is no workflow that runs ESLint on pull requests and posts inline code annotations. Adding one would surface JS style and correctness issues directly in PR review, consistent with the Python linting already enforced via .pylintrc.

• [ ] Create .github/workflows/lint_js.yaml triggered on pull_request targeting relevant JS paths (javascript/**, extensions-builtin

Good first issues

1. Add unit tests for extensions-builtin/Lora/network_hada.py and network_lokr.py — the math for Hadamard and Kronecker product weight decomposition is untested per the visible test structure. 2) Add JSDoc comments to the JavaScript files (175k+ JS with ESLint but no documentation tooling configured) — start with the most-used UI interaction handlers. 3) Write a contributor guide specifically for adding new LoRA network types, documenting the interface defined in network.py that network_lora.py, network_oft.py, etc. must implement — currently there is no such guide.

Top contributors

• @AUTOMATIC1111 — 58 commits
• @w-e-w — 20 commits
• @v0xie — 8 commits
• @light-and-ray — 6 commits
• @AndreyRGW — 2 commits

Recent commits

• 82a973c — changelog (AUTOMATIC1111)
• 1d7e9ec — Merge pull request #16275 from AUTOMATIC1111/fix-image-upscale-on-cpu (AUTOMATIC1111)
• c19d044 — Merge branch 'release_candidate' (AUTOMATIC1111)
• 8b3d98c — update CHANGELOG (AUTOMATIC1111)
• 5bbbda4 — Merge branch 'dev' into release_candidate (AUTOMATIC1111)
• 9f5a98d — Merge pull request #16166 from richardtallent/dev (AUTOMATIC1111)
• 986c31d — Merge pull request #16180 from light-and-ray/do_not_send_image_size_on_paste_inpaint (AUTOMATIC1111)
• 5096c16 — Merge pull request #16173 from AUTOMATIC1111/robust-sysinfo (AUTOMATIC1111)
• 7b99e14 — Merge pull request #16194 from light-and-ray/fix_cannot_write_mode_P_as_jpeg (AUTOMATIC1111)
• 7c8a4cc — Merge pull request #16202 from light-and-ray/do_not_break_progressbar_on_non-job_actions (AUTOMATIC1111)

Security observations

• High · Outdated ESLint Dependency with Potential Supply Chain Risk — package.json. The project uses ESLint ^8.40.0 as a dev dependency. While not directly a runtime vulnerability, pinning to a caret range allows minor/patch updates that could introduce malicious code via supply chain attacks. ESLint 8.x also has known advisories in some sub-versions. Additionally, no lockfile enforcement or integrity verification is mentioned. Fix: Pin exact versions (remove ^ prefix), use npm audit regularly, and commit a package-lock.json or yarn.lock to enforce reproducible builds. Consider using npm ci in CI pipelines.
• High · Arbitrary Code Execution via Model Loading — extensions-builtin/LDSR/ldsr_model_arch.py, extensions-builtin/Lora/lora.py, extensions-builtin/ScuNET/scripts/scunet_model.py, extensions-builtin/SwinIR/scripts/swinir_model.py. Stable Diffusion web UIs commonly load model files (e.g., .ckpt, .pt, .safetensors) which can contain serialized Python objects (pickle). Loading untrusted .ckpt or .pt files can result in arbitrary code execution on the host machine. The file structure includes LoRA, LDSR, ScuNET, SwinIR model loaders (e.g., lora.py, ldsr_model_arch.py, scunet_model.py, swinir_model.py) which likely use torch.load() or similar deserialization. Fix: Always use safetensors format where possible. If loading .ckpt/.pt files, use torch.load() with weights_only=True (PyTorch >= 1.13). Warn users prominently about risks of loading untrusted model files. Consider sandboxing the model loading process.
• High · Potential Path Traversal in File Handling — extensions-builtin/postprocessing-for-training/scripts/, extensions-builtin/Lora/scripts/lora_script.py, embeddings/. The application handles user-supplied file paths for embeddings, extensions, models, and output images. Without proper path sanitization, an attacker could craft inputs using sequences like '../' to read or write files outside intended directories. Extension scripts (e.g., postprocessing scripts, LoRA scripts) that process user-supplied filenames are particularly at risk. Fix: Validate and sanitize all file paths using os.path.realpath() and ensure paths are within expected base directories before any file operations. Use allow-lists for file extensions.
• High · Extension System Allows Arbitrary Code Execution — extensions/, extensions-builtin/. The extensions system loads Python scripts from the extensions and extensions-builtin directories at runtime. If an attacker can place or modify files in these directories (e.g., via a compromised extension repository, a path traversal, or local access), they can achieve arbitrary code execution with the privileges of the web UI process. Fix: Implement code signing or checksum verification for extensions. Warn users strongly about installing extensions from untrusted sources. Consider sandboxing extension execution. Document the security implications clearly.
• High · Missing Authentication and Authorization — README.md, launch configuration. The web UI, by default, runs without authentication. If exposed on a network interface (not just localhost), any user can access all functionality including file uploads, model loading, script execution via extensions, and system configuration changes. The README and configuration suggest it can be launched and exposed publicly. Fix: Enable authentication by default or enforce it when binding to non-localhost interfaces. Implement proper access controls. Use --share flag with caution and ensure Gradio's authentication features are documented and encouraged.
• Medium · XSS Risk via JavaScript Extensions — extensions-builtin/canvas-zoom-and-pan/javascript/zoom.js, extensions-builtin/prompt-bracket-checker/javascript/prompt-bracket-checker.js, extensions-builtin/mobile/javascript/mobile.js. Multiple JavaScript files are loaded as extensions (zoom.js, mobile.js, prompt-bracket-checker.js). These scripts interact with the DOM and user inputs. If user-provided prompt text is rendered without proper sanitization in the UI (e.g., in canvas-zoom-and-pan or prompt-bracket-checker), it could lead to Cross-Site Scripting attacks. Fix: Ensure all user input rendered in the DOM uses text

LLM-derived; treat as a starting point, not a security audit.

Where to read next

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

---

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.