WAIT — Stale — last commit 2y ago

• 14 active contributors
• Distributed ownership (top contributor 40% of recent commits)
• MIT licensed
• ⚠ Stale — last commit 2y ago
• ⚠ No CI workflows detected
• ⚠ No test directory detected
• ⚠ Scorecard: marked unmaintained (0/10)
• ⚠ Scorecard: default branch unprotected (0/10)

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests + OpenSSF Scorecard}

This is the official PyTorch implementation of Swin Transformer, a hierarchical vision transformer architecture that uses shifted window attention to efficiently process image data. It serves as a foundational backbone for image classification, object detection, instance segmentation, video understanding, and masked image modeling tasks, with multiple variants (Swin v1/v2, SwinMLP, SwinMoE) optimized for different scales and domains. Single-repo structure with configs/ directory containing 50+ YAML files organized by model variant (swin/, swinv2/, simmim/, swinmoe/, swinmlp/). Core implementation in Python (~275K LOC) split between models/ directory (architecture definitions), data/ (loaders), and training scripts. CUDA kernels (9.7K LOC) optimize shifted window attention, with C++ wrapper code (3.7K LOC) for compute binding.

Computer vision researchers and ML engineers building state-of-the-art image understanding systems who need a production-grade, well-documented transformer backbone with pre-trained checkpoints for ImageNet/ImageNet-22K classification and downstream task fine-tuning (detection, segmentation, action recognition).

Highly mature and actively developed. This is a Microsoft official implementation with papers published in top venues (ICCV 2021, ICCV 2022 for v2), extensive pre-trained model checkpoints across configs/swin/, configs/swinv2/, configs/simmim/, and configs/swinmoe/ directories. Recent updates noted in README (December 2022, November 2022) indicate ongoing maintenance. Multiple companion repos for downstream tasks suggest established ecosystem adoption.

Low risk for core image classification tasks. The codebase is well-maintained by Microsoft with clear versioning (Swin v1, v2). Primary risks: (1) heavyweight dependencies on CUDA for optimal performance (9,693 lines CUDA code), requiring GPU access; (2) config complexity with 50+ YAML configurations across variants that can lead to reproducibility issues if hyperparameters are incorrectly specified; (3) downstream task implementations live in separate repos (object detection, segmentation), creating fragmentation.

Active development focused on: (1) SwinV2 improvements with continuous training on larger resolutions (window size 12→24, image size 192→384); (2) Mixture-of-Experts variant (configs/swinmoe/) with 8, 16, 32, 64 expert configurations; (3) SimMIM masked image modeling pre-training support; (4) Integration with NVIDIA FasterTransformer for inference optimization (noted in 12/29/2022 update).

git clone https://github.com/microsoft/Swin-Transformer.git
cd Swin-Transformer
pip install -r requirements.txt  # Create this if missing; verify torch, torchvision, timm
python -m torch.utils.collect_env  # Verify CUDA setup
# For quick classification test:
python main.py --cfg configs/swin/swin_tiny_patch4_window7_224.yaml --data-path /path/to/imagenet

See get_started.md for detailed setup with specific ImageNet download instructions.

Daily commands: For training from scratch:

python main.py --cfg configs/swin/swin_tiny_patch4_window7_224.yaml --data-path /path/to/imagenet --batch-size 128 --epochs 300

For fine-tuning ImageNet-22K→1K:

python main.py --cfg configs/swin/swin_base_patch4_window7_224_22kto1k_finetune.yaml --resume /path/to/22k_pretrained.pth --data-path /path/to/imagenet1k

For evaluation:

python main.py --eval --cfg configs/swin/swin_base_patch4_window7_224.yaml --resume checkpoint.pth --data-path /path/to/imagenet

• models/swin_transformer.py — Core Swin Transformer architecture implementation; defines the hierarchical vision transformer with shifted windows—essential for understanding the model backbone.
• main.py — Primary training entry point with config loading, distributed training setup, and training loop—required reading for running experiments.
• models/build.py — Model factory that instantiates different Swin variants (Swin, SwinV2, SwinMLP, SwinMoE) from configs—critical for model loading and architecture selection.
• data/build.py — Data pipeline factory that constructs dataloaders for ImageNet and ImageNet-22K—essential for understanding data flow into training.
• config.py — Configuration system using YAML and argument parsing; defines all hyperparameters and training settings across the entire codebase.
• models/swin_transformer_v2.py — Swin Transformer V2 with scaled-up capacity and cosine attention; important variant adding continuous positional embeddings and layer normalization improvements.
• kernels/window_process/swin_window_process_kernel.cu — CUDA kernel for efficient window partitioning—critical performance component for shifted window computation on GPU.

Add a new Swin Transformer variant

1. Create new model class inheriting from base Swin structure in models/swin_transformer.py or create new file following naming convention (models/swin_transformer.py)
2. Register model in models/build.py in the build_model() function with a unique model name (models/build.py)
3. Create YAML config file under configs/swin/ with hyperparameters (depths, dims, num_heads, window_size) (configs/swin/swin_new_variant.yaml)
4. Run training using main.py with --cfg pointing to new config file (main.py)

Add support for a new dataset

1. Create dataset class in data/ directory implementing getitem and len following imagenet22k_dataset.py pattern (data/imagenet22k_dataset.py)
2. Add dataset instantiation logic to data/build.py in build_dataset() function with conditional imports (data/build.py)
3. Add dataset name parameter to config.py under dataset section (config.py)
4. Update main.py or relevant training script to handle dataset-specific preprocessing if needed (main.py)

Implement a custom training routine (e.g., contrastive learning)

1. Create new entry point script following main_simmim_pt.py pattern with custom loss computation (main_simmim_pt.py)
2. Add model building and configuration loading using models/build.py and config.py infrastructure (config.py)
3. Leverage optimizer.py and lr_scheduler.py for optimization, and utils.py for checkpointing and metrics (optimizer.py)
4. Create corresponding YAML config file under configs/ with training hyperparameters (configs/simmim/simmim_pretrain__swin_base__img192_window6__800ep.yaml)

Optimize window processing with custom kernels

1. Implement CUDA kernel in kernels/window_process/ following swin_window_process_kernel.cu structure (kernels/window_process/swin_window_process_kernel.cu)
2. Add Python wrapper in kernels/window_process/window_process.py with PyTorch tensor interface (kernels/window_process/window_process.py)
3. Update setup.py to compile new kernel and ensure it's imported in models/swin_transformer.py (kernels/window_process/setup.py)
4. Test with kernels/window_process/unit_test.py before integration (kernels/window_process/unit_test.py)

• PyTorch — Industry-standard deep learning framework with CUDA support, distributed training (DDP), and extensive ecosystem for computer vision.
• CUDA/C++ — Custom kernels for window partitioning provide 2-3× speedup over pure PyTorch implementations for the critical shifted window operation.
• YAML config files — Declarative configuration enables reproducible experiments and hyperparameter sweeps without code changes; supports inheritance and overrides.
• Distributed Data Parallel (DDP) — Native PyTorch multi-GPU/multi-node training with minimal overhead, essential for ImageNet-22K pretraining at

(1) CUDA compatibility: shifted window attention CUDA kernels require specific CUDA versions; verify with python -m torch.utils.collect_env before training. (2) ImageNet path conventions: configs expect data-path pointing to imagenet/ with train/ and val/ subdirectories following specific structure; incorrect layout causes silent training failures. (3) Window size constraints: not all window sizes are valid; window size must divide image resolution; e.g., window_size=14 requires image size divisible by 14, not 12. (4) Mixed precision training: some configs use AMP; ensure NVIDIA APEX is installed if using older PyTorch versions. (5) Multi-GPU setup: SwinMoE configs explicitly require 32 or 64 GPUs as hardcoded in filename; scaling down requires careful router loss rebalancing.

• Shifted Window Attention — Core innovation of Swin Transformer that reduces computational complexity from O(n²) to O(n log n) by partitioning image into local windows and shifting them between layers; essential to understand for any modifications
• Relative Position Bias — Swin replaces absolute positional encodings with learnable relative position biases within each window; crucial for achieving resolution generalization (as seen in SwinV2 window scaling configs)
• Hierarchical Multi-Scale Feature Maps — Unlike standard ViTs, Swin progressively halves spatial dimensions across stages while increasing channels; this design enables use as backbone for detection/segmentation (see downstream task repos)
• Cyclic Shift & Padding for Efficiency — Shifted windows are realized via cyclic shift of feature maps with masked padding; understanding this trick is essential for implementing custom window attention variants or debugging CUDA kernels
• Mixture-of-Experts (MoE) Routing — SwinMoE configs show sparse expert routing; necessary for understanding large-scale model scaling and load balancing across 32+ experts in configs/swinmoe/
• Masked Image Modeling (SimMIM) — Self-supervised pre-training approach where random image patches are masked and reconstructed; implemented in configs/simmim/; critical for understanding modern SSL training supported in this repo
• Continuous Position Bias Interpolation — SwinV2 enables resolution and window size generalization by interpolating position biases learned at one resolution to different resolutions (window_size 12→24); seen in configs/swinv2/to configs

• facebookresearch/dino — Self-supervised vision transformer learning method that can use Swin as backbone; complementary to SimMIM pre-training support in this repo
• SwinTransformer/Swin-Transformer-Object-Detection — Official downstream task implementation using Swin backbone for MMDetection-based object detection; essential companion for practitioners building detection systems
• SwinTransformer/Swin-Transformer-Semantic-Segmentation — Official semantic segmentation variant using Swin; demonstrates hierarchical feature integration for dense prediction tasks
• SwinTransformer/Video-Swin-Transformer — Extends Swin architecture to video/temporal domain with 3D shifted windows; shows how base architecture generalizes beyond images
• timm/pytorch-image-models — Contains optimized implementations of Swin variants and interoperable pre-trained weights; useful for model hub and alternative implementations

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add comprehensive unit tests for window_process CUDA kernel

The kernels/window_process/ directory contains critical CUDA implementations (swin_window_process_kernel.cu) and a unit_test.py file, but there's no evidence of integrated test coverage or CI validation. Given this is a performance-critical component used across all Swin variants, adding rigorous unit tests that validate correctness on different window sizes, batch sizes, and GPU architectures would prevent regressions and improve contributor confidence.

• [ ] Expand kernels/window_process/unit_test.py with parametrized tests covering edge cases (window_size=7,8,12,14,16,24, batch sizes 1-32)
• [ ] Add tests validating numerical correctness against PyTorch reference implementations
• [ ] Create kernels/window_process/test_backward_pass.py for gradient validation
• [ ] Add GitHub Actions workflow (.github/workflows/test-kernels.yml) to run these tests on GPU runners

Implement configuration validation and schema documentation for YAML configs

The configs/ directory has 40+ YAML files across swin, swinv2, swinmlp, simmim, and swinmoe variants, but there's no validation layer or schema documentation in config.py. New contributors frequently make config mistakes (typos in parameter names, invalid combinations). Adding a schema validator with clear error messages would reduce friction and catch bugs early.

• [ ] Create configs/schema.py defining a Pydantic or dataclass-based schema for all valid config parameters across model variants
• [ ] Update config.py to validate loaded YAML against schema before model instantiation
• [ ] Add docstring examples in config.py showing valid configurations for each model type (swin_base, swinv2_large, swin_moe, etc.)
• [ ] Include validation for interdependent parameters (e.g., patch_size must divide image_size; window_size must be <= image_size after patching)

Add integration tests and benchmarking suite for model inference across variants

With 5 model families (Swin, SwinV2, SwinMLP, SwinMoE, SimMIM) and multiple sizes (tiny/small/base/large), there's no automated verification that all variants load, run inference, and produce expected output shapes. This prevents silent breakage when core model code changes and makes it hard for contributors to validate their changes don't break specific variants.

• [ ] Create tests/test_model_loading.py with fixtures that dynamically load each config from configs/ directories
• [ ] Add parametrized tests that instantiate each model variant, run forward pass with dummy input (batch_size=1, img_size from config)
• [ ] Validate output shapes match expected dimensions for classification (B, num_classes) and feature extraction modes
• [ ] Create tests/test_inference_speed.py with timing benchmarks and add a GitHub Actions workflow to track performance regressions across PRs

• Add comprehensive documentation to models/ directory explaining WindowAttention mechanism, shifted window indexing logic, and how relative position bias is computed; this core innovation needs walkthrough for new contributors
• Create minimal inference script (inference.py) demonstrating how to load pre-trained checkpoints and run single-image predictions with visualization; currently only training is well-documented in get_started.md
• Implement unit tests for CUDA kernel correctness by comparing shifted window attention outputs against pure PyTorch reference implementation for small tensors; validate gradients for training stability

The Swin-Transformer repository demonstrates a generally secure posture as a machine learning research project. The codebase is free of obvious critical vulnerabilities, SQL injection risks, or exposed secrets. However, there are several areas for improvement: (1) Native C/C++ extensions in the window_process kernels require security review to prevent memory safety issues; (2) YAML configuration loading should use safe loading functions; (3) Data loading utilities should implement stricter input validation and path traversal protection; (4) Missing dependency pinning documentation increases supply chain risk. The project benefits from Microsoft's security policies and the SECURITY.md file indicating responsible disclosure practices. Recommendations focus on defensive programming practices and supply chain security management typical for ML projects handling variable input data.

• Low · Native C++ Extensions Without Clear Security Review — kernels/window_process/swin_window_process.cpp, kernels/window_process/swin_window_process_kernel.cu. The repository includes native CUDA kernels and C++ extensions (kernels/window_process/) that are compiled as part of the build process. These compiled extensions could introduce security risks if not properly reviewed and maintained, including potential memory safety issues, buffer overflows, or other C/C++ specific vulnerabilities. Fix: Conduct thorough security review of native code, enable Address Sanitizer (ASAN) and Undefined Behavior Sanitizer (UBSAN) during testing, implement static analysis tools (clang-tidy, cppcheck), and document the security review process.
• Low · Potential Path Traversal in Data Loading — data/zipreader.py. The zipreader.py module (data/zipreader.py) handles zip file operations. If not carefully implemented, zip file extraction could be vulnerable to path traversal attacks where malicious zip files contain entries with '../' paths that extract outside the intended directory. Fix: Ensure proper validation of zip file paths before extraction, sanitize all extracted paths, use secure extraction functions that prevent path traversal, and validate that all extracted files are within the intended directory.
• Low · YAML Configuration Files Without Input Validation — configs/ directory (all YAML files), config.py. The repository uses YAML configuration files extensively (configs/ directory). If these files are loaded without proper validation, they could be vulnerable to YAML deserialization attacks, code injection, or arbitrary code execution through unsafe YAML loading. Fix: Use safe YAML loading functions (yaml.safe_load instead of yaml.load), implement schema validation for all configuration files, sanitize any user-provided configuration values, and document expected configuration structure.
• Low · Missing Dependency Pinning Information — Root directory (missing requirements.txt or equivalent). No package dependency file (requirements.txt, setup.py, pyproject.toml, etc.) was provided in the file structure. Without pinned dependency versions, the project could be vulnerable to supply chain attacks or compatibility issues from unvetted dependency updates. Fix: Create and maintain a requirements.txt file with pinned versions (==), use lock files (pip-compile, poetry.lock), regularly audit and update dependencies, implement dependency scanning in CI/CD pipeline, and document minimum required versions.
• Low · Data Loader Security Considerations — data/data_simmim_ft.py, data/data_simmim_pt.py, data/imagenet22k_dataset.py, data/cached_image_folder.py. Multiple data loading modules (data_simmim_ft.py, data_simmim_pt.py, imagenet22k_dataset.py, cached_image_folder.py) handle image data and file operations. Insufficient input validation could lead to file access issues or unexpected behavior with malformed datasets. Fix: Implement robust error handling for file operations, validate file paths and formats, implement size limits for images and batches, add checks for corrupted or malformed image files, and use safe file path operations.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/microsoft/Swin-Transformer shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live microsoft/Swin-Transformer repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/microsoft/Swin-Transformer.

What it runs against: a local clone of microsoft/Swin-Transformer — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in microsoft/Swin-Transformer | Confirms the artifact applies here, not a fork | | 2 | License is still MIT | Catches relicense before you depend on it | | 3 | Default branch main exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 686 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of microsoft/Swin-Transformer. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/microsoft/Swin-Transformer.git
#   cd Swin-Transformer
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of microsoft/Swin-Transformer and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "microsoft/Swin-Transformer(\\.git)?\\b" \\
  && ok "origin remote is microsoft/Swin-Transformer" \\
  || miss "origin remote is not microsoft/Swin-Transformer (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(MIT)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"MIT\"" package.json 2>/dev/null) \\
  && ok "license is MIT" \\
  || miss "license drift — was MIT at generation time"

# 3. Default branch
git rev-parse --verify main >/dev/null 2>&1 \\
  && ok "default branch main exists" \\
  || miss "default branch main no longer exists"

# 4. Critical files exist
test -f "models/swin_transformer.py" \\
  && ok "models/swin_transformer.py" \\
  || miss "missing critical file: models/swin_transformer.py"
test -f "main.py" \\
  && ok "main.py" \\
  || miss "missing critical file: main.py"
test -f "models/build.py" \\
  && ok "models/build.py" \\
  || miss "missing critical file: models/build.py"
test -f "data/build.py" \\
  && ok "data/build.py" \\
  || miss "missing critical file: data/build.py"
test -f "config.py" \\
  && ok "config.py" \\
  || miss "missing critical file: config.py"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 686 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~656d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/microsoft/Swin-Transformer"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.