If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/projectdiscovery/subfinder shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

GO — Healthy across the board

• Last commit 1w ago
• 17 active contributors
• Distributed ownership (top contributor 38% of recent commits)
• MIT licensed
• CI configured
• Tests present

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live projectdiscovery/subfinder repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/projectdiscovery/subfinder.

What it runs against: a local clone of projectdiscovery/subfinder — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in projectdiscovery/subfinder | Confirms the artifact applies here, not a fork | | 2 | License is still MIT | Catches relicense before you depend on it | | 3 | Default branch dev exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 40 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of projectdiscovery/subfinder. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/projectdiscovery/subfinder.git
#   cd subfinder
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of projectdiscovery/subfinder and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "projectdiscovery/subfinder(\\.git)?\\b" \\
  && ok "origin remote is projectdiscovery/subfinder" \\
  || miss "origin remote is not projectdiscovery/subfinder (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(MIT)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"MIT\"" package.json 2>/dev/null) \\
  && ok "license is MIT" \\
  || miss "license drift — was MIT at generation time"

# 3. Default branch
git rev-parse --verify dev >/dev/null 2>&1 \\
  && ok "default branch dev exists" \\
  || miss "default branch dev no longer exists"

# 4. Critical files exist
test -f "cmd/subfinder/main.go" \\
  && ok "cmd/subfinder/main.go" \\
  || miss "missing critical file: cmd/subfinder/main.go"
test -f "pkg/runner/runner.go" \\
  && ok "pkg/runner/runner.go" \\
  || miss "missing critical file: pkg/runner/runner.go"
test -f "pkg/passive/passive.go" \\
  && ok "pkg/passive/passive.go" \\
  || miss "missing critical file: pkg/passive/passive.go"
test -f "pkg/subscraping/sources" \\
  && ok "pkg/subscraping/sources" \\
  || miss "missing critical file: pkg/subscraping/sources"
test -f "pkg/resolve/resolve.go" \\
  && ok "pkg/resolve/resolve.go" \\
  || miss "missing critical file: pkg/resolve/resolve.go"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 40 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~10d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/projectdiscovery/subfinder"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

subfinder is a fast passive subdomain enumeration tool written in Go that discovers valid subdomains for target domains by querying multiple online passive data sources (like certificate transparency, DNS records, and public archives). It focuses purely on passive reconnaissance—gathering public information without active probing—making it ideal for bug bounty hunting and penetration testing workflows that prioritize speed and stealth. Modular Go project: cmd/subfinder/main.go is the CLI entry point wrapping pkg/runner (orchestrates enumeration), pkg/passive (manages 20+ passive sources), pkg/resolve (handles DNS resolution and wildcard elimination), and pkg/subscraping (HTTP agent for source queries). Configuration flows through pkg/runner/config.go; passive sources defined in pkg/passive/sources.go.

Bug bounty hunters, penetration testers, and security researchers who need to quickly enumerate subdomains during reconnaissance phases. They use subfinder as a first-stage recon tool that feeds into larger attack workflows via STDIN/STDOUT integration.

Production-ready. The project uses Go 1.24+, has comprehensive CI/CD pipelines (.github/workflows/ includes build-test, codeql-analysis, dockerhub-push, release-binary), and includes test files for core components (sources_test.go, sources_w_auth_test.go, ratelimit_test.go, enumerate_test.go). Active ProjectDiscovery maintenance with regular binary releases and Docker image builds.

Low risk for core functionality, moderate operational risk. The codebase has minimal but critical external dependencies (dnsx, chaos-client, ratelimit from ProjectDiscovery; widely-used Go stdlib packages). Risk factors: reliance on 20+ passive data sources (APIs that may rate-limit, change, or go offline); requires valid API keys for enhanced sources (stored in config); single-organization maintenance by ProjectDiscovery. No recent breaking change warnings visible.

Active releases and CI automation visible: automated binary releases via release-binary.yml, Docker pushes via dockerhub-push.yml, dependency auto-merging via dep-auto-merge.yml, and compatibility checks (compat-checks.yml). The project uses .goreleaser.yml for cross-platform binary distribution, indicating ongoing active maintenance.

git clone https://github.com/projectdiscovery/subfinder.git
cd subfinder
make build
./subfinder -h

Daily commands: Build: make build (defined in Makefile) Run: ./subfinder -d example.com or ./subfinder -dL domains.txt for batch enumeration Usage: Supports STDIN/STDOUT piping for workflow integration

• cmd/subfinder/main.go — Entry point for the CLI application; all execution begins here and sets up argument parsing and runner initialization.
• pkg/runner/runner.go — Core orchestrator that manages the enumeration workflow; understand this to see how sources are invoked and results aggregated.
• pkg/passive/passive.go — Primary passive subdomain enumeration engine that coordinates all data sources and handles deduplication logic.
• pkg/subscraping/sources — Directory containing 30+ abstracted source implementations (Censys, Shodan, crtsh, etc.); adding sources requires following patterns here.
• pkg/resolve/resolve.go — DNS resolution layer that validates discovered subdomains; critical for filtering false positives.
• pkg/runner/options.go — Configuration structure and CLI flag definitions; required reading for understanding all tunable parameters.
• go.mod — Dependency manifest showing all external libraries and Go version requirements; critical for build and environment setup.

• Runner (pkg/runner/runner.go) (Go concurrency (goroutines), flag parsing, file I/O) — Orchestrates the complete enumeration pipeline: initializes config, invokes passive enumeration, resolves results, formats output
  • Failure mode: Invalid config flags, missing API keys → exits with error message before enumeration starts

Add a New Passive Reconnaissance Source

1. Create a new source directory under pkg/subscraping/sources/{sourcename}/ (pkg/subscraping/sources)
2. Implement the Source interface with Enumerate() method in {sourcename}.go (pkg/subscraping/sources/crtsh/crtsh.go)
3. Register your source in the sources map within pkg/passive/sources.go (pkg/passive/sources.go)
4. Add API key handling (if required) following the pattern in pkg/subscraping/sources/chaos/chaos.go (pkg/subscraping/sources/chaos/chaos.go)
5. Write tests following pkg/passive/sources_w_auth_test.go or sources_wo_auth_test.go pattern (pkg/passive/sources_w_auth_test.go)

Add a New CLI Flag or Configuration Option

1. Define the new flag struct field in the Options struct in pkg/runner/options.go (pkg/runner/options.go)
2. Add command-line flag binding in pkg/runner/initialize.go using flag package patterns (pkg/runner/initialize.go)
3. Implement validation logic in pkg/runner/validate.go if the flag needs constraints (pkg/runner/validate.go)
4. Use the option in pkg/runner/enumerate.go to control enumeration behavior (pkg/runner/enumerate.go)

Integrate a New DNS Resolver or Validation Method

1. Extend the client configuration in pkg/resolve/client.go to support new resolver parameters (pkg/resolve/client.go)
2. Add validation logic in pkg/resolve/resolve.go following existing DNS query patterns (pkg/resolve/resolve.go)
3. Expose the resolver option through pkg/runner/options.go and pkg/runner/initialize.go (pkg/runner/options.go)
4. Call the resolver in pkg/runner/enumerate.go at the appropriate stage in the pipeline (pkg/runner/enumerate.go)

• Go 1.24 — Enables fast concurrent source queries, minimal memory footprint, and single-binary deployment across OS platforms.
• Passive API sources (Censys, crtsh, GitHub, etc.) — Avoids active network scanning, evasion detection, and legal liability; sources are freely public data.
• DNS resolution with dnsx — Filters false positives by validating subdomain existence without triggering target alerts.
• Rate limiting (projectdiscovery/ratelimit) — Respects API quotas and prevents temporary bans from enumeration sources.
• YAML configuration — Human-readable API credential storage and reproducible enumeration configurations.

• Passive enumeration only (no active scanning)

  • Why: Reduces legal risk and detection surface
  • Consequence: Cannot discover subdomains not previously indexed by public sources; misses very new subdomains.

• Optional DNS resolution as separate phase

  • Why: Allows trade-off between speed (skip validation) and accuracy (validate all)
  • Consequence: Results may include stale/non-existent subdomains if resolution is skipped.

• Source API authentication optional per-source

  • Why: Works without any API keys; enhanced results with credentials
  • Consequence: Users must maintain multiple API accounts; enumeration quality varies by configured credentials.

• Single Go binary distribution

  • Why: Simplifies deployment (no runtime dependencies)
  • Consequence: Larger binary size; cross-compilation required for non-native platforms.

• Active subdomain enumeration (port scanning, brute-forcing, zone transfers)
• Real-time threat monitoring or continuous background scanning
• DNS server implementation or authoritative DNS query handling
• Authentication bypass or credential testing of discovered services
• Web scraping beyond public API interfaces
• Automated reporting or integration with SIEM platforms

API key management: Many sources require valid API keys (Chaos, Censys, Shodan, etc.) stored in ~/.config/subfinder/config.yaml or via env vars—missing keys will silently skip those sources. Rate limiting: ProjectDiscovery ratelimit library enforces source-specific rate limits; threads/concurrency flags affect enumeration speed and may trigger bans. Wildcard elimination: pkg/resolve requires DNS resolution; if all resolvers fail, legitimate subdomains may be filtered as wildcards. Source reliability: passive sources change, deprecate endpoints, or become unavailable; enumeration results vary based on source health (visible in GitHub issues).

• Passive Reconnaissance — subfinder's core philosophy—all enumeration uses public, already-published data sources (certificate logs, DNS records, archives) avoiding detection; understanding this paradigm explains architectural choices like no active probing in resolve.go
• Certificate Transparency (CT) Logs — One of the primary passive sources (crtsh integration); CT logs are freely queryable public records of all SSL certificates issued, exposing subdomains embedded in cert Subject Alternative Names
• Wildcard DNS Records — Enumeration returns false positives when targets have wildcard records (*.domain.tld resolves to same IP); pkg/resolve/resolve.go detects and filters these via NXDOMAIN testing
• Token Bucket Rate Limiting — projectdiscovery/ratelimit implements token buckets for per-source concurrency; required to avoid overwhelming public APIs and getting banned, visible in pkg/passive/ratelimit_test.go
• Deduplication Hashing — Multiple sources return overlapping subdomains; pkg/passive/passive.go uses result deduplication to merge findings from 20+ sources and avoid duplicates in output
• DNS Canonical Name (CNAME) Chains — Some enumeration results are CNAMEs aliasing to other subdomains; resolution must follow CNAME chains to validate true endpoints, affecting filtering in pkg/resolve
• Modular Source Architecture — Each passive source (GitHub, Chaos, Censys, etc.) is pluggable via Source interface in pkg/passive/sources.go; allows adding new sources without modifying core logic, enabling community contributions

• projectdiscovery/dnsx — Dependency for DNS resolution; used in pkg/resolve for wildcard elimination and validation
• projectdiscovery/chaos-client — Passive source integration for Chaos API queries; one of the primary enumeration sources
• projectdiscovery/nuclei — Companion tool in ProjectDiscovery ecosystem; consumes subfinder output for vulnerability scanning workflows
• projectdiscovery/ratelimit — Rate limiting library; enforces source-specific concurrency and request throttling in pkg/passive
• OWASP/Amass — Alternative subdomain enumeration tool with active enumeration; solves same problem but includes DNS brute-force (vs subfinder's passive-only approach)

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add comprehensive unit tests for pkg/subscraping/sources directory

The repo has 30+ subdomain source integrations (alienvault, anubis, bevigil, bufferover, builtwith, c99, censys, certspotter, etc.) but only censys_test.go exists in the file structure. Each source needs isolated unit tests to verify parsing logic, error handling, and API response processing. This is critical for a passive enumeration tool where source reliability directly impacts results.

• [ ] Create *_test.go files for untested sources in pkg/subscraping/sources/ (alienvault, anubis, bevigil, bufferover, builtwith, c99, certspotter, etc.)
• [ ] Add mock HTTP responses and test cases for successful parsing, malformed responses, and API errors
• [ ] Verify authentication flows for API key-based sources match the existing sources_w_auth_test.go patterns
• [ ] Ensure tests run in CI via .github/workflows/build-test.yml

Add integration tests for pkg/resolve/client.go DNS resolution logic

The resolve package is critical for validating discovered subdomains, but there are no visible test files (*_test.go) for resolve/client.go or resolve/resolve.go. Without tests, DNS resolution bugs (timeouts, NXDOMAIN handling, recursive queries) could silently produce false positives/negatives. This directly impacts the tool's accuracy.

• [ ] Create pkg/resolve/client_test.go with test cases for valid DNS responses, NXDOMAIN, timeouts, and concurrent resolution
• [ ] Create pkg/resolve/resolve_test.go with end-to-end resolution tests using example domains
• [ ] Test custom resolver configurations and error handling paths
• [ ] Verify tests are included in the compat-checks workflow for multiple Go versions

Add missing API authentication documentation and example configs for pkg/subscraping sources

The codebase supports 30+ sources with varying auth requirements (API keys, credentials), evidenced by sources_w_auth_test.go and sources_wo_auth_test.go split. However, no example configuration files or auth setup guides exist in the examples/ directory. Contributors and users lack clear guidance on which sources require auth, how to configure them, and what credentials are needed.

• [ ] Create examples/config.yaml or examples/auth-config.example.yaml showing all ~30 sources with their required/optional auth fields
• [ ] Document each source's API key source (Shodan, Censys, BinaryEdge, etc.) in a new AUTHENTICATION.md file
• [ ] Add code comments to pkg/subscraping/sources/*/[source].go files linking to auth examples
• [ ] Update README.md with a quick reference table: Source | Auth Required | Env Var | Example

• Add test coverage for pkg/resolve/resolve.go wildcard elimination logic—examine resolve_test.go or create one with edge cases (*.domain.tld patterns, NXDOMAIN handling)
• Extend pkg/passive/sources.go to add a new passive source (e.g., BuiltWith, PublicWWW, or URLhaus) by implementing the Source interface and adding integration tests in sources_test.go
• Improve output formatting in pkg/runner/outputter.go to support CSV or XML formats alongside existing JSON/plain-text, with tests in enumerate_test.go

• High · Outdated Go Toolchain Version — go.mod. The project specifies Go 1.24.0 with toolchain 1.24.1. While these are recent versions, Go 1.24.x is still in early lifecycle. Security patches may be delayed compared to stable LTS versions. Additionally, the go.mod shows dependency on golang.org/x/exp which is an experimental package not intended for production use. Fix: Consider using a stable Go LTS version (1.22.x or later stable release) and minimize use of experimental packages from golang.org/x/exp. Regularly update dependencies using 'go get -u' and 'go mod tidy'.
• High · Dependency on Experimental Packages — go.mod - golang.org/x/exp dependency. The go.mod includes 'golang.org/x/exp' which is explicitly not recommended for production use. Experimental packages have no stability guarantees and may contain security issues. Fix: Replace golang.org/x/exp with stable alternatives or implement required functionality directly. If unavoidable, pin to a specific commit hash and review regularly.
• Medium · Incomplete go.sum Verification — go.sum. The go.sum file appears truncated in the provided context ('github.com/dsnet/compress v0.0.'). This prevents full verification of dependency integrity. Incomplete checksums could mask supply chain attacks. Fix: Ensure go.sum is complete and all dependencies are properly checksummed. Run 'go mod verify' to validate integrity. Commit complete go.sum to version control.
• Medium · Base Image Not Pinned to Specific Version — Dockerfile - line 'FROM alpine:latest'. The Dockerfile uses 'FROM alpine:latest' for the release stage. Using 'latest' tag means the base image can change unexpectedly between builds, potentially introducing vulnerabilities or breaking changes. Fix: Pin the alpine image to a specific version (e.g., 'alpine:3.20') and regularly update after testing. Example: 'FROM alpine:3.20'.
• Medium · Potential Privilege Escalation in Container — Dockerfile. The Dockerfile does not explicitly create a non-root user. The entrypoint runs as root, which increases risk if the application or dependencies have vulnerabilities. Fix: Add a non-root user and switch to it before the ENTRYPOINT. Example: RUN addgroup -g 1000 subfinder && adduser -D -u 1000 -G subfinder subfinder USER subfinder
• Medium · Missing Security Headers in Build Process — Dockerfile - 'RUN go build ./cmd/subfinder'. The build stage uses 'go build' without security flags like '-ldflags=-s -w' for stripping or build-time flag injection for version/commit tracking. Fix: Add security and debugging flags: RUN go build -v -ldflags="-s -w -X main.version=$(git describe --tags)" ./cmd/subfinder Also consider using CGO_ENABLED=0 for static binaries.
• Low · No Health Check Defined — Dockerfile. The Dockerfile does not define a HEALTHCHECK instruction, making it harder to monitor container health in orchestrated environments. Fix: Add a HEALTHCHECK instruction to enable automated monitoring: HEALTHCHECK --interval=30s --timeout=5s --start-period=5s --retries=3 CMD subfinder -h || exit 1
• Low · Build Dependencies Not Optimized — Dockerfile - 'RUN apk add build-base'. The build stage installs 'build-base' which includes many unnecessary tools. This increases attack surface in the build container and image size. Fix: Specify only required build dependencies: RUN apk add --no-cache musl-dev gcc Consider using a multi-stage build with cleanup to further reduce final image size.
• Low · Missing SBOM and Provenance Information — undefined. The release workflow and Docker build process do not appear to generate Software Bill of Materials ( Fix: undefined

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.