If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/freeok/so-novel shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

WAIT — Single-maintainer risk — review before adopting

• Last commit 2w ago
• 3 active contributors
• AGPL-3.0 licensed
• CI configured
• Tests present
• ⚠ Small team — 3 contributors active in recent commits
• ⚠ Single-maintainer risk — top contributor 98% of recent commits
• ⚠ AGPL-3.0 is copyleft — check downstream compatibility

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live freeok/so-novel repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/freeok/so-novel.

What it runs against: a local clone of freeok/so-novel — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in freeok/so-novel | Confirms the artifact applies here, not a fork | | 2 | License is still AGPL-3.0 | Catches relicense before you depend on it | | 3 | Default branch main exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 45 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of freeok/so-novel. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/freeok/so-novel.git
#   cd so-novel
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of freeok/so-novel and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "freeok/so-novel(\\.git)?\\b" \\
  && ok "origin remote is freeok/so-novel" \\
  || miss "origin remote is not freeok/so-novel (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(AGPL-3\\.0)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"AGPL-3\\.0\"" package.json 2>/dev/null) \\
  && ok "license is AGPL-3.0" \\
  || miss "license drift — was AGPL-3.0 at generation time"

# 3. Default branch
git rev-parse --verify main >/dev/null 2>&1 \\
  && ok "default branch main exists" \\
  || miss "default branch main no longer exists"

# 4. Critical files exist
test -f "src/main/java/com/pcdd/sonovel/Main.java" \\
  && ok "src/main/java/com/pcdd/sonovel/Main.java" \\
  || miss "missing critical file: src/main/java/com/pcdd/sonovel/Main.java"
test -f "src/main/java/com/pcdd/sonovel/core/Crawler.java" \\
  && ok "src/main/java/com/pcdd/sonovel/core/Crawler.java" \\
  || miss "missing critical file: src/main/java/com/pcdd/sonovel/core/Crawler.java"
test -f "src/main/java/com/pcdd/sonovel/core/Source.java" \\
  && ok "src/main/java/com/pcdd/sonovel/core/Source.java" \\
  || miss "missing critical file: src/main/java/com/pcdd/sonovel/core/Source.java"
test -f "src/main/java/com/pcdd/sonovel/core/AppConfigLoader.java" \\
  && ok "src/main/java/com/pcdd/sonovel/core/AppConfigLoader.java" \\
  || miss "missing critical file: src/main/java/com/pcdd/sonovel/core/AppConfigLoader.java"
test -f "src/main/java/com/pcdd/sonovel/parse/BookParser.java" \\
  && ok "src/main/java/com/pcdd/sonovel/parse/BookParser.java" \\
  || miss "missing critical file: src/main/java/com/pcdd/sonovel/parse/BookParser.java"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 45 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~15d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/freeok/so-novel"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

So Novel is a universal web content extraction and export tool that scrapes novel chapters and structured web content, then converts them into EPUB, TXT, PDF, and other standard e-book formats. It provides three interfaces (CLI, TUI, WebUI) to download and process Chinese web novels from aggregated sources defined in JSON rule files (bundle/rules/*.json). Monolithic Java application built with Maven (pom.xml, pom-native-image.xml for GraalVM). Main entry point is src/main/java/com/pcdd/sonovel/Main.java; action classes in src/main/java/com/pcdd/sonovel/action/ (e.g., AggregatedSearchAction.java) handle different UI modes. Extraction rules bundled as JSON in bundle/rules/ (main.json, cloudflare.json, proxy-required.json, etc.) define how to parse each novel source.

Chinese web novel readers and content aggregators who want to download serialized fiction from multiple novel websites without manual chapter collection; also useful for anyone needing to convert web content to standard e-book formats programmatically or via GUI.

Production-ready with active maintenance: version 1.10.1 released, full CI/CD via GitHub Actions (.github/workflows/ has docker-release.yml, release-jpackage.yml, release.yml), multi-platform distribution (Linux/macOS/Windows installers in bin/), and packaged releases across Scoop, Homebrew, and Docker. However, test suite presence is not visible in file listing.

Moderate risk: single maintainer (freeok), heavy dependency on third-party novel sources (defined in bundle/rules/) which may break if sites change structure, and relies on web scraping which is legally gray in some jurisdictions. Java 21 requirement may limit deployment environments. No visible test coverage could hide regression issues.

Active development on multi-platform distribution: release workflows for jpackage (native installers), Docker containerization, and platform-specific scripts (release-linux.sh, release-macos.sh, release-windows.sh). GitHub Issues configured with templates for feature requests, feedback, and source recommendations (ISSUE_TEMPLATE/), suggesting ongoing community contributions for new novel sources.

git clone https://github.com/freeok/so-novel.git && cd so-novel && mvn clean package -DskipTests. For release artifacts, download from https://github.com/freeok/so-novel/releases/latest or use platform installers (Scoop: scoop bucket add freeok https://github.com/freeok/scoop-bucket && scoop install freeok/so-novel; Docker: docker pull per Dockerfile).

Daily commands: mvn spring-boot:run (if Spring configured) or mvn exec:java -Dexec.mainClass='com.pcdd.sonovel.Main' (inferred). For built JAR: java -jar target/so-novel-1.10.1.jar [args]. WebUI likely accessible at http://localhost:8080 (Jetty default). CLI/TUI mode via command-line flags (see bundle/readme.txt for user docs).

• src/main/java/com/pcdd/sonovel/Main.java — Entry point for the application; orchestrates CLI, TUI, and WebUI launchers.
• src/main/java/com/pcdd/sonovel/core/Crawler.java — Core web scraping engine that downloads chapters from novel sources using rules and handles multi-threading.
• src/main/java/com/pcdd/sonovel/core/Source.java — Represents a novel source with parsing rules; critical abstraction for supporting multiple book websites.
• src/main/java/com/pcdd/sonovel/core/AppConfigLoader.java — Loads JSON rule definitions and application configuration; enables dynamic source management.
• src/main/java/com/pcdd/sonovel/parse/BookParser.java — Parses book metadata and table of contents from HTML using Jsoup; foundation for content extraction.
• src/main/java/com/pcdd/sonovel/handle/PostProcessingHandler.java — Post-processing pipeline factory for merging chapters into EPUB, PDF, or TXT formats.
• bundle/rules/main.json — Default source rule definitions; defines parsing selectors and endpoints for all supported novel websites.

Add a new novel source

1. Define parsing rules in JSON: add entry to bundle/rules/main.json with CSS selectors for book info, TOC, and chapters (bundle/rules/main.json)
2. If the source requires special handling, add proxy/Cloudflare config to bundle/rules/proxy-required.json or bundle/rules/cloudflare.json (bundle/rules/cloudflare.json)
3. Source is auto-loaded at startup by AppConfigLoader; no code changes needed (src/main/java/com/pcdd/sonovel/core/AppConfigLoader.java)

Add a new output format

1. Create a new handler class extending PostProcessingHandler in src/main/java/com/pcdd/sonovel/handle/, e.g., MobiMergeHandler.java (src/main/java/com/pcdd/sonovel/handle/PostProcessingHandler.java)
2. Register the handler in PostHandlerFactory.createHandler() switch statement, mapping ContentType to your new class (src/main/java/com/pcdd/sonovel/handle/PostHandlerFactory.java)
3. Add the new ContentType enum value to src/main/java/com/pcdd/sonovel/model/ContentType.java (src/main/java/com/pcdd/sonovel/model/ContentType.java)

Customize chapter parsing for a specific source

1. If CSS selectors alone fail, create a custom parser class extending ChapterParser with overridden parse() method (src/main/java/com/pcdd/sonovel/parse/ChapterParser.java)
2. In bundle/rules/main.json, add a js field to the source rule with inline JavaScript to extract chapter content using DOM APIs (bundle/rules/main.json)
3. The Crawler calls JsCaller to evaluate the JS rule at parse time, bypassing ChapterParser entirely (src/main/java/com/pcdd/sonovel/util/JsCaller.java)

Add a new CLI command or action

1. Create a new Action class in src/main/java/com/pcdd/sonovel/action/, e.g., ExportMetadataAction.java, implementing a run() method (src/main/java/com/pcdd/sonovel/action/DownloadAction.java)
2. Register the action in CliLauncher by adding a command-line argument case and instantiating your action (src/main/java/com/pcdd/sonovel/launch/)

Cloudflare blocking: bundle/rules/cloudflare.json and proxy-required.json suggest some sources need proxies or Cloudflare bypass (Javet may handle JavaScript challenges). Javet is a heavyweight C++ JNI bridge—ensure native libraries available for platform. Rate limiting: bundle/rules/rate-limit.json indicates per-source throttling required; violating it breaks scraping. .env file present but not checked in—likely contains API keys, proxy configs, or feature flags needed at runtime. Java 21 minimum (not Java 17) may surprise users on older systems. Novel source fragility: rules in bundle/rules/ can break if target websites change HTML structure; no schema validation visible.

• CSS Selector Extraction — jsoup uses CSS selectors (defined in bundle/rules/*.json) to locate and extract content from arbitrary HTML; understanding this is key to adding new novel sources
• Web Scraping & HTML Parsing — Core skill for rule maintenance; you must understand DOM structure and selectors to debug why a source breaks when a website redesigns
• EPUB (Electronic Publication) Format — epub4j-core generates EPUB files; understanding EPUB structure (XML manifest, spine, metadata) helps diagnose export issues
• JavaScript Engine Bridging (JNI) — Javet 5.0.5 bridges Java to V8; some novel sites require JavaScript execution to load content, and this library handles that integration
• Rate Limiting & Backoff Strategies — bundle/rules/rate-limit.json enforces per-source throttling to avoid IP bans; critical for reliable long-term scraping
• GraalVM Native Image Compilation — pom-native-image.xml enables AOT compilation to native executables; reduces startup time and dependency footprint for distributed releases
• Cloudflare Bypass & Proxy Handling — cloudflare.json and proxy-required.json rules indicate some sources need bypasses or proxies; Javet + OkHttp combination enables this

• lovemanyi/novel — Another Chinese web novel downloader; similar problem domain and rule-based architecture for handling multiple sources
• Alanpoi/NovelSpider — Web novel scraping framework; offers alternative parsing approach and community-driven source rules
• calibre-ebook/calibre — Industry-standard e-book management and format conversion tool; complementary for post-download processing and metadata management
• arthurredfern/OfflineLibrary — Similar EPUB/e-book generation from web content; demonstrates alternative architecture for structured export
• freeok/scoop-bucket — Companion repo: Scoop installer bucket for So Novel distribution on Windows

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add unit tests for Source.java and Crawler.java core components

The repo has no visible test files (src/test directory is missing) despite being a Maven project with junit dependency configured. Source.java and Crawler.java are critical for parsing book sources and fetching content. Unit tests would ensure scraper reliability across source updates and prevent regressions. This is especially important given the project downloads from multiple external sources.

• [ ] Create src/test/java/com/pcdd/sonovel/core/ directory structure
• [ ] Add SourceTest.java with tests for source parsing, validation, and configuration loading from bundle/rules/*.json files
• [ ] Add CrawlerTest.java with mocked HTTP tests using OkHttpClientContext to verify chapter extraction and filtering logic
• [ ] Update pom.xml to include junit-jupiter test dependencies and maven-surefire-plugin configuration
• [ ] Add test resources directory (src/test/resources) with sample HTML pages and rule configurations for realistic test scenarios

Implement GitHub Actions workflow to validate book source rules (bundle/rules/*.json)

The project maintains multiple JSON rule files (cloudflare.json, main.json, proxy-required.json, rate-limit.json) with a rule-template.json5 reference. There's no automated validation that new/updated rules conform to the schema and don't break the parser. A CI workflow would catch malformed rules before merge and prevent runtime failures when users update sources.

• [ ] Create .github/workflows/validate-rules.yml GitHub Actions workflow
• [ ] Add a Java validator class (src/main/java/com/pcdd/sonovel/core/RuleValidator.java) that loads and validates all JSON rules against the template schema
• [ ] Configure the workflow to run on pull requests modifying bundle/rules/*.json files
• [ ] Add schema validation to catch missing required fields, type mismatches, and invalid regex patterns
• [ ] Output detailed validation errors in the workflow log so contributors know exactly what needs fixing

Extract and document ChapterFilter.java and ChapterFormatter.java filtering/formatting logic

The core/ChapterFilter.java and core/ChapterFormatter.java classes are critical for content processing but lack documentation on how filtering rules work and what formatting transformations are applied. This creates a barrier for contributors wanting to add new filters or formatters. Adding javadocs and a FILTERING_GUIDE.md would unlock community contributions.

• [ ] Add comprehensive JavaDoc comments to ChapterFilter.java methods explaining filter types, regex syntax, and precedence rules
• [ ] Add comprehensive JavaDoc comments to ChapterFormatter.java methods explaining transformations (HTML cleanup, encoding, etc.)
• [ ] Create docs/CHAPTER_PROCESSING.md documenting the full pipeline: filtering → formatting → rendering with examples
• [ ] Add integration test examples showing before/after filtering scenarios (src/test/resources/chapter-samples/)
• [ ] Document how custom filters can be added via configuration (reference bundle/config.ini options)

• Add test coverage for rule parsing: Create unit tests in src/test for JSON rule loading and CSS selector validation against sample HTML (bundle/rules/rule-template.json5 lacks tests)
• Document new novel source addition workflow: Write a contributor guide extending BOOK_SOURCES.md with step-by-step instructions, example rule JSON, and a checklist (currently only rule-template.json5 exists)
• Implement health check for bundled rules: Add a CLI flag to validate all rules in bundle/rules/*.json against their target websites to detect broken sources before release

Failed to generate security analysis.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.