Onboarding: sql-js/sql.js

Generated by RepoPilot · 2026-05-06 · Source

Agent protocol

If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/sql-js/sql.js shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

Verdict

WAIT — Mixed signals — read the receipts

• Last commit 2mo ago
• 26+ active contributors
• Other licensed
• CI configured
• Tests present
• ⚠ Concentrated ownership — top contributor handles 66% of recent commits
• ⚠ Non-standard license (Other) — review terms

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

Verify before trusting

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live sql-js/sql.js repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/sql-js/sql.js.

What it runs against: a local clone of sql-js/sql.js — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in sql-js/sql.js | Confirms the artifact applies here, not a fork | | 2 | License is still Other | Catches relicense before you depend on it | | 3 | Default branch master exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 93 days ago | Catches sudden abandonment since generation |

<details> <summary><b>Run all checks</b> — paste this script from inside your clone of <code>sql-js/sql.js</code></summary>

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of sql-js/sql.js. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/sql-js/sql.js.git
#   cd sql.js
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of sql-js/sql.js and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "sql-js/sql.js(\\.git)?\\b" \\
  && ok "origin remote is sql-js/sql.js" \\
  || miss "origin remote is not sql-js/sql.js (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(Other)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"Other\"" package.json 2>/dev/null) \\
  && ok "license is Other" \\
  || miss "license drift — was Other at generation time"

# 3. Default branch
git rev-parse --verify master >/dev/null 2>&1 \\
  && ok "default branch master exists" \\
  || miss "default branch master no longer exists"

# 4. Critical files exist
test -f "src/api.js" \\
  && ok "src/api.js" \\
  || miss "missing critical file: src/api.js"
test -f "src/exported_functions.json" \\
  && ok "src/exported_functions.json" \\
  || miss "missing critical file: src/exported_functions.json"
test -f "src/worker.js" \\
  && ok "src/worker.js" \\
  || miss "missing critical file: src/worker.js"
test -f "Makefile" \\
  && ok "Makefile" \\
  || miss "missing critical file: Makefile"
test -f "package.json" \\
  && ok "package.json" \\
  || miss "missing critical file: package.json"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 93 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~63d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/sql-js/sql.js"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

</details>

TL;DR

sql.js compiles SQLite to WebAssembly and JavaScript (via Emscripten) to run a full relational database entirely in the browser with no server. It allows you to create tables, run SQL queries, and import/export SQLite database files as typed arrays—but stores everything in memory and doesn't persist changes to disk by default. Thin wrapper architecture: src/api.js exposes the high-level SQL API, src/worker.js handles threaded execution, src/shell-pre.js and src/shell-post.js wrap the Emscripten-compiled SQLite binary. Build pipeline in Makefile orchestrates Emscripten compilation to dist/ (outputs both sql-wasm.js for Node.js and sql-wasm-browser.js for browsers). Examples in examples/ and test/ demonstrate real usage patterns (GUI demo, persistent.html, repl.html).

Who it's for

Web developers building data-heavy browser applications (dashboards, data analysis tools, offline-first apps) who need SQL querying capabilities without a backend database; also Electron and Node.js developers who prefer pure JavaScript over native SQLite bindings for portability.

Maturity & risk

Production-ready and actively maintained. The project has a full CI pipeline (GitHub Actions in .github/workflows/CI.yml), comprehensive test suite across multiple build targets (asm, wasm, wasm-browser variants in test/all.js), published on npm as sql.js@1.14.1, and JSDoc-generated API documentation. Last visible activity shows active development with release workflows.

Relatively low risk for a mature compiled project. Primary concern: entire database must fit in browser memory (no streaming), so unsuitable for large datasets. Emscripten is the single critical dependency—version mismatches during rebuild could break the WebAssembly compilation. No native persistence means users must manually serialize databases. Single repository maintainer pattern (sql-js org) means contribution review velocity depends on maintainer availability.

Active areas of work

The project appears to be in steady maintenance mode. Version 1.14.1 is current, CI is passing, and the repo includes modern tooling (ESLint with flat config in eslint.config.cjs, JSDoc generation via .jsdoc.config.json, Docker dev environment in .devcontainer/). Release workflows suggest periodic updates but the file list doesn't show recent major feature additions—focus is on stability and bug fixes.

Get running

git clone https://github.com/sql-js/sql.js.git
cd sql.js
npm install
npm run build
npm test

Daily commands:

# Build from source
make

# Run all test suites
npm test

# Run specific test variant
node test/all.js wasm

# Start local demo server
python examples/start_local_server.py

# Generate API docs
npm run doc

Map of the codebase

• src/api.js — Core public API surface exposing Database, Statement, and utility classes; all external interactions flow through this module.
• src/exported_functions.json — Defines which functions from the Emscripten-compiled SQLite are exposed to JavaScript; critical for understanding capability boundaries.
• src/worker.js — Implements Web Worker support for async query execution; essential for non-blocking database operations in browsers.
• Makefile — Orchestrates the build pipeline including Emscripten compilation, dependency management, and dist artifact generation.
• package.json — Declares the library's public exports and test suite; defines build targets and dependency versions.

Components & responsibilities

• Database (src/api.js) (JavaScript, Emscripten Runtime) — Manages Emscripten instance lifecycle, prepared statements, and result iteration
  • Failure mode: Memory exhaustion on large datasets; failed WASM instantiation if binary is corrupted or unsupported
• Prepared Statement Handler (JavaScript, SQLite C API (via Emscripten)) — Binds parameters, executes queries, and manages cursor state
  • Failure mode: Unmatched parameter counts, type mismatches, or SQL syntax errors; statement finalization leaks
• Web Worker Proxy (src/worker.js) (Web Workers, MessagePort, JSON serialization) — Serializes DB operations across thread boundary; provides async interface
  • Failure mode: Serialization overhead for large result sets; message queue saturation under high query load
• Emscripten/WASM Runtime (WebAssembly, asm.js, Emscripten) — Virtual filesystem, memory management, and SQLite C library execution
  • Failure mode: Out-of-memory errors, unsupported browser features, or stack overflow on deeply nested queries
• Build Pipeline (Makefile + GitHub — undefined

How to make changes

Add a new SQL function or custom aggregate

1. Create a test file in test/ (e.g., test/test_new_function.js) that validates your function behavior with SQL queries (test/test_new_function.js)
2. Implement the function in src/api.js under the Database or Statement class (e.g., database.createFunction()) (src/api.js)
3. If wrapping a native SQLite C function, add it to src/exported_functions.json with its signature (src/exported_functions.json)
4. Run 'npm run test' to validate the new function across all build variants (asm.js, WebAssembly, debug, release)

Build and release a new version

1. Update version in package.json following semantic versioning (package.json)
2. Run 'make clean && make' to rebuild all artifacts (asm.js and WebAssembly, debug and release) (Makefile)
3. Run 'npm run test' to validate all test suites pass before release (test/all.js)
4. Push version tag to GitHub; the CI/CD workflow in .github/workflows/release.yml automatically publishes to npm (.github/workflows/release.yml)

Enable Web Worker support for a custom use case

1. Review the worker API in src/worker.js to understand message protocol (initSqlJs, run, close) (src/worker.js)
2. In your application code, instantiate a new Worker pointing to the dist/sql-wasm-worker.js file (dist/sql-wasm-worker.js)
3. Send async messages to the worker (e.g., {type: 'run', sql, params}) and handle responses (examples/GUI/gui.js)

Why these technologies

• Emscripten + LLVM — Compiles SQLite C source to WebAssembly and asm.js, enabling SQL execution in browsers without network round-trips
• WebAssembly (WASM) and asm.js — Provides near-native performance for compute-intensive SQL operations while maintaining cross-browser compatibility
• Web Workers — Offloads database queries from the main thread to prevent UI freezing during large data operations
• Virtual Filesystem (Emscripten) — Allows SQLite to operate on an in-memory database with a familiar file API without persistent storage

Trade-offs already made

• In-memory only, no persistence by default

  • Why: Simplifies browser security model and avoids IndexedDB/localStorage fragmentation
  • Consequence: Users must explicitly export/import database state; data is lost on page reload unless manually saved

• Single synchronous context per Database instance

  • Why: Matches SQLite's threading model and simplifies the API
  • Consequence: Long-running queries block the main thread unless the optional Worker interface is used

• Full SQLite feature set exposed

  • Why: Maximizes compatibility with existing SQL applications and libraries
  • Consequence: Large WASM payload (~1–3 MB uncompressed); requires users to manage memory for large datasets

• Emscripten-based compilation pipeline

  • Why: Avoids maintaining custom C→JS bindings and leverages SQLite's official C source
  • Consequence: Build process is complex and requires Emscripten toolchain; hard to fork and customize

Non-goals (don't propose these)

• Does not provide automatic persistence to IndexedDB, Local Storage, or cloud backends; export/import is manual
• Does not implement distributed replication or multi-user synchronization
• Does not support server-side SQL execution or network-based databases
• Does not provide built-in encryption or authentication mechanisms
• Does not offer real-time change notifications across tabs or workers

Traps & gotchas

Emscripten version pinning: The Makefile likely has a specific Emscripten version requirement (not visible in snippet)—installing wrong version breaks WASM compilation. Memory limits: Browsers have heap size caps (varies by browser/device); large databases crash silently if they exceed available memory. WASM file location: Runtime must be configured with locateFile to find the .wasm binary—missing or incorrect path causes silent module init failures. File system: Uses Emscripten MEMFS (memory-backed virtual filesystem), not real disk—persistence requires explicit export to typed array. Test parallelization: Some test files may have shared database state; running tests in parallel can cause flakiness. Browser compatibility: asm.js fallback requires explicit build path; some bundlers don't pick up browser export automatically.

Architecture

Concepts to learn

• Emscripten & WebAssembly compilation — sql.js's core mechanism—Emscripten translates SQLite's C code to WASM; understanding the compilation pipeline (source → LLVM → WASM/asm.js) is essential to debugging build failures and understanding performance characteristics
• Virtual file system (MEMFS) — sql.js uses Emscripten's in-memory filesystem to store the SQLite database; databases are never persisted to disk unless explicitly exported as typed arrays—critical to understanding data durability implications
• Prepared statements — sql.js exposes SQLite's prepared statement API (bind parameters, step through results); this is the core query execution pattern and is essential to both performance and security (SQL injection prevention)
• Web Workers & threading — src/worker.js provides optional threaded query execution to avoid blocking the main JavaScript thread; understanding when and how to offload queries to workers is critical for responsive browser UIs with heavy SQL workloads
• JavaScript typed arrays — sql.js imports and exports databases as typed arrays (Uint8Array); this is how persistence and file I/O work—understanding the binary serialization format is essential for importing existing SQLite files or exporting for storage
• Module initialization & locateFile — sql.js's runtime requires async initialization with proper WASM file location configuration; failure to set locateFile correctly causes silent module loading failures—a common gotcha for first-time users

Related repos

• kripken/emscripten — The compiler toolchain that sql.js depends on; responsible for translating SQLite C code to WebAssembly and JavaScript
• mapbox/node-sqlite3 — Native SQLite binding for Node.js; the recommended alternative when you need speed and direct file access instead of in-memory databases
• sql-js/react-sqljs-demo — Official example integration of sql.js with React and Webpack; shows production bundler setup and state management patterns
• sqlite/sqlite — The upstream SQLite C library that sql.js compiles; understanding SQLite source helps debug edge cases in sql.js behavior
• vlcn-io/cr-sqlite — Alternative in-browser SQLite with built-in CRDT sync capabilities; solves the persistence and multi-device sync problem that sql.js doesn't address

PR ideas

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add comprehensive test suite for Worker API (src/worker.js)

The src/worker.js file exists but there are no dedicated tests in test/ directory for Worker functionality. Given that sql.js is designed for browser environments, testing the Worker interface is critical for validating off-main-thread database operations. This would ensure the Worker properly marshals data, handles errors, and maintains state correctly.

• [ ] Create test/test_worker.js with tests for Worker instantiation and message passing
• [ ] Test data serialization/deserialization between main thread and Worker
• [ ] Add tests for error handling and exception propagation in Worker context
• [ ] Verify Worker correctly loads WASM and executes SQL statements
• [ ] Add test to CI.yml workflow to run the new Worker tests

Add integration tests for JSON1 extension edge cases (test/test_json1.js enhancement)

test/test_json1.js exists but appears minimal. JSON1 is a powerful extension for SQLite, and given sql.js targets browser use cases where JSON manipulation is common, this test file should have more comprehensive coverage. Current test naming pattern suggests it may lack edge cases for nested JSON, type coercion, and NULL handling.

• [ ] Expand test/test_json1.js with tests for json_extract with nested paths
• [ ] Add tests for json_set and json_insert with conflicting paths
• [ ] Test JSON functions with NULL values and type mismatches
• [ ] Add tests for json_group_array/json_group_object in aggregate scenarios
• [ ] Verify compatibility between json() type checking and standard SQLite behavior

Create regression tests for reported issues in test/ directory

The repository has issue-specific test files (test/test_issue55.js, test/test_issue73.js, etc.) showing good practice, but test/disabled_test_memory_leak_on_error.js is disabled. Additionally, there are issues mentioned in the file structure (issue128, issue325, issue55, issue73, issue76) with some having tests but this suggests a pattern of bugs that should have permanent regression tests. Adding a structured approach to capture unresolved issues would prevent regressions.

• [ ] Review GitHub issues for bugs without corresponding test/test_issueX.js files
• [ ] Create test/test_issue[N].js for each open/closed critical issue without test coverage
• [ ] Document the original issue number and description in each test file header
• [ ] Re-enable test/disabled_test_memory_leak_on_error.js if the memory leak was fixed, or create a new memory leak test
• [ ] Add all new regression tests to test/all.js and ensure they run in CI pipeline

Good first issues

• Add comprehensive test coverage for src/worker.js Web Worker threading—currently no dedicated test file in test/ for worker-based queries, only examples in examples/
• Document the exact Emscripten version and flags used in the Makefile with inline comments, and add a CONTRIBUTING.md troubleshooting section for 'my WASM build failed' (currently Makefile is undocumented)
• Create a TypeScript .d.ts definition file for the public API in src/api.js (JSDoc exists but no TS support)—would unblock TypeScript users and improve IDE autocomplete

Top contributors

• @lovasoa — 66 commits
• @Taytay — 6 commits
• @llimllib — 2 commits
• @kaizhu256 — 2 commits
• @abetomo — 2 commits

Recent commits

• 8088a68 — Fix browser bundle (#625) (lovasoa)
• ee29605 — Update devcontainer + migrate linting to ESLint 10 (#623) (lovasoa)
• c2f5c17 — feat: add browser-only WASM build to eliminate Node.js require() warnings in bundlers (#622) (rxliuli)
• 52e5649 — Update CONTRIBUTING.md (#610) (aidant)
• ee67aeb — Remove usage of emscripten's allocate function. (#606) (sbc100)
• b645ca0 — make updateHook return the db object (lovasoa)
• 8dd13e5 — fix jsdoc for updatehook (lovasoa)
• b78df58 — fix execBtn (lovasoa)
• 293d57d — mono (lovasoa)
• bdd3cda — syntax in snippet (lovasoa)

Security observations

The sql.js codebase has a solid security posture with minimal critical vulnerabilities. The project follows good practices by being dependency-light (dev-only deps), maintaining a clean code structure, and providing browser-based SQLite without network exposure. Main concerns

• Low · Outdated ESLint Configuration Format — .eslintrc.js, eslint.config.cjs. The codebase uses both .eslintrc.js and eslint.config.cjs files, with the newer eslint.config.cjs being the recommended flat config format. The presence of .eslintrc.js suggests older ESLint configuration that may not enforce all modern security linting rules. Fix: Migrate fully to the new ESLint flat config format (eslint.config.cjs) and remove .eslintrc.js to ensure consistent and modern linting rules are applied.
• Low · Missing Security Headers in HTML Files — GUI/index.html, examples/GUI/index.html, examples/simple.html, examples/repl.html, examples/requireJS.html, examples/persistent.html. HTML files in the examples and GUI directories (e.g., examples/simple.html, examples/GUI/index.html, GUI/index.html) may not include security headers like Content-Security-Policy. While sql.js runs client-side, CSP headers would help prevent XSS if dynamic content is loaded. Fix: Add Content-Security-Policy and other security headers to HTML files. At minimum, implement strict CSP policies that whitelist only necessary resources and prevent inline script execution.
• Low · Python Development Server Without Security Warnings — examples/start_local_server.py. The file examples/start_local_server.py provides a simple HTTP server for testing. This development server is not secure and should never be used in production. There's no warning in the file about this risk. Fix: Add prominent warnings in documentation and code comments that this server is for development only. Consider adding a warning message when the server starts, or recommend using a production-grade web server.
• Medium · Potential XSS Risk in Web Worker Implementation — src/worker.js, test/test_worker.js. The file src/worker.js handles data from web workers. If user-supplied SQL or other input is passed through web workers and later rendered without proper sanitization, it could lead to XSS vulnerabilities. Web workers can be a vector for injecting malicious code. Fix: Ensure all data received from web workers is treated as untrusted. Implement proper input validation and output encoding. Never use eval() or similar dynamic code execution on worker messages.
• Low · Missing SBOM and Dependency Audit — package.json, .github/workflows/CI.yml, .github/workflows/release.yml. While devDependencies are minimal and current, there's no evidence of regular dependency auditing, lock file integrity verification, or SBOM generation in the CI/CD pipeline. The npm package.json has only development dependencies, but supply chain security should be verified. Fix: Implement 'npm audit' checks in the CI pipeline. Generate and maintain a Software Bill of Materials (SBOM). Use npm ci instead of npm install in CI/CD for lock file integrity. Consider adding dependabot for automated dependency vulnerability checks.
• Low · Exported Functions May Expose Unsafe APIs — src/exported_functions.json, src/exported_runtime_methods.json. The files src/exported_functions.json and src/exported_runtime_methods.json define the API surface compiled from SQLite via Emscripten. If unsafe memory manipulation functions are exposed, it could allow security bypasses. Fix: Review the exported functions to ensure only safe, high-level database APIs are exposed. Avoid exposing low-level memory manipulation functions (malloc, free, write to arbitrary memory). Document the safety guarantees of exported functions.
• Low · No Release Signing or Integrity Verification — .github/workflows/release.yml. The release workflow (.github/workflows/release.yml) doesn't show evidence of cryptographic signing of releases or integrity verification mechanisms for published packages. Fix: Implement package signing using npm's provenance feature. Add integrity checksums to release notes. Consider using GPG signatures for release artifacts. Enable npm's 'publish-access: public' with provenance enabled.

LLM-derived; treat as a starting point, not a security audit.

Where to read next

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

---

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.