GO — Healthy across the board

• Last commit 4d ago
• 7 active contributors
• Distributed ownership (top contributor 30% of recent commits)
• GPL-3.0 licensed
• CI configured
• Tests present
• ⚠ GPL-3.0 is copyleft — check downstream compatibility

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

Marker is a document conversion engine that transforms PDFs, images, PPTX, DOCX, XLSX, HTML, and EPUB files into markdown, JSON, HTML, and chunked formats with high accuracy. It uses deep learning models to preserve complex formatting (tables, equations, code blocks, forms, headers/footers removal) and works on GPU/CPU/MPS, optionally enhanced with LLM-based post-processing for accuracy boosts. Modular plugin architecture: core conversion logic in marker/ package (inferred from .py structure), benchmarking/validation in benchmarks/ with pluggable methods (benchmarks/overall/methods/ contains marker.py, docling.py, llamaparse.py, mathpix.py, mistral.py, olmocr.py). Scoring system separated into heuristic/LLM/clean scorers (benchmarks/overall/scorers/). CI/CD orchestrates via GitHub Actions workflows.

Document processing teams and ML engineers building document intelligence pipelines who need fast, accurate PDF-to-markdown conversion at scale (25 pages/sec on H100). Users include companies with high-volume document workloads seeking alternatives to cloud services like Llamaparse and Mathpix.

Production-ready and actively developed. The codebase is substantial (620KB+ Python), has comprehensive CI/CD workflows (.github/workflows/ with benchmarks, publish, and scripts), includes a managed platform offering, and benchmarks against commercial competitors. Visible benchmark infrastructure (benchmarks/ directory with multiple scoring methods and dataset handling) indicates serious production use.

Moderate risk: GPL-3.0 license with OpenRAIL-M model license requires careful commercial licensing (mentioned: 'Commercial self-hosting requires a license'). Large Python codebase with external model dependencies creates maintenance surface area. Dependency on GPU/compute resources and multiple model formats could create environment setup friction. Single maintainer (Datalab team) concentration is mitigated by commercial backing.

Active development around document format support expansion (PPTX, DOCX, XLSX, EPUB handling), LLM-enhanced accuracy features ('Hybrid Mode' mentioned in README), and competitive benchmarking against emerging competitors (docling, ollmocr). Benchmark infrastructure is being actively maintained with multiple scoring strategies (benchmarks/overall/scorers/).

git clone https://github.com/datalab-to/marker.git
cd marker
pip install -e .  # or pip install -r requirements.txt if present
python -m marker.convert path/to/document.pdf  # or see CLI in marker/ package

Daily commands: Exact commands not visible from file structure alone, but likely python -m marker or marker CLI command. Check marker/main.py or setup.py entry_points. For benchmarks: python -m benchmarks.overall.overall for comparative analysis. GPU acceleration likely requires CUDA/torch setup.

• benchmarks/overall/overall.py: Main orchestrator for competitive benchmarking—entry point for understanding accuracy/speed comparisons
• benchmarks/overall/methods/marker.py: Reference implementation of how Marker's own inference is wrapped for benchmarking—shows API contract
• benchmarks/overall/scorers/llm.py: LLM-based scoring demonstrates the 'Hybrid Mode' accuracy-boosting mechanism mentioned in README
• benchmarks/overall/schema.py: Defines output schema validation for all converters—critical for understanding expected JSON/markdown structure
• .github/workflows/benchmarks.yml: Automated benchmark pipeline configuration showing how quality gates are enforced

Add document format support: extend benchmarks/overall/methods/ with new converter class (see methods/marker.py pattern). Add scoring metrics: add new scorer in benchmarks/overall/scorers/ (see clean.py, heuristic.py, llm.py as templates). Fix conversion bugs: likely in marker/ package core logic. Benchmark regression: run benchmarks/overall/ suite before/after changes. LLM integration: modify scoring or add new scorer using benchmarks/overall/scorers/llm.py pattern.

Model licensing complexity: GPL-3.0 code + OpenRAIL-M models require license agreement for commercial use (MODEL_LICENSE file critical). GPU/CUDA environment setup not visible but likely required—CPU fallback may be slow. Benchmarking infrastructure expects dataset presence (benchmarks/overall/download/ suggests data fetching required). LLM scoring features may require API keys (OpenAI/Anthropic). Pre-commit hooks configured (.pre-commit-config.yaml)—must be respected for contributions.

• Document Layout Understanding (DLU) — Marker's core capability is decomposing PDF spatial layout into semantic blocks (tables, text, equations)—understanding how models segment and classify layout regions is essential to extending format support
• Optical Character Recognition (OCR) + Neural Post-Processing — Marker combines OCR with deep learning for accuracy—the benchmarks/ structure compares OCR quality metrics, so understanding OCR confidence scores and error recovery is critical for debugging conversions
• Table Structure Recognition (TSR) — README explicitly mentions table formatting preservation—Marker's table extraction likely uses graph-based cell detection, critical for accurate markdown table generation from PDFs
• Structured Output / JSON Schema Validation — README mentions 'structured extraction, given a JSON schema (beta)'—benchmarks/overall/schema.py defines validation, requiring understanding of schema-guided generation for constrained outputs
• Semantic Segmentation with Vision Transformers — Modern document models use ViT backbones for pixel-level layout classification—understanding how Marker tokenizes document images and assigns semantic labels is key to model architecture changes
• Batch Processing / Throughput Optimization — README claims '25 pages/second on H100'—achieving this requires understanding GPU batching, pipeline parallelism, and memory management; benchmarks/throughput/ likely contains profiling code
• LLM-based Post-Processing / Chain-of-Thought Correction — Hybrid Mode uses LLMs to refine Marker output (benchmarks/overall/scorers/llm.py)—understanding prompt engineering and few-shot correction patterns is needed to extend LLM integration

• datalab-to/chandra — Successor model to Marker with higher accuracy—directly mentioned in README as managed platform alternative
• unstructured-io/unstructured — Competing document parsing library with similar multi-format support and structured extraction capabilities
• deepdoctection/deepdoctection — Open-source document intelligence framework with table/form extraction—overlapping use cases for complex layout handling
• PixelsDB/pixelsdb — Related document/image processing in the broader Datalab ecosystem for columnar data extraction
• phillipi/pix2code — Predecessor/inspiration for neural document understanding—uses similar CNN-based layout understanding patterns

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add comprehensive unit tests for benchmark scorer modules

The benchmarks/overall/scorers/ directory contains three scorer implementations (clean.py, heuristic.py, llm.py) with a shared schema.py, but there's no evidence of unit tests. Given that scorers are critical for evaluating conversion quality, adding tests would ensure reliability across the benchmark pipeline and prevent regressions when scoring logic is modified.

• [ ] Create tests/benchmarks/scorers/ directory structure
• [ ] Add test_clean.py with tests for clean scorer edge cases (empty documents, special characters, formatting preservation)
• [ ] Add test_heuristic.py with tests for heuristic scoring logic and weighted calculations
• [ ] Add test_llm.py with mock LLM calls to test scorer initialization and output validation
• [ ] Add test_schema.py to validate schema definitions and serialization
• [ ] Integrate new tests into .github/workflows/ci.yml to run on pull requests

Add integration tests for convert.py and convert_single.py with example PDFs

The repo includes example outputs in data/examples/ (multicolcnn, switch_trans, thinkpython) but no visible integration tests that verify the conversion pipeline works end-to-end. Adding tests that process these known examples and validate output structure would catch regressions in the core conversion logic and ensure consistency with stored expected outputs.

• [ ] Create tests/integration/ directory for end-to-end tests
• [ ] Add test_convert_single.py that processes data/examples PDFs and validates JSON schema against data/examples/json outputs
• [ ] Add test_convert_batch.py that validates chunk_convert.py handles multiple files correctly
• [ ] Add fixtures to load and compare expected JSON/markdown outputs from data/examples/
• [ ] Add test documentation explaining how to add new example PDFs for regression testing
• [ ] Update .github/workflows/ci.yml to run integration tests on relevant file changes

Add documentation and tests for benchmark method implementations

The benchmarks/overall/methods/ directory has implementations for multiple extraction methods (marker.py, llamaparse.py, mathpix.py, mistral.py, docling.py, olmocr.py) with a shared schema.py, but there's no per-method documentation or tests. New contributors can't easily understand how to add a new benchmark method or verify existing ones work correctly.

• [ ] Create tests/benchmarks/methods/ directory for method-specific tests
• [ ] Add test_marker.py, test_llamaparse.py, etc. with mock tests for method initialization and output validation
• [ ] Create benchmarks/overall/methods/README.md documenting: (1) the Method interface/schema, (2) how to add a new extraction method, (3) required configuration for each existing method
• [ ] Add validation that each method's output conforms to benchmarks/overall/schema.py definitions
• [ ] Verify all methods in registry.py have corresponding documentation and tests
• [ ] Add CI workflow step to validate no methods are missing documentation

• Add missing output format tests: benchmarks/overall/scorers/ has clean.py, heuristic.py, and llm.py but no test file (benchmarks/overall/scorers/test_scorers.py missing)—write unit tests for each scorer class with mock document inputs.
• Implement benchmark results CSV export: benchmarks/overall/display/table.py handles display but no CSV/JSON export for CI integration—add export_results() method to persist benchmark runs for trend analysis.
• Document LLM integration pattern: README mentions 'Optionally boost accuracy with LLMs' but no examples in benchmarks/overall/—create benchmarks/overall/examples/llm_hybrid_workflow.py showing Marker → LLM scorer → JSON output pipeline.

The codebase appears to be a PDF-to-Markdown conversion tool with benchmarking capabilities. Without access to actual source code and dependencies, a complete assessment is limited. Key concerns include: (1) Inability to audit dependencies for known CVEs due to missing package files, (2) Potential input validation gaps in PDF processing workflows, (3) Lack of visible security controls around external API integrations, and (4) No apparent security configurations if components are web-exposed. The project uses GPL-3.0 license and OpenRAIL-M for models, indicating open-source transparency. Recommend providing dependency files, implementing comprehensive input validation for file handling, securing API credentials, and conducting thorough code review for injection vulnerabilities in PDF processing logic.

• Medium · Missing Dependencies and Package Management File — Repository root - package management files. The security analysis cannot fully assess dependency vulnerabilities as no package management files (requirements.txt, pyproject.toml, setup.py, Pipfile, etc.) were provided. This prevents identification of known CVEs in third-party libraries. Fix: Provide and regularly audit requirements.txt, pyproject.toml, or equivalent files. Use tools like pip-audit, safety, or dependabot to scan for known vulnerabilities in dependencies.
• Low · Benchmark Code with External API Integration — benchmarks/overall/download/ and benchmarks/overall/methods/ directories. The benchmarks directory contains code for integrating with external APIs (llamaparse, mathpix, mistral) without visible API key management patterns. Credentials might be passed via environment variables, but no verification of secure credential handling is visible. Fix: Ensure API credentials are never hardcoded. Use environment variables or secure vaults. Implement credential rotation policies. Document secure credential handling in contributing guidelines.
• Low · No Visible Input Validation in Conversion Scripts — convert.py, convert_single.py, chunk_convert.py, chunk_convert.py. Files like convert.py, convert_single.py, and chunk_convert.py suggest PDF processing and conversion. Without access to source code, potential risks include improper handling of malformed PDFs, path traversal through file operations, or injection attacks through processed content. Fix: Implement strict input validation for all file paths and PDF content. Use sandboxed processing for untrusted PDFs. Validate output paths to prevent directory traversal attacks.
• Low · No Visible Security Headers or CORS Configuration — benchmarks/overall/display/ (if web-exposed). If this is a web-exposed service (suggested by benchmark/display modules), there's no visible configuration for security headers, CORS policies, or rate limiting. Fix: If web-exposed, implement security headers (CSP, X-Frame-Options, etc.), proper CORS policies, rate limiting, and input validation. Use a security middleware.
• Low · Benchmark Code May Download External Content — benchmarks/overall/download/ directory. The benchmarks/overall/download/ directory suggests downloading content from external sources (LlamaParse, Mathpix, Mistral APIs). This could pose supply chain risks if not properly validated. Fix: Verify cryptographic signatures of downloaded content. Implement checksum validation. Use secure, authenticated connections (HTTPS with certificate pinning where applicable). Log all downloads for audit trails.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/datalab-to/marker shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live datalab-to/marker repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/datalab-to/marker.

What it runs against: a local clone of datalab-to/marker — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in datalab-to/marker | Confirms the artifact applies here, not a fork | | 2 | License is still GPL-3.0 | Catches relicense before you depend on it | | 3 | Default branch master exists | Catches branch renames | | 4 | Last commit ≤ 34 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of datalab-to/marker. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/datalab-to/marker.git
#   cd marker
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of datalab-to/marker and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "datalab-to/marker(\\.git)?\\b" \\
  && ok "origin remote is datalab-to/marker" \\
  || miss "origin remote is not datalab-to/marker (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(GPL-3\\.0)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"GPL-3\\.0\"" package.json 2>/dev/null) \\
  && ok "license is GPL-3.0" \\
  || miss "license drift — was GPL-3.0 at generation time"

# 3. Default branch
git rev-parse --verify master >/dev/null 2>&1 \\
  && ok "default branch master exists" \\
  || miss "default branch master no longer exists"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 34 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~4d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/datalab-to/marker"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.