Onboarding: sjmoran/satire-classifier

Generated by RepoPilot · 2026-05-06 · Source

Agent protocol

If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/sjmoran/satire-classifier shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

Verdict

AVOID — Stale and unlicensed — last commit 3y ago

• 2 active contributors
• Tests present
• ⚠ Stale — last commit 3y ago
• ⚠ Small team — 2 contributors active in recent commits
• ⚠ Concentrated ownership — top contributor handles 67% of recent commits
• ⚠ No license — legally unclear to depend on
• ⚠ No CI workflows detected
• ⚠ Scorecard: marked unmaintained (0/10)
• ⚠ Scorecard: default branch unprotected (0/10)

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests + OpenSSF Scorecard}

Verify before trusting

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live sjmoran/satire-classifier repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/sjmoran/satire-classifier.

What it runs against: a local clone of sjmoran/satire-classifier — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in sjmoran/satire-classifier | Confirms the artifact applies here, not a fork | | 2 | Default branch master exists | Catches branch renames | | 3 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 4 | Last commit ≤ 1275 days ago | Catches sudden abandonment since generation |

<details> <summary><b>Run all checks</b> — paste this script from inside your clone of <code>sjmoran/satire-classifier</code></summary>

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of sjmoran/satire-classifier. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/sjmoran/satire-classifier.git
#   cd satire-classifier
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of sjmoran/satire-classifier and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "sjmoran/satire-classifier(\\.git)?\\b" \\
  && ok "origin remote is sjmoran/satire-classifier" \\
  || miss "origin remote is not sjmoran/satire-classifier (artifact may be from a fork)"

# 3. Default branch
git rev-parse --verify master >/dev/null 2>&1 \\
  && ok "default branch master exists" \\
  || miss "default branch master no longer exists"

# 4. Critical files exist
test -f "README.md" \\
  && ok "README.md" \\
  || miss "missing critical file: README.md"
test -f "requirements.txt" \\
  && ok "requirements.txt" \\
  || miss "missing critical file: requirements.txt"
test -f "logging.conf" \\
  && ok "logging.conf" \\
  || miss "missing critical file: logging.conf"
test -f "eval.prl" \\
  && ok "eval.prl" \\
  || miss "missing critical file: eval.prl"
test -f "test-class" \\
  && ok "test-class" \\
  || miss "missing critical file: test-class"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 1275 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~1245d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/sjmoran/satire-classifier"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

</details>

TL;DR

A Naive Bayes-based binary classifier that detects satirical news articles by combining multiple feature types (word2vec embeddings, Chi2-filtered unigrams, punctuation/capitalization counts, sentiment polarity, intensifiers, and interjections). It uses a two-stage ensemble: Multinomial NB for discrete features and Gaussian NB for embeddings, then meta-learns a final Gaussian NB on the combined probability outputs, achieving 0.96 cross-validation F-score and 0.72 test F-score. Single-file Python library structure with a main classifier module, separate Perl evaluation script (eval.prl), configuration via logging.conf, and requirements.txt for dependencies. Test suite comprises 54 numbered test cases (test-0001 through test-0054) plus a test-class file, suggesting structured test data rather than unit tests. No visible src/, models/, or data/ directories, implying core logic and model artifacts are at root level.

Who it's for

NLP researchers and practitioners building satire/sarcasm detection systems who need a working baseline model with feature engineering examples; teams tackling irony and humor detection who can adapt the multi-feature approach.

Maturity & risk

Experimental/prototype stage. Version 1.0.0 is released with a comprehensive test suite (54 test cases in test/ directory) and clear documentation, but the commit history and CI setup are not visible in the file list, suggesting this is a research project rather than actively maintained production code. The old dependency versions (scikit-learn 0.19.0, gensim 3.0.1, numpy 1.13.3 from ~2017) indicate no recent updates.

High dependency risk: 21 pinned dependencies from 2017 with known security vulnerabilities and API changes in downstream libraries (scikit-learn, gensim, nltk have all evolved significantly). No evidence of CI/CD pipeline, and the presence of only Perl eval.prl (not Python evaluation) suggests evaluation infrastructure may be incomplete. The 54 test files indicate reasonable coverage but lack documentation on which models they exercise.

Active areas of work

No active development signals visible in the file list. The repository appears dormant: version is pinned at 1.0.0, dependencies are frozen to 2017 versions, and there is no indication of recent commits, open PRs, or active issues. The extensive test suite suggests the project reached a stable baseline and was archived.

Get running

git clone https://github.com/sjmoran/satire-classifier.git
cd satire-classifier
pip install -r requirements.txt
python -m satire_classifier.train  # (assumed entry point)

Daily commands: No Makefile or setup.py visible. Likely: python classifier.py or python -m satire_classifier (exact entry point unclear from file list). Evaluation via perl eval.prl against test/ cases. Configure logging via logging.conf.

Map of the codebase

• README.md — Documents the entire Naive Bayes satire detection system, feature types, and model selection rationale that every contributor must understand.
• requirements.txt — Specifies all dependencies including scikit-learn, nltk, gensim, and pandas which are load-bearing for model training and feature extraction.
• logging.conf — Configures logging behavior across the codebase; essential for debugging training and inference pipelines.
• eval.prl — Evaluation script that likely benchmarks model performance; critical for validating changes against baseline metrics.
• test-class — Test classification entry point; demonstrates how to run inference on test data and validate model output.

Components & responsibilities

• Feature Extractors (Unigrams, Embeddings, Punctuation, Sentiment, Intensifiers) (NLTK, gensim, scikit-learn) — Transforms raw text into 6 distinct feature representations; implements Chi2 filtering for unigrams and pre-trained sentiment lexicons.
  • Failure mode: Malformed text or missing NLTK data causes feature extraction to crash; affects all downstream predictions.
• Naive Bayes Classifier (scikit-learn MultinomialNB) — Learns P(satire|features) and P(non-satire|features) from training data; applies Bayes rule at inference time.
  • Failure mode: Unbalanced training data or sparse features lead to biased probability estimates; threshold selection affects precision/recall tradeoff.
• Evaluation Pipeline (eval.prl) (scikit-learn metrics, test/ documents) — Benchmarks model on held-out test set; computes classification metrics (accuracy, precision, recall, F1).
  • Failure mode: Test data contamination or metric miscalculation invalidates performance claims; blocks deployment decisions.
• Inference Service (test-class) (Python, joblib or pickle for model serialization) — Loads trained model, applies feature extractors to new documents, returns satire probability.
  • Failure mode: Model version mismatch or missing feature preprocessing causes incorrect predictions at scale.

Data flow

• Raw training documents → Feature Extractors — Text is tokenized, stemmed, and split into features (unigrams, embeddings, punctuation, sentiment, intensifiers).
• Feature Extractors → Feature Matrix (training) — 6 feature types aggregated into dense/sparse matrices; Chi2 filters top-k unigrams by relevance.
• Feature Matrix + class labels → Naive Bayes Model — Model learns class priors and feature likelihoods P(feature|class) during fit phase.
• Trained Model → eval.prl — Model serialized to disk; evaluation script loads model and test documents (test-0001 to test-0114).
• Test documents + Trained Model → Predictions — eval.prl applies feature extraction and inference to each test document; computes confusion matrix and metrics.
• Trained Model → test-class (production) — Model loaded at startup; new documents flow through feature extraction → prediction → confidence score.

How to make changes

Add a new feature extraction method

1. Review existing feature types in README.md (embeddings, unigram counts, punctuation, capitalization, sentiment, intensifiers) (README.md)
2. Create feature extraction logic in a new module following the pattern of existing extractors (requirements.txt)
3. Add test cases to validate the new feature on sample documents in the test/ directory (test/test-0001)
4. Update eval.prl to measure impact of new feature on model performance (eval.prl)

Train the model on new data

1. Prepare training dataset with satire/non-satire labels following the structure of test documents (test)
2. Create training script that loads data and applies all feature extractors (word2vec, chi2-filtered unigrams, punctuation, etc.) (requirements.txt)
3. Configure logging in logging.conf to track training progress and feature statistics (logging.conf)
4. Run eval.prl after training to measure accuracy, precision, recall on validation set (eval.prl)

Deploy the classifier to production

1. Serialize trained Multinomial Naive Bayes model and feature extractors to disk (test-class)
2. Implement model loading and inference in test-class script (test-class)
3. Verify all dependencies in requirements.txt are installed in target environment (requirements.txt)
4. Test inference on sample documents from test/ directory to validate deployment (test/test-0001)

Why these technologies

• Multinomial Naive Bayes (scikit-learn) — Explicitly selected as superior performer for discrete count-based features; simple, interpretable, and computationally efficient.
• word2vec (gensim) — Generates dense document embeddings as one of six feature types; captures semantic relationships between words.
• NLTK — Provides natural language processing tools for tokenization, sentiment analysis, and linguistic feature extraction.
• scikit-learn — Core ML framework providing Naive Bayes classifier, Chi2 feature selection, and evaluation metrics.
• pandas & numpy — Data manipulation and numerical computation for feature matrices and model training.

Trade-offs already made

• Multinomial Naive Bayes over more sophisticated models (SVM, deep learning)

  • Why: Count-based features performed very well and simpler models are preferred for interpretability and speed.
  • Consequence: May leave performance gains on the table; limited ability to capture complex non-linear patterns.

• Six manually-engineered features rather than learned representations

  • Why: Allows domain expertise (punctuation, sentiment, intensifiers) to guide the model; easier to debug.
  • Consequence: Requires significant feature engineering effort; may miss latent patterns that deep learning could discover.

• Binary classification (satire vs. non-satire) rather than multi-class

  • Why: Simplifies problem scope and training; aligns with core use case.
  • Consequence: Cannot detect nuanced satire types or sarcasm degrees; one-shot prediction only.

Non-goals (don't propose these)

• Does not handle real-time streaming classification
• Does not provide model retraining on user feedback
• Does not support multiple languages (English-centric feature design)
• Does not perform cross-domain transfer learning
• Does not include user authentication or access control

Anti-patterns to avoid

• Possible data leakage from test set (High) — test/ directory and eval.prl: If Chi2 unigram filtering is computed on all data before train/test split, test set features leak into feature selection, inflating accuracy metrics.
• Hardcoded feature extraction logic (Medium) — Feature extractors (inferred from README): Six feature types manually engineered and integrated; refactoring or adding features requires modifying multiple code paths.
• No cross-validation reported — eval.prl: Single

Traps & gotchas

1. Python version constraint: Dependencies pin to 2017-era libraries incompatible with Python 3.8+; likely requires Python 3.5–3.6. 2. Word2vec model not in repo: gensim.models.word2vec is a separate pre-trained asset; code will fail if model file is missing (check for .bin or .model files not in file list). 3. Perl/Python integration: eval.prl orchestrates evaluation; ensure Perl 5 is installed and test case format matches parser expectations. 4. Chi2 feature selection: scikit-learn 0.19.0 has deprecated chi2 API; upgrading dependencies will break feature extraction. 5. No explicit entry point: No setup.py or main.py visible; may require manual inspection of root-level .py files to find training/inference entry points.

Architecture

Concepts to learn

• Chi-squared (χ²) feature selection — This repo uses Chi2 scoring to filter unigrams by relevance to satire vs. non-satire class; understanding this statistical test is critical for modifying feature selection thresholds.
• Gaussian Naive Bayes (continuous features) — Used for word2vec embeddings (continuous vectors) in the first stage and for meta-learning on probability outputs; assumes Gaussian distribution of features per class.
• Multinomial Naive Bayes (count-based features) — Handles discrete unigram counts, punctuation counts, and capitalization counts; this model shows the strongest performance in the ensemble and is the primary predictor.
• Probability calibration / meta-learning — Two base classifiers output probabilities that are stacked as features into a meta-Gaussian NB; understanding probability combination is key to the ensemble architecture.
• Word2Vec embeddings — Dense vector representations of words trained via gensim; provide semantic features that complement count-based features and feed the Gaussian NB stage.
• Intensifiers and interjections — Linguistic features (e.g., 'very', 'extremely', 'oh no') are counted as signals of satirical tone; understanding their role clarifies why linguistic depth matters in satire.
• 10-fold cross-validation — The reported 0.96 F-score is from 10-fold CV; this evaluation technique is critical to understanding the gap between CV performance (0.96) and test performance (0.72).

Related repos

• cjhutto/vaderSentiment — Sentiment polarity extraction used as a feature in this classifier; complementary to satire detection and mentioned in the feature description.
• nltk/nltk — Direct dependency (nltk 3.2.5) providing tokenization and potentially sentiment/intensifier word lists; source of truth for underlying NLP primitives.
• scikit-learn/scikit-learn — Core ML library providing Multinomial NB and Gaussian NB implementations; pinned at 0.19.0 but repo would benefit from version audit against 1.0+.
• RamiKrispin/coronavirus — Unrelated data project but represents similar structured test data organization; demonstrates alternative evaluation patterns.
• openai/gpt-2 — Potential successor approach: modern transformer-based satire detection could benchmark against this classical Naive Bayes baseline.

PR ideas

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add unit tests for feature extraction pipeline

The repo contains 74 test data files (test/test-0001 through test/test-0074) but no visible test suite or unit tests validating the feature extraction modules (word2vec embeddings, Chi2 filtering, punctuation/capitalization/sentiment/intensifier counts). A proper test suite would ensure each feature extractor produces expected outputs and prevent regressions.

• [ ] Create test/unit_tests/ directory with test files for each feature type mentioned in README
• [ ] Add tests for word2vec embedding generation using test data files
• [ ] Add tests for Chi2-filtered unigram extraction with known expected outputs
• [ ] Add tests for punctuation, capitalization, sentiment, intensifier/interjection counters
• [ ] Update requirements.txt to include pytest (currently missing)
• [ ] Document test execution in README.md

Implement missing Naive Bayes model training and evaluation script

The repo references eval.prl (Perl script) but the core Python training script is missing. README mentions both Gaussian and Multinomial Naive Bayes variants and 6 feature types, but there's no visible train.py or model.py to orchestrate feature extraction, model fitting, and cross-validation. This is the core deliverable.

• [ ] Create src/train.py that orchestrates feature extraction from test data files using all 6 feature types
• [ ] Implement model selection between GaussianNB and MultinomialNB with configurable parameters
• [ ] Add cross-validation and evaluation metrics (precision, recall, F1) using the test dataset
• [ ] Integrate logging.conf for training progress tracking
• [ ] Add argparse for CLI parameters (feature selection, model type, output path)
• [ ] Document training pipeline in README with example usage

Create feature extraction module with modular design

README describes 6 distinct feature types but no modular feature extraction code is visible in the file structure. Currently, dependencies like gensim, nltk, scikit-learn, and pandas are in requirements.txt but their usage is unclear. Creating a features/ package with separate modules would improve maintainability and testability.

• [ ] Create src/features/init.py and organize into: embeddings.py (word2vec), unigrams.py (Chi2), punctuation.py, capitalization.py, sentiment.py (polarity), intensifiers.py
• [ ] Implement each feature extractor as a class with fit() and transform() methods following scikit-learn conventions
• [ ] Add feature_extraction.py to chain all 6 extractors into a FeatureUnion pipeline
• [ ] Create data/lexicons/ directory for sentiment polarity list and intensifier/interjection word lists (referenced in README but not provided)
• [ ] Add docstrings referencing the papers cited in README (ACL P15-2124 for sentiment)
• [ ] Update requirements.txt comments to clarify which dependencies are used by which modules

Good first issues

• Add unit tests for feature extraction modules: The test/ directory contains 54 test data cases but no visible pytest/unittest files. Create tests/test_features.py to unit-test punctuation counting, capitalization counting, and unigram extraction functions, validating edge cases (empty docs, all-caps, Unicode).
• Document the test case format and eval.prl schema: test-0001 through test-0054 files exist but their format (JSON, CSV, plain text?) is not documented. Add a test/README.md explaining the format, expected columns, and how eval.prl parses results.
• Create a reproducibility guide for dependency management: requirements.txt pins 2017 versions; add a REPRODUCIBILITY.md with (a) Python 3.5 installation steps, (b) pre-trained word2vec model download URL, (c) workarounds for deprecated scikit-learn APIs, and (c) Docker setup as fallback.

Top contributors

• [@Sean Moran](https://github.com/Sean Moran) — 10 commits
• @sjmoran — 5 commits

Recent commits

• b715f30 — Minor update to the README.md (sjmoran)
• cc977b8 — 1) Added tf-idf weighting in the word2vec computation (sjmoran)
• f4f1116 — Fixed the rendering of asterix in markdown (sjmoran)
• 336d68b — Updated the top words (sjmoran)
• ca22cd7 — Trimmed the requirements.txt file (sjmoran)
• 7abdbf6 — Fixed the output directory creation (Sean Moran)
• 03a580d — Added experiment name to output directory name (Sean Moran)
• dc5fb87 — Fixed output directory creation (Sean Moran)
• dd8fbd8 — Minor README.md update (Sean Moran)
• 7dd34fb — Minor updates to the README.md (Sean Moran)

Security observations

• High · Outdated Dependency: urllib3 with Known Vulnerabilities — requirements.txt - urllib3==1.22. urllib3==1.22 (released 2017) contains multiple known security vulnerabilities including CVE-2018-20225 (CRLF injection) and CVE-2019-11236 (CRLF injection in request headers). This is a transitive dependency used by requests and other packages. Fix: Upgrade to urllib3>=1.26.0 to address known security vulnerabilities
• High · Outdated Dependency: requests with Known Vulnerabilities — requirements.txt - requests==2.18.4. requests==2.18.4 (released 2017) contains multiple known security vulnerabilities including potential session hijacking and information disclosure issues. Current secure version is 2.31.0+. Fix: Upgrade to requests>=2.31.0 to patch known security issues
• High · Outdated Dependency: certifi Expired Certificate Bundle — requirements.txt - certifi==2017.7.27.1. certifi==2017.7.27.1 contains an outdated certificate bundle. SSL/TLS connections may fail or be vulnerable to man-in-the-middle attacks due to expired or untrusted certificates. Fix: Upgrade to certifi>=2023.7.22 to ensure current CA certificate bundle
• High · Outdated Dependency: scikit-learn with Potential Vulnerabilities — requirements.txt - scikit-learn==0.19.0. scikit-learn==0.19.0 (released 2017) is significantly outdated and may contain unpatched security or stability issues. Current secure version is 1.0.0+. Fix: Upgrade to scikit-learn>=1.3.0 to address known issues and get security patches
• High · Outdated Dependency: gensim with Known Vulnerabilities — requirements.txt - gensim==3.0.1. gensim==3.0.1 (released 2017) is outdated and may contain unpatched vulnerabilities. No security updates are provided for this version. Fix: Upgrade to gensim>=4.3.0 for security patches and bug fixes
• Medium · Outdated Dependency: boto with Deprecated Status — requirements.txt - boto==2.48.0. boto==2.48.0 is the old Python 2 AWS SDK (deprecated in 2020). While not inherently a security risk, it receives no security updates. If AWS integration is present, this is problematic. Fix: Migrate to boto3 (modern AWS SDK) for security updates and Python 3 support
• Medium · Outdated Dependency: numpy with Known Vulnerabilities — requirements.txt - numpy==1.13.3. numpy==1.13.3 (released 2017) is severely outdated. Multiple CVEs exist for this version range including memory disclosure issues. Fix: Upgrade to numpy>=1.24.0 to address known vulnerabilities and compatibility issues
• Medium · Outdated Dependency: pandas with Security Issues — requirements.txt - pandas==0.20.3. pandas==0.20.3 (released 2017) is outdated and may have known security or stability issues. Current versions are 2.0.0+. Fix: Upgrade to pandas>=2.0.0 for security patches and bug fixes
• Medium · Outdated Dependency: scipy — requirements.txt - scipy==0.19.1. scipy==0.19.1 (released 2017) is severely outdated and may contain unpatched vulnerabilities or compatibility issues. Fix: Upgrade to scipy>=1.10.0 to address known issues and get security updates
• Low · Unnecessary Dummy Package: sklearn==0.0 — undefined. The package 'sklearn==0.0' is a dummy met Fix: undefined

LLM-derived; treat as a starting point, not a security audit.

Where to read next

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

---

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.