If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/EvanLi/Github-Ranking shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

WAIT — Solo project — review before adopting

• Last commit today
• MIT licensed
• ⚠ Solo or near-solo (1 contributor active in recent commits)
• ⚠ No CI workflows detected
• ⚠ No test directory detected

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live EvanLi/Github-Ranking repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/EvanLi/Github-Ranking.

What it runs against: a local clone of EvanLi/Github-Ranking — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in EvanLi/Github-Ranking | Confirms the artifact applies here, not a fork | | 2 | License is still MIT | Catches relicense before you depend on it | | 3 | Default branch master exists | Catches branch renames | | 4 | 3 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 30 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of EvanLi/Github-Ranking. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/EvanLi/Github-Ranking.git
#   cd Github-Ranking
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of EvanLi/Github-Ranking and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "EvanLi/Github-Ranking(\\.git)?\\b" \\
  && ok "origin remote is EvanLi/Github-Ranking" \\
  || miss "origin remote is not EvanLi/Github-Ranking (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(MIT)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"MIT\"" package.json 2>/dev/null) \\
  && ok "license is MIT" \\
  || miss "license drift — was MIT at generation time"

# 3. Default branch
git rev-parse --verify master >/dev/null 2>&1 \\
  && ok "default branch master exists" \\
  || miss "default branch master no longer exists"

# 4. Critical files exist
test -f "README.md" \\
  && ok "README.md" \\
  || miss "missing critical file: README.md"
test -f "Data/github-ranking-2019-05-01.csv" \\
  && ok "Data/github-ranking-2019-05-01.csv" \\
  || miss "missing critical file: Data/github-ranking-2019-05-01.csv"
test -f ".gitignore" \\
  && ok ".gitignore" \\
  || miss "missing critical file: .gitignore"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 30 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~0d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/EvanLi/Github-Ranking"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

Github-Ranking is an automated daily snapshot system that crawls GitHub's API and publishes ranked lists of repositories sorted by stars and forks across 40+ programming languages. It generates dated CSV files (stored in Data/) and maintains a comprehensive markdown README with top 100 lists per language, updated via CI/CD at 2026-05-07T04:07:06Z. The core capability is tracking GitHub ecosystem trends in real-time. Simple flat structure: Data/ directory contains dated CSV snapshots (github-ranking-YYYY-MM-DD.csv), a root README.md with markdown tables for each language, and .gitignore. Language breakdown is detected at collection time (Python 13,924 lines, Shell 434, Ruby 43), suggesting Python is the primary scraper. No src/ or lib/ folders visible—scripts likely run as cron jobs outside the repo.

Data analysts and researchers studying open-source trends, GitHub explorers looking for popular projects by language, and maintainers benchmarking their repositories' visibility against peers. Used by developers making technology selection decisions based on community adoption signals.

Actively developed and stable: the repo has 499k+ stars on the most-starred project tracked (build-your-own-x), continuous daily updates through 2026, and a 7+ year history (data from Dec 2018 onward). Last automatic update was 2026-05-07, indicating reliable automation. However, no visible test suite or CI config in the file list suggests it relies on simple, battle-tested scraping logic.

Low technical risk but maintenance-dependent: the project depends entirely on GitHub API stability and scraping patterns—if GitHub's HTML/API structure changes, parsing breaks. Single maintainer (EvanLi) visible in the description with no apparent fallback or team. The 60+ CSV files are raw data snapshots with no versioning system, so data integrity relies on append-only collection discipline.

Daily automated updates: CSV files show continuous collection from Dec 2018 through Feb 2019 (and metadata shows updates through May 2026), with occasional skipped days (e.g., 2018-12-29, 2019-01-26, 2019-01-27 gaps suggest job failures or manual pauses). The README reflects the most recent snapshot. No open issues or PRs are visible in the provided data.

git clone https://github.com/EvanLi/Github-Ranking.git
cd Github-Ranking
# No install needed: read the Data/ CSVs or view README.md directly

The repo is data-only; no build step required. To regenerate data, you would run the (untracked) Python scraper scripts with GitHub API credentials.

Daily commands: This repo doesn't run as a service. To view the data: open Data/github-ranking-LATEST.csv in a spreadsheet or cat it in a terminal. To regenerate rankings, you would execute the hidden Python script (not in the file list) with export GITHUB_TOKEN=xxx && python scraper.py, which would fetch current stars/forks and rebuild the Data/ CSVs and README.md.

• README.md — Primary entry point documenting the project's purpose, structure, and links to all language-specific rankings; essential for understanding the repo's scope and navigation
• Data/github-ranking-2019-05-01.csv — Example of the daily CSV data output that forms the core deliverable; shows the data schema and ranking format that the automation produces
• .gitignore — Defines which files are excluded from version control; critical for understanding how the automated daily updates are managed

• GitHub API Client (GitHub REST API v3) — Queries GitHub for repository metadata (stars, forks, language) across all tracked languages
  • Failure mode: Rate limit exceeded or API downtime causes daily snapshot to fail; previous day's data becomes stale
• Ranking Engine (Custom sorting algorithm (likely Python or shell script)) — Sorts repositories by star count and fork count, groups by language, and selects Top 100 per category
  • Failure mode: Incorrect sorting logic produces misordered rankings; manual verification required to detect
• Data Persistence (CSV) (CSV file format, git version control) — Stores daily snapshots as date-stamped CSV files for historical analysis and trend tracking
  • Failure mode: Corrupted CSV file is committed; requires git revert or manual cleanup
• Documentation (Markdown) (Markdown, GitHub rendering) — Renders Top 100 rankings per language and overall stats in human-readable markdown format on GitHub
  • Failure mode: Broken table formatting or missing language sections reduce discoverability

• GitHub API → Ranking Engine — Raw repository metadata (name, stars, forks, language) flows from GitHub for all queried languages
• Ranking Engine → CSV Generator — Sorted and ranked repository lists flow to CSV output for storage as daily snapshot
• Ranking Engine → Markdown Generator — Ranked data flows to markdown formatter to produce language-specific Top 100 tables
• CSV Generator & Markdown Generator → Git Repository — Generated files are committed and pushed to GitHub to update README.md and Data/ directory
• Git Repository → Browser / End User — Static markdown files and CSV snapshots are served via GitHub Pages and browsed by users

Add a new programming language ranking

1. Update the automation script to query GitHub API for repositories in the new language (README.md (reference the automation process documented here))
2. Generate a new markdown file for the language in the Top100 directory following the existing language naming convention (Top100/Top-100-[language].md (create following the pattern of other language files))
3. Add a new entry to the Table of Contents in README.md linking to the new language section (README.md)

Update the daily automated ranking data

1. The automation script queries GitHub API for current star and fork counts across all tracked languages (Data/ (directory where new dated CSVs are written daily))
2. A new CSV file is generated with today's date in the format github-ranking-YYYY-MM-DD.csv (Data/github-ranking-2019-05-01.csv (example of expected output format))
3. Commit and push the new daily snapshot to maintain the historical record (README.md (see 'Last Automatic Update Time' field))

Add a new repository to the rankings

1. No manual intervention required; repositories appear automatically based on GitHub API query results (Data/github-ranking-2019-05-01.csv (rankings are algorithmically generated))
2. If a new language ecosystem should be tracked, update the language list in the automation script (README.md (document the new language in the Table of Contents))

• GitHub API — Provides real-time access to repository metadata (stars, forks, language) without needing to clone or maintain local mirrors
• CSV format for daily snapshots — Lightweight, version-control friendly, and easily queryable for historical trend analysis across months of data
• Markdown documentation — Human-readable, renders natively on GitHub, supports easy linking and navigation for users browsing language-specific rankings

• Daily snapshot approach rather than real-time API

  • Why: Reduces GitHub API quota consumption and provides stable, reproducible historical records without rate-limit pressure
  • Consequence: Rankings are at most 24 hours stale; users cannot see minute-to-minute changes in star counts

• Store entire Top 100 per language in separate markdown files rather than a single database

  • Why: Simple file-based structure that requires no backend infrastructure, database, or hosting beyond GitHub Pages
  • Consequence: No real-time filtering, aggregation, or complex queries; users must browse static files

• Separate CSV files per day rather than appending to a single growing file

  • Why: Prevents any single file from becoming unwieldy and allows git to efficiently store historical diffs
  • Consequence: Directory becomes large over time (600+ files as shown); requires date-based navigation to find specific snapshots

• Does not provide real-time ranking updates or streaming notifications
• Does not include user authentication or private repository rankings
• Does not perform sentiment analysis, quality metrics, or code-quality scoring on repositories
• Does not offer an interactive web UI or search functionality—rankings are static markdown/CSV files
• Does not track forks or stars over intra-day intervals; captures only daily snapshots
• Does not archive deleted or deprecated repositories

• Avg cyclomatic complexity: ~2 — Codebase is primarily data-driven with simple sorting and CSV/markdown generation; no complex algorithms or state management
• Largest file: README.md (5,000 lines)
• Estimated quality issues: ~3 — Inferred issues: lack of documented error handling, no automated testing visible, manual sync between markdown and CSV outputs, unbounded data growth

• Manual synchronization of markdown and CSV data (Medium) — README.md + Data/*.csv: Top 100 markdown tables and CSV files likely contain overlapping data with no programmatic sync; risk of divergence if one is updated without the other
• No explicit error handling or retry logic documented (Medium) — Daily automation process (inferred from file list): If GitHub API rate limits or fails, the automation may skip a day or produce incomplete data; no visible error reporting mechanism
• Unbounded directory growth (Low) — Data/: 600+ daily CSV files accumulate without pruning; no documented archival or cleanup strategy. Over 5+ years this could exceed practical limits

• GitHub API query step (Ranking Engine) (I/O bound) — Querying top 100 repos across 40+ languages multiplies API calls; could hit rate limits (5,000 req/hour for authenticated requests)
• Git push of large CSV and markdown files daily (Storage bound) — Pushing new files daily to the repository; repository size grows by ~600 files, increasing clone time and storage

The scraper is not version-controlled in this repo—only its output (CSVs and README) is tracked. Reproducing rankings requires the missing Python script with GitHub API authentication (GITHUB_TOKEN env var likely required). CSV schema is implicit; column order and format are not documented. Gaps in the Data/ timeline (missing dates) indicate unhandled failures in the automation—expect incomplete historical data for those days. The README's 'Last Automatic Update Time' is hardcoded metadata, not auto-generated, so it can drift.

• GitHub REST API v3 Pagination — The scraper must handle GitHub's paginated search results to fetch top 100+ repos per language efficiently; understanding rate limits (60 req/hr unauthenticated, 5000/hr authenticated) is critical to avoid scraper failures
• Time-series Data Snapshots — This repo uses dated immutable CSVs as a time-series—each day's data is a point-in-time snapshot allowing trend analysis; understanding snapshot semantics (whether repos removed from top 100 are lost forever) affects analytical validity
• Idempotent Daily Cron Jobs — The automation runs once per day; must handle edge cases like duplicate runs, skipped days, and midnight timezone boundaries without corrupting historical data
• Language Detection in GitHub — GitHub assigns a 'primary language' to each repo based on code analysis; this ranking filters results by that label, so understanding GitHub's linguistic heuristics (e.g., why TypeScript vs. JavaScript repos cluster) affects result interpretation
• Markdown Table Generation — The README contains 40+ language sections, each with auto-generated markdown tables; the build process must render CSV data into sorted, linked tables without manual effort
• GitHub Stars as Popularity Proxy — Stars are a noisy signal of project quality (gamed by trending bots, skewed toward visual/entertaining projects); using them as a ranking metric requires acknowledging survivorship bias and the difference between popularity and technical merit
• CSV as Immutable Data Lake — Storing rankings as append-only CSVs (not overwriting) creates an audit trail; enables historical trend queries (e.g., 'how many repos entered top 100 since 2019?') but requires careful file naming and schema stability

• sindresorhus/awesome — The #2 most-starred repo in this ranking (463k stars); a curated list of awesome projects that complements trend-based discovery
• github/gitignore — Canonical .gitignore templates; relevant if this project wants to auto-suggest ignore patterns for new GitHub users
• donnemartin/system-design-primer — The #8 most-starred repo tracked here; exemplifies the kind of educational content repositories that dominate these rankings
• kamranahmedse/developer-roadmap — Similar trending project tracker and learning resource; competes in the educational/reference category alongside repos ranked in this list
• EvanLi/Github-Ranking-Analysis — Likely companion repo (if it exists) for statistical analysis of ranking trends over time using the CSV snapshots

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add automated data quality validation script for CSV files

The Data/ directory contains 60+ CSV files with daily snapshots spanning from 2018-12-18 to 2019-03-15. There are visible gaps (e.g., 2018-12-29, 2019-01-26, 2019-02-23, 2019-03-06/07/09-11), but no validation mechanism exists to ensure data consistency, prevent duplicates, or detect schema changes across files. A validation script would help maintainers catch data issues before the next daily update runs.

• [ ] Create scripts/validate_csv_data.py to check all CSV files in Data/ directory for consistent columns, missing values, and duplicate rows
• [ ] Add validation checks for repository URL/ID uniqueness within each daily file
• [ ] Add a GitHub Actions workflow (.github/workflows/validate-data.yml) to run this script on each commit that modifies Data/
• [ ] Document in README.md how to run the validation manually and what issues it detects

Create a changelog/data schema documentation file

The README.md is truncated and doesn't document the actual CSV schema (column names, data types, what each represents). New contributors won't understand the data structure or how to properly analyze the rankings. A DATA_SCHEMA.md file would clarify what each column represents and how the daily snapshots are meant to be used.

• [ ] Create docs/DATA_SCHEMA.md documenting all CSV column headers (rank, repository name, stars, forks, language, etc.)
• [ ] Include example rows and explain what each metric represents
• [ ] Add information about how the daily snapshot schedule works and why certain days are missing
• [ ] Link this file prominently from README.md in a 'Data Format' or 'Understanding the Data' section

Add data analysis/export utilities for contributors

With 60+ daily CSV files, potential contributors might want to analyze trends (e.g., which repos gained most stars over time) or export language-specific rankings to markdown. Currently there's no utility code to make this easy. Providing Python scripts for common analysis tasks would lower the barrier for contributions.

• [ ] Create scripts/analyze_trends.py to compare rankings across two dates and identify biggest gainers/losers
• [ ] Create scripts/export_language_rankings.py to generate markdown tables for each programming language from the raw CSV data
• [ ] Add usage examples in CONTRIBUTING.md (or create it) showing how to run these scripts
• [ ] Include requirements.txt with pandas dependency so contributors can easily install needed packages

• Document the CSV schema in a SCHEMA.md file listing column names, data types, and examples from an actual row in Data/github-ranking-2019-02-16.csv—currently undocumented and inferred only.
• Add missing dates to the Data/ directory by backfilling gaps (e.g., 2018-12-29, 2019-01-26/27 are missing)—requires re-running the scraper against the Wayback Machine or GitHub's API archive if available.
• Create a Data/README.md index listing all CSV files with row counts and date ranges to help explorers understand coverage and find relevant snapshots for analysis.

The 'Github-Ranking' repository appears to be primarily a data collection and storage project with limited apparent security risks. However, critical information is missing: no dependency files were provided, making it impossible to assess vulnerable dependencies. The project consists mainly of CSV data files with automated daily updates. Key concerns include: (1) Unknown dependencies that may contain vulnerabilities, (2) Lack of visible security configuration for any automation or APIs, (3) No evidence of input validation for externally sourced GitHub data. The project would benefit from dependency manifest files, security scanning tools integration, and documented security practices for the automated update process. For a data-focused repository with no apparent web services or complex logic, the security posture is reasonably adequate, but dependency management must be verified.

• Medium · Missing Dependency File — Repository root. No package.json, requirements.txt, Gemfile, or other dependency manifest file was provided in the analysis. This makes it impossible to assess whether the project uses vulnerable dependencies. The codebase likely has dependencies that should be tracked and audited. Fix: Provide and maintain a dependency manifest file (package.json for Node.js, requirements.txt for Python, etc.). Use dependency scanning tools like npm audit, pip-audit, or Snyk to regularly check for vulnerable dependencies.
• Low · No Security Configuration Files Detected — Repository root. No security-related configuration files (.env, security headers config, CORS policy, etc.) were found in the provided file structure. While this could indicate minimal security configuration needs for a data repository, it may also indicate missing security hardening. Fix: If the project includes any web services or APIs, implement proper security configurations including environment variable management, CORS policies, security headers, and input validation.
• Low · Large Number of CSV Data Files — Data/ directory. The Data directory contains numerous CSV files with GitHub ranking data. If these files are large or frequently updated through automated processes, ensure data handling doesn't introduce injection vulnerabilities when parsing or processing CSV content. Fix: Implement safe CSV parsing libraries with proper input validation. Avoid using string concatenation or eval-like functions when processing CSV data. Validate all data before use in any downstream systems.
• Low · Automated Daily Updates Without Visible Security Controls — Automation/CI-CD pipeline (not shown). The README indicates 'Automatically update daily' but no automation, CI/CD, or version control security mechanisms are visible in the provided file structure. Automated data collection from external sources (GitHub API) could introduce risks if not properly validated. Fix: Implement signed commits, branch protection rules, and audit logs for automated updates. Validate all data from GitHub API before storing. Use API rate limiting and authentication. Implement integrity checks on data sources.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.