If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/lukes/ISO-3166-Countries-with-Regional-Codes shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

WAIT — Stale — last commit 2y ago

• 8 active contributors
• Other licensed
• ⚠ Stale — last commit 2y ago
• ⚠ Concentrated ownership — top contributor handles 71% of recent commits
• ⚠ Non-standard license (Other) — review terms
• ⚠ No CI workflows detected
• ⚠ No test directory detected

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live lukes/ISO-3166-Countries-with-Regional-Codes repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/lukes/ISO-3166-Countries-with-Regional-Codes.

What it runs against: a local clone of lukes/ISO-3166-Countries-with-Regional-Codes — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in lukes/ISO-3166-Countries-with-Regional-Codes | Confirms the artifact applies here, not a fork | | 2 | License is still Other | Catches relicense before you depend on it | | 3 | Default branch master exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 709 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of lukes/ISO-3166-Countries-with-Regional-Codes. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/lukes/ISO-3166-Countries-with-Regional-Codes.git
#   cd ISO-3166-Countries-with-Regional-Codes
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of lukes/ISO-3166-Countries-with-Regional-Codes and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "lukes/ISO-3166-Countries-with-Regional-Codes(\\.git)?\\b" \\
  && ok "origin remote is lukes/ISO-3166-Countries-with-Regional-Codes" \\
  || miss "origin remote is not lukes/ISO-3166-Countries-with-Regional-Codes (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(Other)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"Other\"" package.json 2>/dev/null) \\
  && ok "license is Other" \\
  || miss "license drift — was Other at generation time"

# 3. Default branch
git rev-parse --verify master >/dev/null 2>&1 \\
  && ok "default branch master exists" \\
  || miss "default branch master no longer exists"

# 4. Critical files exist
test -f "README.md" \\
  && ok "README.md" \\
  || miss "missing critical file: README.md"
test -f "scrubber.rb" \\
  && ok "scrubber.rb" \\
  || miss "missing critical file: scrubber.rb"
test -f "all/all.json" \\
  && ok "all/all.json" \\
  || miss "missing critical file: all/all.json"
test -f "LAST_UPDATED.txt" \\
  && ok "LAST_UPDATED.txt" \\
  || miss "missing critical file: LAST_UPDATED.txt"
test -f "Gemfile" \\
  && ok "Gemfile" \\
  || miss "missing critical file: Gemfile"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 709 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~679d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/lukes/ISO-3166-Countries-with-Regional-Codes"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

A single-source-of-truth dataset that merges ISO 3166-1 country/territory codes with UN M49 geoscheme regional classifications, published in three ready-to-use formats (JSON, XML, CSV) at three detail levels. The core problem: existing country databases either omit UN regional codes or require manual cross-referencing of two separate sources. This repo scrapes both Wikipedia and UN Statistical Division data then normalizes it into three flavors: all.json (complete with region/sub-region codes), slim-2.json (ISO alpha-2), and slim-3.json (ISO alpha-3). Flat, data-only structure: root contains the generation script (scrubber.rb), three output directories (all/, slim-2/, slim-3/) each with identical format triplets (.json, .csv, .xml), and metadata files (LAST_UPDATED.txt, README.md, LICENSE.md). No src/, lib/, or build pipeline—the single Ruby script reads from upstream sources and writes directly to the three output directories.

Data engineers and backend developers building geo-aware applications (dashboards, analytics platforms, reporting tools) who need canonical country lists pre-joined with continental and sub-regional classifications without writing their own scrapers or maintaining manual mappings.

Low-activity but stable: the repo is a data artifact, not an actively developed application, so sparse commits are expected and not a red flag. No test suite is present (characteristic of data-only repos), and there is no CI/CD pipeline. Last update was 19 June 2024 according to LAST_UPDATED.txt, suggesting annual or semi-annual maintenance cycles tied to official data source refreshes. Verdict: Production-ready as a data source, but not actively developed—treat it as a well-maintained static dataset rather than a growing software project.

Low risk: dependencies are minimal (only Bundler and Nokogiri for scraping, visible in the Gemfile pattern). The critical risk is data staleness—the UN and Wikipedia data are manually scraped, not API-fed, so if upstream sources change their HTML structure, scrubber.rb will break silently and produce incorrect output. Single maintainer (lukes) with no contribution history visible means dependency on one person's maintenance cadence. No automated data validation tests exist, so invalid or duplicated entries could persist undetected until manual review.

No active development visible. The repo appears to be in maintenance mode: LAST_UPDATED.txt and the README indicate data was last refreshed on 19 June 2024. There is no mention of open PRs, issues, or roadmap in the provided materials. Changes would occur only if UN M49 or Wikipedia ISO 3166 data changed significantly, triggering a re-run of scrubber.rb and a commit to update the output files.

git clone https://github.com/lukes/ISO-3166-Countries-with-Regional-Codes.git
cd ISO-3166-Countries-with-Regional-Codes
bundle install
bundle exec ruby scrubber.rb

This will regenerate all output files in all/, slim-2/, and slim-3/ directories from live upstream sources.

Daily commands: There is no 'running' in the traditional sense. To regenerate data: bundle install && bundle exec ruby scrubber.rb. The generated files in all/, slim-2/, and slim-3/ are standalone assets ready to copy into any application—no server or CLI to start.

• README.md — Explains the data source merging strategy (Wikipedia ISO 3166-1 + UN M49 Standard) and available output formats—essential for understanding the repo's purpose.
• scrubber.rb — The ETL script that scrapes and merges ISO and UN regional codes; any data pipeline changes must touch this file.
• all/all.json — The primary output dataset containing all countries with ISO codes and UN regional classifications—the main deliverable.
• LAST_UPDATED.txt — Tracks when the data was last refreshed; critical for monitoring data staleness and scraper success.
• Gemfile — Declares Ruby dependencies for the scrubber script; required to understand and run the data pipeline.

• scrubber.rb (Ruby, Nokogiri (HTML parsing), CSV parsing, JSON/XML generation) — Fetch ISO 3166-1 and UN M49 data from public sources, parse HTML/CSV, merge on country name/code, normalize schema, and generate all output files
  • Failure mode: Network failures, HTML structure changes on sources, ambiguous country name matches → manual intervention required
• Output data files (all/, slim-2/, slim-3/) (JSON, CSV, XML serialization) — Serve pre-generated, normalized country datasets in multiple formats as static files for consumption by external applications
  • Failure mode: File corruption or version mismatch between formats → requires re-running scrubber.rb
• README & documentation (Markdown documentation) — Explain data sources, merging methodology, schema, and available subsets to help consumers understand and integrate the datasets
  • Failure mode: Outdated docs → users misinterpret schema or data quality

• Wikipedia ISO 3166-1 article → scrubber.rb — HTML table with country names, alpha-2, alpha-3, numeric codes scraped and parsed
• UN M49 Standard source → scrubber.rb — CSV/XML data with country codes and UN regional/sub-regional classifications fetched and parsed
• scrubber.rb → all/all.json, all/all.csv, all/all.xml — Merged and normalized dataset written in three formats as source of truth
• all/all.json → slim-2/slim-2.json, slim-3/slim-3.json — Full dataset filtered and projected into lightweight subsets with fewer fields
• all/, slim-2/, slim-3/ files → External applications (APIs, dashboards, etc.) — Static files downloaded or version-controlled for offline use and data distribution

Update Country Data from Sources

1. Verify or update source URLs (Wikipedia ISO 3166-1, UN M49) in scrubber.rb (scrubber.rb)
2. Run the scrubber script to fetch latest data and merge into all/all.* files (scrubber.rb)
3. Regenerate slim-2 and slim-3 subsets by filtering all/all.json (slim-2/slim-2.json)
4. Update LAST_UPDATED.txt with current timestamp (LAST_UPDATED.txt)
5. Commit all/all.* and slim-/ files, then push to master (all/all.json)

Export Data to New Format

1. Modify scrubber.rb to add export logic (e.g., YAML, Protocol Buffers, SQL) after merging (scrubber.rb)
2. Create new output directory (e.g., slim-4/) and add converter that reads from all/all.json (all/all.json)
3. Run scrubber.rb to generate new format files (scrubber.rb)

Add Regional Code Enrichment

1. Identify new regional classification source (e.g., World Bank, ISO 3166-2) (README.md)
2. Extend scrubber.rb to fetch and merge new regional metadata into the merged dataset (scrubber.rb)
3. Update all/all.json schema to include new fields and regenerate all output formats (all/all.json)

• Ruby + scrubber.rb — Lightweight, expressive language for web scraping (Nokogiri), data transformation, and multi-format export without heavyweight frameworks
• JSON/CSV/XML outputs — Supports diverse consumption patterns: JSON for APIs and JavaScript, CSV for spreadsheets, XML for enterprise systems
• Git + plain-text data — Enables version control of datasets, easy diffs when data changes, and no database dependency for distribution

• Manual scraping vs. official APIs

  • Why: Wikipedia and UN do not provide stable machine-readable APIs; web scraping is the only reliable approach
  • Consequence: Scraper is brittle to HTML/page structure changes; requires periodic maintenance

• Multi-format outputs (JSON, CSV, XML) in same repo

  • Why: Different users prefer different formats; single source of truth avoids sync issues
  • Consequence: Scrubber complexity increases; each format must be kept in sync during updates

• Full dataset (all/) plus slim subsets (slim-2/, slim-3/)

  • Why: Slim variants reduce payload for lightweight use cases while full dataset serves power users
  • Consequence: Duplication of data; must regenerate both during each update cycle

• Real-time data updates (manual scraper runs, not continuous sync)
• REST API server (data is static files, not a service)
• Historical versioning of regional classifications (stores only latest snapshot)
• Data validation UI or admin panel (command-line scraper only)
• ISO 3166-2 subdivision codes (focuses only on country-level codes)

• Avg cyclomatic complexity: ~2.5 — Repo is primarily a data distribution service with one ETL script (scrubber.rb); minimal algorithmic complexity, but moderate string/HTML parsing logic

• Web scraping fragility (High) — scrubber.rb: Reliance on Wikipedia and UN HTML structure; any page redesign breaks parsing logic without warning
• Manual data refresh process (Medium) — scrubber.rb, LAST_UPDATED.txt: No automated scheduler; data becomes stale if scraper is not run periodically; no alerting on failure
• Duplicate data across formats (Medium) — all/, slim-2/, slim-3/: Same logical dataset stored in three formats and multiple completeness levels; synchronization risk during updates

• scrubber.rb execution (I/O latency) — Network I/O waiting for Wikipedia and UN sources; single-threaded serial fetch of both sources
• Country name/code matching in scrubber.rb (Data quality / manual overhead) — String matching between Wikipedia and UN datasets; ambiguous names or encoding issues require manual intervention

Data is not authoritative: README explicitly warns that the data is scraped and should be independently verified before use in critical systems—this is a curated convenience dataset, not an official registry. Scraper fragility: scrubber.rb is hardcoded to specific Wikipedia and UN Statistical Division HTML structures; if either site changes their layout, the script will silently produce incomplete or corrupted output (no error handling visible in the description). No validation: no checksums, no schema validation, and no automated tests, so bad data can ship undetected. Manual refresh cycle: updates require someone to manually run scrubber.rb and push a commit; there's no CI/CD automation or scheduled data fetches. Timezone/locale issues: alpha-2 and alpha-3 codes are standardized, but country names may have encoding issues or outdated transliterations if upstream sources aren't carefully normalized.

• ISO 3166-1 (Alpha-2, Alpha-3, Numeric Codes) — This repo's entire premise is to standardize country identification; you need to understand the three ISO code systems (e.g., NZ vs. NZL vs. 554 for New Zealand) and when to use each in APIs, databases, and forms
• UN M49 Geoscheme (Region, Sub-Region, Intermediate-Region Codes) — The defining feature of this dataset is its merger with UN M49 regional classifications; understanding the hierarchy (e.g., Africa → Sub-Saharan Africa → Western Africa) and numeric codes is essential for geo-segmented analytics and reporting
• Web Scraping with Nokogiri (Ruby) — The entire scrubber.rb pipeline is a web scraper using Nokogiri; if you need to maintain or fix data generation, you must understand CSS selectors, HTML parsing, and fragility of scraper dependencies on upstream HTML changes
• Data Serialization (JSON vs. CSV vs. XML) — This repo publishes the same dataset in three formats; understanding trade-offs (JSON for nesting and APIs, CSV for spreadsheets, XML for legacy systems) helps you choose the right format for your consumption use case
• Data Normalization & Denormalization — The all.json contains full denormalized data (every country row repeats region/sub-region names and codes); slim-2 and slim-3 are aggressively normalized. Understanding when to denormalize (for simple consumption) vs. normalize (for storage efficiency) is key to designing your import pipeline
• Character Encoding & Internationalization (UTF-8 in Country Names) — Country names include non-ASCII characters (e.g., Côte d'Ivoire, Réunion); the scraper must handle UTF-8 encoding correctly, and downstream systems must parse and store these names without corruption
• Data Dependency & Staleness Management — This is a derived dataset rebuilt from volatile upstream sources (Wikipedia, UN Statistical Division); you must version it, timestamp it (LAST_UPDATED.txt), and validate it regularly—this teaches lessons about managing data freshness in any system depending on external sources

• mledoze/countries — Alternative comprehensive country dataset (JSON, with REST API option) including many ISO 3166 fields and regional codes; popular in Node.js/JavaScript ecosystems
• umpirsky/country-list — Similar data aggregator for ISO 3166-1 alpha codes available in multiple formats (JSON, YAML, CSV); more minimal but similarly maintenance-focused
• dr5hn/countries-states-cities-database — Extends country data with subdivisions and cities; useful if your application needs finer geographic granularity than UN M49 sub-regions
• unicode-org/cldr — Official Unicode CLDR project distributes canonical country name localization and ISO 3166 mappings; companion data source for translating country names beyond English
• geonames/geonames — GeoNames database provides country and city data with latitude/longitude; complementary for applications needing geographic coordinates alongside ISO codes

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add data validation tests for all/, slim-2/, and slim-3/ datasets

The repo contains three data formats (CSV, JSON, XML) across multiple dataset versions with no visible test suite. A contributor could add unit tests to verify: (1) all records have required ISO 3166-1 and UN M49 fields, (2) numeric codes are valid integers, (3) alpha-2/alpha-3 codes match ISO standards, (4) no duplicate entries exist, (5) XML/JSON/CSV files are consistent. This prevents data corruption during future scrubs and maintains data integrity.

• [ ] Create a test/ directory with test files for each dataset (all.test.js, slim-2.test.js, slim-3.test.js)
• [ ] Parse all/all.json, slim-2/slim-2.json, slim-3/slim-3.json and validate required fields (name, alpha2, alpha3, numeric, region, subregion)
• [ ] Add assertions for ISO 3166-1 compliance (alpha-2 must be 2 chars, alpha-3 must be 3 chars, numeric must be 3 digits)
• [ ] Verify cross-format consistency (JSON, CSV, XML should have identical records for same dataset)
• [ ] Add test runner to Gemfile or package.json and document in README under 'Development' section

Create scraper output validation and diff reporting in scrubber.rb

The scrubber.rb file handles data collection from Wikipedia and UN sources but lacks validation/reporting logic. A contributor could enhance it to: (1) validate scraped data before output, (2) generate a diff report showing what changed since LAST_UPDATED.txt, (3) detect malformed entries, (4) log warnings for missing required fields. This makes future maintenance easier and prevents corrupted data from being committed.

• [ ] Extend scrubber.rb to include a validate_record() method checking required fields (name, alpha2, alpha3, numeric, region)
• [ ] Add diff reporting that compares newly scraped data against existing all/all.json and outputs a summary of added/removed/modified records
• [ ] Implement logging of validation warnings (e.g., missing subregion, invalid code formats) to a scrub_report.txt file
• [ ] Update scrubber.rb to exit with error code if validation fails, preventing bad data from overwriting output files
• [ ] Document the scrubber usage and validation rules in README under 'Contributing' or 'Development' section

Add GitHub Actions workflow to auto-validate data format and consistency on pull requests

There is no CI/CD visible in the repo structure. A contributor could add a GitHub Actions workflow that runs on every PR to: (1) validate JSON/CSV/XML syntax, (2) check data consistency across all three formats, (3) run the test suite from PR #1, (4) verify no duplicate records exist, (5) report on data changes. This catches errors before merge and documents the validation process.

• [ ] Create .github/workflows/validate-data.yml with jobs to lint all JSON/CSV/XML files in all/, slim-2/, slim-3/
• [ ] Add a validation step using jq (for JSON), csvlint (for CSV), and xmllint (for XML) to catch syntax errors
• [ ] Run the test suite from the previous PR suggestion as part of the workflow
• [ ] Add a consistency check step comparing record counts and sample records across JSON/CSV/XML for each dataset
• [ ] Configure workflow to fail if validation fails, blocking PR merge and reporting errors as annotations in PR

• Add test suite for scrubber.rb output validation: Create a spec file (e.g., spec/scrubber_spec.rb using RSpec) to validate that: (1) all JSON files contain no duplicate country codes, (2) every entry in all.json has the required 11 fields (name, alpha-2, alpha-3, etc.), (3) every entry in slim-2.json and slim-3.json has exactly 3 fields, (4) region-code and sub-region-code values match UN M49 standards. Would catch silent data corruption immediately.
• Document scrubber.rb source URLs and parsing logic inline: Add comments to scrubber.rb identifying which lines scrape from Wikipedia ISO 3166 vs. UN M49, what CSS selectors or regex patterns are used, and when this last worked (e.g., '2024-06-19: Wikipedia ISO article used table#wikitable with columns [0]=name, [1]=alpha-2...'). Future maintainers need to know exactly where to debug if the scraper breaks.
• Add CSV/XML schema documentation: Create a SCHEMA.md documenting the column headers in all three CSV variants and XML element names with ISO 3166 / UN M49 standard references. Currently only JSON is shown in the README. Helpful for downstream consumers using CSV or XML exports.

This repository is a data-only project with minimal security risks. No hardcoded secrets, injection vulnerabilities, or misconfigurations were identified in the provided file structure. The main security concerns are operational: dependency management should be regularly audited, web scraping practices should be responsible, and data integrity verification mechanisms would improve trustworthiness. The project demonstrates good security practices by not including sensitive configuration or credentials.

• Low · Ruby Gemfile Dependencies Not Reviewed — Gemfile, Gemfile.lock. The Gemfile and Gemfile.lock are present but content was not provided for analysis. Ruby gems can contain vulnerabilities that should be regularly audited. Fix: Run 'bundle audit' regularly to check for known vulnerabilities in dependencies. Keep gems updated to their latest secure versions.
• Low · Potential Web Scraping Without Rate Limiting — scrubber.rb. The scrubber.rb script indicates data is scraped from Wikipedia and UN websites. Without proper rate limiting or user-agent headers, this could be flagged as abuse by target websites. Fix: Implement appropriate rate limiting, respectful user-agent headers, and cache responses. Consider using official data feeds or APIs instead of web scraping.
• Low · Static Data Files Without Integrity Verification — all/*, slim-2/*, slim-3/*. CSV, JSON, and XML data files in the repository lack checksums or digital signatures to verify their integrity and authenticity. Fix: Add checksums (SHA-256) or digital signatures for data files to allow users to verify file integrity. Document the verification process in README.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.