RepoPilotOpen in app →

thiagopradi/octopus

Database Sharding for ActiveRecord

Concerns

Stale and unlicensed — last commit 2y ago

worst of 4 axes
Use as dependencyConcerns

no license — legally unclear; last commit was 2y ago

Fork & modifyConcerns

no license — can't legally use code; last commit was 2y ago

Learn fromHealthy

Documented and popular — useful reference codebase to read through.

Deploy as-isConcerns

no license — can't legally use code; last commit was 2y ago

  • 15 active contributors
  • CI configured
  • Tests present
Show 3 more →
  • Stale — last commit 2y ago
  • Concentrated ownership — top contributor handles 58% of recent commits
  • No license — legally unclear to depend on
What would change the summary?
  • Use as dependency ConcernsMixed if: publish a permissive license (MIT, Apache-2.0, etc.)
  • Fork & modify ConcernsMixed if: add a LICENSE file
  • Deploy as-is ConcernsMixed if: add a LICENSE file

Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests

Informational only. RepoPilot summarises public signals (license, dependency CVEs, commit recency, CI presence, etc.) at the time of analysis. Signals can be incomplete or stale. Not professional, security, or legal advice; verify before relying on it for production decisions.

Embed the "Great to learn from" badge

Paste into your README — live-updates from the latest cached analysis.

RepoPilot: Great to learn from
[![RepoPilot: Great to learn from](https://repopilot.app/api/badge/thiagopradi/octopus?axis=learn)](https://repopilot.app/r/thiagopradi/octopus)

Paste at the top of your README.md — renders inline like a shields.io badge.

Preview social card (1200×630)

This card auto-renders when someone shares https://repopilot.app/r/thiagopradi/octopus on X, Slack, or LinkedIn.

Onboarding doc

Onboarding: thiagopradi/octopus

Generated by RepoPilot · 2026-05-10 · Source

🤖Agent protocol

If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

  1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
  2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
  3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/thiagopradi/octopus shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

🎯Verdict

AVOID — Stale and unlicensed — last commit 2y ago

  • 15 active contributors
  • CI configured
  • Tests present
  • ⚠ Stale — last commit 2y ago
  • ⚠ Concentrated ownership — top contributor handles 58% of recent commits
  • ⚠ No license — legally unclear to depend on

<sub>Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests</sub>

Verify before trusting

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live thiagopradi/octopus repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/thiagopradi/octopus.

What it runs against: a local clone of thiagopradi/octopus — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in thiagopradi/octopus | Confirms the artifact applies here, not a fork | | 2 | Default branch master exists | Catches branch renames | | 3 | Last commit ≤ 843 days ago | Catches sudden abandonment since generation |

<details> <summary><b>Run all checks</b> — paste this script from inside your clone of <code>thiagopradi/octopus</code></summary>
#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of thiagopradi/octopus. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/thiagopradi/octopus.git
#   cd octopus
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of thiagopradi/octopus and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "thiagopradi/octopus(\\.git)?\\b" \\
  && ok "origin remote is thiagopradi/octopus" \\
  || miss "origin remote is not thiagopradi/octopus (artifact may be from a fork)"

# 3. Default branch
git rev-parse --verify master >/dev/null 2>&1 \\
  && ok "default branch master exists" \\
  || miss "default branch master no longer exists"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 843 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~813d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/thiagopradi/octopus"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

</details>

TL;DR

ar-octopus (Octopus) is a Ruby gem that adds transparent database sharding and master-slave replication support to ActiveRecord. It lets Rails applications distribute queries across multiple databases and read replicas while maintaining a simple, unified ORM interface—solving the problem of scaling single-database applications without rewriting data access logic. Monolithic gem with a proxy-pattern architecture: lib/octopus/proxy.rb and lib/octopus/relation_proxy.rb intercept ActiveRecord queries; lib/octopus/model.rb extends Rails models with shard-aware methods; lib/octopus/shard_tracking/ handles attribute-based and dynamic shard assignment; lib/octopus/load_balancing/ implements round-robin replica selection; lib/octopus/migration.rb manages cross-shard schema changes. sample_app/ contains a minimal Rails 4/5 integration demo.

👥Who it's for

Rails developers (Rails 4–5, moving to maintenance mode for Rails 6+) who need to scale beyond a single database; specifically backend engineers building multi-tenant systems, horizontally-sharded databases, or high-read-volume applications with master-slave replication.

🌱Maturity & risk

Production-ready but entering maintenance mode: the gem has CI (Travis), code climate tracking, test coverage (Gherkin feature specs in spec/), and supports multiple Rails versions via Appraisals gemfiles. However, the README explicitly states Octopus will enter maintenance mode once Rails 6 ships (Rails 6 includes native sharding support), signaling the project's active development has plateaued.

Moderate risk: single primary maintainer (thiagopradi), no recent commit visibility in file list suggests potential staleness, and Rails 6+ will supersede it. The gem deeply patches ActiveRecord (proxy-based interception in lib/octopus/proxy.rb, lib/octopus/relation_proxy.rb), making upgrades to new Rails versions fragile. Upgrading from pre-0.5.0 requires running schema version migrations (rake task), adding operational complexity.

Active areas of work

Project is in maintenance mode awaiting Rails 6 release; no active development visible from file list. The README indicates a migration guide will be published once Rails 6 beta lands, but the gem itself is not actively being enhanced beyond bug fixes.

🚀Get running

Clone and install: git clone https://github.com/thiagopradi/octopus.git && cd octopus && bundle install. To run specs: bundle exec rspec (uses .rspec config). To try the sample app: cd sample_app && bundle install && rails s.

Daily commands: bundle exec rspec to run full test suite; cd sample_app && rails s to run example Rails app on localhost:3000 (after configuring config/database.yml with multiple shards).

🗺️Map of the codebase

  • lib/octopus/proxy.rb: Core proxy object that intercepts all ActiveRecord calls and routes them to the correct shard/replica; understanding this is essential to grasp how Octopus works
  • lib/octopus/model.rb: Extends ActiveRecord::Base with shard-aware class and instance methods (using_shard, current_shard); the main API users interact with
  • lib/octopus/relation_proxy.rb: Wraps ActiveRecord::Relation to propagate shard context through query chains; critical for making shard selection work across .where().order().limit() calls
  • lib/octopus/shard_tracking/attribute.rb: Implements attribute-based shard assignment (e.g., shard_id column determines which shard a record lives on); a primary use case for automatic shard routing
  • lib/octopus/proxy_config.rb: Manages the configuration object that holds shard definitions, replica groups, and connection pools; loaded from database.yml/octopus.yml at boot
  • lib/octopus/migration.rb: Provides rake tasks to run migrations across multiple shards; enables schema synchronization critical for multi-shard deployments
  • lib/octopus/load_balancing/round_robin.rb: Implements read-replica selection strategy; shows how replicas are chosen for read queries in replication setups
  • sample_app/config/octopus.yml: Example configuration showing shards, replicas, and slave groups; the canonical reference for how to configure Octopus in a real app

🛠️How to make changes

Start in lib/octopus/model.rb to add new shard-aware query methods. For replication logic, edit lib/octopus/load_balancing/round_robin.rb. For shard assignment strategies, modify lib/octopus/shard_tracking/attribute.rb or lib/octopus/shard_tracking/dynamic.rb. Add feature specs to spec/features/ following Gherkin format. Database config tested via sample_app/config/octopus.yml.

🪤Traps & gotchas

  1. Rails 6+ incompatible: Gem will not work with Rails 6+; requires immediate migration to Rails native sharding. 2. Database.yml must define master + all shards: missing shard configs silently fail; no validation errors. 3. Shard selection is per-query-context: forgetting using_shard() in a controller or model method will query the default shard, causing silent data leaks. 4. Schema migration complexity: upgrading from < 0.5.0 requires running rake octopus:copy_schema_versions to move schema tracking from master to individual shards; missing this step breaks schema management. 5. Proxy pattern overhead: all model instantiation goes through lib/octopus/proxy.rb, adding latency; no documented performance benchmarks.

💡Concepts to learn

  • Database Sharding / Horizontal Partitioning — Octopus's core problem domain; you need to understand that sharding distributes rows across multiple physical databases by a key (shard_id) to grasp why shard selection is mandatory
  • Master-Slave Replication (Read Replicas) — Octopus routes writes to master and reads to replicas; understanding replication lag and consistency implications is essential for using replication mode correctly
  • Proxy Pattern — Octopus uses proxies (lib/octopus/proxy.rb, relation_proxy.rb) to intercept ActiveRecord method calls without modifying the ORM itself; understanding this pattern explains how shard context flows through query chains
  • Connection Pooling — Octopus maintains separate connection pools per shard (managed in proxy_config.rb); misconfiguration can exhaust database connections or cause deadlocks
  • Thread-Local Storage (Context Variables) — Octopus uses thread-local variables to track current shard context across request/response cycles; understanding thread-safety is critical for concurrent request handling
  • Round-Robin Load Balancing — lib/octopus/load_balancing/round_robin.rb distributes read queries across replicas in sequence; understanding its fairness guarantees (and lack thereof for skewed workloads) matters for production deployments
  • ActiveRecord Query Interception / Relation Chaining — Octopus patches ActiveRecord::Relation to preserve shard context across chained methods (.where, .joins, .limit); understanding how Relation lazy-loads queries is key to debugging shard selection bugs
  • rails/rails — Rails 6+ includes native database sharding (PR #34052 mentioned in README); Octopus users will migrate here
  • ankane/dexter — Complementary tool for Rails that analyzes query performance and suggests indexes; pairs well with Octopus to optimize sharded queries
  • palkan/database_consistency — Validates schema consistency across multiple databases; critical for maintaining coherent schema across Octopus shards during deployments
  • noelboss/activerecord-typedstore — Handles polymorphic attribute storage in Rails models; useful for shard-agnostic data modeling in Octopus multi-shard contexts
  • collectiveidea/interactor — Service object pattern library; recommended for encapsulating shard-aware business logic outside of models (common Octopus antipattern)

🪄PR ideas

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add comprehensive test coverage for load balancing strategies in lib/octopus/load_balancing/

The load_balancing directory only contains round_robin.rb, but there's no visible test file for load balancing logic. Given that load balancing is critical for shard distribution, adding unit tests would catch regressions and document expected behavior. This is especially important since the repo enters maintenance mode with Rails 6.

  • [ ] Create spec/octopus/load_balancing/round_robin_spec.rb with tests for round-robin distribution logic
  • [ ] Add edge case tests: empty shard lists, single shard, concurrent access patterns
  • [ ] Test the load_balancing module integration with proxy.rb and relation_proxy.rb
  • [ ] Verify tests pass against multiple Rails versions (4.2, 5.0, 5.1, 5.2 from Appraisals)

Add integration tests for shard_tracking strategies in lib/octopus/shard_tracking/

The shard_tracking directory has two implementations (attribute.rb and dynamic.rb) but there's no visible test coverage. These are core to determining which shard operations hit, making this high-value. Tests should verify both strategies work correctly with association_shard_tracking.rb and singular_association.rb.

  • [ ] Create spec/octopus/shard_tracking/attribute_spec.rb with tests for attribute-based shard resolution
  • [ ] Create spec/octopus/shard_tracking/dynamic_spec.rb with tests for dynamic shard resolution
  • [ ] Add tests that verify shard tracking persists correctly through model.rb and persistence.rb
  • [ ] Test edge cases: missing attributes, nil shards, associations crossing shard boundaries

Add Rails 6 compatibility check and deprecation warnings in lib/octopus/railtie.rb

The README explicitly states maintenance mode begins with Rails 6 beta, but there's no code enforcement of this. Adding runtime warnings when Rails 6+ is detected would help users understand the transition path early, improving the user experience during the deprecation period mentioned in the README.

  • [ ] Modify lib/octopus/railtie.rb to detect Rails version at initialization
  • [ ] Add a Rails.logger.warn when Rails 6.0+ is detected, pointing to migration guide
  • [ ] Create spec/octopus/railtie_spec.rb to test the warning behavior
  • [ ] Link deprecation warning to a MIGRATION_GUIDE.md file in the root (create if missing)

🌿Good first issues

  • Add integration tests for Rails 5.2 shard-switching in deeply-nested associations (sample_app lacks comprehensive association.rb test coverage for sharding across has_many through)
  • Document the ProxyConfig YAML schema with validation examples; lib/octopus/proxy_config.rb accepts arbitrary YAML but README lacks troubleshooting section for typos in octopus.yml (e.g., shard names vs. shard IDs confusion)
  • Extend lib/octopus/load_balancing/ with a least-connections strategy or configurable failover retry logic; currently only round_robin.rb exists, limiting operators' ability to tune replica selection for uneven load

Top contributors

Click to expand

📝Recent commits

Click to expand
  • 1c0abed — Updating Wiki Links (Thiago Pradi)
  • 43c367e — Octopus 0.10.2 (thiagopradi)
  • 7cb9aa6 — Merge pull request #513 from craigmcnamara/patch-1 (thiagopradi)
  • 26ff9eb — Merge pull request #516 from kyuden/mantain_shard_after_where_not (thiagopradi)
  • 49c9b80 — Merge pull request #510 from jkhart/master (thiagopradi)
  • 53cede2 — Fix a compatibility for sqlite3 gem (kyuden)
  • eab334f — Mantain current shard with enum method after where.not (kyuden)
  • 2aa9468 — Update README.mkdn (Craig McNamara)
  • f7d658d — bugfix: transaction should not reassign @klass variable (jkhart)
  • b900beb — Octopus 0.10.1 (thiagopradi)

🔒Security observations

  • High · Outdated Rails Version Support — gemfiles/ directory, ar-octopus.gemspec. The codebase supports Rails 4.2, 5.0, 5.1, and 5.2 based on gemfiles. These versions are no longer receiving security updates. Rails 4.2 reached EOL in April 2016, Rails 5.0 in June 2018, and Rails 5.1 in December 2019. Running outdated Rails versions exposes the application to known security vulnerabilities. Fix: Update to Rails 6.0+ or at minimum Rails 5.2 LTS. Establish a policy for upgrading to supported versions within 12 months of release.
  • High · Maintenance Mode Status — README.mkdn. According to the README, Octopus will enter maintenance mode once Rails 6 is released. This means the project is not actively maintained and may not receive security updates for new vulnerabilities discovered after maintenance mode begins. Fix: Plan migration to Rails 6+ built-in database sharding features. Do not rely on this gem for new projects. For existing projects, establish a timeline for migration within 12-18 months.
  • Medium · Potential SQL Injection in Dynamic Query Building — lib/octopus/finder_methods.rb, lib/octopus/relation_proxy.rb, lib/octopus/proxy.rb. The codebase appears to handle database sharding and query routing through multiple files (finder_methods.rb, relation_proxy.rb, proxy.rb). Dynamic query construction in sharding systems can be prone to SQL injection if user input is not properly parameterized. Fix: Review all SQL query construction to ensure parameterized queries are used exclusively. Conduct static analysis with tools like Brakeman. Never concatenate user input directly into SQL queries.
  • Medium · Missing Security Configuration Best Practices — sample_app/config/initializers/. The sample app config files (sample_app/config/initializers/secret_token.rb, session_store.rb) may contain hardcoded or insecurely configured secrets. Rails initializers often contain sensitive configuration that should be environment-based. Fix: Use environment variables for all secrets. Implement secret management using Rails credentials or a dedicated secret management system. Never commit secrets to version control.
  • Medium · Insufficient Input Validation in Shard Selection — lib/octopus/shard_tracking/, lib/octopus/association_shard_tracking.rb. The shard tracking and selection mechanism (lib/octopus/shard_tracking/, lib/octopus/association_shard_tracking.rb) dynamically determines which shard to query. Improper validation of shard identifiers could allow attackers to access unintended data partitions. Fix: Implement strict validation and whitelisting of shard identifiers. Use an allowlist of valid shards. Implement access controls to ensure users can only access permitted shards.
  • Medium · Load Balancing Configuration Risk — lib/octopus/load_balancing/, lib/octopus/load_balancing/round_robin.rb. The load balancing module (lib/octopus/load_balancing/) implements round-robin distribution across replica databases. Misconfiguration could lead to uneven load distribution, denial of service, or unintended replica access patterns. Fix: Document load balancing configuration thoroughly. Implement monitoring for replica lag and unavailability. Add circuit breaker patterns for failed replicas. Use health checks before routing queries.
  • Low · Insufficient Logging of Sensitive Operations — lib/octopus/log_subscriber.rb, lib/octopus/migration.rb. The log_subscriber.rb and migration.rb files handle logging for shard operations. Insufficient logging of shard selection, migration, and failover events could hinder security incident investigation. Fix: Implement comprehensive audit logging for all shard-related operations including shard selection, data migration, and failover events. Log relevant context without exposing sensitive data.
  • Low · Missing CORS and Security Headers Configuration — ``. While this is a gem, the sample app configuration does not show explicit security header configuration. If used in a web application, missing security headers could expose the application to common attacks. Fix: undefined

LLM-derived; treat as a starting point, not a security audit.


Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.

Concerning signals · thiagopradi/octopus — RepoPilot