WAIT — Stale — last commit 3y ago

• 9 active contributors
• MIT licensed
• ⚠ Stale — last commit 3y ago
• ⚠ Single-maintainer risk — top contributor 91% of recent commits
• ⚠ No CI workflows detected
• ⚠ No test directory detected

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

ML-From-Scratch is a bare-bones NumPy implementation library covering machine learning models from linear regression to deep learning (CNNs, RNNs, reinforcement learning). It prioritizes transparency and educational accessibility over performance, making the inner workings of algorithms visible through pure NumPy code without framework abstractions. Monolithic package structure: mlfromscratch/ root with four main subdirectories—supervised_learning/, unsupervised_learning/, deep_learning/, reinforcement_learning/—each containing algorithm implementations. deep_learning/ is most complex (activation_functions.py, layers.py, loss_functions.py, optimizers.py, neural_network.py). examples/ directory contains 40+ standalone runnable scripts (one per algorithm) that demonstrate usage with synthetic or included data (TempLinkoping2016.txt).

Machine learning students and practitioners who want to understand how algorithms work internally; educators teaching ML fundamentals; developers who need reference implementations before applying scikit-learn or TensorFlow to production problems.

Actively maintained educational project with 251K lines of Python code and comprehensive examples covering 40+ algorithms, but designed explicitly for learning rather than production use. Single-maintainer repo (eriklindernoren) with stable structure—no indication of breaking changes or abandonment, though commit recency data is not visible in the provided snapshot.

Low production risk because it's not intended for production use. Primary risks: single maintainer (abandonment concern if inactive), dependencies include legacy packages (progressbar33, cvxopt, scipy) with varying maintenance levels, and NumPy-only implementations lack distributed computing or GPU acceleration. No visible CI/CD pipeline or test suite in the file structure, suggesting testing may be manual.

Unable to determine from static file snapshot alone. Check the repository's GitHub page for recent commits, open issues, and pull requests to assess current activity status.

git clone https://github.com/eriklindernoren/ML-From-Scratch
cd ML-From-Scratch
python setup.py install

Then run an example: python mlfromscratch/examples/polynomial_regression.py

Daily commands: Run individual algorithm examples directly: python mlfromscratch/examples/{algorithm_name}.py (e.g., linear_regression.py, convolutional_neural_network.py, deep_q_network.py). Each example is self-contained and prints results or displays plots. No development server—all are batch scripts.

• mlfromscratch/__init__.py — Package entry point that exposes all major ML modules; contributors must understand the public API surface.
• mlfromscratch/deep_learning/neural_network.py — Core neural network abstraction used by CNN, RNN, and MLP implementations; foundational to deep learning examples.
• mlfromscratch/supervised_learning/regression.py — Implements linear, polynomial, ridge, lasso, and elastic-net regression; heavily reused base for supervised learning.
• mlfromscratch/utils/data_operation.py — Shared utility functions for data preprocessing and manipulation; used across all examples and models.
• mlfromscratch/deep_learning/layers.py — Layer implementations (Dense, Conv, RNN, Dropout) that form building blocks for all neural network models.
• mlfromscratch/deep_learning/activation_functions.py — Activation function definitions with forward/backward passes; essential for understanding gradient flow.
• mlfromscratch/deep_learning/loss_functions.py — Loss function implementations (MSE, CrossEntropy, etc.) that define training objectives across all models.

Add a New Supervised Learning Model

1. Create a new class in mlfromscratch/supervised_learning/ that implements fit() and predict() methods, following the naming convention used by LogisticRegression, DecisionTree, etc. (mlfromscratch/supervised_learning/<model_name>.py)
2. Add your model to the init.py to export it from the module. (mlfromscratch/supervised_learning/__init__.py)
3. Create an example script that loads data, instantiates your model, trains it, and evaluates accuracy/metrics using utils. (mlfromscratch/examples/<model_name>.py)

Add a New Neural Network Layer

1. Implement a new layer class inheriting from base Layer pattern in layers.py, with forward() and backward() methods. (mlfromscratch/deep_learning/layers.py)
2. Update the NeuralNetwork class if the layer requires special parameter initialization or learning rate scheduling. (mlfromscratch/deep_learning/neural_network.py)
3. Create an example that uses the new layer in a network (e.g., for CNN or RNN) to demonstrate its usage. (mlfromscratch/examples/convolutional_neural_network.py or mlfromscratch/examples/recurrent_neural_network.py)

Add a New Unsupervised Learning Model

1. Create a new class in mlfromscratch/unsupervised_learning/ with fit() and predict() or fit_transform() methods. (mlfromscratch/unsupervised_learning/<model_name>.py)
2. Add your model to the init.py to export it. (mlfromscratch/unsupervised_learning/__init__.py)
3. Create an example that demonstrates clustering or dimensionality reduction, possibly visualizing results with matplotlib. (mlfromscratch/examples/<model_name>.py)

Add a New Activation or Loss Function

1. Add the forward() and backward() method pair to the appropriate module (activation_functions.py or loss_functions.py). (mlfromscratch/deep_learning/activation_functions.py or mlfromscratch/deep_learning/loss_functions.py)
2. Reference the new function in neural_network.py where activations or losses are selected during model initialization. (mlfromscratch/deep_learning/neural_network.py)
3. Optionally add a unit test or small example that verifies gradient computation is correct. (mlfromscratch/examples/multilayer_perceptron.py)

• NumPy — Core computational engine for all linear algebra, matrix operations, and numerical computations; enables pure Python implementations without heavy C dependencies like TensorFlow.
• Matplotlib — Visualization library used in examples to plot decision boundaries, loss curves, and clustering results for interpretability.
• undefined — undefined

No test suite: validation is done by visual inspection of example outputs (plots) or manual comparison—adding a test breaks the repo. cvxopt dependency: required for SVM but can be difficult to install on Windows without a C compiler; consider documenting this. NumPy version constraints: no explicit pinning; NumPy API changes (e.g., random module in v1.17+) may break examples without notice. Data format assumptions: examples assume specific input shapes (e.g., convolutional_neural_network.py expects (1, 8, 8) images); reshaping is not always explicit. No logging: debug-friendly print() statements scattered throughout; no centralized logging strategy for tracing training progress.

• Backpropagation — Core to all neural network training in deep_learning/neural_network.py and layers.py; understanding the chain rule applied layer-by-layer is essential to modifying or extending the implementations
• Information Gain & Entropy (ID3/C4.5) — Decision tree splitting criterion in supervised_learning/decision_tree.py; fundamental to understanding how trees recursively partition data
• Gradient Descent Variants (SGD, Adam, RMSprop) — Implemented in deep_learning/optimizers.py and used by all neural network training; understanding momentum and adaptive learning rates explains training convergence behavior
• Convolutional Filters & Feature Maps — deep_learning/layers.py implements Conv2D with manual stride and padding logic; critical to understanding how CNNs extract spatial patterns without fully connected layers
• Recurrent Hidden State & BPTT — RNN/LSTM implementations in deep_learning/layers.py maintain hidden state across time steps and backpropagate through time; essential for sequence modeling
• Q-Learning & Bellman Equation — Deep Q-Network in reinforcement_learning/deep_q_network.py combines Q-learning with neural networks; core to understanding value-based reinforcement learning
• Kernel Methods (SVM) — SVM implementation uses cvxopt for quadratic programming with kernel trick support; necessary for understanding non-linear classification without explicit feature expansion

• scikit-learn/scikit-learn — Production-grade machine learning library with optimized implementations; use after understanding algorithms via this repo
• pytorch/pytorch — Deep learning framework with similar layer abstractions (nn.Module) and optimizer patterns; this repo's neural_network.py is a pedagogical precursor
• d2l-ai/d2l-en — Companion resource: Dive into Deep Learning textbook with code implementations and visual explanations of the same algorithms
• ageron/handson-ml — Hands-On Machine Learning textbook repo with scikit-learn and TensorFlow implementations; best next step after understanding this repo's fundamentals
• ujjwalkarn/Machine-Learning-Basics — Similar educational repo focused on algorithm intuition; alternative reference for comparing implementation approaches

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add unit tests for supervised_learning models with pytest

The repo has 15+ supervised learning implementations (regression.py, decision_tree.py, svm.py, etc.) but no test suite. This is critical for a 'from scratch' educational repo where correctness is paramount. Adding pytest tests would verify that implementations match expected behavior on toy datasets and catch regressions as contributors modify code.

• [ ] Create tests/test_supervised_learning/ directory structure
• [ ] Add test_regression.py covering LinearRegression, PolynomialRegression, Ridge, Lasso, ElasticNet from mlfromscratch/supervised_learning/regression.py
• [ ] Add test_decision_tree.py for DecisionTreeClassifier/Regressor from mlfromscratch/supervised_learning/decision_tree.py
• [ ] Add test_svm.py for SupportVectorMachine from mlfromscratch/supervised_learning/support_vector_machine.py
• [ ] Add test_ensemble.py for AdaBoost, RandomForest, GradientBoosting from supervised_learning/
• [ ] Create conftest.py with common test fixtures (synthetic datasets, expected outputs)
• [ ] Add pytest to requirements or setup.py
• [ ] Add pytest configuration to setup.cfg or pytest.ini

Add integration tests and validation for deep_learning/neural_network.py against examples

The deep_learning module has complex interdependencies (neural_network.py uses layers.py, activation_functions.py, loss_functions.py, optimizers.py) but no tests verify they work together correctly. With 6+ example scripts (multilayer_perceptron.py, convolutional_neural_network.py, recurrent_neural_network.py), automated tests would ensure API compatibility and catch breaking changes across the deep learning stack.

• [ ] Create tests/test_deep_learning/ directory
• [ ] Add test_neural_network_integration.py that instantiates NeuralNetwork with various layer combinations from layers.py
• [ ] Add test_activation_functions.py to verify forward/backward pass correctness for all functions in activation_functions.py
• [ ] Add test_optimizers.py to verify SGD, Adam, RMSprop, etc. from optimizers.py reduce loss on toy problems
• [ ] Add test_loss_functions.py for MSE, CrossEntropy, etc. from loss_functions.py
• [ ] Create a simple end-to-end test that trains a small network and verifies loss decreases
• [ ] Add tests that mirror logic from examples/multilayer_perceptron.py and examples/convolutional_neural_network.py

Add comprehensive docstrings and type hints to mlfromscratch/supervised_learning/support_vector_machine.py and mlfromscratch/supervised_learning/xgboost.py

These are complex algorithms (SVM with kernel methods, XGBoost with gradient boosting) that lack clear documentation. Adding docstrings with algorithm summaries, parameter descriptions, return types, and inline comments explaining key math operations would make the code more accessible—directly aligned with the repo's stated goal of 'transparency and accessibility.' Type hints also enable better IDE support and catch bugs early.

• [ ] Add module-level docstring to support_vector_machine.py explaining SVM theory and kernel methods
• [ ] Add class docstring to SupportVectorMachine with Parameters, Attributes, and Methods sections
• [ ] Add parameter descriptions and return type hints to init, fit(), predict(), and helper methods
• [ ] Add inline comments explaining the dual formulation, SMO algorithm, and kernel computations
• [ ] Repeat docstring and type hint additions for xgboost.py, covering GradientBoostingClassifier/Regressor
• [ ] Document the loss function update rules and tree building process in xgboost.py
• [ ] Verify examples/support_vector_machine.py and examples/xgboost.py run without errors post-documentation

• Add docstrings and type hints to mlfromscratch/supervised_learning/decision_tree.py—currently lacks documentation of the information gain calculation and splitting criteria, making it hard for learners to understand the core algorithm. A PR adding comprehensive docstrings would be immediately useful.
• Create a test suite for mlfromscratch/deep_learning/layers.py validating forward/backward pass correctness using numerical gradient checking (finite differences). Currently no tests exist; even a pytest file with 5-10 layer tests would catch NumPy API drift.
• Add visualization examples to mlfromscratch/examples/ for unsupervised algorithms (K-means cluster centers, DBSCAN density plots, PCA variance explained). Currently most unsupervised examples print text output only—matplotlib visualizations would make these more educational.

The ML-From-Scratch project is a educational codebase with moderate security posture. The primary concerns are dependency management (unpinned versions, deprecated packages) and lack of explicit input validation in ML models. The project does not expose web services, databases, or handle sensitive data directly, which reduces the attack surface. However, as a library that processes arbitrary input data and uses third-party dependencies, it should implement better dependency pinning, input validation, and security documentation. The codebase itself does not contain hardcoded secrets, SQL injection risks, or infrastructure misconfigurations. Developers integrating this library should be aware of pickle deserialization risks if model persistence is implemented.

• Medium · Outdated and Vulnerable Dependencies — requirements.txt. The requirements.txt file specifies dependencies without pinned versions, allowing installation of potentially vulnerable versions. Notable concerns: 'sklearn' is deprecated (should use 'scikit-learn'), 'progressbar33' is unmaintained, and 'gym' (OpenAI Gym) has known vulnerabilities in older versions. No version constraints mean automatic updates to potentially breaking or vulnerable versions. Fix: Pin all dependencies to specific versions (e.g., numpy==1.21.0). Use 'scikit-learn' instead of 'sklearn'. Regularly audit dependencies using 'pip-audit' or 'safety' tools. Consider using 'pip-compile' or 'Poetry' for better dependency management.
• Low · Missing Input Validation in Machine Learning Models — mlfromscratch/supervised_learning/, mlfromscratch/unsupervised_learning/. The codebase implements various ML algorithms that accept user input (features, parameters) without apparent input validation. While not a traditional security vulnerability, malformed or adversarial inputs could cause DoS conditions, unexpected behavior, or information disclosure through error messages. Fix: Implement input validation for all model training and prediction methods. Validate data types, shapes, and value ranges. Use NumPy's error handling to catch dimension mismatches. Sanitize error messages to avoid leaking sensitive information about data or model internals.
• Low · Pickle Serialization Security Risk — mlfromscratch/ (entire package). If the codebase uses pickle for model serialization (common in ML projects), untrusted pickle files can execute arbitrary code. The file structure suggests model persistence capabilities that may rely on pickle. Fix: If pickle is used, clearly document that only trusted model files should be loaded. Consider using safer alternatives like joblib with protocol limits, or JSON-based serialization. Add security warnings in documentation about loading untrusted model files.
• Low · Lack of Security Documentation — Repository root. The project lacks a SECURITY.md file or security guidelines. No mention of secure practices, vulnerability reporting procedures, or security considerations for users integrating this library. Fix: Create a SECURITY.md file documenting: responsible disclosure procedures, security considerations for users, known limitations, and best practices for using ML models safely. Add security headers to documentation.
• Low · No Type Hints or Input Sanitization — mlfromscratch/ (entire package). The codebase lacks type hints which could help catch type-related security issues at development time. Without explicit type checking, unexpected input types could cause failures or unexpected behavior. Fix: Add type hints using Python's typing module to all function signatures. Use tools like 'mypy' for static type checking in CI/CD. Implement runtime type validation for critical functions, especially those accepting external input.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/eriklindernoren/ML-From-Scratch shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live eriklindernoren/ML-From-Scratch repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/eriklindernoren/ML-From-Scratch.

What it runs against: a local clone of eriklindernoren/ML-From-Scratch — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in eriklindernoren/ML-From-Scratch | Confirms the artifact applies here, not a fork | | 2 | License is still MIT | Catches relicense before you depend on it | | 3 | Default branch master exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 967 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of eriklindernoren/ML-From-Scratch. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/eriklindernoren/ML-From-Scratch.git
#   cd ML-From-Scratch
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of eriklindernoren/ML-From-Scratch and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "eriklindernoren/ML-From-Scratch(\\.git)?\\b" \\
  && ok "origin remote is eriklindernoren/ML-From-Scratch" \\
  || miss "origin remote is not eriklindernoren/ML-From-Scratch (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(MIT)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"MIT\"" package.json 2>/dev/null) \\
  && ok "license is MIT" \\
  || miss "license drift — was MIT at generation time"

# 3. Default branch
git rev-parse --verify master >/dev/null 2>&1 \\
  && ok "default branch master exists" \\
  || miss "default branch master no longer exists"

# 4. Critical files exist
test -f "mlfromscratch/__init__.py" \\
  && ok "mlfromscratch/__init__.py" \\
  || miss "missing critical file: mlfromscratch/__init__.py"
test -f "mlfromscratch/deep_learning/neural_network.py" \\
  && ok "mlfromscratch/deep_learning/neural_network.py" \\
  || miss "missing critical file: mlfromscratch/deep_learning/neural_network.py"
test -f "mlfromscratch/supervised_learning/regression.py" \\
  && ok "mlfromscratch/supervised_learning/regression.py" \\
  || miss "missing critical file: mlfromscratch/supervised_learning/regression.py"
test -f "mlfromscratch/utils/data_operation.py" \\
  && ok "mlfromscratch/utils/data_operation.py" \\
  || miss "missing critical file: mlfromscratch/utils/data_operation.py"
test -f "mlfromscratch/deep_learning/layers.py" \\
  && ok "mlfromscratch/deep_learning/layers.py" \\
  || miss "missing critical file: mlfromscratch/deep_learning/layers.py"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 967 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~937d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/eriklindernoren/ML-From-Scratch"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.