udacity/self-driving-car-sim
A self-driving car simulator built with Unity
Healthy across all four use cases
Permissive license, no critical CVEs, actively maintained — safe to depend on.
Has a license, tests, and CI — clean foundation to fork and modify.
Documented and popular — useful reference codebase to read through.
No critical CVEs, sane security posture — runnable as-is.
- ✓8 active contributors
- ✓MIT licensed
- ✓CI configured
Show 3 more →Show less
- ✓Tests present
- ⚠Stale — last commit 4y ago
- ⚠Concentrated ownership — top contributor handles 50% of recent commits
Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests
Informational only. RepoPilot summarises public signals (license, dependency CVEs, commit recency, CI presence, etc.) at the time of analysis. Signals can be incomplete or stale. Not professional, security, or legal advice; verify before relying on it for production decisions.
Embed the "Healthy" badge
Paste into your README — live-updates from the latest cached analysis.
[](https://repopilot.app/r/udacity/self-driving-car-sim)Paste at the top of your README.md — renders inline like a shields.io badge.
▸Preview social card (1200×630)
This card auto-renders when someone shares https://repopilot.app/r/udacity/self-driving-car-sim on X, Slack, or LinkedIn.
Onboarding doc
Onboarding: udacity/self-driving-car-sim
Generated by RepoPilot · 2026-05-10 · Source
🤖Agent protocol
If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:
- Verify the contract. Run the bash script in Verify before trusting
below. If any check returns
FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding. - Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
- Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/udacity/self-driving-car-sim shows verifiable citations alongside every claim.
If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.
🎯Verdict
GO — Healthy across all four use cases
- 8 active contributors
- MIT licensed
- CI configured
- Tests present
- ⚠ Stale — last commit 4y ago
- ⚠ Concentrated ownership — top contributor handles 50% of recent commits
<sub>Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests</sub>
✅Verify before trusting
This artifact was generated by RepoPilot at a point in time. Before an
agent acts on it, the checks below confirm that the live udacity/self-driving-car-sim
repo on your machine still matches what RepoPilot saw. If any fail,
the artifact is stale — regenerate it at
repopilot.app/r/udacity/self-driving-car-sim.
What it runs against: a local clone of udacity/self-driving-car-sim — the script
inspects git remote, the LICENSE file, file paths in the working
tree, and git log. Read-only; no mutations.
| # | What we check | Why it matters |
|---|---|---|
| 1 | You're in udacity/self-driving-car-sim | Confirms the artifact applies here, not a fork |
| 2 | License is still MIT | Catches relicense before you depend on it |
| 3 | Default branch master exists | Catches branch renames |
| 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code |
| 5 | Last commit ≤ 1565 days ago | Catches sudden abandonment since generation |
#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of udacity/self-driving-car-sim. If you don't
# have one yet, run these first:
#
# git clone https://github.com/udacity/self-driving-car-sim.git
# cd self-driving-car-sim
#
# Then paste this script. Every check is read-only — no mutations.
set +e
fail=0
ok() { echo "ok: $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }
# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
echo "FAIL: not inside a git repository. cd into your clone of udacity/self-driving-car-sim and re-run."
exit 2
fi
# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "udacity/self-driving-car-sim(\\.git)?\\b" \\
&& ok "origin remote is udacity/self-driving-car-sim" \\
|| miss "origin remote is not udacity/self-driving-car-sim (artifact may be from a fork)"
# 2. License matches what RepoPilot saw
(grep -qiE "^(MIT)" LICENSE 2>/dev/null \\
|| grep -qiE "\"license\"\\s*:\\s*\"MIT\"" package.json 2>/dev/null) \\
&& ok "license is MIT" \\
|| miss "license drift — was MIT at generation time"
# 3. Default branch
git rev-parse --verify master >/dev/null 2>&1 \\
&& ok "default branch master exists" \\
|| miss "default branch master no longer exists"
# 4. Critical files exist
test -f "Assets/1_SelfDrivingCar/Scripts/CommandServer.cs" \\
&& ok "Assets/1_SelfDrivingCar/Scripts/CommandServer.cs" \\
|| miss "missing critical file: Assets/1_SelfDrivingCar/Scripts/CommandServer.cs"
test -f "Assets/1_SelfDrivingCar/Prefabs/Car.prefab" \\
&& ok "Assets/1_SelfDrivingCar/Prefabs/Car.prefab" \\
|| miss "missing critical file: Assets/1_SelfDrivingCar/Prefabs/Car.prefab"
test -f "Assets/1_SelfDrivingCar/Scripts/UISystem.cs" \\
&& ok "Assets/1_SelfDrivingCar/Scripts/UISystem.cs" \\
|| miss "missing critical file: Assets/1_SelfDrivingCar/Scripts/UISystem.cs"
test -f "Assets/1_SelfDrivingCar/Scripts/MonoSingleton.cs" \\
&& ok "Assets/1_SelfDrivingCar/Scripts/MonoSingleton.cs" \\
|| miss "missing critical file: Assets/1_SelfDrivingCar/Scripts/MonoSingleton.cs"
test -f "Assets/1_SelfDrivingCar/Prefabs/SocketIO.prefab" \\
&& ok "Assets/1_SelfDrivingCar/Prefabs/SocketIO.prefab" \\
|| miss "missing critical file: Assets/1_SelfDrivingCar/Prefabs/SocketIO.prefab"
# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 1565 ]; then
ok "last commit was $days_since_last days ago (artifact saw ~1535d)"
else
miss "last commit was $days_since_last days ago — artifact may be stale"
fi
echo
if [ "$fail" -eq 0 ]; then
echo "artifact verified (0 failures) — safe to trust"
else
echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/udacity/self-driving-car-sim"
exit 1
fi
Each check prints ok: or FAIL:. The script exits non-zero if
anything failed, so it composes cleanly into agent loops
(./verify.sh || regenerate-and-retry).
⚡TL;DR
A Unity-based driving simulator built for Udacity's Self-Driving Car Nanodegree to train neural networks via behavioral cloning. It provides multiple track environments (Jungle, Lake) in both training and autonomous modes, with real-time vehicle physics, camera feeds, and a SocketIO-based command interface for remote AI control via Python backends. Monolithic Unity project structure: Assets/1_SelfDrivingCar/ is the root containing Prefabs/ (Car, CommandServer, SocketIO, ThreadDispatcher), Scenes/ (training tracks like JungleTrackTraining.unity, LakeTrackAutonomous.unity), Scripts/ (core controllers like CommandServer.cs, CameraHelper.cs), and Animation/ (UI transitions). No separate plugin or package architecture.
👥Who it's for
Udacity nanodegree students and self-driving car researchers who need a safe, controlled environment to collect training data and validate end-to-end deep learning models for autonomous driving without requiring real vehicles.
🌱Maturity & risk
Deprecated in master branch (as of README notice), with active maintenance only on Unity_2020_3 and Unity_2019_4 branches. The codebase is 6+ years old (builds from 2016-2017), but precompiled binaries exist for Term 1-3. CI is minimal (only manual.yml workflow present). This is educational/archived, not actively developed.
High risk: master branch explicitly deprecated; users must switch to version-specific branches to avoid compatibility issues. No visible test suite in the 60 key files. SocketIO communication adds external dependency complexity. Last commit age unknown from provided data, but deprecation notice suggests stalled maintenance. Single-point-of-failure: CommandServer.cs handles all remote training commands.
Active areas of work
Repository is in maintenance mode. Master branch explicitly marked deprecated with guidance to use Unity_2020_3 or Unity_2019_4 branches (PR#134, PR#133 referenced). No active development visible; repo serves as historical reference and archive for legacy course materials.
🚀Get running
- Clone the repo:
git clone https://github.com/udacity/self-driving-car-sim.git. 2. Switch to compatible Unity version branch:git checkout Unity_2020_3(or Unity_2019_4). 3. Open Assets/ in Unity Editor matching that version. 4. Open one of the scene files (e.g., Assets/1_SelfDrivingCar/Scenes/JungleTrackTraining.unity) and hit Play.
Daily commands:
- In Unity Editor: File → Open Scene → select Assets/1_SelfDrivingCar/Scenes/JungleTrackTraining.unity. 2. Click Play button. 3. Simulator spawns car and waits for SocketIO connection from Python training client (default localhost:4567). Alternatively, run precompiled binaries from Releases page.
🗺️Map of the codebase
Assets/1_SelfDrivingCar/Scripts/CommandServer.cs— Core network communication handler that bridges the simulator with external ML training scripts via SocketIO, essential for understanding how training data flows in/out.Assets/1_SelfDrivingCar/Prefabs/Car.prefab— Primary vehicle GameObject prefab that encapsulates car physics, sensors, and control logic—all autonomous driving behavior originates here.Assets/1_SelfDrivingCar/Scripts/UISystem.cs— Central UI state manager that orchestrates menu navigation, scene transitions, and recording/playback modes across all game scenes.Assets/1_SelfDrivingCar/Scripts/MonoSingleton.cs— Singleton base class used throughout codebase to ensure single instances of critical managers (SocketIO, CommandServer, ThreadDispatcher).Assets/1_SelfDrivingCar/Prefabs/SocketIO.prefab— WebSocket communication prefab that enables real-time bidirectional messaging between Unity simulator and Python training environment.Assets/1_SelfDrivingCar/Scripts/CameraHelper.cs— Manages camera capture and image streaming for neural network input—handles multiple camera angles (left/center/right) for training data collection.
🛠️How to make changes
Add a New Training Track
- Create a new scene in Assets/1_SelfDrivingCar/Scenes/ (e.g., DesertTrackTraining.unity) by duplicating JungleTrackTraining.unity and editing the road mesh/layout (
Assets/1_SelfDrivingCar/Scenes/JungleTrackTraining.unity) - Create corresponding autonomous variant (e.g., DesertTrackAutonomous.unity) with identical track but no manual steering controls (
Assets/1_SelfDrivingCar/Scenes/JungleTrackAutonomous.unity) - Register both scenes in ControlMenu.unity button handlers within MenuOptions.cs to enable track selection (
Assets/1_SelfDrivingCar/Scripts/MenuOptions.cs)
Add a New Sensor or Data Source
- Extend CameraHelper.cs or create a new sensor helper (e.g., LidarHelper.cs) to capture additional sensor modalities (
Assets/1_SelfDrivingCar/Scripts/CameraHelper.cs) - Add sensor data serialization in CommandServer.cs SendMessage() method to include new sensor frames in the JSON payload sent to training scripts (
Assets/1_SelfDrivingCar/Scripts/CommandServer.cs) - Update your Python training client to parse the extended message format and feed new sensor data to your ML model
Modify Vehicle Control Parameters
- Open Car.prefab in the Inspector and locate the vehicle physics component to adjust wheel colliders, suspension, mass, and drag coefficients (
Assets/1_SelfDrivingCar/Prefabs/Car.prefab) - Edit the steering and throttle input mapping in Car.prefab's control script to change sensitivity curves and max acceleration/steering angle (
Assets/1_SelfDrivingCar/Prefabs/Car.prefab) - Verify CommandServer.cs correctly reads steering/throttle commands from external ML script and applies them via RPC to the car's control methods (
Assets/1_SelfDrivingCar/Scripts/CommandServer.cs)
Change Network Communication Protocol
- Replace or extend the SocketIO.prefab with a different networking transport (e.g., gRPC, REST) by creating a new prefab and communication manager (
Assets/1_SelfDrivingCar/Prefabs/SocketIO.prefab) - Refactor CommandServer.cs OnMessage() and SendMessage() methods to serialize/deserialize using your new protocol instead of JSON over WebSocket (
Assets/1_SelfDrivingCar/Scripts/CommandServer.cs) - Update UISystem.cs or MenuOptions.cs to display connection status and errors from the new transport layer (
Assets/1_SelfDrivingCar/Scripts/UISystem.cs)
🔧Why these technologies
- Unity 2017+ (game engine) — Provides 3D physics simulation, rendering, and cross-platform deployment for accessible simulator; pre-built vehicle dynamics and wheel collider support
- SocketIO (WebSocket) — Enables low-latency, bidirectional real-time communication between Unity simulator and Python ML training scripts without HTTP polling overhead
- C# (scripting language) — Native to Unity; tight integration with engine APIs for physics, rendering, and scene management; strong type safety for networked command serialization
- JSON (serialization) — Language-agnostic, human-readable format for transmitting telemetry (images, steering, speed) and commands between simulator and Python training environment
⚖️Trade-offs already made
-
Single SocketIO connection bridging all simulator-to-training communication
- Why: Simplifies handshake and connection lifecycle; reduces network overhead
- Consequence: All data (images, telemetry, commands) flows through single bottleneck; no data parallelism across multiple connections
-
Three fixed camera angles (left, center, right) rather than arbitrary sensor placement
- Why: Mirrors human behavioral cloning use case (center view mimics driver perspective); reduces complexity of dynamic camera configuration
- Consequence: Cannot easily add additional sensor modalities (LiDAR, radar) without extending protocol; fixed FOV may not suit all network architectures
-
CommandServer processes commands synchronously on main thread via ThreadDispatcher
- Why: Ensures physics determinism and frame-locked vehicle updates; avoids race conditions on steering/throttle state
- Consequence: Network latency (~50ms round-trip) introduces perceptible lag between prediction
🪤Traps & gotchas
Version mismatch: master branch does not compile; must use Unity_2020_3 or Unity_2019_4 branches or building will fail. SocketIO dependency: CommandServer expects Python client on localhost:4567 by default; simulator hangs if training script never connects. Prefab references: Car.prefab referenced in multiple scenes; deleting/renaming breaks track instantiation. ThreadDispatcher singleton: Assets/1_SelfDrivingCar/Prefabs/ThreadDispatcher.prefab must exist for SocketIO callbacks from background threads. Scene-specific setup: each track scene (Jungle vs. Lake) has slightly different lighting/physics; copy-pasting scenes without re-tuning causes physics instability.
🏗️Architecture
💡Concepts to learn
- Behavioral Cloning — The core training methodology this simulator enables: learning driving policy directly from human demonstrations via supervised learning, rather than reinforcement learning
- End-to-End Learning — This simulator trains single neural networks mapping raw camera images directly to steering/throttle, skipping explicit lane detection or path planning
- SocketIO / WebSocket duplex communication — The CommandServer.cs and SocketIO.prefab use SocketIO for bidirectional real-time messaging; essential for low-latency command/sensor feedback loop between simulator and Python backend
- Unity MonoBehaviour and GameObject-Component pattern — All simulation logic (vehicle physics, camera management, command handling) is implemented as MonoBehaviours attached to GameObjects; understanding this is required to modify car behavior
- Data augmentation and generalization — LightConditions.cs and multiple track scenes (Jungle/Lake training/autonomous) provide varying environments to prevent overfitting and ensure robust autonomous driving across conditions
- Prefab instantiation and scene management — Car.prefab, CommandServer.prefab, and scene files use Unity's prefab system for reusable, configurable GameObject templates; modifying prefabs automatically updates all instances
- Thread-safe event dispatch (ThreadDispatcher pattern) — SocketIO callbacks arrive on background threads but must update Unity GameObjects on main thread; ThreadDispatcher.prefab queues events safely across thread boundaries
🔗Related repos
udacity/CarND-Behavioral-Cloning-P3— The actual project assignment that this simulator was built for; contains Python training code and model architecture guidanceudacity/CarND-Capstone— Capstone project using a similar simulator for system integration; references advanced autonomous stack beyond behavioral cloningcarla-simulator/carla— Modern alternative open-source driving simulator with more advanced graphics, sensor realism, and active development; spiritual successor to this projectAirSim/AirSim— Microsoft's drone/car simulator built on Unreal Engine; offers comparable SocketIO integration and multi-agent support for autonomous researchudacity/self-driving-car— Main Udacity self-driving car course repository; contains curriculum and project specs that use this simulator
🪄PR ideas
To work on one of these in Claude Code or Cursor, paste:
Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.
Add CI/CD workflow to build and test Unity scenes across supported versions
The README explicitly directs users to Unity_2020_3 and Unity_2019_4 branches, but there's no automated validation that the master branch or versioned branches build successfully. The manual.yml workflow exists but appears unused. Add a GitHub Actions workflow that builds the Unity project for both LTS versions to catch breaking changes early and ensure scene integrity (ControlMenu.unity, JungleTrackTraining.unity, LakeTrackTraining.unity, etc.).
- [ ] Create .github/workflows/unity-build.yml that uses unity-builder action
- [ ] Configure matrix strategy for Unity 2019.4 LTS and 2020.3 LTS
- [ ] Add build steps for both Training and Autonomous scenes in Assets/1_SelfDrivingCar/Scenes/
- [ ] Include scene validation to ensure no missing references in Car.prefab, CommandServer.prefab, SocketIO.prefab
- [ ] Document in README the build status badge and how contributors should validate locally
Create unit tests for core server/networking scripts (CommandServer.cs, SocketIO integration)
CommandServer.cs and SocketIO.prefab are critical for simulator-to-training communication, but no test files exist in the repository. Add a test suite that validates message parsing, server initialization, and thread-safe communication patterns used in ThreadDispatcher.prefab to prevent regressions in the trainer interface.
- [ ] Create Assets/1_SelfDrivingCar/Tests/ directory structure with EditMode tests
- [ ] Add CommandServer_Tests.cs to test server initialization, message handling, and socket communication mocking
- [ ] Add ThreadDispatcher_Tests.cs to validate thread-safe queuing and dispatcher behavior
- [ ] Create a mock socket implementation to test SocketIO.prefab integration without live connections
- [ ] Configure test runner in .github/workflows to run via 'unity -runTests' in the workflow
Add input validation and scene state documentation for UI and menu scripts
MenuOptions.cs, UISystem.cs, and FadeImageController.controller manage critical scene transitions (ControlMenu → JungleTrackTraining/LakeTrackTraining, etc.), but there's no validation contract or state machine documentation. Create inline documentation and input guards to prevent race conditions during scene loads and menu navigation.
- [ ] Add XML documentation comments to MenuOptions.cs and UISystem.cs detailing state transitions between LauncherScene → MenuScene → ControlMenu → Training/Autonomous scenes
- [ ] Add null-checks and input validation in MenuOptions.cs for scene loading parameters
- [ ] Document the FadeImageController.controller animation states and their trigger conditions
- [ ] Create SCENE_FLOW.md in Assets/1_SelfDrivingCar/ showing the navigation graph and expected state during training vs autonomous modes
- [ ] Add assertions to prevent invalid scene transitions (e.g., prevent skipping ControlMenu)
🌿Good first issues
- Add unit tests for CommandServer.cs message parsing: currently no test coverage visible for SocketIO command deserialization (throttle/steering protocol). Create Assets/Tests/CommandServerTests.cs.
- Document the exact SocketIO message format and Python client API: README has no protocol specification; new contributors must reverse-engineer from CommandServer.cs. Add PROTOCOL.md with example JSON payloads.
- Add configuration UI for training parameters (learning rate, batch size, timeout): currently hardcoded in scripts or external Python only; create MenuOptions.cs extension or new scene for hyperparameter tuning.
⭐Top contributors
Click to expand
Top contributors
- [@Aaron Brown](https://github.com/Aaron Brown) — 13 commits
- @SudKul — 5 commits
- [@Michael Virgo](https://github.com/Michael Virgo) — 2 commits
- @olivercameron — 2 commits
- @mvirgo — 1 commits
📝Recent commits
Click to expand
Recent commits
7aeb57c— Update README.md (SudKul)8cf01e0— Typo (SudKul)4fe607f— Add deprecation note (SudKul)4a7bb62— Add Github Action (SudKul)e6e182e— Create CODEOWNERS (SudKul)522bd7a— docs: Change Term 1 V2 sim links to AWS (Michael Virgo)75bb9dc— Docs: Add info for non-Term 1 sims (Michael Virgo)bdcd588— Merge pull request #48 from sshiv/patch-1 (mvirgo)e63530e— add info/workarounds for some known simulator issues (baumanab)814587d— Fixing typo in README.md (sshiv)
🔒Security observations
The codebase is a deprecated Unity-based self-driving car simulator with moderate security concerns. Primary issues include: (1) deprecated status indicating lack of ongoing security maintenance, (2) inability to assess dependency vulnerabilities due to missing/empty dependency files, and (3) potential risks in network communication and threading components that require code-level review. The simulator appears to be an educational project without obvious critical vulnerabilities in the file structure, but the deprecated status and lack of visible dependency management significantly reduce confidence in its security posture. Recommendation: Either archive the master branch entirely or migrate to maintained branches with regular security audits.
- Medium · Deprecated Repository - Outdated Dependencies —
README.md and overall repository status. The master branch is marked as deprecated in the README. This indicates the codebase may contain outdated Unity versions and dependencies that could have known vulnerabilities. Users are directed to use Unity_2020_3 or Unity_2019_4 branches instead, suggesting the master branch may not receive security updates. Fix: Archive the deprecated master branch or clearly mark it as unmaintained. Ensure all active development occurs on maintained branches (Unity_2020_3/2019_4) with regular dependency updates and security patches. - Medium · No Visible Dependency Management File —
Assets/manifest.json or Packages/manifest.json (not provided). The provided dependency file content is empty. For a Unity project, package dependencies should be defined in 'Assets/manifest.json' or 'Packages/manifest.json'. Without visibility into dependencies, it's impossible to assess whether third-party packages contain known vulnerabilities. Fix: Provide and review the manifest.json file. Regularly audit dependencies using tools like npm audit (if using .NET/npm packages). Keep all packages updated to versions without known CVEs. - Low · Network Communication Component Present —
Assets/1_SelfDrivingCar/Scripts/CommandServer.cs and Assets/1_SelfDrivingCar/Prefabs/SocketIO.prefab. The file structure includes 'CommandServer.cs' and 'SocketIO.prefab', indicating network communication capabilities. Without code review, potential vulnerabilities could include unencrypted communication, lack of input validation, or insufficient authentication. Fix: Review CommandServer.cs for: 1) Encrypted communication (TLS/SSL), 2) Input validation and sanitization, 3) Authentication mechanisms, 4) Rate limiting, 5) Error handling that doesn't leak sensitive information. - Low · Threading Component - Potential Race Conditions —
Assets/1_SelfDrivingCar/Prefabs/ThreadDispatcher.prefab. The presence of 'ThreadDispatcher.prefab' suggests multi-threaded operations. Improper thread synchronization could lead to race conditions, data corruption, or unpredictable behavior. Fix: Review thread-safe implementations: use proper locking mechanisms, immutable data structures where possible, and ensure thread-safe access to shared resources. Consider using thread-safe collections from the .NET framework.
LLM-derived; treat as a starting point, not a security audit.
👉Where to read next
- Open issues — current backlog
- Recent PRs — what's actively shipping
- Source on GitHub
Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.