WAIT — Mixed signals — read the receipts

• Last commit 2mo ago
• 24+ active contributors
• Other licensed
• ⚠ Concentrated ownership — top contributor handles 61% of recent commits
• ⚠ Non-standard license (Other) — review terms
• ⚠ No CI workflows detected
• ⚠ No test directory detected

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

MockingBird is a voice cloning and speech synthesis system that can clone a target speaker's voice in ~5 seconds and generate arbitrary speech in real-time using that cloned voice. It combines three neural network modules (encoder for speaker embedding, PPG2Mel for acoustic feature conversion, and a vocoder for waveform generation) trained on multiple Chinese datasets (aidatatang_200zh, magicdata, aishell3, data_aishell) and supports both Mandarin and other languages. Modular training pipeline: control/cli/ contains separate CLI trainers for encoder_train.py, synthesizer_train.py, ppg2mel_train.py, and vocoder_train.py; control/mkgui/ provides Streamlit-based UI (app.py, app_vc.py) and FastAPI server wrapper; control/toolbox/ wraps the inference interface (utterance.py, ui.py). Two UIs compete: legacy PyQt5 (control/toolbox/ui.py) and newer Streamlit (control/mkgui/base/ui/streamlit_ui.py).

Researchers and developers building Chinese text-to-speech (TTS) and voice conversion systems; audio engineers prototyping speaker cloning features; organizations needing fast speaker adaptation without retraining large models. Contributors are likely audio ML engineers familiar with PyTorch and acoustic modeling.

Experimental / archived maintenance mode. The repo has significant star history (3869 on Trendshift) and was actively maintained through 2021, but the maintainer explicitly states 'I no longer actively update this repo' in the README and directs users to their commercial version (noiz.ai). No active CI/CD pipeline visible (.github/workflows absent), and no test directory in file structure suggests limited automated testing infrastructure.

High maintenance risk: single-author repo with explicit abandonment notice; PyTorch and dependency versions pinned to August 2021 (1.9.0) creating compatibility issues with modern Python 3.9+ and CUDA; 796KB of Python code with no visible test suite increases regression risk. The 'archived_untest_files/' directory name itself signals untested code paths. Dependency on webrtcvad, espnet, and transformers adds transitive dependency risk.

No active development. The README redirect to noiz.ai indicates the maintainer has moved to a commercial offering. No visible recent activity, open PRs, or milestones in the provided data. The 'archived_untest_files/' directory suggests incomplete refactoring was the final state before maintenance ceased.

Check README for instructions.

Daily commands: For Streamlit UI: streamlit run control/mkgui/app.py. For FastAPI server: inferred from control/mkgui/base/api/fastapi_utils.py but exact command not documented in file list. For training: python control/cli/encoder_train.py (or synthesizer_train.py, ppg2mel_train.py, vocoder_train.py depending on which module). Inference via control/toolbox/ui.py (PyQt5) or control/mkgui/app.py (Streamlit).

• demo_toolbox.py — Primary entry point for the voice cloning pipeline; demonstrates the complete inference workflow from audio input to speech synthesis
• models/encoder/model.py — Core speaker encoder that extracts voice embeddings in 5 seconds; foundational to the voice cloning capability
• models/ppg2mel/train/train_linglf02mel_seq2seq_oneshotvc.py — One-shot voice conversion training logic that bridges linguistic features to mel-spectrograms; critical for real-time synthesis
• models/ppg_extractor/e2e_asr_common.py — Extracts phonetic posteriorgrams (PPG) from audio; essential preprocessing stage for the synthesis pipeline
• control/mkgui/app.py — Main GUI application entry point using Streamlit; orchestrates user interaction with the voice cloning system
• control/toolbox/ui.py — Toolbox UI implementation providing interactive interface for real-time voice conversion and synthesis
• gen_voice.py — Inference script for end-to-end voice generation; shows how components integrate from speaker embedding to vocoded audio

Add a new voice cloning interface

1. Create a new UI class inheriting from the Streamlit or PyQt5 base classes (control/mkgui/base/ui/streamlit_ui.py)
2. Register the new interface in the main app by adding import and route (control/mkgui/app.py)
3. Implement speaker encoder inference callback (models/encoder/inference.py)
4. Add any custom audio processing in the audio module (models/encoder/audio.py)

Extend the synthesis pipeline with a new vocoder

1. Create preprocessing logic for the vocoder in a new file under models/vocoder/ (models/ppg2mel/preprocess.py)
2. Add vocoder model class following the pattern in rnn_decoder_mol.py (models/ppg2mel/rnn_decoder_mol.py)
3. Create a CLI training command using the existing PPG2Mel CLI as template (control/cli/ppg2mel_train.py)
4. Integrate vocoder into the main inference pipeline in gen_voice.py (gen_voice.py)

Add support for a new language

1. Extend the PPG extractor or create language-specific encoder in ppg_extractor/encoder/ (models/ppg_extractor/e2e_asr_common.py)
2. Add language-specific text processing (similar to pypinyin for Chinese) (models/encoder/audio.py)
3. Create new dataset preprocessing for the language (models/ppg2mel/preprocess.py)
4. Add a new CLI training command or extend existing one with language flag (control/cli/encoder_preprocess.py)

Implement real-time synthesis optimization

1. Add streaming inference to the encoder using batching strategies (models/encoder/inference.py)
2. Optimize the synthesis solver for low-latency mel generation (models/ppg2mel/train/solver.py)
3. Implement caching for speaker embeddings in the toolbox (control/toolbox/ui.py)
4. Add performance metrics to the main inference script (gen_voice.py)

• PyTorch — Flexible deep learning framework for training speaker encoders, sequence-to-sequence models, and neural vocoders with GPU acceleration
• Streamlit + PyQt5 dual UI — Streamlit for rapid web-based prototyping and collaboration; PyQt5 for local desktop app with lower latency and real-time synthesis
• ESPnet (Transformer ASR backbone) — Provides pretrained multilingual ASR for phonetic posteriorgram extraction without retraining from scratch
• RNN + Attention (seq2seq) — Proven architecture for sequence-to-sequence voice conversion; attention mechanism allows speaker-conditioned mel generation
• Mixture-of-Logistics (MOL) decoder — Enables probabilistic mel-spectrogram prediction with better modeling of multimodal distributions in acoustic features

• One-shot voice conversion (5-second reference) vs. multi-utterance enrollment

  • Why: Fast user experience and minimal data requirement
  • Consequence: Lower voice fidelity than enrollment with 10+ utterances; may struggle with rare phonemes not in reference

• ---

  • Why: undefined
  • Consequence: undefined

Critical version constraints: must use PyTorch 1.9.0 and Python 3.7–3.8 (3.9+ may fail torch installation per README); numpy pinned differently per OS (1.19.3 on Windows, 1.20.3 elsewhere). webrtcvad installation is optional but fails silently on Windows without webrtcvad-wheels. Streamlit UI (control/mkgui/) and PyQt5 UI (control/toolbox/) both exist but are not feature-parity; unclear which is canonical. No documented GPU/CPU fallback or device selection. Training requires large datasets (aidatatang_200zh, aishell3, etc.) that must be downloaded separately; no data download scripts visible. Dockerfile present but not documented in README, suggesting it may be outdated.

• Speaker Embedding / Speaker Verification — The encoder (control/cli/encoder_train.py) must learn speaker-discriminative embeddings from short audio clips to enable voice cloning; understanding triplet loss and speaker verification is essential to grok why 5 seconds of audio suffices
• Phonetic Posteriorgram (PPG) — PPG2Mel conversion (control/cli/ppg2mel_train.py, control/cli/pre4ppg.py) maps phonetic units to acoustic features, enabling voice conversion independent of content; central to the pipeline's modularity
• Mel-Spectrogram and Vocoder — The synthesizer outputs Mel-spectrograms, which a vocoder (e.g., HiFi-GAN) converts to raw waveforms; understanding this two-stage acoustic→waveform pipeline is critical for debugging audio quality issues
• Fine-tuning vs. Retraining — MockingBird reuses pretrained encoder and vocoder, training only the synthesizer on new speaker data; this transfer learning approach is why voice cloning requires minimal data and compute, and is explicitly highlighted in the README as a key feature
• Real-time Synthesis and Streaming — The repo claims 'generate arbitrary speech in real-time'; understanding latency bottlenecks (encoder inference, mel generation, vocoding) and streaming constraints is necessary for production deployment via FastAPI (control/mkgui/base/api/fastapi_utils.py)
• Multilingual Text Normalization (Unidecode, pypinyin) — Dependencies on Unidecode and pypinyin (Mandarin pinyin conversion) show the system is optimized for Chinese; understanding grapheme-to-phoneme and language-specific text preprocessing is crucial for extending to other languages
• Voice Activity Detection (webrtcvad) — webrtcvad-wheels is listed as optional but likely used in preprocessing audio clips to extract speech segments and remove silence, improving encoder training and inference quality

• mozilla/TTS — Multi-speaker, multi-lingual TTS with voice cloning; more actively maintained alternative using Glow-TTS and HiFi-GAN
• CorentinJ/Real-Time-Voice-Cloning — Inspirational predecessor using speaker embeddings (SpeakerEncoder) + Tacotron2; likely a direct ancestor of MockingBird's encoder+synthesizer design
• espnet/espnet — MockingBird depends on espnet for ASR and phonetic posteriorgrams (PPG) computation; core dependency for the PPG2Mel pipeline
• jik876/hifi-gan — State-of-the-art vocoder likely used in MockingBird's waveform generation stage (implicit dependency via vocoder_train.py)
• babysor/Parakeet — Companion repo by same author providing alternative TTS components and training utilities for speech synthesis

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add unit tests for audio preprocessing pipeline (encoder/audio.py and vocoder_preprocess.py)

The repo has critical audio processing code in models/encoder/audio.py and control/cli/vocoder_preprocess.py with no visible test coverage. Given the project's core functionality depends on correct audio preprocessing (5-second voice cloning), adding tests would prevent regressions and document expected behavior for contributors.

• [ ] Create tests/test_audio_preprocessing.py with test cases for models/encoder/audio.py functions
• [ ] Create tests/test_vocoder_preprocess.py covering control/cli/vocoder_preprocess.py edge cases
• [ ] Add pytest to dependencies and create tests/init.py
• [ ] Test handling of various sample rates, audio formats, and edge cases (silent audio, corrupted files)

Add GitHub Actions CI workflow to validate core training pipelines on PRs

With multiple training entry points (encoder_train.py, ppg2mel_train.py, synthesizer_train.py, vocoder_train.py in control/cli/), there's no automated validation that these don't break. A lightweight CI workflow testing imports and basic initialization would catch breaking changes early.

• [ ] Create .github/workflows/test-training-pipelines.yml
• [ ] Add workflow steps to validate all control/cli/training scripts can import without errors
• [ ] Include Python dependency validation and PyTorch installation checks
• [ ] Run on Python 3.8+ with both Linux and Windows (due to platform-specific numpy in dependencies)

Extract and document the FastAPI server implementation from control/mkgui/base/api/fastapi_utils.py

The repo has a FastAPI implementation (fastapi_utils.py) but no visible API documentation or example usage in README. This is valuable for users wanting to deploy MockingBird as a service. Creating a dedicated API guide and example server startup script would significantly improve accessibility.

• [ ] Create docs/API.md documenting endpoints available in control/mkgui/base/api/fastapi_utils.py
• [ ] Add example usage: create examples/api_server_example.py showing how to start the FastAPI server
• [ ] Document the relationship between fastapi_utils.py, app.py, and app_vc.py in control/mkgui/
• [ ] Include curl/Python examples for voice cloning requests in the documentation

• Add a test suite for control/toolbox/utterance.py to verify encoder→synthesizer→vocoder pipeline correctness on mock audio; currently archived_untest_files/ signals this gap
• Document the FastAPI server startup in README with exact curl examples; control/mkgui/base/api/fastapi_utils.py exists but 'Webserver Ready' claim lacks deployment docs
• Create a minimal example script (e.g., examples/quick_clone_voice.py) showing how to clone a speaker's voice and generate speech in 10 lines, since README lacks concrete code snippets

Failed to generate security analysis.

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/babysor/MockingBird shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live babysor/MockingBird repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/babysor/MockingBird.

What it runs against: a local clone of babysor/MockingBird — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in babysor/MockingBird | Confirms the artifact applies here, not a fork | | 2 | License is still Other | Catches relicense before you depend on it | | 3 | Default branch main exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 97 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of babysor/MockingBird. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/babysor/MockingBird.git
#   cd MockingBird
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of babysor/MockingBird and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "babysor/MockingBird(\\.git)?\\b" \\
  && ok "origin remote is babysor/MockingBird" \\
  || miss "origin remote is not babysor/MockingBird (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(Other)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"Other\"" package.json 2>/dev/null) \\
  && ok "license is Other" \\
  || miss "license drift — was Other at generation time"

# 3. Default branch
git rev-parse --verify main >/dev/null 2>&1 \\
  && ok "default branch main exists" \\
  || miss "default branch main no longer exists"

# 4. Critical files exist
test -f "demo_toolbox.py" \\
  && ok "demo_toolbox.py" \\
  || miss "missing critical file: demo_toolbox.py"
test -f "models/encoder/model.py" \\
  && ok "models/encoder/model.py" \\
  || miss "missing critical file: models/encoder/model.py"
test -f "models/ppg2mel/train/train_linglf02mel_seq2seq_oneshotvc.py" \\
  && ok "models/ppg2mel/train/train_linglf02mel_seq2seq_oneshotvc.py" \\
  || miss "missing critical file: models/ppg2mel/train/train_linglf02mel_seq2seq_oneshotvc.py"
test -f "models/ppg_extractor/e2e_asr_common.py" \\
  && ok "models/ppg_extractor/e2e_asr_common.py" \\
  || miss "missing critical file: models/ppg_extractor/e2e_asr_common.py"
test -f "control/mkgui/app.py" \\
  && ok "control/mkgui/app.py" \\
  || miss "missing critical file: control/mkgui/app.py"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 97 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~67d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/babysor/MockingBird"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.