GO — Healthy across the board

• Last commit 2mo ago
• 15 active contributors
• Distributed ownership (top contributor 45% of recent commits)
• Apache-2.0 licensed
• Tests present
• ⚠ No CI workflows detected

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

Tongyi DeepResearch is an open-source agentic LLM (30.5B parameters, 3.3B activated per token) built by Alibaba-NLP that performs long-horizon, deep information-seeking tasks through web browsing and research automation. It combines three specialized agents (AgentFounder, AgentScaler, WebAgent) with tool integration (MCP, code execution) to execute multi-step research workflows autonomously. Monorepo with three parallel agent modules: Agent/AgentFounder (data scaling & performance optimization), Agent/AgentScaler (SFT-based scaling), and WebAgent/ (containing NestBrowse for hierarchical web exploration, ParallelMuse for reasoning aggregation, and AgentFold for protein folding). Shared dependencies on qwen-agent[gui,rag,code_interpreter,mcp] and sglang[all]. Config via .env.example; deployment via Docker and vLLM shells.

AI researchers and enterprise developers building autonomous research and information-gathering systems who need deep web exploration, multi-step reasoning, and tool use beyond basic RAG pipelines. Also appeals to teams deploying web agents that must handle complex, hierarchical research tasks with parallel execution.

Actively developed (evidenced by three distinct agent subsystems with individual READMEs, HuggingFace/ModelScope deployments, and an accompanying research paper arxiv.org/pdf/2510.24701). Has official model checkpoints and demo integrations (Modelscope studios, HuggingFace spaces, Aliyun Bailian service), suggesting production viability, though the repo is still relatively young with ongoing feature expansion visible in the ParallelMuse and NestBrowse submodules.

Depends heavily on external services (web browsing via MCP, LLM inference via vLLM or qwen-agent, search APIs) creating deployment complexity and latency sensitivity—the README explicitly warns that demos 'may fail intermittently due to model latency and tool QPS limits.' Single organization (Alibaba-NLP) maintains it; no visible community contribution history in file structure. Tool ecosystem (MCP client, browser toolkit) is custom-built and not battle-tested at scale.

Active development of web agent capabilities: NestBrowse toolkit (browser.py, mcp_client.py, tool_search.py) is being refined with async support (infer_async_nestbrowse.py); ParallelMuse extends this with compressed_reasoning_aggregation.py for multi-path inference. AgentScaler focuses on SFT vs. base model performance comparison. Tech report and paper recently published (arxiv.org/pdf/2510.24701).

git clone https://github.com/Alibaba-NLP/DeepResearch.git
cd DeepResearch
cp .env.example .env  # Configure API keys and model endpoints
pip install qwen-agent[gui,rag,code_interpreter,mcp] sglang[all]
cd WebAgent/NestBrowse
python infer_async_nestbrowse.py  # or vllm_deploy.sh for server mode

Daily commands: Local inference: cd WebAgent/NestBrowse && bash vllm_deploy.sh (starts vLLM server on GPU). For async single-shot queries: python infer_async_nestbrowse.py --query "<research task>". For guided UI: use qwen-agent GUI mode via qwen-agent-gui (requires qwen-agent[gui]). For protein folding: cd WebAgent/AgentFold && bash serve.sh. Demo: pre-hosted on Modelscope (https://www.modelscope.cn/studios/jialongwu/Tongyi-DeepResearch).

• README.md — Primary entry point documenting Tongyi Deep Research's purpose, models, and overall system architecture; essential for understanding project scope.
• Tech_Report.pdf — Comprehensive technical documentation covering the research methodology, model training, and experimental validation behind the deep research system.
• Agent/AgentFounder/README.md — Documents the AgentFounder component architecture, performance metrics, and integration patterns for the research agent foundation.
• WebAgent/WebDancer/demos/agents/search_agent.py — Core implementation of the search agent demonstrating the primary workflow for web-based research task execution.
• WebAgent/NestBrowse/toolkit/browser.py — Browser abstraction layer providing critical web navigation and interaction capabilities for the web agent system.
• WebAgent/ParallelMuse/compressed_reasoning_aggregation.py — Central reasoning aggregation pipeline for parallel research task coordination and result synthesis.
• WebAgent/WebDancer/demos/llm/qwen_dashscope.py — LLM integration point for Qwen models, showing how language models are plugged into the agent framework.

Add a New Web Research Tool

1. Create a new tool module in WebAgent/NestBrowse/toolkit/ or WebAgent/WebDancer/demos/tools/private/ (WebAgent/NestBrowse/toolkit/tool_search.py)
2. Implement tool interface with execute() method matching the existing tool_search.py or tool_explore.py patterns (WebAgent/NestBrowse/toolkit/tool_explore.py)
3. Register the tool in the MCP client via toolkit/mcp_client.py (WebAgent/NestBrowse/toolkit/mcp_client.py)
4. Add tool reference to the agent configuration in WebAgent/WebDancer/demos/agents/search_agent.py (WebAgent/WebDancer/demos/agents/search_agent.py)
5. Update prompts.py to include instructions for the new tool (WebAgent/NestBrowse/prompts.py)

Integrate a New LLM Backend

1. Create new LLM adapter in WebAgent/WebDancer/demos/llm/ following the pattern of qwen_dashscope.py and oai.py (WebAgent/WebDancer/demos/llm/qwen_dashscope.py)
2. Implement required interface: init, inference(), and streaming methods (WebAgent/WebDancer/demos/llm/oai.py)
3. Update search_agent.py to support model selection and instantiation of your new backend (WebAgent/WebDancer/demos/agents/search_agent.py)
4. Add configuration options in WebAgent/WebDancer/demos/assistant_qwq_chat.py for model selection (WebAgent/WebDancer/demos/assistant_qwq_chat.py)

Add a New Research Pipeline Stage

1. Create aggregation logic in WebAgent/ParallelMuse/compressed_reasoning_aggregation.py or create a new stage file (WebAgent/ParallelMuse/compressed_reasoning_aggregation.py)
2. Define stage inputs/outputs following ParallelMuse data format conventions (JSONL in data/ directory) (WebAgent/ParallelMuse/data/browsecomp_200.jsonl)
3. Integrate stage into search agent execution flow in search_agent.py (WebAgent/WebDancer/demos/agents/search_agent.py)
4. Add results storage in WebAgent/ParallelMuse/deepresearch_results/ directory with proper JSON serialization (WebAgent/ParallelMuse/data/browsecomp_200.jsonl)

Add a New UI Feature or Page

1. Create new component in WebAgent/WebDancer/demos/gui/ or extend web_ui.py with new route (WebAgent/WebDancer/demos/gui/web_ui.py)
2. Add HTML formatting utilities to html_decorate.py if custom styling is needed (WebAgent/WebDancer/demos/gui/html_decorate.py)
3. Wire UI component to agent backend via search_agent.py API calls (WebAgent/WebDancer/demos/agents/search_agent.py)
4. Add logging for the new feature using logging utilities from demos/utils/logs.py (WebAgent/WebDancer/demos/utils/logs.py)

• SGLang [all] — Provides high-performance LLM inference and structured generation for multi-stage reasoning pipelines at scale
• Qwen Agent Framework — Foundation for tool-use agent architecture with MCP (Model Context Protocol) support for extensible tool integration
• Python async/await patterns — Enables concurrent web navigation and parallel search execution without blocking
• JSONL data format — Efficient streaming storage for large research datasets and partial results in ParallelMuse

• Monolithic Python codebase rather than microservices

  • Why: Simpler deployment and lower latency for research pipelines; research tasks are typically long-running rather than high-concurrency
  • Consequence: Limited horizontal scaling; tightly-coupled components make isolated testing difficult

• MCP-based tool integration rather than direct tool implementation

  • Why: Provides decoupling and enables reuse of existing tools; easier to sandbox tools
  • Consequence: Added serialization overhead and process communication latency between agent and tools

• Simple file-based caching (cache_utils.py) over distributed cache (Redis)

  • Why: Lower operational complexity for research workflows that are typically single-user or small-scale
  • Consequence: Cache not shared across distributed instances; TTL management requires process-level coordination

• Synchronous LLM API calls despite async I/O elsewhere

  • Why: Most LLM provider SDKs (OpenAI, Qwen DashScope) are blocking; simplifies reasoning flow control
  • Consequence: LLM blocking calls can bottleneck parallel search execution; would benefit from async wrappers

• Real-time collaborative research (no multi-user session management)
• Production-grade authentication and authorization (demos use config-based auth only)
• Mobile-first UI (WebDancer UI targets desktop/web browsers

1. MCP and external service dependencies: infer_async_nestbrowse.py requires a running vLLM server (vllm_deploy.sh) and configured search API credentials in .env (not provided). 2. Async concurrency gotchas: NestBrowse uses Python async but toolkit/browser.py wraps Selenium (synchronous); blocking I/O can starve the event loop—see async wrapper patterns. 3. Memory overhead: ParallelMuse branches multiple inference paths; on <40GB VRAM, compression_aggregation.py may OOM without careful batch sizing. 4. Environment variables: code_interpreter and MCP require specific keys (SEARCH_API_KEY, SERPAPI_KEY inferred from imports) not documented in README. 5. Model quantization: vllm_deploy.sh assumes AWQ or GPTQ quantization is available; unquantized 30.5B model may not fit on single GPU.

• Model Context Protocol (MCP) — DeepResearch's toolkit/mcp_client.py relies on MCP for pluggable tool integration; understanding MCP is essential to extending agent capabilities beyond built-in search and browsing.
• Mixture of Experts (MoE) / Sparse Activation — The 30.5B model with only 3.3B activated tokens per inference is MoE-based; this significantly impacts latency, memory, and cost trade-offs in deployment.
• Hierarchical Task Decomposition — NestBrowse implements nested goal refinement (high-level query → sub-queries → page extraction → aggregation); critical for understanding how multi-step research is orchestrated.
• Supervised Fine-Tuning (SFT) on Agent Trajectories — Agent/AgentScaler focuses on SFT to improve agent decision-making; understanding trajectory data collection and loss functions is key to improving model performance.
• Parallel Inference with Output Aggregation — ParallelMuse's compressed_reasoning_aggregation.py branches reasoning paths in parallel; useful for exploring multiple hypotheses and selecting best-evidence conclusions.
• Web Automation & Dynamic Content Extraction — toolkit/browser.py wraps Selenium for JS-rendered pages and structured parsing; essential for real-world web research where static scraping fails.
• Long-Context Reasoning & In-Context Learning — DeepResearch is optimized for long-horizon tasks requiring memory of prior steps; implicitly uses in-context learning to guide multi-step reasoning.

• qwen-agent/qwen-agent — Core dependency for agentic framework; DeepResearch is built on top of Qwen-Agent's tool-use and code execution infrastructure.
• OpenDevin/OpenDevin — Parallel agentic LLM system for software engineering; similar autonomous task decomposition and tool-use patterns but domain-specific to code.
• geekan/MetaGPT — Multi-agent orchestration framework emphasizing role-based workflows; DeepResearch's AgentFounder/Scaler parallel structure echoes this approach.
• langchain-ai/langchain — LLM tool-chaining and agent framework; DeepResearch could integrate LangChain agents as an alternative to Qwen-Agent, though currently uses native agent protocol.
• anthropic/anthropic-sdk-python — Claude API with tool_use support; represents competitive agentic LLM capability in the same problem space as DeepResearch.

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add integration tests for WebAgent toolkits (NestBrowse, WebDancer, AgentFold)

The WebAgent directory contains three sophisticated sub-agents (NestBrowse, WebDancer, AgentFold) with complex toolkit dependencies (browser.py, mcp_client.py, tool_search.py, tool_explore.py) but no visible test files. Adding integration tests would validate toolkit initialization, tool execution, and inter-component communication, which is critical for web-based agent reliability.

• [ ] Create WebAgent/NestBrowse/tests/ with test_toolkit_initialization.py covering browser.py and mcp_client.py initialization
• [ ] Create WebAgent/NestBrowse/tests/test_tool_execution.py for tool_search.py and tool_explore.py with mock API responses
• [ ] Create WebAgent/WebDancer/tests/ with tests for dataset loading (sample_qa.jsonl, sample_traj.jsonl) and demo validation
• [ ] Create WebAgent/AgentFold/tests/ with tests for infer.py inference pipeline
• [ ] Add pytest configuration (pytest.ini) at WebAgent/ root and document test running in WebAgent/README.md

Create GitHub Actions CI workflow for Agent model inference validation

The repo has models deployed on HuggingFace and multiple inference entry points (AgentFounder, AgentScaler, WebAgent//infer.py, WebAgent/ParallelMuse/functionality_specified_partial_rollout.py) but no visible CI pipeline. Adding a GitHub Action workflow would catch inference regressions and model loading issues early, especially important given the multi-agent architecture.

• [ ] Create .github/workflows/inference-tests.yml with job to download lightweight model checkpoint or use mock model for testing
• [ ] Add inference validation step for Agent/AgentFounder (check infer.py if present) using synthetic input
• [ ] Add inference validation for WebAgent/NestBrowse/infer_async_nestbrowse.py and WebAgent/ParallelMuse/functionality_specified_partial_rollout.py
• [ ] Include dependency installation from sglang[all] and qwen-agent[gui,rag,code_interpreter,mcp]
• [ ] Add cache for model downloads and document in CONTRIBUTING.md (create if missing)

Add API documentation and type hints for ParallelMuse reasoning pipeline

WebAgent/ParallelMuse contains a sophisticated compressed reasoning aggregation system (compressed_reasoning_aggregation.py) but lacks docstrings and type hints. The module coordinates parallel reasoning paths and result merging—core to the 'Deep Research' value proposition—making it critical for contributor onboarding and correctness validation.

• [ ] Add comprehensive docstrings (Google/NumPy style) to WebAgent/ParallelMuse/compressed_reasoning_aggregation.py with input/output schemas
• [ ] Add Python type hints to compressed_reasoning_aggregation.py function signatures (especially aggregation functions)
• [ ] Add docstrings and type hints to WebAgent/ParallelMuse/functionality_specified_partial_rollout.py with examples of partial rollout workflow
• [ ] Create WebAgent/ParallelMuse/README_API.md documenting the reasoning aggregation pipeline, with examples using the provided data files (browsecomp_200.jsonl, hle_search_157.jsonl)
• [ ] Document the relationship between ParallelMuse and NestBrowse/WebDancer in WebAgent/README.md

• Add unit tests for WebAgent/NestBrowse/toolkit/tool_search.py and tool_explore.py (currently no test directory visible); would ensure search result parsing is robust across web page variations.
• Document the MCP protocol schema and add examples for custom tool registration in WebAgent/NestBrowse/toolkit/mcp_client.py; currently no examples of extending with new external services.
• Create an end-to-end example script showing ParallelMuse multi-path reasoning with tracing and visualization output (e.g., decision tree of reasoning branches); current data/browsecomp_200.jsonl has examples but no tutorial.

• High · Exposed Environment Configuration Template — .env.example. .env.example file contains detailed configuration for NCCL, TORCH, and application settings. While this is a template, it reveals the application's infrastructure dependencies and configuration structure, which could aid attackers in reconnaissance. Fix: Ensure .env.example does not contain production-like values. Use generic placeholders (e.g., 'your-api-key-here', 'set-to-value'). Document security-sensitive settings separately in internal documentation only.
• High · Missing Secret Management for API Keys — .env.example, WebAgent/WebDancer/demos/llm/qwen_dashscope.py, WebAgent/WebDancer/demos/llm/oai.py. The .env.example file shows configuration for QWen, Dashscope, and other services that likely require API keys. No evidence of secrets management best practices (e.g., AWS Secrets Manager, HashiCorp Vault, or environment variable validation). Fix: Implement a secrets management system. Use tools like python-dotenv with strict validation, AWS Secrets Manager, or HashiCorp Vault. Never commit .env files. Add pre-commit hooks to prevent credential commits.
• Medium · Potential Web UI Security Issues — WebAgent/WebDancer/demos/gui/web_ui.py, WebAgent/WebDancer/demos/gui/html_decorate.py. WebAgent/WebDancer/demos/gui/web_ui.py and html_decorate.py suggest web interface implementation. Without code review, there's risk of XSS, CSRF, and security header misconfiguration common in web UI frameworks. Fix: Implement Content Security Policy (CSP) headers, CSRF protection, input validation, and output encoding. Use framework security features (e.g., Flask/FastAPI security middleware). Conduct security code review of web interface components.
• Medium · Browser Automation Security Risks — WebAgent/NestBrowse/toolkit/browser.py, WebAgent/NestBrowse/toolkit/tool_explore.py, WebAgent/NestBrowse/toolkit/tool_search.py. WebAgent/NestBrowse/toolkit/browser.py and tool_explore.py suggest browser automation capabilities. This creates risks for SSRF attacks, XSS via navigated pages, and exposure to malicious content. Fix: Implement strict URL validation and allowlisting. Sanitize all user-provided URLs. Use headless browser sandboxing. Implement timeout protections. Log all navigation activities. Consider running browsers in isolated containers.
• Medium · MCP Client Communication Risks — WebAgent/NestBrowse/toolkit/mcp_client.py. WebAgent/NestBrowse/toolkit/mcp_client.py indicates Model Context Protocol client implementation. Unencrypted or unauthenticated MCP communication could allow man-in-the-middle attacks or unauthorized access. Fix: Ensure MCP communication uses TLS/SSL encryption. Implement mutual authentication. Validate certificates. Use secure serialization formats. Add request signing/HMAC verification.
• Medium · Data Cache Security — WebAgent/WebDancer/demos/tools/private/cache_utils.py. WebAgent/WebDancer/demos/tools/private/cache_utils.py suggests caching mechanism. If cache stores sensitive data (API responses, search results, user queries), it could expose PII or sensitive information. Fix: Encrypt sensitive cache data at rest. Implement cache expiration policies. Avoid caching PII, API keys, or sensitive tokens. Use secure cache backends (Redis with AUTH). Implement cache isolation per user/session.
• Medium · Third-Party Dependency Vulnerabilities — Package dependencies (sglang, qwen-agent). Dependencies like 'sglang[all]' and 'qwen-agent[gui,rag,code_interpreter,mcp]' with [all] and multiple extras increase the attack surface. Pin versions and audit transitive dependencies. Fix: Use pinned versions (e.g., sglang==0.x.x instead of sglang[all]). Implement dependency scanning with tools like Snyk, OWASP Dependency-Check, or pip-audit. Run regular vulnerability assessments. Use Software Composition Analysis (SCA)

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/Alibaba-NLP/DeepResearch shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live Alibaba-NLP/DeepResearch repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/Alibaba-NLP/DeepResearch.

What it runs against: a local clone of Alibaba-NLP/DeepResearch — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in Alibaba-NLP/DeepResearch | Confirms the artifact applies here, not a fork | | 2 | License is still Apache-2.0 | Catches relicense before you depend on it | | 3 | Default branch main exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 101 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of Alibaba-NLP/DeepResearch. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/Alibaba-NLP/DeepResearch.git
#   cd DeepResearch
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of Alibaba-NLP/DeepResearch and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "Alibaba-NLP/DeepResearch(\\.git)?\\b" \\
  && ok "origin remote is Alibaba-NLP/DeepResearch" \\
  || miss "origin remote is not Alibaba-NLP/DeepResearch (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(Apache-2\\.0)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"Apache-2\\.0\"" package.json 2>/dev/null) \\
  && ok "license is Apache-2.0" \\
  || miss "license drift — was Apache-2.0 at generation time"

# 3. Default branch
git rev-parse --verify main >/dev/null 2>&1 \\
  && ok "default branch main exists" \\
  || miss "default branch main no longer exists"

# 4. Critical files exist
test -f "README.md" \\
  && ok "README.md" \\
  || miss "missing critical file: README.md"
test -f "Tech_Report.pdf" \\
  && ok "Tech_Report.pdf" \\
  || miss "missing critical file: Tech_Report.pdf"
test -f "Agent/AgentFounder/README.md" \\
  && ok "Agent/AgentFounder/README.md" \\
  || miss "missing critical file: Agent/AgentFounder/README.md"
test -f "WebAgent/WebDancer/demos/agents/search_agent.py" \\
  && ok "WebAgent/WebDancer/demos/agents/search_agent.py" \\
  || miss "missing critical file: WebAgent/WebDancer/demos/agents/search_agent.py"
test -f "WebAgent/NestBrowse/toolkit/browser.py" \\
  && ok "WebAgent/NestBrowse/toolkit/browser.py" \\
  || miss "missing critical file: WebAgent/NestBrowse/toolkit/browser.py"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 101 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~71d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/Alibaba-NLP/DeepResearch"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.