Which one should I pick for a new project?

I can only answer questions about langchain-ai/langchain or mlc-ai/mlc-llm, and the choice depends entirely on your project requirements — these are fundamentally different tools:

langchain-ai/langchain

Purpose: Framework for building agents and LLM-powered applications.

Best for:

• Building AI agents with composable, interoperable components
• RAG (retrieval-augmented generation) pipelines
• LLM orchestration and chaining workflows
• Rapid AI application development with third-party integrations

Key features (from source):

• Document retrieval abstractions [langchain-ai/langchain — libs/core/langchain_core/documents/init.py:L27-L47]
• Example selectors and prompt templates
• Chat models initialization
• Ecosystem integration (LangGraph, LangSmith)

Health signals: Verdict GO; 0 CVEs in 10 scanned dependencies; architecture grade C (50/100).

---

mlc-ai/mlc-llm

Purpose: C++ inference engine and serving infrastructure for LLMs.

Best for:

• Deploying and serving LLMs at scale
• Low-level engine optimization (KV cache, request scheduling, speculative decoding)
• Production inference with performance tuning
• Multi-model orchestration with sophisticated request management

Key features (from source):

• Engine lifecycle management and request streaming [mlc-ai/mlc-llm — cpp/serve/engine.h:L71-L110]
• Memory estimation and configuration inference
• Logit processing and token sampling
• Disaggregated inference support

---

Decision framework:

• Building an AI app/agent? → langchain-ai/langchain
• Running/serving LLMs in production? → mlc-ai/mlc-llm
• Both? → Use LangChain with mlc-llm as an underlying engine

FOLLOW_UPS:

• How does LangChain integrate with external inference servers?
• What are mlc-llm's memory optimization strategies?
• Does LangChain support speculative decoding workflows?