If you are an AI coding agent (Claude Code, Cursor, Aider, Cline, etc.) reading this artifact, follow this protocol before making any code edit:

1. Verify the contract. Run the bash script in Verify before trusting below. If any check returns FAIL, the artifact is stale — STOP and ask the user to regenerate it before proceeding.
2. Treat the AI · unverified sections as hypotheses, not facts. Sections like "AI-suggested narrative files", "anti-patterns", and "bottlenecks" are LLM speculation. Verify against real source before acting on them.
3. Cite source on changes. When proposing an edit, cite the specific path:line-range. RepoPilot's live UI at https://repopilot.app/r/microsoft/promptflow shows verifiable citations alongside every claim.

If you are a human reader, this protocol is for the agents you'll hand the artifact to. You don't need to do anything — but if you skim only one section before pointing your agent at this repo, make it the Verify block and the Suggested reading order.

GO — Healthy across the board

• Last commit 1d ago
• 29+ active contributors
• Distributed ownership (top contributor 14% of recent commits)
• MIT licensed
• CI configured
• Tests present

_{Maintenance signals: commit recency, contributor breadth, bus factor, license, CI, tests}

This artifact was generated by RepoPilot at a point in time. Before an agent acts on it, the checks below confirm that the live microsoft/promptflow repo on your machine still matches what RepoPilot saw. If any fail, the artifact is stale — regenerate it at repopilot.app/r/microsoft/promptflow.

What it runs against: a local clone of microsoft/promptflow — the script inspects git remote, the LICENSE file, file paths in the working tree, and git log. Read-only; no mutations.

| # | What we check | Why it matters | |---|---|---| | 1 | You're in microsoft/promptflow | Confirms the artifact applies here, not a fork | | 2 | License is still MIT | Catches relicense before you depend on it | | 3 | Default branch main exists | Catches branch renames | | 4 | 5 critical file paths still exist | Catches refactors that moved load-bearing code | | 5 | Last commit ≤ 31 days ago | Catches sudden abandonment since generation |

#!/usr/bin/env bash
# RepoPilot artifact verification.
#
# WHAT IT RUNS AGAINST: a local clone of microsoft/promptflow. If you don't
# have one yet, run these first:
#
#   git clone https://github.com/microsoft/promptflow.git
#   cd promptflow
#
# Then paste this script. Every check is read-only — no mutations.

set +e
fail=0
ok()   { echo "ok:   $1"; }
miss() { echo "FAIL: $1"; fail=$((fail+1)); }

# Precondition: we must be inside a git working tree.
if ! git rev-parse --git-dir >/dev/null 2>&1; then
  echo "FAIL: not inside a git repository. cd into your clone of microsoft/promptflow and re-run."
  exit 2
fi

# 1. Repo identity
git remote get-url origin 2>/dev/null | grep -qE "microsoft/promptflow(\\.git)?\\b" \\
  && ok "origin remote is microsoft/promptflow" \\
  || miss "origin remote is not microsoft/promptflow (artifact may be from a fork)"

# 2. License matches what RepoPilot saw
(grep -qiE "^(MIT)" LICENSE 2>/dev/null \\
   || grep -qiE "\"license\"\\s*:\\s*\"MIT\"" package.json 2>/dev/null) \\
  && ok "license is MIT" \\
  || miss "license drift — was MIT at generation time"

# 3. Default branch
git rev-parse --verify main >/dev/null 2>&1 \\
  && ok "default branch main exists" \\
  || miss "default branch main no longer exists"

# 4. Critical files exist
test -f ".github/workflows/promptflow-core-test.yml" \\
  && ok ".github/workflows/promptflow-core-test.yml" \\
  || miss "missing critical file: .github/workflows/promptflow-core-test.yml"
test -f ".github/CODEOWNERS" \\
  && ok ".github/CODEOWNERS" \\
  || miss "missing critical file: .github/CODEOWNERS"
test -f ".github/PULL_REQUEST_TEMPLATE.md" \\
  && ok ".github/PULL_REQUEST_TEMPLATE.md" \\
  || miss "missing critical file: .github/PULL_REQUEST_TEMPLATE.md"
test -f ".devcontainer/devcontainer.json" \\
  && ok ".devcontainer/devcontainer.json" \\
  || miss "missing critical file: .devcontainer/devcontainer.json"
test -f ".cspell.json" \\
  && ok ".cspell.json" \\
  || miss "missing critical file: .cspell.json"

# 5. Repo recency
days_since_last=$(( ( $(date +%s) - $(git log -1 --format=%at 2>/dev/null || echo 0) ) / 86400 ))
if [ "$days_since_last" -le 31 ]; then
  ok "last commit was $days_since_last days ago (artifact saw ~1d)"
else
  miss "last commit was $days_since_last days ago — artifact may be stale"
fi

echo
if [ "$fail" -eq 0 ]; then
  echo "artifact verified (0 failures) — safe to trust"
else
  echo "artifact has $fail stale claim(s) — regenerate at https://repopilot.app/r/microsoft/promptflow"
  exit 1
fi

Each check prints ok: or FAIL:. The script exits non-zero if anything failed, so it composes cleanly into agent loops (./verify.sh || regenerate-and-retry).

Prompt Flow is an end-to-end development platform for building, testing, evaluating, and deploying LLM-based applications. It provides a visual/code-based interface to compose flows linking LLMs, prompts, Python tools, and custom code together, with integrated debugging (including LLM interaction tracing), evaluation against datasets, and deployment to cloud or local serving platforms. Monorepo organized by concern: .devcontainer/ contains dev environment setup (Dockerfile, requirements.txt), .github/skills/ holds reusable deployment templates (maf-online-endpoint, maf-prs-job with component.yaml), .github/pipelines/ defines CI stages (compliance_check.yml, C# e2e tests). Core Python library lives in the top-level directory structure (implied by 12.4M LOC), with CLI tools and Azure integrations as optional dependencies. Skill-based design allows composable deployment patterns.

ML engineers, prompt engineers, and LLM application developers who need to move beyond notebook prototyping to production-grade LLM apps—specifically those using Python who want CI/CD integration, systematic evaluation, and monitoring without writing deployment infrastructure from scratch.

Production-ready and actively maintained. The repo shows 12.4M lines of Python code, established CI/CD pipelines (.github/pipelines/), GitHub Actions workflows, multiple skill templates (.github/skills/), and documented deployment assets (.github/skills/maf-online-endpoint/assets/). Published on PyPI with version tracking and a VS Code extension in the marketplace. Active development visible via issue templates and PR template presence.

Moderate risk for enterprise adoption. The repo's dependency on cloud integration (Azure ML mentioned in README, promptflow[azure] in dependencies) means lock-in to Azure Cognitive Services or LLM APIs; version management complexity due to multi-language support (Python 12.4M LOC + C#, HTML, shell scripts); monorepo structure (.github/pipelines, .github/skills/) may cause larger blast radius on refactoring. No single-maintainer risk visible (Microsoft-backed), but rapid LLM ecosystem evolution could make some integrations stale.

Active development on cloud-native deployment skills (maf-online-endpoint skill with managed identity and deployment patterns), C# integration testing (promptflow-csharp-e2e-test pipelines), and compliance/quality automation (.github/pipelines/compliance_check.yml). GitHub Actions reusable workflows being used for SDK setup, environment creation, and test result publishing. Recent work includes multi-language support and production deployment automation.

Clone the repo, set up Python environment (3.9+ inferred from PyPI badges), install with pip: git clone https://github.com/microsoft/promptflow.git && cd promptflow && pip install -e . && pip install -e .[azure] (based on README dependencies). For dev: use .devcontainer/Dockerfile for containerized setup or run pip install -r .devcontainer/requirements.txt locally. Run pf --help to verify CLI installation.

Daily commands: As a CLI tool: pf flow run --file flow.yaml --data data.jsonl. For SDK: from promptflow import PFClient; client = PFClient(). For development/testing: implied Makefile or pytest (Makefile listed in file structure) — likely make test or python -m pytest. For cloud deployment, use .github/skills/maf-online-endpoint/assets/deploy.sh or pf flow deploy --flow . --name my-flow.

• .github/workflows/promptflow-core-test.yml — Primary CI/CD pipeline for core promptflow—defines test matrix, dependencies, and release validation that gates all contributions.
• .github/CODEOWNERS — Documents ownership and review requirements across modules; essential for understanding who approves changes in each component.
• .github/PULL_REQUEST_TEMPLATE.md — Standard PR template enforced on all contributions; establishes baseline expectations for commit quality, testing, and documentation.
• .devcontainer/devcontainer.json — Configures reproducible development environment; required reading for local setup and understanding dependency resolution strategy.
• .cspell.json — Spell-check configuration for documentation and code; prevents false positives and maintains consistency in naming conventions.
• CONTRIBUTING.md — Contribution guidelines covering branching, testing, documentation, and submission workflows—mandatory for any code submission.

Add a New E2E Test Workflow

1. Create a new workflow YAML file in .github/workflows/ following the naming pattern promptflow-<component>-e2e-test.yml (.github/workflows/promptflow-core-test.yml)
2. Define the test matrix with Python versions, OS targets, and dependency variants; reference reusable actions like step_sdk_setup and step_publish_test_results (.github/actions/step_sdk_setup/action.yml)
3. Add test trigger conditions (on.push.paths, on.pull_request) to target specific directories and avoid redundant runs (.github/workflows/promptflow-core-test.yml)
4. Update .github/CODEOWNERS if your component requires new reviewers (.github/CODEOWNERS)

Add a New Sample or Skill Integration

1. Create a new skill directory under .github/skills/<skill-name>/ with SKILL.md documentation explaining the integration goal (.github/skills/maf-online-endpoint/SKILL.md)
2. Place example configs (YAML, Python scripts) in assets/ subdirectory; include conda environment files if custom dependencies are needed (.github/skills/maf-online-endpoint/assets/conda.yml)
3. Add step-by-step guides in .github/skills/<skill-name>/references/ to help users troubleshoot or understand advanced scenarios (.github/skills/maf-online-endpoint/references/troubleshooting.md)
4. If the skill requires deployment automation, include scripts in .github/skills/<skill-name>/scripts/ with test harness (.github/skills/maf-online-endpoint/scripts/test_skill.py)

Add or Update GitHub Action Helper

1. Create a new action directory in .github/actions/step_<action-name>/ with action.yml defining inputs, outputs, and runs configuration (.github/actions/step_sdk_setup/action.yml)
2. Implement the action logic (shell scripts or composite steps) in the same directory; document parameters in action.yml with examples (.github/actions/step_sdk_setup/action.yml)
3. Reference the new action in CI workflows using uses: ./.github/actions/step_<action-name> for local reusability (.github/workflows/promptflow-core-test.yml)

Update Development Environment Setup

1. Modify .devcontainer/requirements.txt to add or pin new Python dev dependencies (.devcontainer/requirements.txt)
2. If base image or system tools need updates, edit .devcontainer/Dockerfile and note changes in .devcontainer/README.md (.devcontainer/Dockerfile)
3. Update .devcontainer/devcontainer.json to add VS Code extensions, mount points, or feature flags (e.g., for new Python versions) (.devcontainer/devcontainer.json)
4. Test locally: gh codespace create or use VS Code Remote Containers to validate the updated environment (.devcontainer/devcontainer.json)

• GitHub Actions — Native CI/CD for GitHub repos; allows declarative workflow-as-code (YAML), reusable actions, and matrix testing across Python versions/OS without external SaaS.
• VS Code Dev Containers — Standardizes local development environment using Docker; eliminates 'works on my machine' issues and reduces onboarding friction for contributors.
• Python 3.9–3.11 matrix — Ensures promptflow compatibility across multiple Python LTS versions; critical for broad user adoption and long-term support.
• Azure ML / MAF integration — Positions promptflow as production-ready for enterprise users deploying to Azure; skills and templates lower barrier to cloud adoption.
• Composite GitHub Actions (step_*) — Reduces duplication across workflows; centralizes environment setup, test result reporting, and merge logic for maintainability.

• Monorepo for SDK, CLI, evals, and executor components

  • Why: Shared dependencies, consistent versioning, and atomic releases across related modules; easier cross-component testing.
  • Consequence: Larger CI/CD matrix; slower per-workflow execution if all tests run on every commit; requires careful CODEOWNERS setup to avoid bottlenecks.

• Dev containers + GitHub Codespaces as primary dev environment

  • Why: Frictionless onboarding; reproducible setup; optional cloud-hosted development for resource-constrained contributors.
  • Consequence: Higher upfront learning curve for contributors unfamiliar with containerization; local Docker/Podman required for offline development.

• Separate e2e and unit test workflows (e.g., promptflow-core-test.yml vs. -unit-test.yml)

  • Why: Allows parallel execution; e2e tests can be slower and run on schedule; unit tests run on every PR for fast feedback.
  • Consequence: More workflow files to maintain; requires clear documentation of which tests run when and on which triggers.

Azure dependency: promptflow[azure] expects Azure subscription + credentials (AZURE_SUBSCRIPTION_ID, etc.) if using cloud features; local-only flows work without it. YAML versioning: flow.yaml format may change; check schema version in component.yaml. Multi-language complexity: C# tests in .github/pipelines/ imply .NET tooling required for full CI (not just Python). DevContainer quirk: .devcontainer/Dockerfile has custom Python setup—don't assume system Python works. Jinja templates in flows are not standard Python f-strings—can cause subtle escaping issues. No explicit tests/ directory visible in file list—tests may be embedded or in unshown subdirectories.

• DAG (Directed Acyclic Graph) Flow Composition — Prompt Flow's core abstraction—flows are DAGs where nodes are tools/LLMs and edges are data dependencies; understanding DAG semantics is essential to design correct flows
• Prompt Templating with Jinja2 — Flows use .jinja files for dynamic prompt generation; Jinja's variable interpolation, filters, and loops are critical for parametric LLM calls in Prompt Flow
• YAML-based Infrastructure as Code (IaC) — All flows, components, and deployments are defined in YAML (component.yaml, flow.yaml, deployment.yaml); this is Prompt Flow's serialization and version-control format
• LLM Trace / Observability — Prompt Flow includes built-in tracing of LLM calls (mentioned in README); this is critical for debugging non-deterministic LLM outputs and monitoring production apps
• Provider Pattern / Pluggable Backends — Prompt Flow supports multiple LLM providers (Azure OpenAI, OpenAI, HuggingFace, etc.) via a provider abstraction; new contributors need to understand how to add a new LLM backend
• Azure ML Component SDK / MLflow Models — Flows are packaged as Azure ML components (component.yaml) and deployed via MLflow; integration with Azure's managed inference is core to Prompt Flow's production story
• GitHub Actions Reusable Workflows — Prompt Flow's CI/CD uses modular Action files (.github/actions/step_*) for SDK setup, environment creation, and test publishing; understanding this pattern is needed to extend the build pipeline

• langchain-ai/langchain — Primary LLM orchestration alternative; Prompt Flow integrates with LangChain tools and shares the flow/chain-composition paradigm
• openai/openai-cookbook — Companion repo showing real-world examples of using OpenAI APIs; Prompt Flow's default LLM backend for many flows
• Azure/azure-sdk-for-python — Dependency for Azure ML integration (promptflow[azure]); contains the MLClient used for cloud deployment
• huggingface/transformers — Alternative LLM backend for Prompt Flow; enables local/open-source model execution without Azure dependency
• microsoft/responsible-ai-toolbox — Companion Microsoft project for evaluation and fairness metrics; integrates with Prompt Flow's evaluation framework

To work on one of these in Claude Code or Cursor, paste: Implement the "<title>" PR idea from CLAUDE.md, working through the checklist as the task list.

Add comprehensive CI/CD workflow for C# SDK end-to-end tests

The repo has C# E2E test pipeline files (.github/pipelines/promptflow-csharp-e2e-test*.yml) but they appear to be incomplete or lack integration with the main CI workflow. There's no evidence of a consolidated GitHub Actions workflow that runs these C# tests on PR/push. This is critical for a multi-language SDK to catch regressions early.

• [ ] Review .github/pipelines/promptflow-csharp-e2e-test-env-setup.yml, promptflow-csharp-e2e-test-tests.yml, and promptflow-csharp-e2e-test.yml for completeness
• [ ] Create a unified .github/workflows/csharp-e2e-tests.yml that triggers on PR events and runs the test suite
• [ ] Integrate test results publishing using the existing .github/actions/step_publish_test_results/action.yml
• [ ] Add conditional skip logic for C# tests when only Python files are modified (optional optimization)

Add unit tests for GitHub Actions custom actions (step_* actions)

The repo defines 6+ custom GitHub Actions in .github/actions/step_* but there's no visible test suite for these actions. These are critical reusable workflows (SDK setup, environment creation, config generation, etc.) that could silently break. Adding tests ensures reliability across all CI jobs.

• [ ] Create .github/actions/tests/ directory structure for action tests
• [ ] Add test YAML files that verify each action's outputs (e.g., step_sdk_setup validation, step_generate_configs output verification)
• [ ] Use act (GitHub Actions local runner) or create a .github/workflows/test-actions.yml workflow to validate actions on PR
• [ ] Document action testing in .github/actions/README.md or similar

Add missing GitHub Skills documentation and test coverage

The repo contains 4 GitHub Skills (.github/skills/maf-*/SKILL.md) with references, examples, and templates, but there's no central registry, validation, or test suite for these skills. A new contributor cannot easily verify if a skill works correctly. Adding a skills validation workflow and index would improve discoverability and reliability.

• [ ] Create .github/skills/README.md with a skills registry/index linking all available skills (maf-online-endpoint, maf-prs-job, maf-tracing, promptflow-to-maf)
• [ ] Add a .github/workflows/validate-skills.yml that tests skill YAML files (schema validation, broken links in references/)
• [ ] Create a simple validation script (e.g., .github/skills/validate.py) that checks for required files (SKILL.md, assets/, references/)
• [ ] Optionally add .github/skills/CONTRIBUTING.md with skill development guidelines

• Add integration tests for the maf-prs-job skill (see .github/skills/maf-prs-job/assets/) to verify component.yaml schema validation and entry.py execution in isolation—currently only documented but no test file visible
• Document the .devcontainer environment setup in CONTRIBUTING.md (not visible in file list) with troubleshooting for common Docker/Python version mismatches on Windows/Mac
• Add a Makefile target for make dev-install that wraps the pip install -e . && pip install -e .[azure] commands and validates all optional dependencies are installed correctly

The codebase demonstrates a reasonable security posture with a structured CI/CD pipeline and organized development practices. However, there are gaps in automated security scanning integration, explicit dependency pinning, and documented secret management. The primary concerns are the incomplete security policy documentation (which could hinder responsible disclosure) and the absence of visible security scanning tools in the CI/CD workflows. Recommendations focus on implementing automated security testing, tightening dependency constraints, and establishing clear security guidelines for contributors.

• Medium · Incomplete Security Policy Documentation — SECURITY.md. The SECURITY.md file appears to be truncated or incomplete. The security vulnerability reporting instructions are cut off mid-sentence, which could confuse users trying to report security issues responsibly. Fix: Complete the SECURITY.md file with full instructions for reporting security vulnerabilities. Ensure the Microsoft Security Response Center (MSRC) reporting URL and process are fully documented.
• Medium · Dependency Version Pinning Not Explicit — Package dependencies (requirements specification). The dependency specification 'promptflow[azure]>=1.9.0' uses a minimum version constraint without an upper bound. This could lead to automatic installation of newer major versions with potentially breaking changes or unvetted features. Fix: Use explicit version pinning with upper bounds (e.g., 'promptflow[azure]>=1.9.0,<2.0.0') or pin to specific tested versions. Regularly audit and update dependencies with proper testing.
• Low · Multiple CI/CD Pipelines Without Visible Security Scanning — .github/workflows/, .github/pipelines/. The repository contains numerous GitHub Actions workflows (.github/workflows/.yml and .github/pipelines/.yml) but no visible SAST (Static Application Security Testing), dependency scanning, or container security scanning steps are evident from the file structure. Fix: Implement automated security scanning in CI/CD pipelines: 1) Enable Dependabot for dependency vulnerability scanning 2) Integrate SAST tools like CodeQL 3) Add container image scanning for Docker images 4) Include license compliance checks
• Low · Docker Configuration Without Security Best Practices Visible — .devcontainer/Dockerfile. The Dockerfile in .devcontainer/Dockerfile is not accessible for review, but no evidence of security best practices (non-root user, minimal base images, layer caching) is visible in the file structure. Fix: Ensure Docker best practices: 1) Use specific base image versions (not 'latest') 2) Run containers as non-root user 3) Use multi-stage builds to minimize image size 4) Scan images for vulnerabilities using tools like Trivy
• Low · No Visible Secret Management Strategy — .github/skills/, .github/actions/, .devcontainer/. No evidence of a secret management system (e.g., GitHub Secrets, Azure Key Vault) documentation is visible. Development and test scripts reference credentials and environment setup without clear secure handling guidelines. Fix: Implement and document a comprehensive secret management strategy: 1) Use GitHub Secrets for workflow secrets 2) Document secret rotation policies 3) Never commit credentials, API keys, or tokens 4) Use environment-based credential management

LLM-derived; treat as a starting point, not a security audit.

• Open issues — current backlog
• Recent PRs — what's actively shipping
• Source on GitHub

Generated by RepoPilot. Verdict based on maintenance signals — see the live page for receipts. Re-run on a new commit to refresh.