PR review pipeline cuts senior-engineer time 4×

Senior engineers were spending 8–12 hours per week each on first-pass PR review across a 6-team monorepo. Junior PRs waited 2+ days for sign-off; velocity stalled; the highest-judgement people were doing the lowest-judgement work.

• 1Senior-engineer time concentrated on first-pass review, not architecture or mentorship
• 2Median PR-open-to-merge time of 41 hours, with the long tail blocking releases
• 3Inconsistent review depth — security and test-coverage gaps caught reactively after merge
• 4Six teams, six different review conventions; cross-team PRs stalled on style debates

A multi-agent CI workflow triggered on every PR open. Three specialist agents run in parallel — a reviewer (Claude Sonnet 4.6) for code-correctness and convention, a security agent for risk patterns, and a test-generator agent for coverage gaps. Outputs are consolidated into a single PR comment within 90 seconds. Humans review the agent's synthesis, not the raw diff.

The agents run as a GitHub Actions workflow triggered on PR open and on every subsequent push. They write to the PR via the GitHub REST API as a bot account scoped to the org — read on all repos, write only to PR comments. Existing branch protections and required reviewers are untouched.

• GitHub Actions runner: existing org runners, no new infrastructure
• Bot account: scoped to PR comments + status checks only
• No changes to branch protection rules — the agent is advisory, not gatekeeping
• Slack notifier piggy-backs on the existing PR-events channel

Runs on the customer's existing GitHub Actions runner pool. No new infrastructure provisioned. The MCP server runs as a container alongside the runner; bot credentials live in GitHub Actions secrets and never enter prompts.

• Runtime: GitHub Actions self-hosted runner (existing pool)
• MCP server: containerised Python service, scoped tokens
• Secret manager: GitHub Actions secrets (not in repo, not in prompts)
• Rollback: feature-flag PR-level skip — single env var pause

Every agent step traces to Langfuse with the PR number as the session key. Per-agent cost, model version, cache-hit, and the integrator's score-delta are all surfaced in a single Langfuse dashboard the team checks daily.

• Per-step Langfuse trace with PR-id session key
• Cost attribution: per-PR · per-agent · per-team
• Cache-hit rate on stable system prompts: 78–84% steady state
• Eval-suite regression CI step on every prompt/model bump

The agent handles first-pass review on every PR. Senior engineers now review the agent's synthesis (and only intervene on the cases it flagged uncertain). The volume of PRs that reach human review unchanged from agent-pass dropped to under 15%.

End-to-end agent pipeline runs in under 90 seconds for the median PR. The largest refactors (10K+ line diffs) take 4–6 minutes, still well inside the team's SLA expectations.

Business outcomes

Six senior engineers reclaimed ~6 hours each per week (≈36 engineer-hours / week across the team). At the customer's loaded-cost rate, the agent pays for itself in under three days of operation.

• 01The eval suite was the unlock. Without 47 adversarial cases scoring every model bump, the team would not trust the agent enough to merge on it.
• 02Three specialists beat one big agent. A single agent with all the tools loaded was 18% less accurate on the same eval set than the three-specialist pattern.
• 03The integrator is the highest-leverage prompt in the system. It is the only one that sees the full picture; iterating on it produced the largest accuracy jumps.
• 04Cost surprised us positively. The cache-hit rate on the stable system prefix turned out to be the dominant cost lever, not the model choice.

The same agent pattern is being extended to release-note generation, dependency-upgrade review, and incident-postmortem drafting. The shared MCP server and observability stack carry over — each new agent is ~1 week to ship instead of 6 weeks.

• Release-note generator using the same GitHub MCP server (in progress)
• Dependency-upgrade review agent reusing the security-scan agent's prompt patterns
• Postmortem-drafter agent fed from incident-channel exports
• Cross-team eval-set library so new agents inherit regression coverage from day one

Want a PR review pipeline like this?

Most engagements like this take 6–8 weeks discovery to handoff. A 60-minute scoping session is enough to tell whether your repo shape and team are a fit.