---

name: engineering-manager-agent-prompts-evals description: | Guides engineering managers leading teams that own agent prompts, tool schemas, golden eval suites, judge programs, and prompt regression CI—org design, hiring, roadmap with PM/risk, release governance, and team KPIs for eval quality and prompt stability. Use when managing prompt/eval engineers, prioritizing eval backlogs, staffing harness and judge programs, or governing prompt releases—not for writing prompts or harnesses (prompt-engineer-agent-prompts-evals), vertical AI squad management (engineering-manager-vertical-ai-products), or AI platform SRE (ai-lead-ops).

Engineering Manager, Agent Prompts & Evals

When to Use

• Build or scale a prompt + eval engineering function (central or embedded)
• Prioritize golden set, harness, and judge work with PM and risk
• Define release policy for prompt/tool changes (gates, waivers, rollback)
• Hire and level prompt engineers, eval engineers, and tech leads
• Resolve capacity conflicts between new agents and eval debt
• Report eval health (pass rate, slice regressions, judge drift) to leadership
• Partner with ai-lead-ops on production incidents tied to prompts

When NOT to Use

• Author system prompts, tool schemas, or eval cases → prompt-engineer-agent-prompts-evals
• General prompt patterns (CoT, few-shot) → prompt-engineer
• Full RAG/agent application code → ai-engineer
• Vertical AI product roadmap and GTM launches → engineering-manager-vertical-ai-products
• Adversarial red-team execution → ai-redteam
• Org-wide data or analytics management → data-manager

Related skills

Need	Skill
IC prompt/eval implementation	prompt-engineer-agent-prompts-evals
Vertical AI EM (broader)	engineering-manager-vertical-ai-products
AI ops and rollouts	ai-lead-ops
Risk tier and policy	ai-risk-governance
Token cost programs	ai-token-improvement-plan-engineer

Core Workflows

1. Org design

Central eval platform vs embedded prompt owners; ratios and interfaces.

See references/team_org_prompt_eval.md.

2. Roadmap and prioritization

Eval debt, new agent coverage, judge calibration, CI investment.

See references/roadmap_prioritization.md.

3. Release governance

Prompt semver gates, waivers, coordination with risk and ops.

See references/release_governance.md.

4. Stakeholder partnerships

PM, applied architect, risk, red-team, platform eng.

See references/stakeholder_partnerships.md.

5. Hiring and development

Levels for prompt/eval specialists.

See references/hiring_development.md.

6. Team metrics

Pass rate, coverage, time-to-golden-case, incident linkage.

See references/team_metrics_accountability.md.

Output standards

• Roadmap items name eval slice, prompt surface, and risk tier
• No prompt prod change without documented baseline vs candidate
• Waivers require risk sign-off for Tier 1–2 customer-facing agents
• Escalations include trade-offs (scope, date, headcount)

When to load references

• Org → references/team_org_prompt_eval.md
• Roadmap → references/roadmap_prioritization.md
• Release → references/release_governance.md
• Stakeholders → references/stakeholder_partnerships.md
• People → references/hiring_development.md
• KPIs → references/team_metrics_accountability.md