skill-systems-architectureDesigns production-ready multi-skill agent architectures with cross-skill coordination patterns, quality gates, deployment pipelines, and monitoring strategies for AI skill ecosystems.
Independent GitHub snapshot · Automated classification · Original documentation governs
name: skill-systems-architecture description: Designs production-ready multi-skill agent architectures with cross-skill coordination patterns, quality gates, deployment pipelines, and monitoring strategies for AI skill ecosystems. license: MIT compatibility: opencode archetypes:
Orchestrates the production deployment of multi-skill agent systems — designing how skills coordinate at runtime, validating quality gates across pipelines, and building monitoring strategies that keep skill ecosystems healthy under real-world load. This skill operates at the system level: it does not create individual SKILL.md files, manage retirement schedules, or design network topology.
Use this skill when:
Avoid this skill for:
skill-engineering instead (that skill handles trigger engineering, stub detection, and constraint design)skill-lifecycle-management insteadskill-ecosystem-design insteadskill-audit instead (that skill validates content against standards)skill-router-system instead┌─────────────────────────────────────────────────────────────────────────────┐
│ MULTI-SKILL SYSTEM ARCHITECTURE │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Author │───▶│ Validate │───▶│ Staging │───▶│ Production│ │
│ │ Skills │ │ Pipeline │ │ Deploy │ │ Deploy │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
│ │ │ │ │
│ [Quality Gate 1] [Quality Gate 2] │ │
│ stub + trigger integration │ │
│ validation tests │ │
│ │ │
│ ┌──────────────┐ │ │
│ │ Monitoring & │◄──────────────────────────────┘ │
│ │ Observability│ (feedback loop from prod) │
│ └──────────────┘ │
│ │
│ Runtime Flow: │
│ Agent Task → Router Index → Skill Match → Load Skill(s) → Execute │
│ ↓ ↓ ↓ │
│ trigger confidence fallback chain │
│ validation scoring on failure │
└──────────────────────────────────────────────────────────────────────────────────┘
Inventory and Classify Existing Skills — Catalog all skills in the ecosystem. Classify each by: role (implementation, orchestration, reference), domain, maturity (draft/beta/stable), and criticality (core vs auxiliary). This creates the baseline inventory for architecture decisions.
Checkpoint: Every skill must have a current metadata.maturity field set. Skills without maturity classification should be flagged as "unclassified" and routed to skill-audit for assessment before inclusion in the production pipeline.
Design Cross-Skill Coordination Patterns — Define how skills interact at runtime. For every group of related skills (e.g., the full risk management suite), specify:
Checkpoint: No single skill should be a blocking dependency for more than 3 other skills. If it is, decompose the dependency into async loading with graceful degradation.
Define Quality Gates — Establish validation stages that every skill passes through before deployment:
Pipeline Stages (quality gates):
Stage 1: Frontmatter Validation
├── YAML parseable?
├── name matches directory kebab-case?
├── triggers count between 3-8 terms?
└── role/scope/output-format are valid enum values?
Stage 2: Stub Detection (Zero-Tolerance)
├── File size >= 3,000 bytes?
├── Zero stub sentinel occurrences?
├── >= 2 real code blocks (for implementation skills)?
└── Core Workflow has domain-specific steps with Checkpoints?
Stage 3: Trigger Space Analysis
├── No shared triggers > 30% with sibling skills?
├── At least one conversational variant present?
└── No ultra-generic single-word triggers?
Stage 4: Integration Testing
├── All related-skills references resolve to existing files?
├── Reciprocity verified (if A lists B, B lists A)?
└── No circular dependencies in the dependency graph?
Checkpoint: All four quality gate stages must return PASS before a skill advances to staging. Any failure produces an actionable report with specific fixes referenced back to AGENTS.md quality checklist.
Build the Deployment Pipeline — Design the CI/CD pipeline for skills:
# Example deployment pipeline structure (GitOps pattern)
pipeline:
name: skill-deployment-pipeline
stages:
- name: validate
triggers: [push, pull_request]
checks:
- validate_skill.sh --llm # Static + LLM validation
- python3 scripts/generate_readme.py # Regenerate catalog
- git diff --exit-code # Verify automation output clean
- name: staging
triggers: [validate_pass]
actions:
- deploy_to_staging_index.json # Update staging index only
- run_integration_tests # Test with sample agent tasks
- measure_load_performance # Record latency metrics
- name: canary
triggers: [staging_pass, manual_approval]
actions:
- deploy_10_percent_traffic # Route 10% of requests to new skill set
- monitor_error_rate # Track failure rate vs baseline
- wait_for_stability_window # 30-minute stability observation
- name: production
triggers: [canary_stable, manual_approval]
actions:
- full_traffic_migration # Route 100% to new skill set
- update_skills-index.json # Publish to live index
- send_deployment_notification # Alert maintainers
Checkpoint: The pipeline must support rollback at every stage. A git revert on the production deployment stage should restore the previous skills-index.json and trigger a router reload within 5 minutes.
Set Up Monitoring and Observability — Define metrics to track skill health in production:
| Metric Category | Specific Metrics | Alert Threshold |
|---|---|---|
| Load Performance | Skill load latency (p50, p95, p99), total system load time | p95 > 2s triggers warning |
| Trigger Accuracy | False positive rate (skill loads when not needed), miss rate (needed skill doesn't load) | False positive > 15% or miss rate > 20% |
| Success Rate | % of times loaded skill produces useful output vs. user re-asking | Success rate < 85% |
| Coverage | % of agent tasks that successfully match at least one skill | Coverage < 70% |
| Quality Decay | Skill performance degradation over time (same query, worse output) | Output score drops > 10% week-over-week |
def compute_skill_health_score(
load_latency_p95: float, # seconds
trigger_accuracy: float, # 0.0 - 1.0
success_rate: float, # 0.0 - 1.0
coverage_pct: float, # 0.0 - 1.0
) -> dict:
"""Compute a composite health score for the skill ecosystem.
Uses weighted averaging with per-metric scoring curves.
Returns individual metric scores and overall rating.
"""
weights = {
"load_latency": 0.25, # Performance matters most
"trigger_accuracy": 0.30, # Accuracy is critical for UX
"success_rate": 0.25, # Output quality drives trust
"coverage": 0.20 # Coverage determines usefulness
}
scores = {}
# Load latency score: linear decay from 1.0 (under 1s) to 0.0 (over 5s)
if load_latency_p95 <= 1.0:
scores["load_latency"] = 1.0
elif load_latency_p95 >= 5.0:
scores["load_latency"] = 0.0
else:
scores["load_latency"] = 1.0 - (load_latency_p95 - 1.0) / 4.0
scores["trigger_accuracy"] = trigger_accuracy
scores["success_rate"] = success_rate
scores["coverage"] = coverage_pct / 100.0
overall = sum(weights[k] * scores[k] for k in weights)
if overall >= 0.9:
rating = "excellent"
elif overall >= 0.75:
rating = "good"
elif overall >= 0.60:
rating = "fair"
else:
rating = "needs_attention"
return {
"metric_scores": scores,
"weighted_score": round(overall, 3),
"rating": rating,
"recommendations": _generate_recommendations(scores)
}
Checkpoint: Monitoring must include a feedback loop — metrics that trigger an alert should automatically create a ticket assigned to the skill's maintainer with specific metric values and suggested investigation steps.
Design Fallback and Recovery Strategies — For every coordination pattern, define what happens during failures:
| Failure Mode | Immediate Action | Recovery Path |
|---|---|---|
| Skill fails to load | Skip to next skill in sequence; log error with context | Retry once after 5s; if still failing, alert maintainer |
| Low confidence match | Load top-3 ranked skills instead of single best | User can explicitly request a different skill via /skill |
| Timeout during execution | Kill the task; return partial results if available | Flag as degraded; skip this skill for 5 minutes (circuit breaker) |
| Trigger collision (multiple skills fire) | Use confidence scoring to pick highest; load secondary in background | Log collision event; review trigger overlap with skill-ecosystem-design |
| Router index corruption | Reload from last known good commit (git checkout HEAD~1) | Alert on-call; manual intervention within 15 minutes |
Checkpoint: Every fallback strategy must be tested under simulated failure conditions before deployment. Use chaos engineering principles — inject failures into staging to verify recovery procedures work as documented.
An orchestrator component manages skill loading decisions, coordinating multiple skills for complex tasks:
┌─────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ Agent │────▶│ Orchestrator │────▶│ Router Index │
│ Request │ │ (Coordinator) │ │ (Skill Catalog)│
└─────────────┘ └──────────────────┘ └─────────────────┘
│ │
┌──────▼─────────┐ ┌─────▼──────┐
│ Skill Loader │───────▶│ Matched │
│ Manager │ │ Skills (N) │
└───────┬────────┘ └────────────┘
│
┌────────▼────────┐
│ Execution │
│ Coordinator │
└────────┬────────┘
│
┌────────▼────────┐
│ Results │
│ Merger & │
│ Response Builder│
└─────────────────┘
The orchestrator pattern is essential when a single agent task requires multiple skills to produce a coherent response. It manages:
Implementation rule (reference code-philosophy): The orchestrator must follow Law 1 (Early Exit) — if any required skill in the chain fails to load, exit immediately rather than producing partial results. Follow Law 4 (Fail Fast) — log detailed context about which skill failed and why.
When deploying changes to a large skill set, use canary deployment to limit blast radius:
Deployment phases (canary strategy):
Phase 1: Isolated Staging → Skills deployed to staging index only
Phase 2: Internal Test → 5-10 trusted testers route through new skills
Phase 3: Canary 10% → 10% of production traffic hits new skill set
Phase 4: Monitor → Observe for 30 minutes (or 100 requests, whichever first)
Phase 5: Canary Expand → If stable, expand to 50% traffic
Phase 6: Full Production → 100% traffic; update skills-index.json permanently
Rollback trigger: Any phase with error rate > baseline by more than 5%
Checkpoint: After canary deployment but before full rollout, verify that the new skill set does not degrade performance of existing skills (cross-contamination check).
For high-criticality deployments (e.g., replacing core skills used in 80%+ of agent tasks), implement multi-round validation:
def run_multi_round_validation(
skill_path: str,
test_tasks: list[str],
rounds: int = 3
) -> dict:
"""Validate a skill across multiple rounds with increasing stringency.
Each round tests the same task set but with progressively stricter criteria.
Args:
skill_path: Path to the SKILL.md being validated.
test_tasks: List of real agent tasks used for testing.
rounds: Number of validation rounds (default 3).
Returns:
Validation results per round with pass/fail status and metrics.
"""
results = {
"skill_path": skill_path,
"rounds": [],
"overall_pass": True,
"pass_rate_per_round": []
}
for round_num in range(1, rounds + 1):
round_results = {
"round": round_num,
"tasks_tested": len(test_tasks),
"tasks_passed": 0,
"avg_confidence": 0.0,
"stricter_threshold": True if round_num > 1 else False
}
# Execute each test task through the skill pipeline
for task in test_tasks:
outcome = evaluate_skill_against_task(
skill_path, task,
stricter=round_num > 1
)
if outcome.passed:
round_results["tasks_passed"] += 1
pass_rate = round_results["tasks_passed"] / len(test_tasks)
round_results["pass_rate"] = pass_rate
round_results["threshold_met"] = (
pass_rate >= 0.95 if round_num > 1 else pass_rate >= 0.85
)
results["rounds"].append(round_results)
results["pass_rate_per_round"].append(pass_rate)
# Fail fast: if any round fails critically, stop early
if not round_results["threshold_met"]:
results["overall_pass"] = False
break
return results
def evaluate_skill_against_task(
skill_path: str,
task: str,
stricter: bool = False
) -> dict:
"""Evaluate a single skill against a test task.
Returns evaluation result with pass/fail and quality score.
"""
# This would integrate with the actual routing engine
# For architecture documentation, this shows the pattern
return {
"matched_skill": "skill-systems-architecture",
"confidence": 0.92,
"response_quality": "high" if not stricter else "medium",
"passed": True # Actual logic depends on evaluation criteria
}
code-philosophy (5 Laws of Elegant Defense) in orchestrator design, especially Law 4 (Fail Fast) for error handlingWhen applying this skill to design or audit a multi-skill system architecture, produce:
| Skill | Purpose |
|---|---|
skill-engineering | Creates individual SKILL.md files with content — this skill orchestrates how those skills are deployed at production scale |
skill-lifecycle-management | Manages versioning, deprecation, and retirement of individual skills — this skill handles the deployment pipeline that lifecycle changes flow through |
skill-ecosystem-design | Designs dependency graphs and reciprocal relationships between skills — this skill builds the operational architecture around those relationships |
skill-audit | Validates individual skill quality against standards — this skill defines the quality gates that audit results feed into for deployment decisions |
skill-router-system | Configures the runtime routing engine and confidence thresholds — this skill designs the broader system that the router operates within |
Authoritative documentation links for multi-skill agent systems architecture. The model follows markdown links at load time to resolve external references.