agent-evaluation-reporting
agent-evaluation-reporting
【效率与协作】Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.
THE SKILLS DIRECTORY
探索全部技能.
一个技能，一个新的可能。找到适合你的 AI 工作流。
ai-prompt-regression-testing
ai-prompt-regression-testing
【数据与研究】Prompt engineering regression test matrix register: baseline outputs, semantic drift thresholds, judge evaluations, and golden dataset.
aws-agentic-ai
aws-agentic-ai
【部署与运维】AWS Bedrock AgentCore comprehensive expert for deploying and managing AI agents at scale. Use when working with any AgentCore service including Gateway, Runtime, Memory, Identity, Code Interpreter, Browser, Observability, Agent Registry, or Evaluations.
azure-ai-projects-dotnet
azure-ai-projects-dotnet
【部署与运维】Azure AI Projects SDK for .NET. High-level client for Azure AI Foundry projects including agents, connections, datasets, deployments, evaluations, and indexes.
azure-ai-projects-java
azure-ai-projects-java
【数据与研究】Azure AI Projects SDK for Java. High-level SDK for Azure AI Foundry project management including connections, datasets, indexes, and evaluations.
azure-ai-projects-ts
azure-ai-projects-ts
【部署与运维】High-level SDK for Azure AI Foundry projects with agents, connections, deployments, and evaluations.
hugging-face-community-evals
hugging-face-community-evals
【效率与协作】Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate.
hugging-face-evaluation
hugging-face-evaluation
【数据与研究】Add and manage evaluation results in Hugging Face model cards. Supports extracting eval tables from README content, importing scores from Artificial Analysis API, and running custom model evaluations with vLLM/lighteval. Works with the model-index metadata format.
hugging-face-evaluation
hugging-face-evaluation
【数据与研究】Add and manage evaluation results in Hugging Face model cards. Supports extracting eval tables from README content, importing scores from Artificial Analysis API, and running custom model evaluations with vLLM/lighteval. Works with the model-index metadata format.
run-deep-swe
run-deep-swe
【效率与协作】Run reproducible DeepSWE coding-agent benchmark evaluations through OpenRouter and mini-swe-agent.
analyzing-offline-evaluations
analyzing-offline-evaluations
【数据与研究】Inspect and compare offline AI evaluation experiments, diagnose case-level regressions, and follow scorer history across application, model, or prompt changes. Use when a user asks whether an offline run improved, why a case failed, how scores changed between runs, or how to publish externally computed offline results. Covers immutable scorer version…
exploring-llm-evaluations
exploring-llm-evaluations
【开发与工程】Investigate AI observability evaluations — `hog` (deterministic code-based), `llm_judge` (LLM-prompt-based), and `sentiment` (user-message sentiment). Find existing evaluations, inspect their configuration, run them against specific generations, query individual results, and set up scheduled reports on an evaluation. Use when the user asks to debug w…
arize-evaluator
arize-evaluator
【效率与协作】Handles LLM-as-judge evaluation workflows on Arize including creating/updating evaluators, running evaluations on spans or experiments, managing tasks, trigger-run operations, column mapping, and continuous monitoring. Use when the user mentions create evaluator, LLM judge, hallucination, faithfulness, correctness, relevance, run eval, score spans, s…
arize-prompt-optimization
arize-prompt-optimization
【数据与研究】Optimizes, improves, and debugs LLM prompts using production trace data, evaluations, and annotations. Extracts prompts from spans, gathers performance signal, and runs a data-driven optimization loop using the ax CLI. Use when the user mentions optimize prompt, improve prompt, make AI respond better, improve output quality, prompt engineering, promp…
evaluating-cosmos-policy
evaluating-cosmos-policy
【效率与协作】Evaluates NVIDIA Cosmos Policy on LIBERO and RoboCasa simulation environments. Use when setting up cosmos-policy for robot manipulation evaluation, running headless GPU evaluations with EGL rendering, or profiling inference latency on cluster or local GPU machines.
phoenix-observability
phoenix-observability
【数据与研究】Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.
huggingface-community-evals
huggingface-community-evals
【效率与协作】Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals automation.
azure-ai-projects-dotnet
azure-ai-projects-dotnet
【部署与运维】Azure AI Projects SDK for .NET. High-level client for Azure AI Foundry projects including agents, connections, datasets, deployments, evaluations, and indexes. Use for AI Foundry project management, versioned agents, and orchestration. Triggers: "AI Projects", "AIProjectClient", "Foundry project", "versioned agents", "evaluations", "datasets", "conne…
18 / 204