---

name: checks-evals description: "Build and run Giskard checks, scenarios, suites, judges, generators, custom checks, serialization, and JUnit export." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

checks-evals

Use this sub-skill for the giskard.checks package: evaluation scenarios, suites, deterministic checks, LLM-as-judge checks, input generators, JSONPath extraction, custom check registration, serialization, and JUnit export.

Route here when

• Building Scenario(...).interact(...).check(...) flows or running Suite.run(...).
• Choosing deterministic checks such as Equals, comparison checks, StringMatching, RegexMatching, JsonValid, AllOf, AnyOf, Not, or one-off from_fn / FnCheck checks.
• Using optional or model-backed checks such as SemanticSimilarity, Readability, RegoPolicy, Groundedness, Conformity, Contradiction, AnswerRelevance, Toxicity, or LLMJudge.
• Generating scenario inputs with LLMGenerator, UserSimulator, or DatasetInputGenerator.
• Debugging Trace / Interaction data and JSONPath selectors such as trace.last.inputs, trace.last.outputs, and trace.last.metadata.context.
• Defining reusable custom checks with @Check.register("..."), JSONPathStr, and MISSING defaults, then round-tripping with model_validate(...).
• Exporting suite results as JUnit XML.

Route elsewhere

• Provider aliases, provider SDK extras, live completion/embedding credentials, and giskard.llm client behavior: use ../llm-providers/SKILL.md.
• Agent workflow construction, prompt templates, tools, and Generator internals: use ../agents-workflows/SKILL.md.
• Vulnerability scans, RAG quality scans, generate_suite, or scan generator registries: use ../scan-redteam/SKILL.md.
• Installation, split-package imports, telemetry controls, and cross-package runtime setup: use ../runtime-setup/SKILL.md.

Operating checklist

1. Import public symbols from giskard.checks; do not use a giskard_checks import namespace.
2. Start with deterministic checks when provider keys or a default generator are unavailable.
3. Bind target output deliberately: pass outputs= to interact(...), or bind a target on the Scenario, Suite, or run(...) call.
4. Prefer trace.last.* JSONPath selectors for current-turn data; use explicit structured field paths when outputs are dictionaries or models.
5. Pass an explicit generator= or configure set_default_generator(...) before running LLM judges or LLM-backed input generators; configure an embedding model before provider-backed SemanticSimilarity runs.
6. Import every custom registered class before model_validate(...) of a serialized scenario, suite, test case, check, interaction spec, or input generator.
7. Use the bundled smoke script for a no-key installed-package sanity check.

References and Script

• API reference for public APIs, defaults, status fields, and result/export surfaces.
• Workflows for Scenario, Suite, judge, generator, serialization, and JUnit recipes.
• Custom checks for custom check authoring, JSONPath field typing, registration, and tests.
• Troubleshooting for failure modes and diagnostics.
• scripts/run_checks_smoke.py for a deterministic installed-package smoke that sets telemetry opt-out before import, makes no network/API calls, and exits nonzero if Scenario, StringMatching, Equals, or Suite.run behavior is broken.

Run the smoke from any current working directory with a Python environment that has giskard.checks installed:

python path/to/sub-skills/checks-evals/scripts/run_checks_smoke.py