技能创建器
skill-creator
创建、改进和评估自定义技能，把专业流程转化为可重复使用的 AI 能力。
探索技能
数据与研究 · SKILL.md
datagen-evaluation-and-benchmarks
datagen-evaluation-and-benchmarks【数据与研究】Use CAMEL-AI synthetic data generation, data collectors, datasets, verifiers, environments, and benchmark wrappers while separating deterministic checks from model-, dataset-, and credential-heavy evaluation runs.
独立 GitHub 快照 · 自动分类 · 以原作者文档为准
技能说明
GitHubname: datagen-evaluation-and-benchmarks description: "Use CAMEL-AI synthetic data generation, data collectors, datasets, verifiers, environments, and benchmark wrappers while separating deterministic checks from model-, dataset-, and credential-heavy evaluation runs." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0
CAMEL Datagen, Evaluation, and Benchmarks
Use this sub-skill when a task involves CAMEL synthetic data generation, dataset wrappers, data collectors, verifier-backed evaluation, environment rollouts, or benchmark wrappers.
Route By Task
- For API signatures, object contracts, and safe local probes, read references/api-reference.md and run scripts/inspect_eval_components.py.
- For CoT, self-instruct, evol-instruct, Source2Synth, collectors, datasets, and verifier-backed task loops, read references/workflows.md.
- For GAIA, APIBank, APIBench, Nexus, BrowseComp, and RAGBench evaluation planning, read references/benchmarks.md.
- For optional dependencies, credentials, schema failures, unsafe code execution, environment contracts, and long-running generation recovery, read references/troubleshooting.md.
Boundaries
- Covers
camel.datagen,camel.data_collectors,camel.datasets,camel.benchmarks,camel.verifiers,camel.extractors,camel.environments, and task-generation/evaluation examples. - Cross-link to sibling model/agent skills for
ChatAgent,RolePlaying,Workforce, model backend, and provider configuration details. - Cross-link to sibling memory/RAG/storage skills for vector stores, retrievers, embeddings, and persistent storage details.
- Treat source repo examples and benchmarks as evidence; do not require the original checkout at runtime.
Safety Defaults
- Prefer no-network local validation first: import checks, extractor normalization, small schema validation, tiny static datasets, and environment reset/step smoke tests.
- Treat data generation and benchmark
run()calls as potentially expensive because they usually call models, download datasets, or require API keys. - Keep generated outputs resumable: write JSON/JSONL checkpoints, include deterministic seeds, track processed input IDs, and rerun only failed records.