---

name: prompt-automation description: "Configure YiVal data generators and prompt variation generators for synthetic examples, document questions, chain-of-density prompts, and self-exemplar prompts." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

YiVal prompt automation

Use this sub-skill when the task involves generating input examples, generating prompt variations, caching generated prompts/data, using yival gen/yival task, or wiring OpenAI/document/chain-of-density/self-exemplar generators into a YiVal config.

Read first

• Generators reference: built-in data and variation generators, config fields, and registry ids.
• Prompt workflows: common auto-prompt experiment patterns.
• Provider and cache notes: credentials, output paths, rate limits, and offline alternatives.
• Prompt automation troubleshooting: generation-specific failures.

Useful helper:

• python sub-skills/prompt-automation/scripts/inspect_prompt_components.py prints built-in generator ids and default configs after importing their modules.

Built-in generator map

Need	YAML location	Registry id
Generate synthetic input examples from a function/task description	dataset.data_generators	openai_prompt_data_generator
Generate questions from text/file/Google Drive document content	dataset.data_generators	document_data_generator
Generate optimized prompt strings with OpenAI	variations[].generator_name	openai_prompt_based_variation_generator
Use the fixed chain-of-density summarization prompt	variations[].generator_name	chain_of_density_prompt_generator
Use a self-exemplar prompt template	variations[].generator_name	self_exemplar

Minimal auto-prompt shape

description: Auto prompt experiment
custom_function: my_task.headline
dataset:
  source_type: machine_generated
  data_generators:
    openai_prompt_data_generator:
      number_of_examples: 2
      chunk_size: 1000
      model_name: gpt-4
      output_path: generated_examples.pkl
      input_function:
        name: headline_generation
        description: Generate a landing page headline for a tech startup business.
        parameters:
          tech_startup_business: str
variations:
  - name: task
    generator_name: openai_prompt_based_variation_generator
    generator_config:
      number_of_variations: 2
      model_name: gpt-4
      output_path: generated_prompts.pkl
      variables:
        - tech_startup_business
      prompt:
        - role: system
          content: Write concise prompts for GPT-4.
    variations:
      - value_type: str
        value: Generate landing page headline for {tech_startup_business}
        instantiated_value: Generate landing page headline for {tech_startup_business}
        variation_id: null

The custom function should use StringWrapper(name="task", variables={"tech_startup_business": tech_startup_business}, state=state) so generated/manual prompt values are applied.

Workflow guidance

1. Start with a tiny run: one or two generated examples and one or two variations.
2. Always set output_path for provider-backed generators so reruns load cached pickles instead of paying/calling again.
3. Keep variables explicit for prompt variation generation; generated prompt outputs must contain {variable} placeholders when configured.
4. For document generation, start with source: text before file or Google Drive sources.
5. Route generated outputs into evaluation-optimization for evaluator, selector, and enhancer design.

Safety and cost boundaries

• OpenAI generators call network services and can bill. Confirm credentials and budget before running.
• Document file/drive sources can invoke loaders, OCR, Google APIs, and local file reads. Use only approved documents.
• Retrieval demos that use embeddings/FAISS/OpenAI are reference patterns, not default runtime paths.
• If the user only needs a static prompt list, use manual variations instead of provider-backed generation.