---

name: data-preparation description: "Prepare DeepKE data for supervised formats, NER weak supervision, and RE distant supervision." disable-model-invocation: true metadata: disco-role: operating license: MIT

DeepKE data preparation

Use this sub-skill when a task asks to prepare or validate data before DeepKE supervised NER, RE, or AE training/prediction. It covers manual annotation schemas, doccano-style exports, conversion into DeepKE text/CSV formats, NER weak supervision from dictionaries, RE distant supervision from triples, train/dev/test split expectations, and data-prep failure diagnosis.

Route by task

• Manual annotation or doccano export planning: read references/data-formats.md for the span/relation fields DeepKE expects and references/workflows.md for when to annotate manually instead of auto-labeling.
• Convert labeled JSON/DOCX/XLSX into DeepKE-ready files: use scripts/convert_supervised_data.py. See references/data-formats.md for accepted schemas and references/troubleshooting.md before assuming offsets or labels are valid.
• Prepare weakly supervised NER data from an entity dictionary: use scripts/prepare_weaksupervised_data.py. See references/workflows.md for dictionary-matching trade-offs and split behavior.
• Label RE examples from a triple table: use scripts/ds_label_data.py. See references/data-formats.md for source and triple schemas.
• Debug bad or empty training data: start with references/troubleshooting.md, then rerun the relevant bundled script with a tiny sample and stricter validation flags.

What this sub-skill owns

• DeepKE standard NER txt, json, and docx preparation.
• DeepKE standard RE and AE-style csv, json, and xlsx conversion where the first row/object keys define the columns.
• Doccano sequence-labeling exports for NER and relation-labeling exports for RE schema planning.
• Dictionary-based NER weak supervision into BIO-tagged train/dev/test text files.
• Distant-supervision RE relation assignment from (head, tail, relation) triples.
• Validation gotchas that commonly produce unusable DeepKE datasets.

What this sub-skill does not own

• Model architecture selection, Hydra config tuning, or running DeepKE training; route those to the supervised-extraction sub-skill.
• LLM instruction-data conversion; route LLM-specific data shaping to the llm-workflows sub-skill.
• Triple-extraction model output post-processing; route that to the triple-extraction sub-skill.
• Downloading large corpora, credentials, model checkpoints, or remote annotation services.

Recommended order

1. Choose the workflow in references/workflows.md.
2. Check the target schema in references/data-formats.md.
3. Run the smallest applicable bundled script command first.
4. Inspect split sizes, labels, offsets, and a few rendered BIO rows before using the data in training.
5. Use references/troubleshooting.md for nonzero script failures or suspicious outputs.