---

name: "project-workflows" description: "Routes scispaCy data conversion, evaluation, UMLS export, and project.yml-driven packaging workflows." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Project Workflows

Use this sub-skill when the task is about scispaCy's data readers, evaluation helpers, or project-yml/config-driven workflow assembly rather than the core pipeline or linker APIs.

Typical triggers

• Convert word-frequency files into spaCy vocabulary JSONL.
• Count token frequencies from a raw corpus.
• Evaluate NER predictions on MedMentions or TSV data.
• Export UMLS META files to JSONL.
• Summarize packaged model metrics.
• Understand or tweak project.yml, configs/*.cfg, base_project_code.py, or the custom spaCy registries.

What belongs here

• Frequency counting and vocabulary conversion helpers.
• MedMentions and BIO TSV readers.
• NER evaluation and per-class scoring.
• UMLS export and package-metrics reporting.
• Config-driven workflow names, reader callbacks, and spaCy registry entries.

What does not belong here

• Tokenization, sentence segmentation, abbreviation handling, or hyponym detection. Use pipeline-components.
• ANN candidate generation or KB construction. Use entity-linking.
• General package installation/model catalog questions. Read the root installation reference.

First things to read or run

• references/api-reference.md for registered readers/callbacks and scoring helpers.
• references/data-formats.md for MedMentions, BIO TSV, frequency-file, and UMLS META layouts.
• references/workflows.md for the command-level workflows.
• references/troubleshooting.md for malformed data, stale helpers, and evaluation issues.
• scripts/convert_freqs.py, scripts/count_word_frequencies.py, scripts/evaluate_ner.py, scripts/export_umls_json.py, and scripts/print_out_metrics.py for reusable helpers.

Fast workflow summary

1. Identify the input format: raw corpus text, BIO TSV, MedMentions, UMLS META, or package metrics JSON.
2. Choose the bundled helper that matches the format.
3. Keep large downloads and long training runs explicit; these workflows are often maintenance-oriented and may require external data.
4. Use the distilled workflow matrix in references/workflows.md when you need to understand how the package builds the small, medium, large, scibert, or specialized NER workflows.

Example route decisions

• If the request says "convert frequency counts into spaCy vocab JSONL", stay here.
• If the request says "evaluate a model on MedMentions", stay here.
• If the request says "change how the tokenizer splits biomedical abbreviations", switch to pipeline-components.
• If the request says "build a custom linker from a KB", switch to entity-linking.

Verification mindset

For this sub-skill, the strongest validation is a safe, fixture-backed workflow:

• convert_freqs.py can transform a tiny frequency file.
• count_word_frequencies.py can process a small raw-corpus directory.
• read_full_med_mentions and read_ner_from_tsv parse the expected layouts.
• evaluate_ner and PerClassScorer compute the expected metrics on a tiny fixture.

If one of those checks fails, read the troubleshooting reference before launching a full corpus conversion or a long evaluation run.