L’interface est localisée ; les présentations et documents des Skills sont traduits automatiquement par IA dans votre langue. Le code, les commandes, chemins et termes techniques restent en version originale, et le document de l’auteur fait foi.
Explorer les Skills
Données et recherche · SKILL.md

jionlp

jionlp

Chinese NLP preprocessing, parsing, augmentation, and language-analysis router for JioNLP.

Independent GitHub snapshot · Automated classification · Original documentation governs

Documentation de la Skill

GitHub

name: jionlp description: "Chinese NLP preprocessing, parsing, augmentation, and language-analysis router for JioNLP." metadata: disco-role: operating disable-model-invocation: true license: Apache 2.0

JioNLP

Use this skill when the task is about JioNLP's Chinese text preprocessing, rule extraction, parsing, augmentation, annotation helpers, or bundled language-analysis utilities.

Quick start

  • Install: pip install jionlp
  • Import: import jionlp as jio
  • Sanity check: print(jio.__version__)
  • Search APIs interactively: jio.help() or jio_help
  • For a silent smoke check, run scripts/smoke_core.py

Importing jionlp prints a banner and loads jiojio's C helpers when available. That is expected.

Route map

User intentUse this sub-skill
Clean raw text, strip URLs/emails/phones, normalize full-width chars, clean HTML, split sentences, read/write line files, or search docstringssub-skills/text-cleaning-and-extraction/
Parse or normalize time, money, location, phone, ID, vehicle plate, pinyin, radicals, idioms, or lunar/solar datessub-skills/parsing-and-normalization/
Augment text with back translation, character swaps, homophone substitution, random add/delete, or entity replacementsub-skills/text-augmentation/
Convert NER/CWS/POS tags, build lexicon NER, split annotated datasets, or batch long NER inputssub-skills/annotation-and-dataset-tools/
Load packaged dictionaries, extract keyphrases or summaries, score sentiment, find new words, use BPE, load the LLM test set, or work with MELLMsub-skills/dictionaries-and-language-analysis/

How to route quickly

  • If the request starts with a noisy paragraph and asks to remove or detect obvious patterns, start with text-cleaning-and-extraction.
  • If the request asks what a time string, money string, address, phone number, ID card, or plate means, start with parsing-and-normalization.
  • If the request mentions augmentation, back translation, or replacing entities, start with text-augmentation.
  • If the request mentions BIO/BIOES tags, entity offsets, CWS/POS conversion, or dataset splits, start with annotation-and-dataset-tools.
  • If the request mentions loaders, keyphrase, summary, sentiment, new words, BPE, LLM test data, or MELLM, start with dictionaries-and-language-analysis.

Practical notes

  • jio.help() is interactive. Use scripts/search_api_docs.py if you want a noninteractive keyword search.
  • jio.bpe.byte_level_bpe lives under the bpe submodule, not as a root attribute.
  • Keyphrase, summary, new-word, NER time extraction, and text-classification helpers are exposed through submodules such as jio.keyphrase, jio.summary, jio.new_word, jio.ner, and jio.text_classification.
  • A NumPy 1.x release older than 1.24 is required for the CWS/POS converters because this repository uses np.unicode.
  • phone_location() is safer on text that contains a leading non-digit boundary before the phone number.
  • llm_test_dataset_loader() expects version strings such as '1.0' or '1.1'.

Read these first when needed

  • references/cli-reference.md for jio_help and the bundled search wrapper.
  • references/troubleshooting.md for import, NumPy, loader, and external-data issues.
  • scripts/smoke_core.py for a fast import and sample-API smoke check.
Skills enregistrées