Die Oberfläche ist lokalisiert; Skill-Beschreibungen und -Dokumente werden per KI automatisch in Ihre Sprache übersetzt. Code, Befehle, Pfade und Fachbegriffe bleiben im Original, maßgeblich ist das Dokument des Autors.
Skills entdecken
Daten und Forschung · SKILL.md

jionlp

jionlp

Chinese NLP preprocessing, parsing, augmentation, and language-analysis router for JioNLP.

Independent GitHub snapshot · Automated classification · Original documentation governs

Skill-Dokumentation

GitHub

name: jionlp description: "Chinese NLP preprocessing, parsing, augmentation, and language-analysis router for JioNLP." metadata: disco-role: operating disable-model-invocation: true license: Apache 2.0

JioNLP

Use this skill when the task is about JioNLP's Chinese text preprocessing, rule extraction, parsing, augmentation, annotation helpers, or bundled language-analysis utilities.

Quick start

  • Install: pip install jionlp
  • Import: import jionlp as jio
  • Sanity check: print(jio.__version__)
  • Search APIs interactively: jio.help() or jio_help
  • For a silent smoke check, run scripts/smoke_core.py

Importing jionlp prints a banner and loads jiojio's C helpers when available. That is expected.

Route map

User intentUse this sub-skill
Clean raw text, strip URLs/emails/phones, normalize full-width chars, clean HTML, split sentences, read/write line files, or search docstringssub-skills/text-cleaning-and-extraction/
Parse or normalize time, money, location, phone, ID, vehicle plate, pinyin, radicals, idioms, or lunar/solar datessub-skills/parsing-and-normalization/
Augment text with back translation, character swaps, homophone substitution, random add/delete, or entity replacementsub-skills/text-augmentation/
Convert NER/CWS/POS tags, build lexicon NER, split annotated datasets, or batch long NER inputssub-skills/annotation-and-dataset-tools/
Load packaged dictionaries, extract keyphrases or summaries, score sentiment, find new words, use BPE, load the LLM test set, or work with MELLMsub-skills/dictionaries-and-language-analysis/

How to route quickly

  • If the request starts with a noisy paragraph and asks to remove or detect obvious patterns, start with text-cleaning-and-extraction.
  • If the request asks what a time string, money string, address, phone number, ID card, or plate means, start with parsing-and-normalization.
  • If the request mentions augmentation, back translation, or replacing entities, start with text-augmentation.
  • If the request mentions BIO/BIOES tags, entity offsets, CWS/POS conversion, or dataset splits, start with annotation-and-dataset-tools.
  • If the request mentions loaders, keyphrase, summary, sentiment, new words, BPE, LLM test data, or MELLM, start with dictionaries-and-language-analysis.

Practical notes

  • jio.help() is interactive. Use scripts/search_api_docs.py if you want a noninteractive keyword search.
  • jio.bpe.byte_level_bpe lives under the bpe submodule, not as a root attribute.
  • Keyphrase, summary, new-word, NER time extraction, and text-classification helpers are exposed through submodules such as jio.keyphrase, jio.summary, jio.new_word, jio.ner, and jio.text_classification.
  • A NumPy 1.x release older than 1.24 is required for the CWS/POS converters because this repository uses np.unicode.
  • phone_location() is safer on text that contains a leading non-digit boundary before the phone number.
  • llm_test_dataset_loader() expects version strings such as '1.0' or '1.1'.

Read these first when needed

  • references/cli-reference.md for jio_help and the bundled search wrapper.
  • references/troubleshooting.md for import, NumPy, loader, and external-data issues.
  • scripts/smoke_core.py for a fast import and sample-API smoke check.
Gespeicherte Skills