界面已本地化;技能介绍与说明文档按所选语言由 AI 自动翻译,代码、命令、路径和专业术语保留原文,以原作者文档为准。
探索技能
数据与研究 · SKILL.md

simpletransformers

simpletransformers

【数据与研究】Use Simple Transformers package workflows for NLP classification, NER, QA, generation, T5, Seq2Seq, retrieval, representations, data validation, and troubleshooting.

独立 GitHub 快照 · 自动分类 · 以原作者文档为准

技能说明

GitHub

name: simpletransformers description: "Use Simple Transformers package workflows for NLP classification, NER, QA, generation, T5, Seq2Seq, retrieval, representations, data validation, and troubleshooting." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Simple Transformers Repo Skill

Use this skill when the user asks about the simpletransformers Python package, Simple Transformers task wrappers, task-specific training/evaluation/prediction data formats, ClassificationModel, NERModel, QuestionAnsweringModel, LanguageModelingModel, LanguageGenerationModel, T5Model, Seq2SeqModel, RepresentationModel, RetrievalModel, ConvAIModel, or package-specific import/runtime errors.

Install and import baseline

Simple Transformers wraps Hugging Face Transformers and uses PyTorch in model modules. Install a compatible PyTorch build before importing task model classes:

pip install torch        # choose CPU or CUDA wheel for the target machine
pip install simpletransformers
python -c "import simpletransformers; print(simpletransformers.name)"

For smoke runs, pass use_cuda=False to model constructors unless the user has already verified a CUDA-capable PyTorch environment.

Route by task

User taskRead
binary/multiclass/regression/sentence-pair/multilabel/LayoutLM/multimodal classification, cross-encoder rerankingclassification
named entity recognition, token classification, CoNLL, LayoutLM token boxes, extractive QA, SQuAD JSON/JSONLtoken-and-qa
language model training/fine-tuning, generation, T5, Seq2Seq, ConvAI, text-to-text tasksgenerative-workflows
sentence/word representations, dense retrieval, DPR, hard negatives, BEIR/MSMARCO/TREC, FAISS/pytrecretrieval-representation

Shared references

  • Package overview maps public tasks to model classes and notes install/backend constraints.
  • Configuration explains shared ModelArgs/task args and safe smoke-run defaults.
  • CLI and viewer explains the simple-viewer Streamlit launcher and why not to run it in automated checks.
  • Troubleshooting covers import/version, PyTorch, CUDA, downloads, outputs/cache, multiprocessing, and optional dependencies.
  • Repository provenance records the source snapshot and refresh baseline.

Bundled helper scripts

  • scripts/check_simpletransformers_env.py: run to check package versions, task-module imports, and optional CUDA visibility.
  • scripts/inspect_model_args.py: run in an environment where Simple Transformers imports cleanly to list args dataclass fields.
  • Sub-skill validators under sub-skills/*/scripts/: use these before model construction to catch data/schema mistakes without downloads or training.

Compatibility warning

Simple Transformers 0.70.8 declares transformers>=4.31.0 but package inspection found runtime hazards with modern Transformers versions: removed SequenceSummary aliases, TransfoXLConfig, and top-level cached_path can break imports before any dataset is read. When import errors mention these symbols, debug dependency compatibility first and use the troubleshooting references; do not rewrite user data prematurely.

Safe operating defaults

  1. Validate data with the nearest bundled validator.
  2. Use CPU (use_cuda=False) and no-save settings for smoke/debug runs.
  3. Avoid full examples/tests unless model downloads, data downloads, training time, and compute budget are approved.
  4. Choose explicit output_dir, cache_dir, and checkpoint policy before production runs.
  5. Install optional dependencies such as ONNX, FAISS, BEIR, pytrec, Streamlit server usage, or WandB only for workflows that need them.
  6. If the current repository/package has changed from provenance, refresh this skill before relying on exact API/version claims.

Non-goals

  • Raw Hugging Face Transformers Trainer/model/tokenizer guidance when the user is not using Simple Transformers.
  • Generic vector database or RAG framework workflows outside Simple Transformers retrieval APIs.
  • Long-running training, benchmark execution, dataset downloads, or Streamlit server launches as default verification steps.
我的收藏