---

name: evaluation-benchmarks description: "Model selection, benchmark interpretation, Spearman/Pearson/QPS analysis, and bounded evaluation planning for text2vec." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

evaluation-benchmarks

Use this sub-skill when you need to choose among text2vec release models, interpret benchmark metrics, summarize existing pair-score files, or plan bounded MTEB / C-MTEB runs.

Use for

• choosing Chinese CoSENT defaults, multilingual, BGE, or Word2Vec fallback;
• reading Spearman, Pearson, and QPS results;
• summarizing local label/score files;
• planning benchmark runs with explicit download and skip conditions.

Route elsewhere

• Training command details or fine-tuning recipes -> training-finetuning
• Embedding generation / batching / CLI vectorization -> embeddings
• Pairwise search / retrieval implementation -> similarity-search
• Serving or deployment wrappers -> serving-deployment

Quick model choice

• Chinese question-question matching: shibing624/text2vec-base-chinese
• Chinese sentence-to-sentence or short semantic matching: shibing624/text2vec-base-chinese-sentence
• Chinese sentence-to-paraphrase / longer text matching: shibing624/text2vec-base-chinese-paraphrase
• Multilingual Chinese-English matching: shibing624/text2vec-base-multilingual
• Stronger short-text discrimination, slower throughput: shibing624/text2vec-bge-large-chinese
• Cold-start or lexical fallback on CPU: w2v-light-tencent-chinese

For large-scale retrieval/search or dedup, hand off to the separate similarities package after choosing a model.

Bundled references

• references/model-overview.md
• references/evaluation-workflows.md
• references/troubleshooting.md

Bundled script

• scripts/summarize_scores.py — summarize CSV/JSONL label/score files into a JSON report.

Example:

python scripts/summarize_scores.py \
  --input-file scores.csv \
  --label-column label \
  --score-column score \
  --output-file summary.json

Guardrails

• Use the source tables and report findings only; do not claim a universal leaderboard winner.
• Treat QPS as hardware-specific throughput, not a portable ranking.
• If benchmark data or model files are unavailable, prefer a bounded summary or mark the run blocked instead of guessing.