---

name: similarity-search description: "Score text pairs and run dense or BM25 retrieval with text2vec." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Similarity Search

Use this sub-skill when you need sentence similarity, pair scoring, semantic search, or lexical BM25 retrieval in text2vec.

Route here for

• Pairwise similarity with Similarity, SimilarityType, EmbeddingType, get_score, get_scores, and similarity.
• Dense retrieval helpers with cos_sim and semantic_search for tensor or NumPy embeddings.
• Raw-text BM25 retrieval with Jieba segmentation and top-k ranking.
• Aligned pair scoring from JSONL, CSV, or TSV without accidentally building a full cross-product matrix.
• Query-to-corpus retrieval workflows where you want either no-network BM25 now or dense search from cached embeddings later.

Do not handle here

• Embedding initialization, batch vector generation, or CLI embedding export: route to embeddings.
• Fine-tuning, training data validation, and negative mining: route to training-finetuning.
• Benchmark-style model quality evaluation and leaderboard comparison: route to evaluation-benchmarks.
• HTTP serving, FastAPI, Jina, or Gradio deployment: route to serving-deployment.

Operating workflow

1. Decide whether the task is pair scoring or retrieval.
2. For pair scoring, choose vector-only scoring or model-backed scoring.
3. For retrieval, choose BM25 when you want a no-network lexical path, or dense search when you already have embeddings or a cached model.
4. Read the API shapes in references/api-reference.md.
5. Use the bundled helpers in scripts/ for safe JSONL/CSV/TSV workflows.
6. Check references/troubleshooting.md if scores, shapes, or downloads look wrong.

Bundled scripts

• scripts/score_pairs.py: score aligned text pairs from JSONL, CSV, or TSV using supplied vectors or Similarity.
• scripts/search_corpus.py: search a corpus with BM25 or dense embeddings and return top-k hits.

Start with references/workflows.md for ready-to-run patterns and references/api-reference.md for return shapes.