---

name: install-and-runtime description: "Install Pyserini, select Python/Java/Torch/Faiss/runtime options, and diagnose import, JVM, and optional dependency failures." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Pyserini Install And Runtime

When To Use

Use this sub-skill when the task is about installing Pyserini, preparing a safe runtime environment, or diagnosing failures before an indexing, search, dense retrieval, evaluation, REST, or MCP workflow can start.

Natural triggers include install Pyserini, Python version errors, Java/JVM errors, pyjnius failures, No matching jar file found, missing faiss, Torch/CUDA mismatch, source checkout setup, pip check conflicts, or REST/MCP imports failing before a server starts.

Quick Decisions

• Prefer Python 3.12 and Java 21. Pyserini 2.3.0 requires Python >=3.12, but the project is built and documented around Python 3.12 with Java 21.
• Use the PyPI package for normal usage; use an editable source checkout only when the user needs unreleased changes or development.
• Install Torch deliberately: CPU wheels are sufficient for CPU-only sparse search, Lucene dense checks, and many setup tasks; CUDA wheels are only needed when the user will run GPU encoders or GPU Faiss.
• Install Faiss separately only when needed. Pyserini metadata does not include Faiss because users must choose faiss-cpu, a conda Faiss package, or a GPU build that matches their platform.
• Do not run large prebuilt-index, model-download, or benchmark commands as installation checks unless the user explicitly asks for functional retrieval verification.

Safe Install Recipes

Minimal PyPI Environment

python3.12 -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install pyserini==2.3.0
python -m pip check

If the user wants a current PyPI release instead of the repo-matched version, omit the version pin after confirming reproducibility is not required.

Conda-Friendly Binary Stack

conda create -n pyserini python=3.12 -y
conda activate pyserini
conda install -c conda-forge openjdk=21 -y
python -m pip install --upgrade pip
python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
python -m pip install pyserini==2.3.0

For CUDA, replace the CPU Torch command with the PyTorch command matching the machine's CUDA runtime. Do not install CUDA wheels just because Pyserini imports; install them only for GPU encoding/search tasks.

Faiss When Needed

python -m pip install faiss-cpu
python -m pip check

If faiss-cpu wheels are unavailable or the user needs GPU Faiss, prefer a conda/mamba environment with a platform-compatible Faiss package. After installing Faiss, run the bundled checker with --check-faiss.

Optional Multimodal Extra

python -m pip install 'pyserini[optional]'

Use the optional extra only when the user asks for the optional multimodal/UniIR paths. Core Lucene search, most dense setup, evaluation, REST, and MCP setup do not require the extra.

Editable Source Checkout

python3.12 -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e .
python -m pip check

A source checkout can import the Python package while still failing Lucene-backed imports if package resources are missing. Source-development setup may also require the evaluation tools submodule and an Anserini fatjar; see ../repo-development/SKILL.md for maintainer build/test workflows.

Verify Before Workflow Handoff

Run the bundled checker from this sub-skill directory or copy it into the target environment:

python scripts/check_pyserini_runtime.py --help
python scripts/check_pyserini_runtime.py
python scripts/check_pyserini_runtime.py --check-lucene
python scripts/check_pyserini_runtime.py --check-faiss --check-server

Interpretation:

• Default checks confirm Python version, Java version, package metadata, importability, PyJNIus configuration, and core neural dependencies without starting retrieval jobs.
• --check-lucene starts the Java-backed Lucene import path and catches missing Java, PyJNIus, and Anserini fatjar problems.
• --check-faiss verifies the optional Faiss Python package and Pyserini Faiss search import path.
• --check-server verifies REST/MCP import dependencies without binding ports or opening indexes.

Use references/runtime-requirements.md for dependency choices and source checkout caveats. Use references/troubleshooting.md for symptom-to-recovery guidance.

Route After Runtime Is Healthy

• Lucene indexing, sparse search, fetching, analyzers, query builders, and index readers: ../index-search-fetch/SKILL.md.
• Dense encoders, Faiss search, GPU/device choices inside retrieval commands, OpenAI/Hugging Face model behavior, and hybrid search: ../dense-encoding/SKILL.md.
• Run evaluation, qrels, TREC/MS MARCO/KILT formats, fusion, and reproduction matrices: ../evaluation-and-fusion/SKILL.md.
• REST API, MCP server, API keys, cache/load shedding, and server config aliases: ../serving-and-agent-tools/SKILL.md.
• Source checkout maintenance, submodules, Anserini/eval tool builds, and test selection: ../repo-development/SKILL.md.