---

name: indexing description: "Build, inspect, and export bounded paperai indexes from a paperetl-style SQLite corpus with txtai-compatible vector configuration." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Indexing

Use this route when the task is to prepare or inspect the searchable model behind paperai: validate a corpus, choose a txtai/vector configuration, build an embeddings index, train optional static word vectors, or export raw section text. Start with the no-download checker:

python scripts/inspect_corpus.py ./corpus --config ./index.yml --maxsize 1000 --toprank 0

The checker is bundled at scripts/inspect_corpus.py. Run that command from this sub-skill directory, or replace scripts/ with the path where this runtime skill is installed.

The ./corpus argument is a user-owned directory containing the source artifact articles.sqlite; inspect_corpus.py is only a bundled validation helper and is not a replacement for paperai.index.

Route the task

1. Validate inputs first. Read data-formats.md and run the checker before loading a model. It checks tables, required columns, counts, bounds, and YAML shape without importing txtai or downloading a model.
2. Choose configuration. Use api-reference.md and workflows.md to distinguish a model reference, a word-vector database, and a full YAML mapping. Start with a positive maxsize or toprank for an exploratory run; 0 means unbounded in the source implementation.
3. Build and preserve the directory contract. paperai.index reads <path>/articles.sqlite and saves txtai artifacts back into the same directory. Do not delete the database when replacing a failed index. Confirm the output files before handing the directory to another route.
4. Recover deliberately. Use troubleshooting.md for missing schemas, malformed YAML, model cache/download failures, memory pressure, and interrupted full-corpus runs. A Python import or a CPU SQLite check does not prove that a configured model or accelerator can load.
5. Hand off only after local checks. Query execution belongs to the sibling querying route; report task configuration and rendering belong to reporting. This route supplies their shared corpus/model prerequisites, not their presentation behavior.

Core commands

Install paperai==2.6.0 in Python 3.10+ and verify imports before attempting a model load:

python -m pip install "paperai==2.6.0"
python -c "import paperai, paperai.index, paperai.export; print(paperai.__file__)"

The source module accepts these positional forms:

python -m paperai.index CORPUS_DIR [VECTORS_OR_YML] [MAXSIZE] [TOPRANK]
python -m paperai.export OUTPUT.txt CORPUS_DIR

For exact signatures and output artifacts, use the linked references rather than treating either positional interface as a safe full-corpus default.