Designs product and site search for end users. Use when choosing search engines, indexing, reindexing, relevance tuning, facets, autocomplete, merchandising, or search analytics.
Independent GitHub snapshot · Automated classification · Original documentation governs
name: software-search
description: "Designs product and site search for end users. Use when choosing search engines, indexing, reindexing, relevance tuning, facets, autocomplete, merchandising, or search analytics."
compatibility: Portable core. Works on Claude Code and Codex.
version: "1.1"
last_validated: 2026-08-24
Search Engineering
Build search features that return the right results, fast.
Quick Reference
Need
Default
Deviate when
Full-text search, PostgreSQL already in the stack
tsvector/tsquery plus pg_trgm, both on GIN indexes
Ranking quality is the gap (ts_rank is not BM25): add a BM25 extension if the host allows it and its licence fits. Facets with counts, typo tolerance, merchandising, or latency at scale: move to a dedicated engine
Dedicated engine
Managed service when no one owns search-cluster operations
Self-host when data residency, cost at volume, or query control requires it and someone owns the cluster
Embedded or client-side search
Embedded index (such as SQLite FTS5) or a static client-side index
The corpus or its update rate outgrows a shipped index
Semantic search
Vector retrieval as a complement to lexical recall, in the existing database first when it supports vectors
Judged-query evidence shows the in-database option misses the latency or recall target
Hybrid search
Keyword + vector, merged with reciprocal rank fusion
Native fusion is missing from the edition or licence tier you will run: fuse in the application
The chosen engine's client UI library, if it has one
Custom components when the library cannot express the UX
Lookup step before naming an engine. Engine capabilities, licences, and managed-service limits change from release to release, so shortlist by the criteria above, then check each candidate's current documentation and licence. Start from data/sources.json; the capability matrix in references/engine-and-relevance.md is a starting point, not current fact.
When to Use This Skill
Choosing a search engine or evaluating whether PostgreSQL search is sufficient
Building full-text search, autocomplete, or faceted filtering
Designing an indexing pipeline from source data to search index
Tuning relevance scoring, synonyms, or ranking signals
Implementing search analytics to measure and improve quality
Define the corpus, query intents, filters, freshness target, latency budget, and display fields.
Confirm that this is product search; route RAG, database tuning, SEO, generic product-analytics or event-tracking work, or API-architecture work to the adjacent skill.
Choose PostgreSQL, a dedicated engine, vector, or hybrid retrieval with the decision tree.
Load the focused reference for the chosen path; design indexing, schema, ranking, synonyms, and result hydration.
Add instrumentation and a judged-query evaluation before changing ranking, analyzers, synonyms, or business boosts.
Verify current engine capabilities and hosted-service behavior from primary sources, then return tradeoffs, assumptions, and a rollout or rollback plan.
Decision Tree
Which search engine?
├── PostgreSQL already in the stack, and the workload is inside its limits?
│ (limits: "When to outgrow PostgreSQL search" in references/engine-and-relevance.md)
│ └── YES → PostgreSQL full-text search (tsvector + pg_trgm on GIN indexes)
│ ├── Ranking quality the gap? (ts_rank is not BM25)
│ │ └── YES → BM25 extension, if your host allows it and its licence fits
│ └── Outgrowing it? (facets with counts, typo tolerance, merchandising, latency at scale)
│ └── YES → Dedicated engine (below)
├── Dedicated engine: does anyone own search-cluster operations?
│ └── NO → Managed service. YES → self-hosted engine, if residency, cost, or control needs it
├── Static site or small, rarely changing corpus?
│ └── YES → Client-side index
├── Need semantic/meaning-based search?
│ └── YES → Vector retrieval, as a complement to lexical recall
└── Need both keyword AND semantic?
└── YES → Hybrid search (keyword + vector + reciprocal rank fusion;
check the edition or licence tier for native fusion)
Then run the lookup step: verify each shortlisted engine's current capabilities and licence.
Decision and Safety Rules
Start with the least operationally complex engine that meets the corpus, query, latency, and freshness requirements; verify capability claims at use time.
Keep indexing idempotent, search documents explicit about searchable, filterable, and stored fields, and live schema changes behind an alias or equivalent rollback path.
Allowlist filters, validate input before embedding or query construction, rate-limit by authenticated actor where applicable, and treat raw query logging as a data-governance decision.
Treat vector retrieval as a complement to lexical recall until judged-query evidence shows otherwise; do not hand-pick ranking weights from intuition.
Keep user consent and tenant or actor isolation explicit before using personalization or behavioral signals.
Search Quality Evaluation
Analytics are not enough on their own. Keep a judged-query set for the product's most important search intents and re-run it whenever you change ranking, analyzers, synonyms, or business boosts.
Minimum loop:
define representative queries across navigational, informational, autocomplete, and zero-result-recovery paths
rate a small set of expected results for each query
run the engine's native rank-evaluation API or an equivalent offline harness
compare score deltas before and after relevance changes
review failures manually before shipping boosts or synonym changes
Use click data to find candidates for the judged set, but do not let click-through alone define quality. Position bias, sparse traffic, and merchandising effects can hide bad ranking decisions.
Ranking Release Gate
Require an offline win on the judged-query set with no unacceptable regression in protected query classes, then canary the candidate against the current ranker. Compare task completion or downstream conversion alongside latency, zero-result rate, abandonment, and reformulation. Log the ranker version with every impression so a bad release can be isolated and rolled back without rebuilding the index.
Common Anti-Patterns
LIKE '%query%' at scale — a leading wildcard cannot use a B-tree index, so the query scans and slows linearly with data growth. A pg_trgm GIN index can serve it, but still gives no ranking, stemming, or relevance; use full-text search for search features.
No analyzers configured — raw text matching misses stemming, case folding, and accent normalization. Users searching "running" won't find "run."
Ignoring zero-result queries — the cheapest way to improve search quality, and most teams never look at them.
Re-indexing full dataset on every change — use incremental updates (upsert by document ID) for individual changes and reserve full reindex for schema changes.
Coupling search schema to database schema — search documents should be denormalized and optimized for query patterns, not mirror your relational schema.
Not measuring search quality — if you don't track click-through rate, zero-result rate, and query refinement rate, you cannot improve search. Instrument from day one.
Known Traps
Semantic or vector retrieval added as a replacement for lexical recall instead of a complement to it.
Faceting on fields with unbounded cardinality, leading to expensive aggregations and unusable filter UX.
Synonym expansion shipped without governance, creating silent relevance regressions and impossible-to-debug ranking changes.
Index freshness assumed to be real-time when ingestion pipelines are actually batched or eventually consistent.
Verification Gate
Before calling a search design or implementation ready:
Engine matches current corpus size, query shape, and latency budget
Latency budgets defined separately for full search and autocomplete, set from the product's own baseline and UX needs (illustrative starting targets, not universal thresholds: ~200ms p95 full search, ~100ms p95 autocomplete)
Index freshness mechanism documented and measured (real-time, near-real-time, or scheduled batch)
Zero-result rate instrumented and baseline established; set the target from that baseline and the catalog's coverage (an illustrative starting point for a mature catalog is below ~5%, not a universal threshold)
Click position and refinement rate tracked from day one
Judged-query evaluation covers navigational, informational, autocomplete, and zero-result-recovery paths
Representative queries tested — not only happy-path keyword matches
Synonym and analyzer changes logged with rollback plan
Navigation
Focused References
Engine and Relevance Details — capability classes, PostgreSQL, indexing architecture, relevance tuning, and query classification.
Gate before invoking any foundation below: Each foundation has a When to Apply / When to Skip section. If your task matches a skip-condition, route to the foundation it names instead — don't pull in primitives the task doesn't need.
ai-rag — Retrieval-augmented generation and vector retrieval for LLM context
foundations-information-theory — Information-theoretic readings of IDF term weighting and KL-divergence (query-likelihood) scoring; load only when reasoning about term weights from first principles
For engine capabilities, licences, and hosted-service limits, use the lookup step above the workflow.
Record the consulted primary documentation and any unresolved capability in the recommendation.
Learnings Loop
Read relevant entries from learnings.consolidated.md only when continuing prior work, investigating a known pitfall, or debugging. Read raw learnings.md only when the consolidated entry points to it or dated detail is needed.
After applying it, if you encountered a pattern worth remembering, a mistake worth preventing, or a domain fact that surprised you, append one dated bullet to learnings.md via agents-skills-feedback-loop/scripts/append_learning.py. Do not modify SKILL.md itself.