---

name: query-engine description: "Guides DataChain query operations, SQL-style function expressions, schema mapping, vector search, and backend-sensitive behavior." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Query Engine

Use this sub-skill when the user asks about DataChain's native Query Engine: SQL-style chain operations, datachain.func expressions, Column / ColumnExpr, nested signal resolution, schema flattening, vector distance search, or a query bug that may depend on the active warehouse backend.

Trigger Phrases

Load this sub-skill for prompts containing or implying:

• mutate vs map, native SQL expression, datachain.func, dc.C, Column, ColumnExpr, alias, label, or expression type inference;
• filter, select, select_except, order_by, distinct, group_by, merge, union, subtract, diff, file_diff, limit, offset, shuffle, sample, count, sum, avg, min, or max;
• aggregate, array, conditional, numeric, string, path, random, or window functions;
• vector search, cosine/euclidean distance, embedding columns, or similarity_search;
• nested fields, flattened column names, SignalSchema, SQL type conversion, nullability, backend divergence, or behavior that passes locally but fails in Studio / ClickHouse.

First Decision

1. Choosing an operation or expression pattern → read operations-and-functions. Prefer native Query Engine operations when the logic can be expressed with columns and dc.func; reroute Python callables or file-content processing to sibling sub-skill sdk-pipelines.
2. Nested fields, schema flattening, SQL type conversion, or backend parity → read schema-and-backends before proposing a fix or claiming a backend result generalizes.
3. Vector nearest-neighbor / embedding ranking → read the vector sections in operations-and-functions and the backend notes in schema-and-backends.
4. A broken query, missing column, alias conflict, delta error, or backend-only failure → read troubleshooting, then use schema_probe.py or query_smoke.py only as small local diagnostics.

Boundaries and Reroutes

• This sub-skill owns native chain operations, SQL functions, query expression semantics, schema flattening/resolution, SQL type conversion, and backend-sensitive query behavior.
• For raw data ingestion, UDFs (map, gen, agg), checkpoint/delta pipeline recipes, exports, LLM calls, or file download/upload workflows, reroute to sibling sub-skill sdk-pipelines.
• For command-line, Studio auth/job/pipeline, or CLI parser questions, reroute to sibling sub-skill cli-and-studio.
• For DataChain's packaged agent skills or knowledge-base generation, reroute to sibling sub-skill agent-harness.
• For contributor test policy, nox, packaging, or source-maintenance decisions, reroute to sibling sub-skill repo-development.

Safety Rules

• Keep query guidance self-contained; do not require future agents to open the source checkout, repository docs, or repository tests to use this sub-skill.
• Do not say a CPU/local SQLite smoke proves ClickHouse, BigQuery, Snowflake, or Postgres behavior. Treat backend parity as a matrix claim that needs backend evidence.
• Do not materialize a large chain with to_pandas(), to_list(), or Python loops just to filter, project, group, aggregate, rank, or compute SQL distances; use native Query Engine operations when possible.
• Use map only for Python callables, file-content reads, model/API calls, or logic that cannot be expressed as a SQL-style expression.
• Every chain operation returns a new chain. Never assume filter, mutate, select, merge, or union changed the receiver in place.