技能创建器
skill-creator
创建、改进和评估自定义技能,把专业流程转化为可重复使用的 AI 能力。
clickhouse-performance-tuning【数据与研究】Optimize ClickHouse query performance with indexing, projections, settings tuning, and query analysis using system tables. Use when queries are slow, investigating performance bottlenecks, or tuning ClickHouse server settings. Trigger with "clickhouse performance", "optimize clickhouse query", "clickhouse slow query", "clickhouse indexing", "clickhou…
独立 GitHub 快照 · 自动分类 · 以原作者文档为准
name: clickhouse-performance-tuning description: | Optimize ClickHouse query performance with indexing, projections, settings tuning, and query analysis using system tables. Use when queries are slow, investigating performance bottlenecks, or tuning ClickHouse server settings. Trigger with "clickhouse performance", "optimize clickhouse query", "clickhouse slow query", "clickhouse indexing", "clickhouse tuning", "clickhouse projections". allowed-tools: Read, Write, Edit version: 1.7.0 license: MIT author: Jeremy Longshore jeremy@intentsolutions.io tags:
Diagnose and fix ClickHouse performance issues using query analysis, proper indexing,
projections, materialized views, and server settings tuning. Work top-down: measure
first with system.query_log, then apply the single highest-leverage fix (usually the
ORDER BY key), then re-measure to confirm.
clickhouse-core-workflow-a)system.query_log and system.partsThe tuning workflow is seven independent steps. Diagnose first, then reach for the fix that matches the bottleneck. Each step's full SQL lives in references/implementation.md — start there for the complete, copy-paste commands.
system.query_log by
query_duration_ms, then inspect a suspect query with EXPLAIN PLAN /
EXPLAIN PIPELINE.bloom_filter for high-cardinality lookups, set for
low-cardinality columns, minmax for range filters on non-key columns.max_threads, external sort/group-by spill, async_insert,
and friends, set per-query or per-session.AggregatingMergeTree so
dashboard reads hit milliseconds, not seconds.PREWHERE, LIMIT BY, and avoiding FINAL.The essential first move — find the slowest queries:
SELECT event_time, query_duration_ms, read_rows, read_bytes,
substring(query, 1, 300) AS query_preview
FROM system.query_log
WHERE type = 'QueryFinish'
AND event_time >= now() - INTERVAL 24 HOUR
AND query_duration_ms > 1000 -- > 1 second
ORDER BY query_duration_ms DESC
LIMIT 20;
Applying this workflow produces:
read_rows / read_bytes cost.ORDER BY key, added data
skipping indexes, a projection, a materialized view, or tuned session settings.system.query_log proving the change reduced
read_rows, read_bytes, query_duration_ms, or memory_usage.| Issue | Indicator | Solution |
|---|---|---|
| Full table scan | read_rows = total rows | Fix ORDER BY to match filters |
| Memory exceeded | Error 241 | Add LIMIT, use streaming, increase limit |
| Slow GROUP BY | High read_bytes | Add materialized view or projection |
| Merge backlog | Parts > 300 | Reduce insert frequency, increase merge threads |
Worked before/after scenarios — full-scan → ORDER BY fix, slow GROUP BY → projection, confirming a skipping index fires, and the query-cost measurement query — are in references/examples.md. The core measurement, run right after any query you are tuning:
SELECT query_duration_ms, read_rows,
formatReadableSize(read_bytes) AS read_size,
formatReadableSize(memory_usage) AS memory
FROM system.query_log
WHERE query_id = currentQueryId() AND type = 'QueryFinish';
For cost optimization, see clickhouse-cost-tuning.