---

name: syn-spark-checkpointing description: Operate, configure, troubleshoot, and manage Apache Spark. Use for Apache Spark engineering tasks, day-to-day operations, and production support.

Apache Spark: checkpointing and lineage truncation

When to use

Use this skill for checkpointing and lineage truncation in Apache Spark. This skill specifically handles checkpointing and lineage truncation and nothing else in the Apache Spark family.

Procedure

Use checkpoint() to truncate long lineage in iterative jobs, configure the checkpoint dir, and balance cost vs recompute.

Notes

This is the Apache Spark skill dedicated to checkpointing and lineage truncation. Sibling skills cover other Apache Spark capabilities; this one is the right choice only when the task is about checkpointing and lineage truncation.