---

name: training description: "Routes Ultra-Fast-Lane-Detection training, checkpoint, resume, finetune, and multi-GPU launch workflows." disable-model-invocation: true metadata: disco-role: operating license: MIT

training

Use this sub-skill when a task is about fitting the lane detector: choosing optimizer and scheduler settings, launching single-GPU or distributed training, resuming checkpoints, or understanding how the repo logs and saves models.

Read this when

• You need to start a training run from train.py.
• You need to resume or finetune from a checkpoint.
• You need to translate the repo's shell launcher into a safer command.
• You need to explain the repo's losses, metrics, or checkpoint naming.

What this sub-skill owns

• Training command construction.
• DDP/NCCL launch advice.
• Optimizer, scheduler, loss, and metric selection.
• Checkpoint save/resume/finetune behavior.
• TensorBoard logging and code-backup side effects.

What it does not own

• Dataset root preparation and TuSimple conversion: see data-and-config.
• Evaluation, scoring, and demo output files: see evaluation-and-visualization.
• TorchScript export and benchmark timing: see export-and-speed.

Start here

• Read references/training-workflows.md for the practical command patterns.
• Read references/api-reference.md for verified function signatures and model dimensions.
• Read references/troubleshooting.md for CUDA, checkpoint, and launch failures.
• Run scripts/model_cuda_smoke.py when you need a quick proof that the model can build and run on CUDA.
• Use scripts/launch_training_template.sh as the safer replacement for the repo's raw launch shell snippet.

Typical flow

1. Pick the dataset family and matching config from data-and-config.
2. Decide whether the run is single-GPU or distributed.
3. Set data_root, log_path, and any checkpoint path overrides explicitly.
4. Confirm the backbone, griding_num, num_lanes, and use_aux match the chosen dataset family.
5. Start the run and keep the logs outside the repository tree.

Caution points

• The source training loop calls .cuda() directly.
• The launcher assumes NCCL for multi-GPU runs.
• cp_projects can copy a large working tree if log_path points at a directory inside the repo.
• The checkpoint filename convention matters for resume logic.

Reference and script links

• references/training-workflows.md for the command patterns and checkpoint lifecycle.
• references/api-reference.md for the verified signatures.
• references/troubleshooting.md for launch, CUDA, and checkpoint problems.
• scripts/launch_training_template.sh for a parameterized shell launcher.
• scripts/model_cuda_smoke.py for a small model/build smoke check.