---

name: self-supervised description: "Use for PaddleViT's DINO self-supervised vision-transformer pretraining, multi-crop data contracts, teacher/student configuration, single- or multi-GPU launch planning, checkpoint/resume handling, and optional PyTorch-to-Paddle weight-porting boundaries." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

PaddleViT DINO self-supervised learning

Use this sub-skill when a task names self_supervised_learning/dino, DINO pretraining, multi-crop augmentation, a DINO teacher/student checkpoint, or porting a DINO ViT backbone into Paddle. It is an operating guide for the PaddleViT checkout, not a promise that every released script is runnable as-is. Keep the source checkout and this skill's evidence boundary explicit.

Applicability and hard boundaries

• The supported end-to-end data path is the ImageNet2012-style path consumed by ImageNet2012Dataset: DATA.DATA_PATH/train_list.txt and DATA.DATA_PATH/val_list.txt, with image paths relative to that directory. Each line is <relative-image-path> <integer-label>; labels are ignored by DINO but are still required by the dataset reader.
• Actual DINO training needs DATA.DATASET=imagenet2012. The CIFAR branches in datasets.py return a single transformed tensor and therefore do not satisfy the multi-crop contract used by main_*_gpu.py.
• One training sample is transformed into two global views at DATA.IMAGE_SIZE and DATA.LOCAL_CROPS_NUMBER local views at DATA.SMALL_CROP_IMAGE_SIZE. Defaults are 2 global + 10 local views, 224/96 pixels, global scale [0.25, 1.0], and local scale [0.05, 0.25].
• The teacher receives only images[:2]; the student receives all crops. Crop tensors must be batched, normalized with the configured ImageNet mean/std, and grouped by spatial size because MultiCropWrapper groups consecutive equal-width crops.
• This skill does not authorize downloading ImageNet, checkpoints, or external repositories, nor starting benchmark-scale or long-running pretraining. Prefer config checks, import checks, synthetic tensors, and a bounded one-step smoke. State the required dataset, GPU count, wall-time, and output directory before proposing a real run.
• torch, timm, and the PyTorch DINO hub model are optional porting dependencies. They are not required for Paddle DINO training and are absent from the inspected environment; do not install or fetch them implicitly.

Evidence and source map

Primary evidence is the pinned checkout's self_supervised_learning/dino/{README.md,config.py,datasets.py,transformer.py,utils.py,main_dino_single_gpu.py,main_dino_multi_gpu.py,run_train_multi.sh} and self_supervised_learning/dino/port_weights/load_pytorch_weights.py. Cross-cutting launch, configuration, AMP, and porting guidance is in docs/paddlevit-{config,multi-gpu,amp,port-weights}.md. The source is an older Paddle implementation (Paddle 2.1-era assumptions); use the bundled scripts for bounded checks and treat source defects below as evidence, not as silent corrections.

Configuration procedure

1. Start from self_supervised_learning/dino/configs/vit_small_patch16_224.yaml or vit_base_patch16_224.yaml; pass it with -cfg from the DINO directory. YAML is merged into the defaults in config.py, then CLI values override selected fields. Use the same effective config for model construction and record it with the run.
2. Check the coupled dimensions before launch: IMAGE_SIZE % PATCH_SIZE == 0, SMALL_CROP_IMAGE_SIZE % PATCH_SIZE == 0, EMBED_DIM % NUM_HEADS == 0, positive OUT_DIM, two ordered crop-scale bounds in (0, 1], and LOCAL_CROPS_NUMBER >= 1. Run scripts/check_dino_config.py --config <yaml>; it is read-only.
3. Preserve the DINO defaults unless a bounded experiment justifies a change: AdamW, cosine LR/weight-decay schedules, teacher momentum warming toward 1, teacher temperature warmup, FREEZE_LAST_LAYER, and OUT_DIM=65536. Note that the README says weight-decay scheduling was not supported in the 2022 release, while the current code computes a schedule; verify the actual checkout before relying on that behavior.
4. Set DATA.BATCH_SIZE per GPU, not global batch size. Scale effective batch size and learning rate deliberately when changing GPU count; the source does not automatically apply the documented linear-LR convention.
5. Treat MODEL.PRETRAINED as a backbone/model-state input only after checking the exact state-dict shape and naming. A DINO run needs student/teacher/head compatibility, not merely a classification checkpoint.

See references/configuration.md for the field contract and scripts/check_dino_config.py for safe validation.

Teacher/student and multi-crop procedure

The intended construction is two ViTs with shared initial state: the student uses configured stochastic depth and the teacher is rebuilt with DROPPATH=0. Each is wrapped by MultiCropWrapper and a DINOHead; teacher parameters have stop_gradient=True. The student is optimized; the teacher is updated after each step by EMA using MOMENTUM_TEACHER's cosine schedule. DINOLoss sharpens and centers the teacher output, compares the two teacher global views against all nonmatching student views, and maintains a center buffer.

For a synthetic check, use two [B,3,32,32] global tensors and two or more [B,3,16,16] local tensors with a tiny ViT config. Do not infer ImageNet accuracy from this check. Run:

python scripts/dino_model_smoke.py --help
python scripts/dino_model_smoke.py --repo-root /path/to/PaddleViT --device cpu
# On a prepared CUDA host, use --device gpu:0 (no dataset or download).

A successful bounded smoke should show model construction, crop shapes, a student/teacher forward, finite DINO logits/loss, and no parameter update on the teacher. If the checkout's wrapper/API fails, retain the diagnostic and do not claim training compatibility; do not hide it by downloading PyTorch.

Launch procedure and backend gates

Single GPU

Use the single-process entrypoint only after config and smoke checks:

CUDA_VISIBLE_DEVICES=0 python main_dino_single_gpu.py \
  -cfg=./configs/vit_small_patch16_224.yaml \
  -dataset=imagenet2012 -batch_size=32 \
  -data_path=/dataset/imagenet -output=./output -amp

-batch_size is per GPU. AMP uses paddle.amp.auto_cast and GradScaler; repository docs limit FP16 AMP claims to NVIDIA Ampere, Volta, and Turing. Require a CUDA Paddle build and a successful CUDA tensor/layer smoke before using -amp. CPU is suitable for import/tiny checks only, not this training claim.

Single-node multi GPU

The source launcher uses paddle.distributed.spawn, initializes one worker per visible GPU, wraps models in paddle.DataParallel, uses DistributedBatchSampler, and all-reduces loss/center statistics. Use an explicit device list and -ngpus matching it; a distributed run requires NCCL and multiple usable CUDA devices:

CUDA_VISIBLE_DEVICES=0,1 python main_dino_multi_gpu.py \
  -ngpus=2 -cfg=./configs/vit_small_patch16_224.yaml \
  -dataset=imagenet2012 -batch_size=16 -data_path=/dataset/imagenet -amp

The repository's run_train_multi.sh is an eight-GPU, long-running ImageNet launcher and is reference-only. Do not execute it during skill use. Validate world size, rank, sampler partitioning, and rendezvous in a short controlled job before any real run. Do not call one-GPU execution a multi-GPU pass.

Known source hazards to check before a real launch

The inspected scripts contain apparent defects that a user must resolve or patch in a controlled copy and then re-run the bounded checks for:

• single-GPU train returns two values but main unpacks three; later code references local_logger, model, and scheduler inconsistently;
• both entrypoints contain params_gropus/params_groups inconsistency;
• pretrain/resume code references model instead of the student/teacher model;
• resume and save paths have inconsistent DINO-loss suffixes (._dino_loss, _dino_loss.pdparams, and _dino_loss.pdprams);
• the config's MODEL.NORM_LAST_LAYER is not consistently wired into the hard-coded head construction;
• ACCUM_ITER is accepted but the shown loop still clears and steps every batch.

These are not permission to perform an unreviewed rewrite. Record the exact checkout, patch, effective config, and smoke result. If the task is only to inspect or plan, stop at the hazard report.

Checkpoints and resume

The intended save prefix is under SAVE/train-<timestamp>/ and includes a .pdparams model state, .pdopt optimizer state, and a DINO-loss state. The teacher is the meaningful exported representation in the multi-GPU saver; the source's suffix typo means you must inspect actual files rather than guessing. Use TRAIN.LAST_EPOCH consistently with the checkpoint's epoch and preserve optimizer state, teacher center, temperature schedule, and effective config. A model-only load is not an exact resume. Before resuming, assert that all three expected artifacts exist, compare parameter names/shapes, and make a copy or use a new output directory. Never overwrite a checkpoint silently.

Optional PyTorch weight porting

port_weights/load_pytorch_weights.py is a separate, CPU-oriented conversion example. It imports torch, loads facebookresearch/dino:main via torch.hub, maps the ViT backbone names, transposes 2-D linear weights, and checks a batched output with np.allclose. It does not establish a full DINO student/teacher/head conversion or checkpoint resume. Keep this route optional: only use an approved isolated environment with a local PyTorch checkpoint and explicit source/target model specifications. Manually inspect parameters and buffers, preserve non-linear 2-D tensors without transpose, compare batched outputs, and save a new .pdparams; never fetch a hub model as an implicit step. The current environment has no torch/timm, so this path is unverified.

Recovery and stop conditions

• Missing train_list.txt or wrong image-root layout: stop and report the expected layout; do not download or synthesize ImageNet silently.
• Crop count/shape mismatch or non-finite loss: stop before long training; verify transform order, LOCAL_CROPS_NUMBER + 2, normalization, output dimension, teacher temperature, and model state.
• CUDA/AMP failure: rerun the CPU import/tiny smoke, then the CUDA preparation probe; mark GPU-specific behavior blocked if CUDA or the required device is unavailable.
• Distributed hang or unequal ranks: terminate the bounded job, check visible devices, -ngpus, NCCL/rendezvous, and sampler/world-size setup. Do not retry indefinitely.
• Checkpoint mismatch: start a new output directory and use a shape/name report; do not force-load incompatible heads or silently discard the teacher center.

Difficult synthetic usability cases

1. Crop-contract and teacher-EMA case: build a tiny ViT with two 32x32 global views and three 16x16 local views, run one finite DINO step, assert teacher outputs use two views, student outputs use five, teacher parameters remain gradient-free, and EMA changes the teacher without changing crop counts. No filesystem dataset or network may be used.
2. Resume/port boundary case: provide a temporary checkpoint prefix with a model state, optimizer state, and deliberately misspelled loss suffix, then ask the agent to diagnose whether it is an exact resume and to produce a non-destructive repair plan. Separately provide a fake PyTorch state mapping with one transposed linear tensor; the expected result is a blocked optional port report when torch is unavailable, not an install or hub download.

Handoff checklist

Report the source commit, effective YAML/CLI overrides, dataset layout check, backend/device probe, smoke command and result, launch mode, per-GPU batch, AMP/distributed assumptions, checkpoint prefix and suffixes, patches applied (if any), and unresolved source hazards. Do not claim ImageNet pretraining or multi-GPU success from a synthetic smoke.

Bundled references

• workflows.md — bounded checks, launch templates, checkpoint flow, and backend boundaries.
• configuration.md — effective fields and invariants.
• troubleshooting.md — source-specific failures and recovery decisions.