---

name: training description: "Use SageMaker Python SDK v3 training workflows with ModelTrainer, distributed drivers, JumpStart training, HPO, AWS Batch queues, and remote-function training concepts." metadata: disco-role: operating disable-model-invocation: true license: Apache 2.0

SageMaker Python SDK v3 Training Sub-skill

Use this sub-skill when the task is about general SageMaker training with the v3 sagemaker-train package. Keep generated code v3-first and SDK-first. Do not fall back to raw boto3 training-job requests unless the user explicitly asks for low-level API calls or the SDK does not expose the needed feature.

Trigger phrases

Read this sub-skill when the user says or implies:

• "train a model on SageMaker" or "create a training job".
• "use ModelTrainer", "sagemaker-train", or "SDK v3 training".
• "local mode training", "local container training", local_cpu, or local_gpu.
• "distributed training", "torchrun", "MPI", "SMP", "multi-node", or "multi-GPU".
• "custom training container", "training image URI", or "retrieve a framework image".
• "JumpStart training" or ModelTrainer.from_jumpstart_config(...).
• "hyperparameter tuning", "HPO", HyperparameterTuner, tune(), or analytics().
• "AWS Batch training queue", TrainingQueue, fair-share, priority, or quota share.
• "remote function training job", @remote, RemoteExecutor, or container drivers.
• "migrate Estimator/framework estimator/fit to v3 training".

When not to read this sub-skill

• Foundation model customization, evaluation, data mixing, recipe overrides, SFT, DPO, RLVR, RLAIF, CPT, or MultiTurnRL: route to ../model-customization/SKILL.md.
• Model deployment, endpoint prediction, serverless/async real-time hosting, local/in-process inference, or ModelBuilder: route to ../serving/SKILL.md.
• Processing jobs, batch transform, low-level resources, sessions, default buckets, image URI internals, or lineage primitives not tied to training: route to ../core-resources/SKILL.md.
• Pipelines, workflow steps, model registry, feature store, Clarify, or lineage governance around training: route to ../mlops/SKILL.md.
• Deprecated v2 maintenance work only: use the root migration reference first, then return here only for the v3 replacement.

V3 default policy

• Use from sagemaker.train import ModelTrainer for training orchestration.
• Prefer current config imports from sagemaker.core.training.configs for SourceCode, Compute, InputData, StoppingCondition, OutputDataConfig, CheckpointConfig, and Networking. Some examples use sagemaker.train.configs; it is a v3 compatibility shim in this repo.
• Use from sagemaker.core import image_uris and image_uris.retrieve(...) for framework image URI resolution.
• Use Session() and get_execution_role() from sagemaker.core.helper.session_helper rather than hardcoded buckets, regions, account IDs, or role ARNs.
• Use placeholders such as <training-image-uri>, <role-name-or-arn>, and s3://<bucket>/<prefix> when exact customer resources are unknown.
• Do not introduce v2 Estimator, framework estimator classes, fit(), v2 Model, v2 Predictor, v2 Processor, or old workflow imports except in an explicitly labeled deprecated migration note.

Short workflow

1. Classify the training task: single-job ModelTrainer, local container, distributed, JumpStart, HPO, AWS Batch queue, remote function, or migration.
2. Confirm required execution substrate: no-submit config drafting, Docker local mode, SageMaker cloud job, AWS Batch queue, or MLOps pipeline.
3. Establish AWS prerequisites if any cloud job may be submitted: region, credentials, execution role, S3 staging/output location, ECR image access, service quotas, and IAM permissions.
4. Build config objects first: SourceCode, Compute, InputData, StoppingCondition, optional Networking, OutputDataConfig, CheckpointConfig, and optional distributed config.
5. Resolve the training image with image_uris.retrieve(...) or accept a user-provided custom image URI placeholder.
6. Construct ModelTrainer(...); set role explicitly outside SageMaker managed notebooks if automatic role discovery is not guaranteed.
7. Validate before spend: prefer trainer.train(..., dry_run=True) for cloud job configuration checks; for local mode also verify Docker/compose and local paths before calling train().
8. Submit only when authorized: train(wait=True, logs=True) for a blocking job, train(wait=False, logs=False) for fire-and-check-later, or use HPO/Batch entry points when the task requires them.
9. After training, route deployment to ../serving/SKILL.md and route pipeline integration to ../mlops/SKILL.md.

Reference map

• references/modeltrainer-workflows.md: ModelTrainer signatures, imports, region/role/session setup, image retrieval, hyperparameters, local/SageMaker training, JumpStart, and validation.
• references/distributed-training.md: Torchrun, MPI, SMP, custom DistributedConfig, driver environment variables, local multi-container training, and distributed troubleshooting.
• references/hyperparameter-tuning.md: HyperparameterTuner, parameter ranges, metrics, tune(), analytics(), warm starts/autotune, and MLOps handoff.
• references/remote-function-and-advanced-training.md: AWS Batch TrainingQueue, remote function and RemoteExecutor, container driver concepts, queue monitoring, and advanced routing boundaries.
• references/troubleshooting.md: region, credentials, role, Docker/local mode, image URI, source packaging, hyperparameters, distributed, HPO, JumpStart, Batch, and remote-function fixes.

Migration guardrail

Deprecated v2 training patterns map as follows; keep details concise here and route broader migrations to ../../references/migration-v2-to-v3.md.

• Deprecated sagemaker.estimator.Estimator -> v3 sagemaker.train.ModelTrainer.
• Deprecated framework estimators such as PyTorch, TensorFlow, SKLearn, XGBoost, and HuggingFace classes -> v3 ModelTrainer plus image_uris.retrieve(...).
• Deprecated estimator.fit(...) -> v3 model_trainer.train(...) with InputData(channel_name=..., data_source=...).
• Deprecated instance_type/instance_count on estimators -> v3 Compute(instance_type=..., instance_count=...).
• Deprecated entry_point/source_dir estimator parameters -> v3 SourceCode(entry_script=..., source_dir=...).

Quality checks before returning training guidance

• Confirm examples use v3 imports and do not contain real account IDs, role ARNs, bucket names, subnet IDs, security group IDs, regions hardcoded as the only option, or credentials.
• If examples use sagemaker.train.configs, consider replacing with sagemaker.core.training.configs unless compatibility with an existing notebook is the explicit goal.
• Verify SourceCode.source_dir contains the named entry_script and optional requirements file, or explain that cloud validation will fail.
• For local mode, say Docker and Docker Compose are required and set local_container_root deliberately when output location matters.
• For distributed jobs, require source_code.entry_script and pick one driver: Torchrun, MPI, or a custom DistributedConfig.
• For HPO, define metric regexes that match the training script logs and use tuner.analytics() only after a tuning job exists.
• For AWS Batch, require Mode.SAGEMAKER_TRAINING_JOB; Batch queues do not accept local-container ModelTrainer jobs.

Common safe defaults

• Start with Compute(instance_type="ml.m5.xlarge", instance_count=1) for CPU examples unless the user asks for GPU or a framework requires GPU.
• Use StoppingCondition(max_runtime_in_seconds=3600) in examples to bound cost.
• Use Session().boto_region_name for region and Session().default_bucket() for sample S3 URI assembly.
• Use dry_run=True when the user asks to validate configuration without launching a chargeable job.
• Use wait=False only when the user wants asynchronous submission and you also show how to inspect or wait for the resulting job/tuner/queue object.

Handoff after training

• For deployment from a completed training job or model artifact, load ../serving/SKILL.md.
• For a train/tune step inside a SageMaker Pipeline, load ../mlops/SKILL.md.
• For low-level inspection of TrainingJob resources or image URI behavior, load ../core-resources/SKILL.md.
• For foundation model customization/evaluation recipes, do not duplicate here; load ../model-customization/SKILL.md.