Selects, deploys, and customizes AI models on Amazon SageMaker. Training or Processing jobs, fine-tuning (SFT/DPO/RLVR/RLAIF), model selection, evaluation, SageMaker or Bedrock deployment, inference optimization and endpoint diagnostics. Use when fine-tuning models on SageMaker, choosing a base model to customize or deploy from SageMaker JumpStart or Hub, SageMakerPublicHub, or the model catalog, checking data quality (duplicates, languages, PII), transforming or validating data (reformat, clean, filter, dedupe, split, merge, redact), generating synthetic data, labeling data with a model, human review or annotation and applying its corrections, evaluating model quality, deploying or optimizing endpoints, configuring IAM/S3, or managing SageMaker Managed MLflow. Use for endpoint health, latency, logs, metrics, errors. Covers Serverless Model Customization, Nova and OSS deployment paths, and PySDK v3. NOT for SageMaker Ground Truth labeling jobs, Feature Store, or general AWS infrastructure.
87%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
Failed to scan
The risk profile of this skill
Domain expertise for fine-tuning and deploying models on Amazon SageMaker. Covers the full model customization lifecycle from planning through production deployment.
Match the user's intent to the appropriate reference folder and load only that content.
| User intent | Reference | When to use |
|---|---|---|
| Plan a model customization project, discover scope of work, resume or modify a plan | references/planning/ | User's request relates to model customization or deployment (fine-tuning, training, building, customizing, reviewing data, deploying or standing up a model — including selecting or deploying an off-the-shelf or base model with no training — or getting advice on approach). Always co-activate with other intents to discover full scope. Load this reference FIRST when the request matches multiple rows in this table — read its plan templates before routing to a single-action reference. |
| Define the business problem, success criteria, or use case spec | references/use-case-specification/ | User says "define my use case", "capture requirements", "what should I decide up front", or as default first step in any plan. Skip only if user explicitly declines. |
| Select or change a base model | references/model-selection/ | User asks which model to use, mentions a model name or family, or wants to evaluate what's available. Always activate model-selection even for known model names because the exact Hub model ID must be resolved. Recommended: route to use-case-specification first to capture requirements — this produces better filtering results. Routing to use-case-specification first is not required if user provides a specific model name/ID or declines. If intent is ambiguous (fine-tune vs deploy as-is), model-selection MUST confirm which path before proceeding. Base model filtering for deployment MUST go through select-for-deployment.md and its scripts for any final recommendation. |
| Choose a fine-tuning technique (SFT, DPO, RLVR, RLAIF) | references/finetuning-technique/ | User has decided to fine-tune and needs to choose a technique, or technique needs validation against the selected model's recipes. Requires a base model to be selected first. |
| Validate dataset quality and format, run quality metrics, assess diversity or toxicity | references/dataset-quality/ | MUST LOAD when the user wants to check, validate, or evaluate a dataset. It checks the dataset's format, runs one dataset quality evaluation job (built-in metrics and spec-driven checks) and ends in a report; duplicate questions go to its deduplication sub-reference (removing duplicates stays with dataset-transformation), composition questions to its taxonomy sub-reference (us-east-1 only). This is the ONLY validated workflow for dataset quality; do not attempt it from general knowledge. Use when the user says "is my dataset okay", "check my training data", "check data quality", "run DQ metrics", "how good is my training data", "check diversity", "check toxicity", "what languages are in my dataset", "is my data all English", "does my data contain PII", "run a scorecard", "I have my own data", "evaluate my dataset", "quality evaluation", "find duplicates", "how many duplicates", "near-duplicates", "audit duplicates before removing", "what's in my dataset", "what topics does my data cover", "taxonomy of my data", "is my data balanced across tasks", or before starting any fine-tuning job. |
| Transform, deduplicate, split, merge, filter, convert schema, or clean a dataset (CodeGen, built-in operations, or BYOC) | references/dataset-transformation/ | User says "transform", "custom transform", "convert", "reformat", "deduplicate", "remove duplicates", "dedupe", "split", "partition", "train/validation/test split", "propose a train/eval/holdout split", "merge", "combine datasets", "filter", "keep only English rows", "filter by language", "remove PII", "redact personal data", "clean", "normalize", "fix encoding", "convert schema", "derive a field", "edit field values", "bring my own code", "run my transformer.py", "run this transformation locally", or asks to replace awk/sort/uniq/jq for a local or S3 dataset. Covers CodeGen, BYOC, and the built-in DataTransformation operators described by the live schema and operator mapping. Always route here instead of writing an ad-hoc transformation script or shell one-liner. |
| Generate synthetic data for training, database seeding, augmentation, tabular data, or any use case requiring realistic generated data | references/synthetic-dataset-generation/ | User says "generate data", "synthetic data", "create training data", "augment data" / "I need more data", "seed database", "mock/fake data (rows, CSV, Parquet, SQLite)", or "generate N rows / records / a dataset". Can run standalone for non-training use cases. |
| Set up human annotation or review cycles via local chat review or Label Studio | references/human-in-the-loop/ | MUST LOAD when user mentions annotation, human review, Label Studio, or applying annotations. Triggers: "set up annotation", "review samples", "set up Label Studio", "annotations are done", "summarize annotation results", "apply the annotations", "incorporate human feedback", "human review", "manual review", "annotate my data", "need human feedback", "annotation project", "annotators", "labeling team", "annotation spec", "corrections from annotators". This reference contains the ONLY validated workflow for human annotation — do not attempt annotation setup from general knowledge. Does NOT modify the dataset directly — always delegates edits to dataset-transformation. |
| Automatically label unlabeled data with a labeling spec and a model jury | references/auto-labeling/ | MUST LOAD; the only validated workflow for labeling by a model. Load when the user mentions labeling, tagging or classifying data at scale, a labeling spec, a refinement loop, or which spec ships, even without the word "label". When people will label or review the rows themselves ("have my team label this", "annotators", "annotate my data"), route to human-in-the-loop instead; auto-labeling hands its own review set there. Triggers: "label my data", "label my dataset", "auto-label", "classify my samples", "I have unlabeled data", "NER tagging", "sentiment labeling", "tag each", "label the rest", "labeling spec", "refine my labeling", "refine my spec", "run the refinement loop", "which spec do I ship", "measure labeling accuracy", "label corpus", "confidence routing", "tag support tickets", "label these rows". |
| Generate fine-tuning code and start training | references/finetuning/ | User says "start training", "fine-tune my model", "I'm ready to train", or plan reaches the finetuning step. Supports SFT, DPO, RLVR, RLAIF trainers. |
| Train a model with a SageMaker Training job | references/training-jobs/ | Use when the goal is to fit or update model parameters and produce model artifacts or checkpoints. Covers recipes, JumpStart traditional ML, custom training scripts or containers, distributed training, and training-job diagnosis. |
| Process data with a SageMaker Processing job | references/processing-jobs/ | Use when the goal is preprocessing, feature engineering, evaluation, transformation, or ETL without model training as the primary operation. Covers scripts, framework processors, Spark, custom containers, Athena, Redshift, and processing-job diagnosis. |
| Evaluate or benchmark a trained model | references/model-evaluation/ | User says "evaluate my model", "run a benchmark", "test model performance", "compare models". Supports LLM-as-Judge and Custom Scorer. |
| Deploy, benchmark, or optimize a model on an endpoint or Bedrock | references/model-deployment/ | User says "deploy my model", "create an endpoint", or "make it available" (plain deploy) — or, for the inference-optimization sub-workflows on SageMaker Real-Time Endpoints only, "benchmark my endpoint" / "compare benchmark runs" (benchmarking), or states a performance/cost/latency/throughput goal for a new deployment such as "find the cheapest instance", or asks for an AI Recommendation Job or to right-size a model already on an endpoint (recommendations). Handles Nova vs OSS deployment pathways. |
| Set up IAM roles, S3 buckets, SDK configuration | references/sdk-getting-started/ | User says "set up", "getting started", "check my environment", "configure SDK", or as first step in any plan involving SageMaker training/evaluation/deployment. |
| Manage project directory and artifacts | references/directory-management/ | Starting a new project, resuming existing one, or when PLAN.md needs to be associated with a project directory. |
| Set up, update, or delete a SageMaker Managed MLflow app | references/manage-mlflow/ | User says "set up MLflow", "create MLflow app", "update my MLflow app", "delete my MLflow app", "I need an MLflow server", asks "what is SageMaker MLflow", or a workflow needs an MLflow backend and none is connected. |
| Diagnose a failing or unhealthy SageMaker endpoint | references/endpoint-diagnostics/ | User reports endpoint errors, latency, inference failures, or a deployment that failed. "What's the status of my endpoint?", "Is my endpoint erroring?", "My endpoint failed — why?", "How many instances are running behind my endpoint?", "Is the latency my model or SageMaker?", "Show me the container logs for my endpoint." NOT for training-job issues, endpoint deletion, scaling changes, or new deployments. |
export AWS_SDK_UA_APP_ID=AWSSkill-SageMaker.This skill can be loaded two ways, and they resolve the skill's own bundled files from different places. Determine how the skill was loaded before reading a reference or running a script:
retrieve_skill tool: The skill is not
installed on the local filesystem. You MUST fetch each reference or script
via retrieve_skill with the file parameter, using its path relative to
the skill root (e.g. file="references/dataset-transformation/overview.md"
or file="references/dataset-transformation/scripts/dataset_split.py"), and
run the script from the returned content. When a reference says to copy a
packaged file into the project, write the returned content unchanged to that
destination. Also fetch every skill module that script imports, so its
imports resolve, and never regenerate a file from context. Do NOT
file_read these paths locally — they do not exist on disk..kiro/skills/aws-ai-ml/ or
~/.claude/skills/aws-ai-ml/): Read and run files from the local skill
directory using relative paths.This distinction applies only to the skill's own packaged files. User data and
session artifacts are always read from and written to the user's working
directory. Never fetch or write customer data through retrieve_skill.
aws/agent-toolkit-for-aws
292edb1
Also appears in
since Sep 6, 2026
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.