aws-ai-ml

Selects, deploys, and customizes AI models on Amazon SageMaker. Fine-tuning (SFT, DPO, RLVR, RLAIF), model selection, dataset preparation, evaluation, deployment to SageMaker endpoints or Bedrock, and endpoint diagnostics. Covers the full lifecycle from planning through production. Use when fine-tuning models on SageMaker, choosing/selecting which base model to customize or fine-tune from SageMaker Hub, finding a model to deploy without fine-tuning, transforming datasets for training, checking data readiness, evaluating model quality, deploying to endpoints, setting up IAM roles and S3 buckets for training jobs, or managing a SageMaker Managed MLflow app. Also use to check endpoint health, diagnose failures, debug latency or errors, or view container logs and CloudWatch metrics. Covers Serverless Model Customization, Nova and OSS deployment paths, and PySDK v3 usage. NOT for Ground Truth labeling, Feature Store, or general-purpose AWS infrastructure.

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

The canonical home for this skill is aws-ai-ml in aws/agent-toolkit-for-aws

Quality

Content

72%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured routing skill body: concise, action-oriented routing with all references verified as real one-level-deep folders. Its weakness is workflow clarity — the body sequences routing but omits explicit validation checkpoints and feedback loops, capping that dimension at 3.

Suggestions

Add an explicit ordered workflow (e.g., plan → select model → prepare dataset → fine-tune → evaluate → deploy) with validation checkpoints between stages so the lifecycle sequence is unambiguous.

For destructive or batch operations referenced by the skill (endpoint deletion, dataset transformation, training job launch), add a validate-then-proceed feedback loop in the relevant routing rows.

Tighten the longer "When to use" cells (e.g., the model-selection and endpoint-diagnostics rows) to remove restated routing advice and keep the routing table scannable.

Dimension	Reasoning	Score
Conciseness	The body is a lean routing table plus three short rules with no concept over-explanation, but a few "When to use" cells carry minor padding (e.g., reiterated routing advice) that could be trimmed — efficient with minor over-explanation, matching score 4 rather than 5.	4 / 5
Actionability	Each routing row pairs a concrete trigger with a real reference link, and some rows name specific artifacts ("select-for-deployment.md and its scripts", the AWS_SDK_UA_APP_ID export); guidance is mostly executable though the body itself delegates the actual code to references, leaving minor gaps versus fully copy-paste-ready inline examples.	4 / 5
Workflow Clarity	Routing provides partial sequencing ("Load this reference FIRST", "Always co-activate with other intents", "route to use-case-specification first") but there is no explicit validate→fix→retry feedback loop or checkpointed multi-step workflow in the body, so it sits at the score-3 anchor with sequence present but checkpoints implicit.	3 / 5
Progressive Disclosure	The body is a clear overview with a well-signaled routing table whose 13 reference links each map to a real one-level-deep reference folder, plus a dedicated Progressive disclosure rule — content is appropriately split and easy to navigate, matching the score-5 anchor.	5 / 5
	Total	16 / 20 Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, specific description with concrete actions, natural trigger phrases, and an explicit use-when clause plus a negative-scope boundary. It cleanly satisfies the what/when pair and is highly distinct from generic AWS skills.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions across the full lifecycle — "Selects, deploys, and customizes", "Fine-tuning (SFT, DPO, RLVR, RLAIF)", "transforming datasets", "evaluating model quality", "diagnose latency or errors" — with comprehensive coverage, matching the score-5 anchor.	5 / 5
Completeness	Explicitly answers both what ("Selects, deploys, and customizes AI models on Amazon SageMaker") and when ("Use when fine-tuning models on SageMaker… Also use to check endpoint health…") with concrete trigger phrases, matching the score-5 anchor.	5 / 5
Trigger Term Quality	Comprehensive natural triggers and synonyms: "fine-tune my model", "start training", "is my dataset okay", "deploy my model", "choosing/selecting which base model", "check endpoint health" — covering the common phrasings a user would actually say.	5 / 5
Distinctiveness Conflict Risk	Occupies a clear SageMaker customization niche and adds an explicit negative boundary ("NOT for Ground Truth labeling, Feature Store, or general-purpose AWS infrastructure"), minimizing conflict risk with adjacent skills.	5 / 5
	Total	20 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: aws/agent-toolkit-for-aws
Path: plugins/aws-core/skills/aws-ai-ml/SKILL.md
Commit: 68d9e85

Reviewed: 3 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.