a-b-test-config-creator

A B Test Config Creator - Auto-activating skill for ML Deployment. Triggers on: a b test config creator, a b test config creator Part of the ML Deployment skill category.

1.02x

Quality

Does it follow best practices?

Impact

100%

1.02x

Average score across 3 eval scenarios

Securityby

Passed

No known issues

Optimize this skill with Tessl

npx tessl skill review --optimize ./planned-skills/generated/08-ml-deployment/a-b-test-config-creator/SKILL.md

Evaluation results

100%

Model Version Rollout: Traffic Split Configuration

Production-ready traffic split config

Criteria

Without context

With context

Config file created

100%

Two variants defined

100%

Traffic percentages present

100%

Percentages sum to 100

100%

No placeholder values

100%

Model endpoint or path

100%

Validation report created

100%

Validation lists checks performed

100%

Production-ready fields

100%

Config is valid YAML or JSON

100%

Without context: $0.2074 · 56s · 13 turns · 14 in / 3,054 out tokens

With context: $0.3991 · 1m 37s · 24 turns · 56 in / 5,049 out tokens

100%

Monitoring Setup for Live A/B Model Comparison

A/B test config with monitoring and validation

Criteria

Without context

With context

Config file present

100%

Monitoring section included

100%

At least 2 metrics defined

100%

Alert threshold present

100%

Success criterion defined

100%

Step-by-step plan created

100%

Plan has sequential steps

100%

Plan covers monitoring phase

100%

Both variants described

100%

Config is valid syntax

100%

No placeholder values

100%

Without context: $0.4905 · 2m 44s · 18 turns · 18 in / 9,607 out tokens

With context: $0.5354 · 2m 54s · 25 turns · 288 in / 8,804 out tokens

100%

MLOps Pipeline Integration for Gradual Model Promotion

MLOps pipeline A/B test configuration

Criteria

Without context

With context

Pipeline config file present

100%

Two variants defined

100%

Traffic allocation specified

100%

Production serving parameter

100%

MLOps pipeline field

100%

No placeholder values

25%

100%

Config is valid syntax

100%

Deployment checklist created

100%

Checklist is binary items

100%

Validation step in checklist

100%

Monitoring or metrics addressed

100%

Without context: $0.2976 · 1m 32s · 15 turns · 15 in / 4,830 out tokens

With context: $0.3462 · 1m 40s · 21 turns · 184 in / 5,384 out tokens

Repository: jeremylongshore/claude-code-plugins-plus-skills
Commit: f17dd51

Evaluated: 18 days ago
Agent: Claude Code
Model: Claude Sonnet 4.6

Table of Contents

Model Version Rollout: Traffic Split Configuration Monitoring Setup for Live A/B Model Comparison MLOps Pipeline Integration for Gradual Model Promotion

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.