CtrlK
BlogDocsLog inGet started
Tessl Logo

model-monitor

Configure TrustyAI model monitoring for bias detection and data drift on deployed InferenceServices. Use when: - "Monitor my model for bias" - "Set up drift detection on my inference endpoint" - "Configure TrustyAI for my deployed model" - "Check if my model has fairness issues" - "I need SPD / DIR metrics for my model" Handles TrustyAIService deployment, bias metric configuration (SPD, DIR), drift metric configuration (MeanShift, FourierMMD, KS-Test, Jensen-Shannon), threshold tuning, and monitoring validation. NOT for deploying models (use /model-deploy first). NOT for input/output content safety guardrails (use /guardrails-config). NOT for infrastructure-level observability (use /ai-observability).

75

Quality

94%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

96%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured operational skill: concrete tool calls with exact parameters, a clearly sequenced 8-step workflow with validation and human-in-the-loop checkpoints, and appropriate deferral of CRD/ConfigMap schema detail to a substantive one-level reference. Remaining improvements are organizational — deduplicating the tool inventory, moving inline troubleshooting to common-issues.md, and wiring up or removing the two unreferenced bundle files.

Suggestions

Consolidate the MCP tool list into the Prerequisites section only and have the Dependencies section reference it once, removing the duplicated inventory.

Move the inline 'Issue 1-3' troubleshooting details into references/common-issues.md, keeping only one-line pointers in SKILL.md.

Either reference known-model-profiles.md and live-doc-lookup.md from the body where relevant, or remove them from the bundle so every shipped file is discoverable.

DimensionReasoningScore

Conciseness

The body is largely lean imperative guidance (tool names, parameters, error handling) with no explanation of concepts Claude already knows. Minor over-explanation could be trimmed: the Step 3 'Document Consultation / Output to user' boilerplate, the MCP tool inventory repeated between Prerequisites and the Dependencies section, and inline Issues 1-3 alongside an existing pointer to common-issues.md. This is the 'efficient; minor instances that could be trimmed' anchor rather than the fully lean anchor at 5.

4 / 5

Actionability

Every step specifies the exact MCP tool plus concrete parameters (apiVersion: "apiextensions.k8s.io/v1", kind: CustomResourceDefinition; labelSelector: "app.kubernetes.io/name=trustyai-service") and copy-paste-ready PromQL such as 'trustyai_spd{model="[isvc-name]"}'. Manifest construction is fully specified via the one-level trustyai-metrics-reference.md, which contains complete CRD, ConfigMap, and threshold schemas, covering the common cases end to end.

5 / 5

Workflow Clarity

Eight clearly sequenced steps with per-step error handling, explicit 'WAIT for user decision' checkpoints, a dedicated validation step (Step 7 checks pods then verifies metrics are flowing), and feedback loops for failure recovery (pod diagnostics via pods_log/events_list, NaN/insufficient-data handling). The HITL summary section consolidates checkpoints and forbids auto-deletion, matching the top anchor including validation for risky operations.

5 / 5

Progressive Disclosure

Good structure: the body is an overview with well-signaled, one-level-deep markdown links (trustyai-metrics-reference.md, skill-conventions.md, openshift-fallback-templates.md, common-issues.md), and all referenced paths exist in references/ with real content in the metrics reference. Minor gaps keep it below the top anchor: the MCP tool list is duplicated between Prerequisites and Dependencies, Issues 1-3 are inlined despite the common-issues.md pointer, and two bundle files (known-model-profiles.md, live-doc-lookup.md) are never referenced from the body.

4 / 5

Total

18

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: concrete third-person capability enumeration, natural quoted trigger phrases, and explicit NOT-for boundaries that steer away from sibling skills. The only gap is a handful of missing natural trigger variations, which keeps trigger term quality at 4 rather than 5.

DimensionReasoningScore

Specificity

The description lists multiple concrete actions in third person: 'Handles TrustyAIService deployment, bias metric configuration (SPD, DIR), drift metric configuration (MeanShift, FourierMMD, KS-Test, Jensen-Shannon), threshold tuning, and monitoring validation.' Named metrics and operations give comprehensive, non-generic coverage; it clearly exceeds the 'several specific actions with minor gaps' anchor at 4.

5 / 5

Completeness

It explicitly answers both: what ('Configure TrustyAI model monitoring for bias detection and data drift on deployed InferenceServices' plus the 'Handles...' enumeration) and when (a five-item 'Use when:' list of concrete trigger phrases). This matches the top anchor exactly; the 'when' is not merely implied.

5 / 5

Trigger Term Quality

Quoted triggers such as 'Monitor my model for bias', 'Set up drift detection on my inference endpoint', and 'Check if my model has fairness issues' are natural user phrasings with good synonym coverage (bias/fairness, drift/SPD/DIR). A few natural variations users might say are still missing (e.g., 'is my model biased', 'fairness audit'), so it sits at 'good keyword coverage; a few natural terms missing' rather than the fully comprehensive anchor at 5.

4 / 5

Distinctiveness Conflict Risk

The niche is distinct (TrustyAI bias/drift monitoring on InferenceServices) and boundary risk is actively minimized with explicit negative triggers: 'NOT for deploying models (use /model-deploy first)', 'NOT for input/output content safety guardrails', 'NOT for infrastructure-level observability'. Minimal conflict risk, matching the top anchor.

5 / 5

Total

19

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository
RHEcosystemAppEng/agentic-plugins
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.