bedrock-agentcore-evaluations

Amazon Bedrock AgentCore Evaluations for testing and monitoring AI agent quality. 13 built-in evaluators plus custom LLM-as-Judge patterns. Use when testing agents, monitoring production quality, setting up alerts, or validating agent behavior.

Quality

71%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./skills/bedrock-agentcore-evaluations/SKILL.md

Quality

Content

53%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is rich with executable boto3 examples and clear operation sections, but it over-inlines reference-grade material, references missing bundle files, and omits validation feedback loops in its batch evaluation workflows.

Suggestions

Create the three referenced files (references/evaluator-reference.md, test-scenarios.md, alerting-patterns.md) and move the bulk API/evaluator code there, keeping SKILL.md as an overview with one-line pointers.

Add an explicit validate-then-retry feedback loop to the batch/on-demand evaluation operation (check failing scores, diagnose, re-run) so destructive/batch workflows clear the workflow-clarity cap.

Tighten the Overview/Key Principles/Quality Targets section and drop restatements Claude already knows about agent testing to improve conciseness.

Dimension	Reasoning	Score
Conciseness	Mostly efficient with executable code blocks, but includes conceptual restatement Claude already knows (e.g., the Purpose/Key Principles overview, Quality Targets block) and could be tightened; some sections repeat evaluator list context already in the table.	3 / 5
Actionability	Provides mostly executable, copy-paste-ready boto3 code across all five operations with concrete parameters and thresholds; minor gaps include undefined helper functions like run_evaluation() and variables (agent_arn, helpfulness_evaluator_id) used without prior assignment.	4 / 5
Workflow Clarity	Operations are sequenced and labeled with time/automation, but batch/destructive-style evaluation workflows lack validation checkpoints — e.g., Operation 3 batch evaluation runs scenarios and aggregates scores with no validate/fix/retry feedback loop, which caps this dimension at 3.	3 / 5
Progressive Disclosure	The body inlines a large amount of content that belongs in separate files (full API reference code, three custom-evaluator examples, dashboard JSON) while listing three references/ files that do not exist in the bundle, so the structure is broken rather than well-navigable.	2 / 5
	Total	12 / 20 Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, specific description that clearly states both what the skill does and when to use it, with concrete trigger phrases. It is mostly distinct within the AgentCore family, with only minor overlap risk against sibling skills.

Dimension	Reasoning	Score
Specificity	Names the domain (AgentCore Evaluations) and lists multiple concrete capabilities: '13 built-in evaluators', 'custom LLM-as-Judge patterns', 'testing and monitoring', 'setting up alerts', and 'validating agent behavior' — comprehensive coverage.	5 / 5
Completeness	Clearly answers 'what' (AgentCore Evaluations for testing/monitoring agent quality, 13 built-in evaluators, custom LLM-as-Judge) and 'when' (explicit 'Use when testing agents, monitoring production quality, setting up alerts, or validating agent behavior') with concrete trigger phrases.	5 / 5
Trigger Term Quality	Includes natural trigger phrases users would say ('testing agents', 'monitoring production quality', 'setting up alerts', 'validating agent behavior'), but lacks common synonyms and the product acronym variants; a few natural terms missing.	4 / 5
Distinctiveness Conflict Risk	The AgentCore Evaluations niche is fairly distinct with specific triggers, and 'When NOT to Use' guidance (in the body) reduces conflict, but the description alone has minor overlap risk with adjacent bedrock-agentcore skills.	4 / 5
	Total	18 / 20 Passed

Validation

81%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

Criteria	Description	Result
skill_md_line_count	SKILL.md is long (678 lines); consider splitting into references/ and linking	Warning
allowed_tools_field	'allowed-tools' contains unusual tool name(s)	Warning
referenced_paths_exist	Referenced path issues: 3 missing	Warning

	Total	13 / 16 Passed

Repository: fernandezbaptiste/Skrillz
Path: skills/bedrock-agentcore-evaluations/SKILL.md
Commit: 93ed392

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.