backtest-diagnose

Diagnose failed or underperforming backtests, locate the root cause, and fix the issue

Quality

87%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

92%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is a tight, highly actionable diagnostic playbook with a clear workflow and explicit validation checkpoints. Its only notable gap is progressive disclosure: everything is inline in one file with no split-out references for the sizable taxonomy and checklists.

Suggestions

Move the Error Taxonomy and/or Hard-Gate Checklist into a one-level-deep reference file (e.g. references/error-taxonomy.md) and link to it from SKILL.md to improve progressive disclosure.

Consider separating the `action_items` writing rules and examples into a reference so the main body stays a concise overview.

Dimension	Reasoning	Score
Conciseness	The body is lean and well-organized into taxonomy tables, checklists, and principles with no padding or explanation of basic concepts Claude already knows; not score 2 because there is no unnecessary explanation to tighten.	3 / 3
Actionability	Provides fully concrete, executable guidance — `bash("pip install xxx")`, `read_file`/`edit_file` calls, a copy-paste AST validation command, and specific action_item examples with line numbers; not score 2 because the instructions are complete and executable rather than pseudocode.	3 / 3
Workflow Clarity	A clear 5-step diagnostic workflow is paired with a hard-gate checklist and post-fix validation (AST parse + rerun backtest) plus an explicit fix-rerun-verify feedback loop and a 3-iteration cap; not score 2 because validation checkpoints are explicit, not implicit.	3 / 3
Progressive Disclosure	It is a well-organized single file with no bundle files, but all reference material (error taxonomy, hard-gate checklist, action_items rules) lives inline in an ~80-line document that could be split into a one-level-deep reference; not score 3 because there are no signaled external references and the simple-skill exception applies only to skills under 50 lines, and not score 1 because the content is clearly sectioned rather than a monolithic wall.	2 / 3
	Total	11 / 12 Passed

Description

82%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific and clearly niche, naming three concrete diagnostic actions with natural trigger terms. Its main weakness is the absence of an explicit 'Use when...' clause, which leaves the trigger context implied rather than stated.

Suggestions

Add an explicit 'when' clause, e.g. 'Use when a backtest failed, raised an error, or produced poor/underperforming results.'

Include common user phrasings such as 'backtest error', 'strategy not profitable', or 'no trades' to broaden natural trigger coverage.

Dimension	Reasoning	Score
Specificity	Lists three concrete actions — 'Diagnose failed or underperforming backtests', 'locate the root cause', and 'fix the issue' — matching the multiple-specific-actions anchor; not score 2 because it is comprehensive rather than naming only some actions.	3 / 3
Completeness	It clearly answers 'what' (diagnose, locate root cause, fix) but lacks any 'Use when...' clause or equivalent explicit trigger guidance, which per the guidelines caps completeness at 2; not score 1 because the 'what' is explicit, and not score 3 because the 'when' is only implied.	2 / 3
Trigger Term Quality	Natural terms a user would say are present — 'backtest', 'failed', 'underperforming', 'root cause', 'fix the issue' — giving good coverage; not score 2 because common user phrasings like 'my backtest failed' are directly represented.	3 / 3
Distinctiveness Conflict Risk	'backtest' is a clear, narrow niche with distinct triggers unlikely to conflict with other skills; not score 2 because the domain is specific rather than broadly overlapping.	3 / 3
	Total	11 / 12 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: HKUDS/Vibe-Trading
Path: agent/src/skills/backtest-diagnose/SKILL.md
Commit: 8643fcd

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.