tooluniverse-variant-predictor-dms-validation

Validate a variant-effect predictor (AlphaMissense, ESM-C SAE, ESM logits, EVE, conservation scores, or any per-variant numeric score) against experimental deep mutational scanning (DMS) data. Computes per-variant predictor scores, splits variants into neutral vs disruptive groups by DMS effect, runs a Mann-Whitney U test on the predictor scores, and sweeps the stratification thresholds for robustness. Use when you need to know whether a predictor's scores track real functional disruption on a specific protein.

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

77%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with a well-sequenced, validated workflow and executable code, but it is a monolithic ~440-line document that inlines multiple predictor implementations and rationale that could be trimmed or split into reference files. Conciseness and progressive disclosure are the weakest dimensions.

Suggestions

Move the full per-predictor implementations (options A–D, ~250 lines) into reference files under references/ (e.g. predictors/sae.md, predictors/alphamissense.md) and keep SKILL.md as an overview with one-level-deep links — this would lift both progressive_disclosure and conciseness.

Trim the rationale paragraphs (Spearman-vs-MWU discussion, DeepMind CSV digression, the 'eval that motivated this skill' anecdote) to a single line each; Claude already understands these tradeoffs.

Consolidate the predictor option tables and cross-reference table so the DMS-matrix contract and tool signatures appear once rather than being restated across options.

Dimension	Reasoning	Score
Conciseness	At ~440 lines the body includes several full Python implementations, multiple predictor options, and rationale paragraphs (e.g. Spearman-vs-MWU discussion, DeepMind CSV digression) that re-explain analysis choices Claude already grasps. Mostly efficient but padded enough to keep it from 4; not severe enough for 2.	3 / 5
Actionability	Provides copy-paste-ready, executable Python with real tool signatures and a concrete worked example (KRAS P01116) across the common cases, e.g. `sae_drop_per_variant`, `am_per_variant_matrix`, and the MWU call. Option C is thinner but the common cases are fully covered.	5 / 5
Workflow Clarity	Steps 1–6 are clearly sequenced with an explicit MANDATORY validation gate (Step 3.5 NaN coverage check) and a sign-convention double-check, plus error-recovery guidance ('do not paper over ... debug the predictor computation step'). Clear checkpoints and feedback loops for a statistical batch operation.	5 / 5
Progressive Disclosure	No references/scripts/assets bundle exists, so the skill is a single ~440-line SKILL.md with multiple full per-predictor implementations inlined. Section headers give some structure, but content that could live in separate files is all inline with no one-level-deep references. Not 2 because sections are organized; not 4 because no external files split the bulk.	3 / 5
	Total	16 / 20 Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, complete, and distinct: it names the domain, enumerates concrete workflow actions, and gives an explicit 'Use when' trigger. The only minor weakness is a conceptually-phrased trigger that could include more natural synonyms.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions — 'Computes per-variant predictor scores, splits variants into neutral vs disruptive groups ... runs a Mann-Whitney U test ... and sweeps the stratification thresholds' — covering the workflow comprehensively. Not the 4 anchor because coverage is complete rather than having minor gaps.	5 / 5
Completeness	Clearly answers both what ('Validate a variant-effect predictor ... Computes ... splits ... runs a Mann-Whitney U test ... sweeps the stratification thresholds') and when ('Use when you need to know whether a predictor's scores track real functional disruption'). Explicit trigger clause present.	5 / 5
Trigger Term Quality	The trigger 'Use when you need to know whether a predictor's scores track real functional disruption on a specific protein' is natural, but it is conceptually phrased and lacks common synonyms or file-extension-style keywords. Above 3 because it is a credible natural phrase a user would say; below 5 because keyword coverage is not comprehensive.	4 / 5
Distinctiveness Conflict Risk	Occupies a clear niche (benchmarking per-variant numeric predictors against DMS data) with named predictor types and a specific statistical framework, making overlap with adjacent skills minimal.	5 / 5
	Total	19 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: mims-harvard/ToolUniverse
Path: plugin/skills/tooluniverse-variant-predictor-dms-validation/SKILL.md
Commit: 1b02920

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.