Name: tooluniverse-variant-predictor-dms-validation
Rating: 73.6 (1 reviews)
Author: mims-harvard

tooluniverse-variant-predictor-dms-validation

Validate a variant-effect predictor (AlphaMissense, ESM-C SAE, ESM logits, EVE, conservation scores, or any per-variant numeric score) against experimental deep mutational scanning (DMS) data. Computes per-variant predictor scores, splits variants into neutral vs disruptive groups by DMS effect, runs a Mann-Whitney U test on the predictor scores, and sweeps the stratification thresholds for robustness. Use when you need to know whether a predictor's scores track real functional disruption on a specific protein.

Quality

90%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

The canonical home for this skill is tooluniverse-variant-predictor-dms-validation in mims-harvard/ToolUniverse

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A high-quality, predictor-agnostic benchmarking skill: fully executable code, a clear gated workflow with mandatory validation and robustness sweep, and honest limitations. The main gap is progressive disclosure — it is a single ~440-line file with long worked examples that could be split into bundled reference files.

Suggestions

Externalize the long AlphaMissense bin-parsing worked example and the per-predictor scoring tables into a references/ file (e.g. PREDICTOR_SOURCES.md), keeping SKILL.md a lean overview with one-level-deep links.

Trim or move the DeepMind bulk-CSV 'higher-resolution alternative' aside out of the main Step 2 flow so the primary path stays scannable.

Consider a scripts/ helper for the repetitive sweep + MWU loop so the body can reference it instead of inlining the full implementation twice (Step 4 and Step 5).

Dimension	Reasoning	Score
Conciseness	Mostly lean with executable, no-preamble code blocks and tight tables, but the worked AlphaMissense bin-parsing example and the alternative-source boilerplate add length that could be trimmed or externalized; assumes Claude's competence for the most part.	4 / 5
Actionability	Copy-paste-ready executable functions cover every predictor option (SAE drop, AlphaMissense matrix, ESM logits, ESM-2 masked-marginal), plus a complete MWU + sweep implementation with concrete example inputs (KRAS P01116, position 12).	5 / 5
Workflow Clarity	Six-step sequence with an explicitly MANDATORY pre-MWU NaN-coverage sanity gate, a sign-convention double-check, error-recovery guidance ('do not paper over… debug the predictor computation step'), and a robustness sweep — clear checkpoints and feedback loops for the batch/statistical operations.	5 / 5
Progressive Disclosure	Well-organized sections with a cross-reference table pointing to sibling skills/tools at one level, but the SKILL.md is monolithic (no references/ scripts/ assets/ bundle files) and inlines several long worked examples (AlphaMissense parsing, two scoring-path tables) that could be externalized to reference files.	4 / 5
	Total	18 / 20 Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it is concrete, third-person, and explicitly pairs the what with a 'Use when' trigger tailored to a narrow predictor-benchmarking niche. Minor room to add a couple of synonyms (DMS benchmark, predictor validation) for fuller trigger coverage.

Dimension	Reasoning	Score
Specificity	Names many concrete actions: computes per-variant predictor scores, stratifies variants into neutral/disruptive groups by DMS effect, runs a one-sided Mann-Whitney U test, and sweeps stratification thresholds for robustness — comprehensive and concrete.	5 / 5
Completeness	Explicitly states both what it does (per-variant scores, stratification, MWU, threshold sweep) and when to use it ('Use when you need to know whether a predictor's scores track real functional disruption on a specific protein').	5 / 5
Trigger Term Quality	Includes strong natural phrases ('does it actually correlate with experimental DMS measurements', 'variant-effect predictor', 'deep mutational scanning') but omits a few synonyms / abbreviation expansions users might say (e.g. 'DMS benchmark', 'predictor validation').	4 / 5
Distinctiveness Conflict Risk	The DMS-validation niche is narrow and the trigger (predictor-vs-DMS correlation on a specific protein) is distinct, with minimal overlap risk against the sibling interpretation skills it names.	5 / 5
	Total	19 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: mims-harvard/ToolUniverse
Path: plugins/tooluniverse/skills/tooluniverse-variant-predictor-dms-validation/SKILL.md
Commit: 1b02920

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.