CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

probast-quality-assessment-for-prediction-model-studies

Assess bias in medical prediction model studies using PROBAST tool. Use when user wants to evaluate the quality or risk of bias of a medical paper (text or PDF).

Quality

71%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./scientific-skills/Data Analysis/probast-quality-assessment-for-prediction-model-studies/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

52%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The core workflow is well designed — a clear four-step sequence correctly delegating detail to a real, well-sectioned reference file and an existing helper script — but roughly a third of the body is meta-boilerplate ('Key Features', 'Implementation Details', generic 'When to Use' bullets) that wastes tokens without instructing. The end-to-end invocation of the PDF extraction script is also never shown with real arguments.

Suggestions

Delete or drastically trim the boilerplate sections ('Key Features', 'Implementation Details', 'Dependencies') and the generic 'When to Use' bullets (e.g., 'needs a packaged method instead of ad-hoc freeform output'), keeping only the task-specific trigger line.

Show a concrete end-to-end command for the helper script in the 'Helper Scripts' section, e.g. `python scripts/extract_pdf.py paper.pdf --output extracted.txt`, and drop or relativize the non-portable '20260316/scientific-skills/...' path in Example Usage.

Add an explicit validation checkpoint in Step 4, such as verifying the generated JSON parses and matches the study_risk_of_bias_schema before returning it, to close the workflow's only validation gap.

DimensionReasoningScore

Conciseness

Several sections are padded boilerplate that adds nothing Claude does not already know: 'Use this skill when a data analytics task needs a packaged method instead of ad-hoc freeform output', 'Structured execution path designed to keep outputs consistent and reviewable', 'See `## Workflow` above for related details', and 'Third-party packages: not explicitly version-pinned in this skill package. Add pinned versions if this skill needs stricter environment control.' The filler spans the 'When to Use', 'Key Features', 'Dependencies', and 'Implementation Details' sections, making this noticeably verbose (anchor 2) rather than mostly efficient (anchor 3).

2 / 5

Actionability

The Workflow steps are concrete (each names its prompt section in references/probast_prompts.md, all verified to exist) and the Example Usage block has runnable commands, but the body never shows a complete invocation of extract_pdf.py with an actual paper, the example cd's into a non-portable dated directory ('20260316/scientific-skills/...'), and the Helper Scripts section only says to 'use extract_pdf.py' without the actual command line. Concrete guidance exists but key execution details are incomplete, matching anchor 3; it is not 4 because the main script's usage is never demonstrated end-to-end in the body.

3 / 5

Workflow Clarity

The four-step assessment sequence is clearly ordered with inputs and outputs stated per step ('Input: The risk ratings (Low/High/Unclear) from Domains 1-4', 'Output Format: A JSON object matching the study_risk_of_bias_schema'), and the run plan includes a review step. Validation is only implicit (schema conformance) rather than an explicit checkpoint, so anchor 4 fits better than 5; this is not a destructive or batch operation, so no cap applies.

4 / 5

Progressive Disclosure

The body acts as an overview with one-level-deep references: every workflow step points to a named section in references/probast_prompts.md (all seven cited sections verified present in that file) and the helper script scripts/extract_pdf.py exists. Structure is good, but the 'Implementation Details' section contains a circular pointer ('See `## Workflow` above for related details') and generic bullets that muddy navigation, keeping this at anchor 4 rather than the cleanly-split anchor 5.

4 / 5

Total

13

/

20

Passed

Description

78%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, compact description that clearly states what the skill does and when to use it, with a genuinely explicit 'Use when' trigger clause. Its main weaknesses are limited action coverage and trigger vocabulary that misses common appraisal synonyms, creating minor overlap risk with adjacent quality-assessment tools.

DimensionReasoningScore

Specificity

The description names the domain ('medical prediction model studies', 'PROBAST tool') and one to two concrete actions ('Assess bias', 'evaluate the quality or risk of bias'), but does not enumerate further capabilities such as the four assessed domains or the JSON report output. It matches the anchor for naming the domain with 1-2 concrete actions, not the several-actions anchor at 4.

3 / 5

Completeness

It explicitly answers both 'what' ('Assess bias in medical prediction model studies using PROBAST tool') and 'when' ('Use when user wants to evaluate the quality or risk of bias of a medical paper (text or PDF)') with concrete trigger phrases, mirroring the anchor-5 example structure. Not 4, because the 'when' clause is fully explicit rather than merely adequate.

5 / 5

Trigger Term Quality

It includes natural phrases users would say — 'risk of bias', 'quality', 'medical paper', 'text or PDF' — giving good keyword coverage, but misses common synonyms such as 'critical appraisal', 'clinical study', or 'prediction model validation'. Good coverage with a few natural terms missing matches anchor 4; the comprehensive-synonyms anchor at 5 is not met.

4 / 5

Distinctiveness Conflict Risk

'PROBAST' and 'prediction model studies' define a clear niche with distinct triggers, but the broader phrase 'evaluate the quality... of a medical paper' could overlap with other evidence-appraisal skills (e.g., QUADAS, ROBINS, generic paper review). Mostly distinct with minor overlap risk matches anchor 4 rather than the minimal-conflict anchor at 5.

4 / 5

Total

16

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

aipoch/medical-research-skills

Path

scientific-skills/Data Analysis/probast-quality-assessment-for-prediction-model-studies/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.