probast-quality-assessment-for-prediction-model-studies

Assess bias in medical prediction model studies using PROBAST tool. Use when user wants to evaluate the quality or risk of bias of a medical paper (text or PDF).

Quality

69%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./scientific-skills/Data Analysis/probast-quality-assessment-for-prediction-model-studies/SKILL.md

Quality

Content

60%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body provides a clear, actionable four-step workflow that correctly delegates detail to a real prompts reference and an extraction script. Its main weakness is verbosity from generic templated boilerplate sections that could be trimmed.

Suggestions

Remove or condense the generic "When to Use", "Key Features", and "Implementation Details" bullets that duplicate the description or state generic process truisms.

Add an explicit final validation step that checks the generated JSON against study_risk_of_bias_schema before returning it.

Consolidate the "Example Usage" compile/help commands with the Helper Scripts section to reduce redundancy.

Dimension	Reasoning	Score
Conciseness	Several padded, templated sections add little value: the six "When to Use" bullets, "Key Features" duplicating the description, and generic "Implementation Details" lines like "validate the request, choose the packaged workflow, and produce a bounded deliverable" are noticeably verbose boilerplate.	2 / 5
Actionability	Each workflow step points to a specific named prompt in references/probast_prompts.md and the PDF step gives a concrete executable command ("python scripts/extract_pdf.py"), with only minor gaps since the prompt contents live in the bundle rather than inline.	4 / 5
Workflow Clarity	The four-step sequence (metadata, parallel domain assessment, overall risk, JSON output) is clearly ordered with per-step inputs and prompt references; the only gap is a missing explicit validation of the final JSON against study_risk_of_bias_schema.	4 / 5
Progressive Disclosure	SKILL.md is an overview pointing one level deep to real bundle files (references/probast_prompts.md and scripts/extract_pdf.py) that are clearly signaled, though redundant boilerplate sections leave minor organization gaps.	4 / 5
	Total	14 / 20 Passed

Description

78%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description clearly answers both what the skill does and when to use it, with a well-defined PROBAST niche and natural trigger terms. It is slightly limited in action specificity and has minor overlap with the broader family of risk-of-bias tools.

Suggestions

Expand the 'what' with one or two more concrete actions (e.g., 'extracts study metadata and produces a structured risk-of-bias JSON report') to lift specificity.

Add a few synonymous trigger terms (e.g., 'clinical prediction model', 'validation study', 'model performance') to broaden natural keyword coverage.

Dimension	Reasoning	Score
Specificity	"Assess bias in medical prediction model studies using PROBAST tool" names the domain and one concrete action (assess bias via PROBAST) but does not enumerate multiple specific actions, matching the anchor that lists 1-2 concrete actions without comprehensive coverage.	3 / 5
Completeness	It explicitly states both what ("Assess bias in medical prediction model studies using PROBAST tool") and when ("Use when user wants to evaluate the quality or risk of bias of a medical paper (text or PDF)") with concrete trigger phrases, matching the top anchor.	5 / 5
Trigger Term Quality	Natural phrases a user would say appear ("evaluate the quality or risk of bias of a medical paper", "PDF", "PROBAST"), giving good keyword coverage, though a few synonyms like "clinical prediction model" or "validation study" are missing.	4 / 5
Distinctiveness Conflict Risk	The PROBAST + prediction-model niche is mostly distinct with clear triggers, but "risk of bias" terminology creates minor overlap risk with related quality-assessment skills (ROB-2, ROBINS-I, QUADAS-2).	4 / 5
	Total	16 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: aipoch/medical-research-skills
Path: scientific-skills/Data Analysis/probast-quality-assessment-for-prediction-model-studies/SKILL.md
Commit: 63c61d3

Reviewed: 8 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.