scholar-evaluation

Implements the ScholarEval framework to evaluate scholarly documents; trigger when the user provides a PDF/DOCX/TXT file or pasted text and requests critique, scoring, or quality assessment.

Quality

72%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./scientific-skills/Evidence Insight/scholar-evaluation/SKILL.md

Quality

Content

61%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is well-structured and points cleanly to a single reference and two scripts, with mostly executable guidance. The main weaknesses are repetition of the extraction command, inlined content that also lives in the reference, and missing validation checkpoints in the evaluation workflow.

Suggestions

Consolidate the extract_text.py invocation so it appears once; the 'File ingestion protocol' section can defer to Example A or vice versa.

Add validation checkpoints to the workflow, e.g. verify extracted text is non-empty before scoring and confirm scores.json contains all 8 dimension keys before running calculate_scores.py.

Move the full 1-5 scoring scale and 8-dimension list to references/evaluation_framework.md and keep only a brief pointer in SKILL.md to reduce inline duplication.

Dimension	Reasoning	Score
Conciseness	Mostly efficient with well-organized sections, but the extract_text.py invocation is repeated in both 'Example Usage' and 'File ingestion protocol', and the 1-5 scale and 8-dimension list are inlined despite also living in the reference.	3 / 5
Actionability	Provides concrete, executable commands ('python scripts/extract_text.py "paper.pdf"', 'python scripts/calculate_scores.py --scores scores.json') and a sample scores.json, though step 4 of Example A ('Use the extracted text plus the rubric to generate the evaluation report') is guidance rather than executable code.	4 / 5
Workflow Clarity	A clear 4-step sequence exists for file evaluation, but there are no validation checkpoints (e.g., confirming extraction produced usable text, or that scores.json is well-formed before computing), leaving checkpoints implicit.	3 / 5
Progressive Disclosure	Good section structure with a clearly signaled one-level-deep reference ('references/evaluation_framework.md') for detailed criteria and bundled scripts; some content duplicated inline (dimensions list, scoring scale) keeps it just short of ideal.	4 / 5
	Total	14 / 20 Passed

Description

83%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description that clearly answers both what the skill does and when to use it, with concrete trigger terms and file extensions. Minor gains are possible from enumerating more specific actions and reducing overlap with generic document skills.

Dimension	Reasoning	Score
Specificity	Names the domain ('ScholarEval framework to evaluate scholarly documents') plus several concrete actions ('critique, scoring, or quality assessment'), but coverage of actions is broad rather than enumerated in full.	4 / 5
Completeness	Explicitly states both what it does ('Implements the ScholarEval framework to evaluate scholarly documents') and when to trigger ('when the user provides a PDF/DOCX/TXT file or pasted text and requests critique, scoring, or quality assessment'), with concrete trigger phrases.	5 / 5
Trigger Term Quality	Includes natural request terms ('critique, scoring, or quality assessment') and file extensions ('PDF/DOCX/TXT'), though it misses a few common synonyms like 'review my paper'.	4 / 5
Distinctiveness Conflict Risk	The ScholarEval / scholarly-critique framing carves a clear niche, but the generic PDF/DOCX/TXT ingestion language creates minor overlap with general document-handling skills.	4 / 5
	Total	17 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: aipoch/medical-research-skills
Path: scientific-skills/Evidence Insight/scholar-evaluation/SKILL.md
Commit: 63c61d3

Reviewed: 5 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.