semantic-consistency-auditor

Use semantic consistency auditor for academic writing workflows that need structured execution, explicit assumptions, and clear output boundaries.

Quality

39%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./scientific-skills/Academic Writing/semantic-consistency-auditor/SKILL.md

Quality

Content

53%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body pairs genuine, actionable domain content (algorithms, executable usage, config, I/O formats) with a generic templated scaffold and several stub cross-references that pad token budget. Workflow steps are coherent but lack concrete validation checkpoints for batch evaluation, and detailed reference material is largely inlined rather than progressively disclosed.

Suggestions

Remove the duplicated description under 'When to Use' and the three stub 'See `## X` above for related details' cross-references that point at existing sections without adding information.

Replace the generic Workflow steps with the real evaluation flow (load BERTScore/COMET models, score the candidate/gold pair, compare against configured thresholds, emit summary) and add an explicit validation checkpoint for batch runs.

Move the full Configuration YAML and Input/Output JSON schemas into a one-level-deep reference file (e.g. references/formats.md) and link to it from SKILL.md to improve progressive disclosure.

Dimension	Reasoning	Score
Conciseness	Several padded sections appear: a duplicated description under 'When to Use', three stub cross-references ('See `## Prerequisites` above...'), and generic boilerplate ('Execution model: validate the request, choose the packaged workflow...') that add nothing Claude does not already know.	2 / 5
Actionability	Provides mostly executable guidance including concrete CLI commands, a full Python API example, config YAML, and input/output JSON formats; minor gaps include the unverified `from semantic_consistency_auditor import ...` path and placeholder `'...'` strings.	4 / 5
Workflow Clarity	A coherent 5-step sequence is present with a fallback path described, but the steps are generic process advice rather than the real evaluation flow, and validation checkpoints for the batch/destructive evaluation operations are missing or implicit.	3 / 5
Progressive Disclosure	The body is well-sectioned and the existing reference (references/audit-reference.md) and script (scripts/main.py) are real and clearly signaled via markdown links, though much detailed material (full config, I/O JSON schemas) is inlined rather than split into one-level-deep reference files.	4 / 5
	Total	13 / 20 Passed

Description

25%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is a templated, circular sentence that restates the skill name with generic process buzzwords and never reveals the skill's real capability (BERTScore/COMET semantic consistency evaluation). It lacks any explicit 'Use when' trigger and carries high conflict risk with other structured-writing skills.

Suggestions

Rewrite the description to state the concrete capability, e.g. 'Evaluates semantic consistency between AI-generated clinical notes and expert gold standards using BERTScore and COMET, outputting precision, recall, F1, and a pass/fail consistency score.'

Add an explicit trigger clause such as 'Use when comparing AI-generated clinical or academic text against a gold standard for semantic entailment, or when the user mentions BERTScore, COMET, or semantic consistency evaluation.'

Drop the generic buzzwords ('structured execution', 'explicit assumptions', 'clear output boundaries') which apply to nearly any skill and dilute distinctiveness.

Dimension	Reasoning	Score
Specificity	Names a domain ('academic writing workflows') but the stated actions ('structured execution', 'explicit assumptions', 'clear output boundaries') are abstract process qualities rather than the skill's concrete capabilities, and the actual function (BERTScore/COMET semantic evaluation) is never named.	2 / 5
Completeness	The 'what' is circular ('Use semantic consistency auditor for academic writing workflows') and there is no 'Use when...' or equivalent trigger clause, leaving both halves weakly addressed.	2 / 5
Trigger Term Quality	Only a couple generic keywords appear ('semantic consistency auditor', 'academic writing workflows') and the natural phrases a user would say ('evaluate semantic consistency', 'BERTScore', 'clinical notes comparison') are absent.	2 / 5
Distinctiveness Conflict Risk	The phrasing is very broad and would overlap with almost any academic-writing or structured-output skill; it does not surface the actual niche (semantic entailment evaluation of clinical notes).	2 / 5
	Total	8 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: aipoch/medical-research-skills
Path: scientific-skills/Academic Writing/semantic-consistency-auditor/SKILL.md
Commit: 63c61d3

Reviewed: 4 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.