changelog-orchestrator
Draft changelog PRs by collecting GitHub/Slack/Git changes, formatting with templates, running quality gates, and preparing a branch/PR. Use when generating weekly/monthly release notes or when the user asks to create a changelog from recent merges. Trigger with "changelog weekly", "generate release notes", "draft changelog", "create changelog PR".
70
64%
Does it follow best practices?
Impact
Pending
No eval scenarios have been run
Advisory
Suggest reviewing before use
Optimize this skill with Tessl
npx tessl skill review --optimize ./plugins/automation/mattyp-changelog/skills/changelog-orchestrator/SKILL.mdQuality
Discovery
100%Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.
This is a strong skill description that excels across all dimensions. It provides specific concrete actions, explicit trigger guidance with both a 'Use when' clause and enumerated trigger phrases, and occupies a clearly distinct niche. The description is concise yet comprehensive, making it easy for Claude to select appropriately from a large skill set.
| Dimension | Reasoning | Score |
|---|---|---|
Specificity | Lists multiple specific concrete actions: collecting GitHub/Slack/Git changes, formatting with templates, running quality gates, and preparing a branch/PR. These are detailed, actionable capabilities. | 3 / 3 |
Completeness | Clearly answers both 'what' (draft changelog PRs by collecting changes, formatting, running quality gates, preparing PR) and 'when' (explicit 'Use when' clause with scenarios plus explicit trigger phrases). | 3 / 3 |
Trigger Term Quality | Includes excellent natural trigger terms that users would actually say: 'changelog weekly', 'generate release notes', 'draft changelog', 'create changelog PR', plus contextual terms like 'weekly/monthly release notes' and 'recent merges'. | 3 / 3 |
Distinctiveness Conflict Risk | Occupies a clear niche combining changelog generation, PR creation, and release notes from specific sources (GitHub/Slack/Git). The specific trigger terms like 'changelog PR' and 'release notes' make it highly distinct and unlikely to conflict with general git or documentation skills. | 3 / 3 |
Total | 12 / 12 Passed |
Implementation
29%Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
The skill has good structural organization with appropriate progressive disclosure to reference files, but suffers from seriously broken workflow numbering that makes the multi-step process confusing and ambiguous. The lack of concrete examples (config format, command invocations, sample output) significantly hurts actionability. The workflow has logical gaps where steps are incomplete or misnumbered, undermining the core purpose of guiding Claude through the changelog generation process.
Suggestions
Fix the numbered list hierarchy—the current numbering restarts multiple times (1,2,3,1,2,3,1,2,1,2,3) making it impossible to follow the sequence. Use consistent nested numbering (e.g., 3a, 3b) or a single flat numbered list.
Add a concrete example of `.changelog-config.json` showing required fields, and show at least one complete command invocation with arguments (e.g., `python ${CLAUDE_SKILL_DIR}/scripts/validate_config.py .changelog-config.json`).
Complete the 'Decide date range' step with actual logic, and clarify what happens when the quality score is below threshold (currently the sub-steps after that condition describe the normal output path, not a remediation path).
Add a brief sample of expected changelog output so Claude knows what the rendered result should look like.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | Mostly efficient but includes some unnecessary sections like the 'Overview' paragraph that restates what the skill does (already clear from context). The prerequisites section is reasonable. Some filler phrases like 'This skill turns raw repo activity... into a publishable changelog draft' could be trimmed. | 2 / 3 |
Actionability | The instructions reference multiple scripts but provide no concrete examples of config file format, command invocations with arguments, or expected outputs. Steps like 'Decide date range' are incomplete (the sub-steps that follow appear to be misnumbered and don't actually explain how to decide the date range). No executable code or copy-paste ready commands are provided. | 1 / 3 |
Workflow Clarity | The numbered steps have broken/inconsistent numbering (1,2,3 then restarts at 1,2,3 then 1,2 then 1,2,3), making the sequence confusing. Step 3 ('Decide date range') has no actual content explaining how to decide it—the sub-steps underneath are about template rendering, not date ranges. The quality gate step says 'If score is below threshold' but never states what to do (the sub-step that follows is about writing the file, which should happen regardless). Critical validation/feedback loops are incomplete. | 1 / 3 |
Progressive Disclosure | Good use of one-level-deep references to implementation.md, errors.md, and examples.md. The main SKILL.md serves as an overview with clear pointers to detailed materials. The Resources section provides a clean inventory of available scripts and templates. | 3 / 3 |
Total | 7 / 12 Passed |
Validation
81%Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.
Validation — 9 / 11 Passed
Validation for skill structure
| Criteria | Description | Result |
|---|---|---|
allowed_tools_field | 'allowed-tools' contains unusual tool name(s) | Warning |
frontmatter_unknown_keys | Unknown frontmatter key(s) found; consider removing or moving to metadata | Warning |
Total | 9 / 11 Passed | |
- Commit
5585c45
- Reviewed
Table of Contents
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.