Set up or align a GitHub Actions release pipeline for a versioned package, library, CLI, or marketplace action. Use when standardizing repos around the verify-then-release shape: push to main → guardrails → semantic-release tags + publishes → version-bump commit back to main with [skip ci].
99
100%
Does it follow best practices?
Impact
98%
1.55xAverage score across 4 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent correctly sets up a release pipeline for a TypeScript GitHub Actions marketplace action, covering the action.yml entrypoint requirement (bundled dist/index.js), moving major tag automation, correct plugin selection for a marketplace action (no npm publish), and the standard workflow structure including [skip ci] guards and concurrency.",
"type": "weighted_checklist",
"checklist": [
{
"name": "action.yml uses dist/index.js",
"description": "The action.yml `runs:` block references `dist/index.js` (or a path under `dist/`) as the main entrypoint — NOT a TypeScript source file like `src/index.ts`",
"max_score": 12
},
{
"name": "dist built in verify",
"description": "The verify job includes a build step that compiles/bundles the TypeScript source into `dist/`, OR the workflow commits `dist/` in the release job",
"max_score": 8
},
{
"name": "Moving major tag step",
"description": "A post-release step force-updates the moving major tag (e.g., `git tag -f v1` and `git push -f origin v1`) so consumers pinning to `@v1` see updates",
"max_score": 12
},
{
"name": "Major tag step is conditional",
"description": "The moving major tag step is conditional on a new release having been published (e.g., `steps.release.outputs.new_release_published == 'true'`)",
"max_score": 8
},
{
"name": "No npm publish plugin",
"description": "The .releaserc.json does NOT include @semantic-release/npm (GitHub Actions are published by tagging, not registry push)",
"max_score": 10
},
{
"name": "SR plugins for action",
"description": "The .releaserc.json includes @semantic-release/git and @semantic-release/github as plugins",
"max_score": 8
},
{
"name": "git before github",
"description": "@semantic-release/git appears before @semantic-release/github in the plugins list",
"max_score": 8
},
{
"name": "skip ci on both jobs",
"description": "Both verify and release jobs have `if:` conditions that skip execution when the commit message contains `[skip ci]`",
"max_score": 8
},
{
"name": "Release concurrency non-cancellable",
"description": "The release job has a concurrency group with `cancel-in-progress: false`",
"max_score": 6
},
{
"name": "fetch-depth: 0",
"description": "Both verify and release job checkout steps specify `fetch-depth: 0`",
"max_score": 6
},
{
"name": "Bot identity in step env",
"description": "GIT_AUTHOR_NAME/EMAIL and GIT_COMMITTER_NAME/EMAIL are set inside the semantic-release step's env, not at the job level",
"max_score": 7
},
{
"name": "SR action v4",
"description": "Uses `cycjimmy/semantic-release-action@v4`",
"max_score": 7
}
]
}