CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/notebook-ci-pipeline-author

The single home for Jupyter notebook testing: wires parameterized execution (papermill), output regression (nbval), function-level unit tests (testbook), output stripping (nbstripout), and artifact upload into one working GitHub Actions CI pipeline, with per-tool depth for papermill (parameters tag, CLI/API, sweeps) and nbval (strict/lax modes, per-cell markers, sanitize config) in references/. Includes a notebook PR review checklist covering untested notebooks, --nbval-lax misuse, hardcoded credentials, non-deterministic output cells, missing parameters tags, and committed outputs, with BLOCK / WARN / INFO severities and a BLOCK-or-PASS verdict. Use when notebooks must run as parameterized regression jobs in CI, when a repo ships .ipynb files whose outputs must stay stable, or when a PR that adds or modifies notebooks needs a structured quality review.

74

Quality

93%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

86%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-engineered, highly actionable skill: nine clearly sequenced steps with executable code throughout, a concrete PR review checklist with severities, hard-reject pre-flight conditions, anti-patterns with fixes, and a clean progressive-disclosure split into three real, one-level-deep reference files. The residual gaps are minor: some repeated rationale within steps (conciseness) and no explicit error-recovery loop for failed nbval/testbook stages (workflow clarity).

Suggestions

Deduplicate repeated rationale: state the module-scoped-fixture justification once in Step 5 and the `if: always()` justification once (Step 6 or 7) instead of twice each — this trims the body without losing information.

Add a short error-recovery subsection for stage failures: when nbval reports diffs, distinguish 'legitimate output change -> regenerate ground truth via papermill + commit' from 'non-deterministic output -> add #NBVAL_IGNORE_OUTPUT or a sanitize.cfg regex', closing the validate-fix-retry loop.

The recurring 'per the [X docs]' citation clause appears in almost every step; citing each source once at the point of first use (or in the References section) would tighten the prose while keeping traceability.

DimensionReasoningScore

Conciseness

The body is dense and skill-specific — no explanations of what Jupyter or these libraries are, no library comparisons, and every step carries executable code. But there are minor instances of over-explanation that could be trimmed: the `scope="module"` fixture rationale is stated twice in Step 5 ("using a module-scoped fixture so the kernel executes once" then "it stops each test re-executing the kernel"), the `if: always()` rationale is repeated across Steps 6 and 7, and the recurring "per the [X docs]" citation phrasing appears in nearly every step. This matches the score-4 anchor (efficient, minor trimmable over-explanation) rather than 5 (every token earns its place).

4 / 5

Actionability

Every step provides copy-paste-ready code: bash for nbstripout install ("nbstripout --install --attributes .gitattributes"), YAML workflow fragments with pinned action versions ("uses: actions/upload-artifact@v4" with name/path/retention-days), a papermill CLI invocation with concrete -p flags, an INI sanitize.cfg, and a working pytest fixture. The hard-reject conditions give concrete detection criteria, and the checklist specifies exact patterns to scan ("api_key =", "://user:pass@", base64 blobs in stream/execute_result blocks). This matches the score-5 anchor: fully executable guidance covering the common cases; the full assembled workflow is delegated to a real reference file rather than left as pseudocode.

5 / 5

Workflow Clarity

Steps 1-9 are clearly sequenced with validation checkpoints: pre-flight hard-reject conditions ("State the blocker to the user and stop"), the nbstripout dry-run action that "fails if any notebook carries uncommitted output", `if: always()` artifact retention on failure, and a full review checklist with severities and a BLOCK-or-PASS verdict. What keeps it at the score-4 anchor rather than 5 is the absence of explicit error-recovery loops: the guidance says the HTML report is "the primary debugging artifact" but never spells out the validate -> fix -> retry cycle (e.g., what to do when nbval diffs — update ground truth or add markers), so checkpoints are present but feedback loops are only implicit.

4 / 5

Progressive Disclosure

The body is a well-signaled overview with one-level-deep references that all exist: "Per-tool depth lives in [references/papermill.md] ... and [references/nbval.md]", the full workflow and test file in "[references/notebook-ci-pipeline.md]", each announced with a summary of its contents. Verified against the actual bundle: all three reference files exist, none reference further files (no nesting), and per-tool depth is appropriately split out while SKILL.md keeps the wiring and integration decisions. This matches the score-5 anchor exactly.

5 / 5

Total

18

/

20

Passed

Description

96%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: concrete capabilities with named tools, an explicit triple-condition "Use when..." clause, natural trigger terms including the .ipynb extension, and third-person voice. Its only weaknesses are a slightly inflated opener ("The single home for") and broad CI-pipeline framing that risks minor overlap with general CI-authoring skills, and its length borders on verbose for a description field.

DimensionReasoningScore

Specificity

The description enumerates concrete, named capabilities: "wires parameterized execution (papermill), output regression (nbval), function-level unit tests (testbook), output stripping (nbstripout), and artifact upload into one working GitHub Actions CI pipeline" plus a checklist covering six named checks ("untested notebooks, --nbval-lax misuse, hardcoded credentials, non-deterministic output cells, missing parameters tags, and committed outputs, with BLOCK / WARN / INFO severities"). Coverage is comprehensive across both pipeline assembly and PR review, matching the score-5 anchor of multiple specific concrete actions; nothing is vague or generic.

5 / 5

Completeness

Both questions are answered explicitly. What: "wires parameterized execution... into one working GitHub Actions CI pipeline" with a described checklist. When: "Use when notebooks must run as parameterized regression jobs in CI, when a repo ships .ipynb files whose outputs must stay stable, or when a PR that adds or modifies notebooks needs a structured quality review" — three concrete trigger conditions, exactly matching the score-5 anchor. It also uses third-person voice throughout, so no voice penalty applies.

5 / 5

Trigger Term Quality

Natural trigger terms a user would actually say are all present: "Jupyter notebook testing", ".ipynb files", "notebook PR", "parameterized regression jobs in CI", plus the tool names (papermill, nbval, testbook, nbstripout) and "GitHub Actions CI pipeline". This matches the score-5 anchor (comprehensive coverage including synonyms and file extensions); it is well above score 4, which requires missing natural terms.

5 / 5

Distinctiveness Conflict Risk

The notebook-specific triggers (papermill, nbval, testbook, .ipynb) carve a clear niche with minimal conflict against other notebook skills. However, the prominent framing "one working GitHub Actions CI pipeline" and "The single home for Jupyter notebook testing" creates minor overlap risk with generic CI/CD or GitHub-Actions pipeline-authoring skills — a request like "set up CI for my repo" could plausibly match. This lands on the score-4 anchor (mostly distinct, minor overlap with closely related skills) rather than 5, since the triggers are not exclusively notebook-scoped.

4 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

Table of Contents