CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl-labs/tessl-skill-review-ci

Implements Tessl skill review CI/CD pipelines through an interactive, configuration-first wizard. Supports GitHub Actions, Jenkins, and Azure DevOps.

70

0.67x
Quality

90%

Does it follow best practices?

Impact

50%

0.67x

Average score across 6 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

task.mdevals/scenario-5/

Task: Add Score Quality Gate to Our Existing GitHub Actions Workflow

We have a repository at ~/project that already has a GitHub Actions workflow for Tessl skill review. The workflow currently treats all review scores as informational only — no pipeline ever fails due to low scores. We need to add enforcement.

Configuration (use these values directly — do not ask configuration questions)

  • Score threshold to add: 75% — any skill scoring below 75% should cause the workflow to fail
  • Cache location: .github/.tessl/skill-review-cache.json
  • All other settings: Keep existing values (do not change triggers, permissions, or branch targets)

Changes needed

  1. Add a score quality gate — after the AVG_SCORE calculation in the review step, add a check: if AVG_SCORE is less than 75, produce an error message that includes the skill name, its score, and the 75% threshold, then set FAILED=1.

  2. Update the PR comment footer — replace the current informational text (Review score is informational) with enforcement text mentioning the 75% threshold (e.g., skills scoring below 75% will fail the check).

  3. Initialize the missing cache file — create .github/.tessl/skill-review-cache.json with the initial JSON structure: {"version": "1", "last_updated": "", "skills": {}}.

Existing workflow file

Write this content to ~/project/.github/workflows/tessl-skill-review.yml first, then apply the modifications:

name: Tessl Skill Review

on:
  pull_request:
    branches: [main]
    paths:
      - '**/SKILL.md'
      - '**/skills/**'
      - '.github/workflows/tessl-skill-review.yml'
  push:
    branches: [main]
    paths:
      - '**/SKILL.md'
      - '**/skills/**'
  workflow_dispatch:

permissions:
  contents: write
  pull-requests: write

jobs:
  review-skills:
    name: Review Skills
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
        with:
          fetch-depth: 0

      - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
        with:
          node-version: '20'

      - run: npm install -g @tessl/cli

      - name: Detect changed skills
        id: detect
        run: |
          CHANGED_SKILLS=$(find . -name "SKILL.md" -not -path "./node_modules/*" -not -path "./.git/*" | xargs -I {} dirname {} | sed 's|^\./||' | sort -u)
          if [[ -z "$CHANGED_SKILLS" ]]; then
            echo "skills=" >> "$GITHUB_OUTPUT"
          else
            EOF_MARKER=$(dd if=/dev/urandom bs=15 count=1 status=none | base64)
            echo "skills<<${EOF_MARKER}" >> "$GITHUB_OUTPUT"
            echo "$CHANGED_SKILLS" >> "$GITHUB_OUTPUT"
            echo "${EOF_MARKER}" >> "$GITHUB_OUTPUT"
          fi

      - name: Run skill reviews
        if: steps.detect.outputs.skills != ''
        id: review
        env:
          SKILLS: ${{ steps.detect.outputs.skills }}
          TESSL_API_KEY: ${{ secrets.TESSL_API_KEY }}
        run: |
          FAILED=0
          TABLE="| Skill | Status | Review Score | Change |"
          TABLE="${TABLE}\n|-------|--------|--------------|--------|"

          while IFS= read -r dir; do
            [[ -z "$dir" ]] && continue
            JSON_OUTPUT=$(tessl skill review --json "$dir" 2>&1)
            JSON=$(echo "$JSON_OUTPUT" | sed -n '/{/,$p')
            PASSED=$(echo "$JSON" | jq -r '.validation.overallPassed // false')
            AVG_SCORE=$(echo "$JSON" | jq -r '
              def avg(obj): (obj.scores | to_entries | map(.value.score) | add) / (obj.scores | length) * 100 / 3;
              ([(.descriptionJudge.evaluation | avg(.)), (.contentJudge.evaluation | avg(.))] | add / 2) | round
            ')
            [[ ! "$AVG_SCORE" =~ ^[0-9]+$ ]] && AVG_SCORE=0

            if [[ "$PASSED" == "true" ]]; then
              STATUS="PASSED"
            else
              STATUS="FAILED"
              FAILED=1
            fi

            DIR_DISPLAY=$(echo "$dir" | tr '|' '/')
            TABLE="${TABLE}\n| \`${DIR_DISPLAY}\` | ${STATUS} | ${AVG_SCORE}% | |"

          done <<< "$SKILLS"

          COMMENT_BODY=$(printf '%b' "<!-- tessl-skill-review -->\n## Tessl Skill Review Results\n\n${TABLE}\n\n---\n_Review score is informational — not used for pass/fail gating._")

          EOF_MARKER=$(dd if=/dev/urandom bs=15 count=1 status=none | base64)
          echo "comment<<${EOF_MARKER}" >> "$GITHUB_OUTPUT"
          echo "$COMMENT_BODY" >> "$GITHUB_OUTPUT"
          echo "${EOF_MARKER}" >> "$GITHUB_OUTPUT"

          if [[ "$FAILED" -eq 1 ]]; then
            exit 1
          fi

Current repository structure

~/project/
  .git/
  .github/
    workflows/
      tessl-skill-review.yml  (content above)
  skills/
    onboarding/SKILL.md

Expected output files

  1. ~/project/.github/workflows/tessl-skill-review.yml — modified workflow with quality gate
  2. ~/project/.github/.tessl/skill-review-cache.json — initialized cache file

Write all changes to ~/project.

evals

scenario-5

criteria.json

task.md

azure-devops.md

circleci.md

github-actions.md

gitlab-ci.md

jenkins.md

README.md

SKILL.md

tessl-skill-review-ci-review.md

TESTING.md

tile.json