CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl-labs/tessl-skill-review-ci

Implements Tessl skill review CI/CD pipelines through an interactive, configuration-first wizard. Supports GitHub Actions, Jenkins, and Azure DevOps.

70

0.67x
Quality

90%

Does it follow best practices?

Impact

50%

0.67x

Average score across 6 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

task.mdevals/scenario-6/

Task: Remove Score Quality Gate and Switch to Informational Mode

We have a repository at ~/project with a GitHub Actions workflow that currently enforces an 80% score quality gate. After running it for several months, the team has decided the gate is causing too many false-positive failures and wants to switch back to informational-only mode.

Configuration (use these values directly — do not ask configuration questions)

  • Score threshold: Change from 80% to None (informational only)
  • All other settings: Keep existing values (do not change triggers, permissions, branch targets, or cache location)

Changes needed

  1. Remove the score quality gate block — find and remove the conditional block that checks AVG_SCORE against the 80% threshold and sets FAILED=1. The pipeline should no longer fail based on review scores (it should still fail on validation failures).

  2. Update the PR comment footer — replace the enforcement text (mentions 80% threshold) with the informational text: _Review score is informational — not used for pass/fail gating._

  3. Verify cache file — the cache file at .github/.tessl/skill-review-cache.json should remain unchanged.

Existing workflow file

Write this content to ~/project/.github/workflows/tessl-skill-review.yml first, then apply the modifications:

name: Tessl Skill Review

on:
  pull_request:
    branches: [main]
    paths:
      - '**/SKILL.md'
      - '**/skills/**'
      - '.github/workflows/tessl-skill-review.yml'
  push:
    branches: [main]
    paths:
      - '**/SKILL.md'
      - '**/skills/**'
  workflow_dispatch:

permissions:
  contents: write
  pull-requests: write

jobs:
  review-skills:
    name: Review Skills
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
        with:
          fetch-depth: 0

      - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
        with:
          node-version: '20'

      - run: npm install -g @tessl/cli

      - name: Detect changed skills
        id: detect
        run: |
          CHANGED_SKILLS=$(find . -name "SKILL.md" -not -path "./node_modules/*" -not -path "./.git/*" | xargs -I {} dirname {} | sed 's|^\./||' | sort -u)
          if [[ -z "$CHANGED_SKILLS" ]]; then
            echo "skills=" >> "$GITHUB_OUTPUT"
          else
            EOF_MARKER=$(dd if=/dev/urandom bs=15 count=1 status=none | base64)
            echo "skills<<${EOF_MARKER}" >> "$GITHUB_OUTPUT"
            echo "$CHANGED_SKILLS" >> "$GITHUB_OUTPUT"
            echo "${EOF_MARKER}" >> "$GITHUB_OUTPUT"
          fi

      - name: Run skill reviews
        if: steps.detect.outputs.skills != ''
        id: review
        env:
          SKILLS: ${{ steps.detect.outputs.skills }}
          TESSL_API_KEY: ${{ secrets.TESSL_API_KEY }}
        run: |
          FAILED=0
          TABLE="| Skill | Status | Review Score | Change |"
          TABLE="${TABLE}\n|-------|--------|--------------|--------|"

          while IFS= read -r dir; do
            [[ -z "$dir" ]] && continue
            JSON_OUTPUT=$(tessl skill review --json "$dir" 2>&1)
            JSON=$(echo "$JSON_OUTPUT" | sed -n '/{/,$p')
            PASSED=$(echo "$JSON" | jq -r '.validation.overallPassed // false')
            AVG_SCORE=$(echo "$JSON" | jq -r '
              def avg(obj): (obj.scores | to_entries | map(.value.score) | add) / (obj.scores | length) * 100 / 3;
              ([(.descriptionJudge.evaluation | avg(.)), (.contentJudge.evaluation | avg(.))] | add / 2) | round
            ')
            [[ ! "$AVG_SCORE" =~ ^[0-9]+$ ]] && AVG_SCORE=0

            # Score quality gate
            if [[ "$AVG_SCORE" -lt 80 ]]; then
              echo "::error::Skill $dir scored ${AVG_SCORE}% (below 80% threshold)"
              FAILED=1
            fi

            if [[ "$PASSED" == "true" ]]; then
              STATUS="PASSED"
            else
              STATUS="FAILED"
              FAILED=1
            fi

            DIR_DISPLAY=$(echo "$dir" | tr '|' '/')
            TABLE="${TABLE}\n| \`${DIR_DISPLAY}\` | ${STATUS} | ${AVG_SCORE}% | |"

          done <<< "$SKILLS"

          COMMENT_BODY=$(printf '%b' "<!-- tessl-skill-review -->\n## Tessl Skill Review Results\n\n${TABLE}\n\n---\n_Review score gate: skills scoring below 80% will fail the check._")

          EOF_MARKER=$(dd if=/dev/urandom bs=15 count=1 status=none | base64)
          echo "comment<<${EOF_MARKER}" >> "$GITHUB_OUTPUT"
          echo "$COMMENT_BODY" >> "$GITHUB_OUTPUT"
          echo "${EOF_MARKER}" >> "$GITHUB_OUTPUT"

          if [[ "$FAILED" -eq 1 ]]; then
            exit 1
          fi

Existing cache file

Write this content to ~/project/.github/.tessl/skill-review-cache.json:

{
  "version": "1",
  "last_updated": "2026-01-15T10:30:00Z",
  "skills": {
    "skills/onboarding": {
      "score": 85,
      "validation_passed": true,
      "timestamp": "2026-01-15T10:30:00Z"
    }
  }
}

Current repository structure

~/project/
  .git/
  .github/
    .tessl/
      skill-review-cache.json  (content above)
    workflows/
      tessl-skill-review.yml  (content above)
  skills/
    onboarding/SKILL.md
    deployment/SKILL.md

Expected output files

  1. ~/project/.github/workflows/tessl-skill-review.yml — modified workflow without quality gate
  2. ~/project/.github/.tessl/skill-review-cache.json — unchanged cache file

Write all changes to ~/project.

evals

azure-devops.md

circleci.md

github-actions.md

gitlab-ci.md

jenkins.md

README.md

SKILL.md

tessl-skill-review-ci-review.md

TESTING.md

tile.json