CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl-labs/tessl-skill-review-ci

Implements Tessl skill review CI/CD pipelines through an interactive, configuration-first wizard. Supports GitHub Actions, GitLab CI, Jenkins, Azure DevOps, and CircleCI.

94

1.75x

Quality

90%

Does it follow best practices?

Impact

100%

1.75x

Average score across 5 eval scenarios

Overview
Skills
Evals
Files

gitlab-ci.md

Tessl Skill Review CI/CD: GitLab CI

Overview

This document provides a complete GitLab CI/CD implementation for automated Tessl skill review. It mirrors the functionality of the GitHub Actions workflows (single-workflow variant) but uses GitLab CI-native concepts: .gitlab-ci.yml configuration, rules: for conditional execution, CI_* predefined variables, artifacts: for passing data between jobs, and the GitLab Notes API for merge request commenting.

The core review logic is identical across all CI platforms:

  1. Detect changed SKILL.md files in merge requests/commits
  2. Run tessl skill review --json <path> on each changed skill
  3. Calculate scores (average of descriptionJudge + contentJudge dimensions, each scored 0-3, normalized to %)
  4. Compare against cached previous scores from .tessl/skill-review-cache.json
  5. Post results as merge request comments with score diff indicators
  6. Update the cache file on main branch merges

Prerequisites

  • GitLab project with CI/CD enabled (Settings > CI/CD > General pipelines)
  • GitLab Runner available (shared runners or project-specific)
  • TESSL_API_KEY stored as a CI/CD variable (Settings > CI/CD > Variables, masked and protected)
  • GITLAB_TOKEN stored as a CI/CD variable (a Project Access Token or Personal Access Token with api scope) — or use CI_JOB_TOKEN if the project permits it for MR comments
  • Node.js 20+ (provided by the node:20 Docker image)
  • jq available in the CI environment (pre-installed in most Node.js Docker images or installable via apt-get)

Architecture Options

Internal Repositories

For internal GitLab projects where all contributors are trusted:

  • Single pipeline (.gitlab-ci.yml) handles review, MR commenting, cache commit, and optional optimization
  • Uses GITLAB_TOKEN or CI_JOB_TOKEN for MR comments via GitLab Notes API
  • Pipeline pushes cache commits directly to the default branch
  • Simplest setup, recommended for most teams

External/Fork Contributions

For projects accepting contributions from external forks:

  • Pipeline 1: Review — Runs on merge request events, produces review results as job artifacts
  • Pipeline 2: Comment — Triggered via trigger: or downstream pipeline, reads artifacts and posts MR comment in the trusted context
  • Secrets are never exposed to fork MR pipelines (GitLab does not expose protected variables to fork pipelines by default)
  • More complex but necessary when untrusted code runs in the pipeline

This document focuses on the internal repository approach. The external approach follows the same pattern as the GitHub Actions two-workflow variant, using GitLab downstream pipelines or multi-project pipelines instead of workflow_run.

Template Variables

These placeholders are replaced with the user's configuration:

  • {{TARGET_BRANCH}} — User's target branch (default: main)
  • {{TRIGGER_PATHS}} — User's file paths for change detection (default: **/SKILL.md, **/skills/**)
  • {{CACHE_FILE}} — User's cache location (default: .tessl/skill-review-cache.json)
  • {{AUTO_OPTIMIZE}} — Whether to auto-optimize skills (default: false)

Pipeline Template

File: .gitlab-ci.yml

# .gitlab-ci.yml
# Tessl Skill Review Pipeline - Reviews SKILL.md files, posts MR comments, caches scores

variables:
  TARGET_BRANCH: "main"
  CACHE_FILE: ".tessl/skill-review-cache.json"
  AUTO_OPTIMIZE: "false"
  # TESSL_API_KEY: set in Settings > CI/CD > Variables (masked)
  # GITLAB_TOKEN: set in Settings > CI/CD > Variables (masked) — or use CI_JOB_TOKEN

stages:
  - review
  - comment
  - cache
  - optimize

# ─── Review Job ────────────────────────────────────────────────────────────────
review-skills:
  stage: review
  image: node:20
  rules:
    # Run on merge request events when skill files change
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      changes:
        - "**/SKILL.md"
        - "**/skills/**"
        - ".gitlab-ci.yml"
    # Run on pushes to main when skill files change
    - if: $CI_COMMIT_BRANCH == "main"
      changes:
        - "**/SKILL.md"
        - "**/skills/**"
    # Allow manual trigger
    - if: $CI_PIPELINE_SOURCE == "web"
  before_script:
    - npm install -g @tessl/cli
    - apt-get update -qq && apt-get install -y -qq jq > /dev/null 2>&1 || true
  script:
    - |
      set -euo pipefail

      # ── Detect changed skills ──────────────────────────────────────────
      if [[ "$CI_PIPELINE_SOURCE" == "merge_request_event" ]]; then
        echo "Running in merge request context (MR !${CI_MERGE_REQUEST_IID})"
        git fetch origin "$CI_MERGE_REQUEST_TARGET_BRANCH_NAME" --depth=100
        CHANGED_SKILLS=$(git diff --name-only --diff-filter=ACMR \
          "origin/${CI_MERGE_REQUEST_TARGET_BRANCH_NAME}"...HEAD \
          -- '**/SKILL.md' '**/skills/**' | \
          grep 'SKILL.md$' | \
          xargs -I {} dirname {} | \
          sort -u) || true
      else
        echo "Running on push to main or manual trigger"
        CHANGED_SKILLS=$(find . -name "SKILL.md" \
          -not -path "./node_modules/*" \
          -not -path "./.git/*" | \
          xargs -I {} dirname {} | \
          sed 's|^\./||' | \
          sort -u) || true
      fi

      if [[ -z "$CHANGED_SKILLS" ]]; then
        echo "No skill changes detected."
        echo "" > changed_skills.txt
        exit 0
      fi

      echo "Skills to review:"
      echo "$CHANGED_SKILLS"
      echo "$CHANGED_SKILLS" > changed_skills.txt

      # ── Read review cache ──────────────────────────────────────────────
      REVIEW_CACHE=""
      if [[ -f "$CACHE_FILE" ]]; then
        echo "Cache file found, loading..."
        if CACHE_CONTENT=$(cat "$CACHE_FILE" 2>&1); then
          if echo "$CACHE_CONTENT" | jq empty 2>/dev/null; then
            echo "Cache is valid JSON"
            REVIEW_CACHE="$CACHE_CONTENT"
          else
            echo "WARNING: Cache file is invalid JSON, ignoring"
          fi
        else
          echo "WARNING: Cache file exists but cannot be read: $CACHE_CONTENT"
        fi
      else
        echo "No cache file found, will create new one"
      fi

      # ── Run skill reviews ──────────────────────────────────────────────
      FAILED=0
      TABLE="| Skill | Status | Review Score | Change |"
      TABLE="${TABLE}\n|-------|--------|--------------|--------|"
      DETAILS=""

      # Create temporary file for cache entries
      CACHE_FILE_TEMP=$(mktemp)
      echo "Cache entries file: $CACHE_FILE_TEMP"

      while IFS= read -r dir; do
        [[ -z "$dir" ]] && continue
        echo "======== Reviewing $dir ========"

        # Build review command based on auto-optimize setting
        REVIEW_CMD="tessl skill review --json"
        if [[ "$AUTO_OPTIMIZE" == "true" ]]; then
          REVIEW_CMD="tessl skill review --json --optimize --yes"
        fi

        JSON_OUTPUT=$($REVIEW_CMD "$dir" 2>&1) || true
        echo "$JSON_OUTPUT"

        # Extract JSON (skip everything before first '{')
        JSON=$(echo "$JSON_OUTPUT" | sed -n '/{/,$p')

        # Look up previous score from cache
        PREV_SCORE=""
        PREV_DESC=""
        PREV_CONTENT=""
        if [[ -n "$REVIEW_CACHE" ]]; then
          CACHE_ENTRY=$(echo "$REVIEW_CACHE" | jq -r --arg path "$dir" '.skills[$path] // empty')
          if [[ -n "$CACHE_ENTRY" ]]; then
            PREV_SCORE=$(echo "$CACHE_ENTRY" | jq -r '.score // empty')
            PREV_DESC=$(echo "$CACHE_ENTRY" | jq -r '.dimensions.description // empty')
            PREV_CONTENT=$(echo "$CACHE_ENTRY" | jq -r '.dimensions.content // empty')
          fi
        fi

        # Validate previous scores are numeric
        if [[ -n "$PREV_SCORE" && ! "$PREV_SCORE" =~ ^[0-9]+$ ]]; then
          echo "WARNING: Invalid previous score for $dir: $PREV_SCORE, ignoring"
          PREV_SCORE=""
        fi
        if [[ -n "$PREV_DESC" && ! "$PREV_DESC" =~ ^[0-9]+$ ]]; then
          echo "WARNING: Invalid previous description score for $dir: $PREV_DESC, ignoring"
          PREV_DESC=""
        fi
        if [[ -n "$PREV_CONTENT" && ! "$PREV_CONTENT" =~ ^[0-9]+$ ]]; then
          echo "WARNING: Invalid previous content score for $dir: $PREV_CONTENT, ignoring"
          PREV_CONTENT=""
        fi

        # Extract validation status
        PASSED=$(echo "$JSON" | jq -r '.validation.overallPassed // false')

        # Calculate average score from all dimensions
        AVG_SCORE=$(echo "$JSON" | jq -r '
          def avg(obj): (obj.scores | to_entries | map(.value.score) | add) / (obj.scores | length) * 100 / 3;
          (
            [(.descriptionJudge.evaluation | avg(.)), (.contentJudge.evaluation | avg(.))] | add / 2
          ) | round
        ')

        # Validate AVG_SCORE is numeric
        if [[ ! "$AVG_SCORE" =~ ^[0-9]+$ ]]; then
          echo "ERROR: Invalid average score calculated for $dir: $AVG_SCORE"
          AVG_SCORE=0
        fi

        # Calculate diff against previous score
        CHANGE=""
        if [[ -n "$PREV_SCORE" ]]; then
          DIFF=$((AVG_SCORE - PREV_SCORE))
          if [[ $DIFF -gt 0 ]]; then
            CHANGE="🔺 +${DIFF}% (was ${PREV_SCORE}%)"
          elif [[ $DIFF -lt 0 ]]; then
            CHANGE="🔻 ${DIFF}% (was ${PREV_SCORE}%)"
          else
            CHANGE="➡️ no change"
          fi
        fi

        # Build status column
        if [[ "$PASSED" == "true" ]]; then
          STATUS="✅ PASSED"
        else
          ERROR=$(echo "$JSON" | jq -r '
            .validation.checks
            | map(select(.status != "passed"))
            | first
            | .message // "Validation failed"
          ' | cut -c1-60)
          STATUS="❌ FAILED — ${ERROR}"
          FAILED=1
        fi

        DIR_DISPLAY=$(echo "$dir" | tr '|' '/')
        TABLE="${TABLE}\n| \`${DIR_DISPLAY}\` | ${STATUS} | ${AVG_SCORE}% | ${CHANGE} |"

        # Calculate dimension scores for cache and details
        DESC_SCORE=$(echo "$JSON" | jq -r '
          (.descriptionJudge.evaluation.scores | to_entries | map(.value.score) | add) * 100 / ((.descriptionJudge.evaluation.scores | length) * 3) | round
        ')
        CONTENT_SCORE=$(echo "$JSON" | jq -r '
          (.contentJudge.evaluation.scores | to_entries | map(.value.score) | add) * 100 / ((.contentJudge.evaluation.scores | length) * 3) | round
        ')

        # Validate dimension scores
        if [[ ! "$DESC_SCORE" =~ ^[0-9]+$ ]]; then
          echo "WARNING: Invalid description score for $dir: $DESC_SCORE, using 0"
          DESC_SCORE=0
        fi
        if [[ ! "$CONTENT_SCORE" =~ ^[0-9]+$ ]]; then
          echo "WARNING: Invalid content score for $dir: $CONTENT_SCORE, using 0"
          CONTENT_SCORE=0
        fi

        # Extract detailed evaluations for collapsible section
        DESC_EVAL=$(echo "$JSON" | jq -r '.descriptionJudge.evaluation |
          "  Description: " + ((.scores | to_entries | map(.value.score) | add) * 100 / ((.scores | length) * 3) | round | tostring) + "%\n" +
          (.scores | to_entries | map("    \(.key): \(.value.score)/3 - \(.value.reasoning)") | join("\n")) + "\n\n" +
          "    Assessment: " + .overall_assessment
        ')

        CONTENT_EVAL=$(echo "$JSON" | jq -r '.contentJudge.evaluation |
          "  Content: " + ((.scores | to_entries | map(.value.score) | add) * 100 / ((.scores | length) * 3) | round | tostring) + "%\n" +
          (.scores | to_entries | map("    \(.key): \(.value.score)/3 - \(.value.reasoning)") | join("\n")) + "\n\n" +
          "    Assessment: " + .overall_assessment
        ')

        # Extract suggestions
        SUGGESTIONS=$(echo "$JSON" | jq -r '
          [.descriptionJudge.evaluation.suggestions // [], .contentJudge.evaluation.suggestions // []]
          | flatten
          | map("- " + .)
          | join("\n")
        ')

        # Build collapsible details block (Markdown)
        DETAILS="${DETAILS}\n\n<details>\n<summary><strong>${DIR_DISPLAY}</strong> — ${AVG_SCORE}% (${STATUS#* })</summary>\n\n"

        # Show score comparison if previous exists
        if [[ -n "$PREV_SCORE" && -n "$PREV_DESC" && -n "$PREV_CONTENT" ]]; then
          DETAILS="${DETAILS}**Previous:** ${PREV_SCORE}% (Description: ${PREV_DESC}%, Content: ${PREV_CONTENT}%)\n"
          DETAILS="${DETAILS}**Current:**  ${AVG_SCORE}% (Description: ${DESC_SCORE}%, Content: ${CONTENT_SCORE}%)\n\n"
          DETAILS="${DETAILS}---\n\n"
        fi

        DETAILS="${DETAILS}\`\`\`\n${DESC_EVAL}\n\n${CONTENT_EVAL}\n\`\`\`\n"

        if [[ -n "$SUGGESTIONS" ]]; then
          DETAILS="${DETAILS}\n**Suggestions:**\n\n${SUGGESTIONS}\n"
        fi

        DETAILS="${DETAILS}\n</details>"

        # Calculate content hash for cache
        if [[ ! -f "$dir/SKILL.md" ]]; then
          echo "ERROR: SKILL.md not found for $dir"
          continue
        fi
        CONTENT_HASH=$(sha256sum "$dir/SKILL.md" 2>&1)
        if [[ $? -ne 0 ]]; then
          echo "ERROR: Failed to calculate hash for $dir: $CONTENT_HASH"
          continue
        fi
        CONTENT_HASH="sha256:$(echo "$CONTENT_HASH" | awk '{print $1}')"

        # Build cache entry (compact to single line)
        if ! CACHE_ENTRY=$(jq -nc \
          --arg score "$AVG_SCORE" \
          --arg passed "$PASSED" \
          --arg hash "$CONTENT_HASH" \
          --arg ts "$(date -u +"%Y-%m-%dT%H:%M:%SZ")" \
          --arg desc "$DESC_SCORE" \
          --arg content "$CONTENT_SCORE" \
          '{
            score: ($score | tonumber),
            validation_passed: ($passed == "true"),
            content_hash: $hash,
            timestamp: $ts,
            dimensions: {
              description: ($desc | tonumber),
              content: ($content | tonumber)
            }
          }'); then
          echo "ERROR: Failed to build cache entry for $dir"
          continue
        fi

        # Write cache entry to file (tab-separated: path<tab>json)
        printf '%s\t%s\n' "$dir" "$CACHE_ENTRY" >> "$CACHE_FILE_TEMP"

      done <<< "$CHANGED_SKILLS"

      # Save cache entries for the update step
      cp "$CACHE_FILE_TEMP" cache_entries.tsv
      echo "Wrote $(wc -l < "$CACHE_FILE_TEMP") cache entries to cache_entries.tsv"

      # ── Update review cache ────────────────────────────────────────────
      mkdir -p "$(dirname "$CACHE_FILE")"

      # Load existing cache or create new structure
      if [[ -f "$CACHE_FILE" ]]; then
        if CACHE=$(cat "$CACHE_FILE" 2>&1); then
          if ! echo "$CACHE" | jq empty 2>/dev/null; then
            echo "WARNING: Cache file is invalid JSON, recreating"
            CACHE='{"version":"1","last_updated":"","skills":{}}'
          fi
        else
          echo "WARNING: Cache file exists but cannot be read: $CACHE"
          CACHE='{"version":"1","last_updated":"","skills":{}}'
        fi
      else
        echo "Creating new cache file..."
        CACHE='{"version":"1","last_updated":"","skills":{}}'
      fi

      # Update timestamp
      TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
      if ! CACHE=$(echo "$CACHE" | jq --arg ts "$TIMESTAMP" '.last_updated = $ts'); then
        echo "ERROR: Failed to update cache timestamp"
        exit 1
      fi

      # Merge cache updates (using TAB delimiter)
      MERGED_COUNT=0
      FAILED_COUNT=0

      while IFS=$'\t' read -r skill_path entry_json; do
        [[ -z "$skill_path" ]] && continue
        if NEW_CACHE=$(echo "$CACHE" | jq --arg path "$skill_path" --argjson entry "$entry_json" \
          '.skills[$path] = $entry' 2>&1); then
          CACHE="$NEW_CACHE"
          MERGED_COUNT=$((MERGED_COUNT + 1))
        else
          echo "WARNING: Failed to merge cache entry for $skill_path: $NEW_CACHE"
          FAILED_COUNT=$((FAILED_COUNT + 1))
          continue
        fi
      done < cache_entries.tsv

      # Write updated cache file
      if ! echo "$CACHE" | jq '.' > "$CACHE_FILE"; then
        echo "ERROR: Failed to write cache file"
        exit 1
      fi

      # Report merge counts
      if [[ $FAILED_COUNT -gt 0 ]]; then
        echo "Cache updated with $MERGED_COUNT entries ($FAILED_COUNT failed)"
      else
        echo "Cache updated with $MERGED_COUNT entries"
      fi

      # Copy cache for artifact
      cp "$CACHE_FILE" skill-review-cache.json

      # ── Build MR comment body ──────────────────────────────────────────
      COMMENT_BODY=$(printf '%b' "<!-- tessl-skill-review -->\n## Tessl Skill Review Results\n\n${TABLE}\n\n---\n\n### Detailed Review\n${DETAILS}\n\n---\n_Checks: frontmatter validity, required fields, body structure, examples, line count._\n_Review score is informational — not used for pass/fail gating._")

      # Save comment body for MR commenting job
      echo "$COMMENT_BODY" > comment.md

      # Save review metadata
      echo "$FAILED" > review_failed.txt

      if [[ "$FAILED" -eq 1 ]]; then
        echo "ERROR: One or more skills failed validation checks."
        exit 1
      fi
  artifacts:
    paths:
      - changed_skills.txt
      - comment.md
      - review_failed.txt
      - cache_entries.tsv
      - skill-review-cache.json
    when: always
    expire_in: 1 week

# ─── MR Comment Job ───────────────────────────────────────────────────────────
post-mr-comment:
  stage: comment
  image: alpine:latest
  needs:
    - job: review-skills
      artifacts: true
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      changes:
        - "**/SKILL.md"
        - "**/skills/**"
        - ".gitlab-ci.yml"
  before_script:
    - apk add --no-cache curl jq
  script:
    - |
      set -euo pipefail

      if [[ ! -f comment.md ]]; then
        echo "No comment file found, skipping MR comment."
        exit 0
      fi

      if [[ ! -s changed_skills.txt ]]; then
        echo "No skills were reviewed, skipping MR comment."
        exit 0
      fi

      COMMENT_BODY=$(cat comment.md)

      # Use GITLAB_TOKEN if set, otherwise fall back to CI_JOB_TOKEN
      TOKEN="${GITLAB_TOKEN:-$CI_JOB_TOKEN}"
      if [[ -z "$TOKEN" ]]; then
        echo "ERROR: Neither GITLAB_TOKEN nor CI_JOB_TOKEN is available. Cannot post MR comment."
        exit 1
      fi

      MR_IID="$CI_MERGE_REQUEST_IID"
      PROJECT_ID="$CI_PROJECT_ID"
      API_URL="${CI_API_V4_URL}/projects/${PROJECT_ID}/merge_requests/${MR_IID}/notes"

      echo "Posting comment to MR !${MR_IID}..."

      # Check for existing tessl-skill-review comment to update
      EXISTING_NOTES=$(curl -s \
        -H "PRIVATE-TOKEN: ${TOKEN}" \
        "${API_URL}?per_page=100&sort=desc")

      # Find existing note with our marker
      EXISTING_NOTE_ID=$(echo "$EXISTING_NOTES" | jq -r '
        .[]
        | select(.body | test("<!-- tessl-skill-review -->"))
        | .id
      ' 2>/dev/null | head -1)

      # Escape comment body for JSON
      ESCAPED_BODY=$(jq -Rs '.' <<< "$COMMENT_BODY")

      if [[ -n "$EXISTING_NOTE_ID" && "$EXISTING_NOTE_ID" != "null" ]]; then
        echo "Found existing review comment (note ID: $EXISTING_NOTE_ID), updating..."

        # Update existing note via PUT
        HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
          -X PUT \
          -H "PRIVATE-TOKEN: ${TOKEN}" \
          -H "Content-Type: application/json" \
          -d "{\"body\": ${ESCAPED_BODY}}" \
          "${API_URL}/${EXISTING_NOTE_ID}")

        if [[ "$HTTP_STATUS" -ge 200 && "$HTTP_STATUS" -lt 300 ]]; then
          echo "Comment updated successfully (HTTP $HTTP_STATUS)."
        else
          echo "ERROR: Failed to update comment (HTTP $HTTP_STATUS)."
          # Fall through to create a new comment
          echo "Creating new comment instead..."
          curl -s \
            -H "PRIVATE-TOKEN: ${TOKEN}" \
            -H "Content-Type: application/json" \
            -d "{\"body\": ${ESCAPED_BODY}}" \
            "${API_URL}"
          echo "New comment posted."
        fi
      else
        echo "No existing review comment found, creating new one..."

        HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
          -H "PRIVATE-TOKEN: ${TOKEN}" \
          -H "Content-Type: application/json" \
          -d "{\"body\": ${ESCAPED_BODY}}" \
          "${API_URL}")

        if [[ "$HTTP_STATUS" -ge 200 && "$HTTP_STATUS" -lt 300 ]]; then
          echo "Comment posted successfully (HTTP $HTTP_STATUS)."
        else
          echo "ERROR: Failed to post comment (HTTP $HTTP_STATUS)."
          exit 1
        fi
      fi

# ─── Cache Commit Job ─────────────────────────────────────────────────────────
commit-cache:
  stage: cache
  image: alpine/git:latest
  needs:
    - job: review-skills
      artifacts: true
  rules:
    # Only run on push to main (not on MR pipelines)
    - if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE != "merge_request_event"
      changes:
        - "**/SKILL.md"
        - "**/skills/**"
    - if: $CI_PIPELINE_SOURCE == "web" && $CI_COMMIT_BRANCH == "main"
  before_script:
    - apk add --no-cache jq
  script:
    - |
      set -euo pipefail

      if [[ ! -f skill-review-cache.json ]]; then
        echo "No cache artifact found, skipping."
        exit 0
      fi

      # Move cache to correct location
      mkdir -p "$(dirname "$CACHE_FILE")"
      cp skill-review-cache.json "$CACHE_FILE"

      # Check if cache actually changed
      if git diff --quiet HEAD -- "$CACHE_FILE" 2>/dev/null; then
        echo "Cache file unchanged, nothing to commit."
        exit 0
      fi

      echo "Cache file has changes, committing..."

      # Configure git for push
      git config user.name "gitlab-ci[bot]"
      git config user.email "gitlab-ci[bot]@${CI_SERVER_HOST}"

      # Use GITLAB_TOKEN for push authentication (CI_JOB_TOKEN has limited push permissions)
      TOKEN="${GITLAB_TOKEN:-$CI_JOB_TOKEN}"
      PUSH_URL="https://gitlab-ci-token:${TOKEN}@${CI_SERVER_HOST}/${CI_PROJECT_PATH}.git"

      git add "$CACHE_FILE"
      git commit -m "chore: update skill review cache [skip ci]"

      if ! git push "$PUSH_URL" HEAD:"$TARGET_BRANCH"; then
        echo "ERROR: Failed to push cache update to $TARGET_BRANCH"
        exit 1
      fi

      echo "Cache committed and pushed to $TARGET_BRANCH."

# ─── Optimize Commit Job ──────────────────────────────────────────────────────
commit-optimized:
  stage: optimize
  image: alpine/git:latest
  needs:
    - job: review-skills
      artifacts: true
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event" && $AUTO_OPTIMIZE == "true"
      changes:
        - "**/SKILL.md"
        - "**/skills/**"
  script:
    - |
      set -euo pipefail

      # Check if there are any optimized file changes
      if git diff --quiet HEAD; then
        echo "No optimized files to commit."
        exit 0
      fi

      git config user.name "gitlab-ci[bot]"
      git config user.email "gitlab-ci[bot]@${CI_SERVER_HOST}"

      git add -A '*.md'

      if git diff --cached --quiet; then
        echo "No staged changes to commit."
        exit 0
      fi

      git commit -m "chore: auto-optimize skills [skip ci]"

      # Push to the MR source branch
      TOKEN="${GITLAB_TOKEN:-$CI_JOB_TOKEN}"
      PUSH_URL="https://gitlab-ci-token:${TOKEN}@${CI_SERVER_HOST}/${CI_PROJECT_PATH}.git"
      SOURCE_BRANCH="$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME"

      if ! git push "$PUSH_URL" HEAD:"$SOURCE_BRANCH"; then
        echo "ERROR: Failed to push optimized skills to $SOURCE_BRANCH"
        exit 1
      fi

      echo "Optimized skills committed and pushed to $SOURCE_BRANCH."

Template Substitution Logic

When creating the .gitlab-ci.yml from this template:

  1. Read the pipeline template above
  2. Replace all occurrences:
    • "main" in TARGET_BRANCH variable and $CI_COMMIT_BRANCH == "main" rules → {{TARGET_BRANCH}}
    • "**/SKILL.md" and "**/skills/**" in changes: blocks → User's {{TRIGGER_PATHS}} (formatted as YAML list items)
    • ".tessl/skill-review-cache.json" in CACHE_FILE variable → {{CACHE_FILE}}
    • "false" in AUTO_OPTIMIZE variable → {{AUTO_OPTIMIZE}}
  3. Write the resulting YAML to .gitlab-ci.yml in the repository root

Setup Instructions

Step 1: Create the Pipeline File

Copy the .gitlab-ci.yml content from the template above into the root of your repository.

Step 2: Initialize the Cache File

mkdir -p .tessl
cat > .tessl/skill-review-cache.json << 'EOF'
{
  "version": "1",
  "last_updated": "",
  "skills": {}
}
EOF
git add .tessl/skill-review-cache.json
git commit -m "feat: initialize skill review cache"
git push

Step 3: Add CI/CD Variables

  1. Go to your GitLab project
  2. Navigate to Settings > CI/CD > Variables
  3. Click Add variable
  4. Add TESSL_API_KEY:
    • Key: TESSL_API_KEY
    • Value: Your Tessl API key from https://tessl.io
    • Type: Variable
    • Flags: Mask variable (checked), Protect variable (optional, for production branches only)
  5. Add GITLAB_TOKEN:
    • Key: GITLAB_TOKEN
    • Value: A Project Access Token or Personal Access Token with api scope
    • Type: Variable
    • Flags: Mask variable (checked)

Step 4: Create a Project Access Token (Recommended)

Using a Project Access Token is more secure than a Personal Access Token:

  1. Go to Settings > Access Tokens
  2. Create a new token:
    • Token name: tessl-skill-review
    • Role: Developer (or Maintainer if pushing to protected branches)
    • Scopes: api, write_repository
    • Expiration: set per your security policy
  3. Copy the token and add it as the GITLAB_TOKEN CI/CD variable

Step 5: Using CI_JOB_TOKEN Instead (Alternative)

If you prefer not to create a separate token, CI_JOB_TOKEN can be used for MR comments in some configurations:

  1. Go to Settings > CI/CD > Token Access
  2. Ensure the project allows the job token to access the API
  3. Note: CI_JOB_TOKEN has limited permissions — it can create MR notes but may not be able to push commits to protected branches

Step 6: Verify Runner Configuration

Ensure a GitLab Runner is available and can pull Docker images:

# Check runner status (admin or maintainer)
# Go to Settings > CI/CD > Runners
# Verify at least one runner is active and online

Secrets Management

Option 1: CI/CD Variables (Simplest)

  1. Go to Settings > CI/CD > Variables
  2. Add TESSL_API_KEY as a masked variable
  3. Add GITLAB_TOKEN as a masked variable
  4. Variables are automatically available to all pipeline jobs

Option 2: Group-Level Variables (Shared Across Projects)

  1. Go to your GitLab group
  2. Navigate to Settings > CI/CD > Variables
  3. Add TESSL_API_KEY and GITLAB_TOKEN at the group level
  4. All projects in the group inherit these variables

Option 3: External Secrets (Enterprise)

GitLab Premium/Ultimate supports external secret managers:

# .gitlab-ci.yml addition for Vault integration
variables:
  VAULT_SERVER_URL: "https://vault.example.com"
  VAULT_AUTH_ROLE: "gitlab-ci"

review-skills:
  secrets:
    TESSL_API_KEY:
      vault: secret/tessl/api_key@production
    GITLAB_TOKEN:
      vault: secret/gitlab/project_token@production

Variable Protection

  • Protected variables: Only available on protected branches and tags. Use this for TESSL_API_KEY if you only want reviews on protected branches.
  • Masked variables: Values are masked in job logs. Always enable this for secrets.
  • Environment scope: Restrict variables to specific environments if needed.

MR Comments

How It Works

The pipeline uses the GitLab Notes API to post and update merge request comments.

API Endpoint:

POST/PUT ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/merge_requests/${CI_MERGE_REQUEST_IID}/notes

Authentication: Uses GITLAB_TOKEN (Project Access Token or Personal Access Token) via the PRIVATE-TOKEN header. Falls back to CI_JOB_TOKEN if GITLAB_TOKEN is not set.

Comment Lifecycle

  1. First run on an MR: Creates a new note (comment) with review results
  2. Subsequent runs on the same MR: Finds the existing note by searching for the <!-- tessl-skill-review --> HTML marker in the note body, then updates it via PUT
  3. Different MRs: Each MR gets its own note

GitLab API Details

List existing notes:

GET ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/merge_requests/${MR_IID}/notes?per_page=100&sort=desc

Create new note:

POST ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/merge_requests/${MR_IID}/notes
Body: {"body": "<comment markdown>"}

Update existing note:

PUT ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/merge_requests/${MR_IID}/notes/${NOTE_ID}
Body: {"body": "<updated comment markdown>"}

Markdown Support

GitLab MR comments support standard Markdown including:

  • Tables
  • Collapsible <details> sections
  • Code blocks
  • Bold, italics, emojis
  • Task lists

This means the same comment format used in GitHub Actions works in GitLab without modification.


Troubleshooting

Pipeline Not Triggering on Merge Requests

Symptom: Pipeline runs on pushes but not on merge requests.

Cause: The rules: may not match the merge request event, or the merge request pipeline feature is not enabled.

Fix:

  1. Verify the rule uses $CI_PIPELINE_SOURCE == "merge_request_event"
  2. Check that the changes: paths match your skill file locations
  3. Go to Settings > CI/CD > General pipelines and ensure merge request pipelines are not disabled
  4. If you see duplicate pipelines (one for branch push, one for MR), use workflow: rules: to control which runs:
workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == "main"
    - if: $CI_PIPELINE_SOURCE == "web"

MR Comment Not Appearing

Symptom: Pipeline succeeds but no comment appears on the merge request.

Causes:

  1. GITLAB_TOKEN is not set or has insufficient permissions
  2. CI_JOB_TOKEN lacks permission to create MR notes
  3. The post-mr-comment job was skipped due to rule mismatch

Fix:

  • Verify the GITLAB_TOKEN variable exists in CI/CD settings and has api scope
  • Check job logs for HTTP status codes (403 = permission denied, 404 = MR not found)
  • Ensure the token owner has at least Developer role on the project

Cache Not Committing

Symptom: Reviews run but cache never gets committed to main.

Causes:

  1. Pipeline is running on an MR (cache commit only runs on push to main)
  2. Token lacks write_repository scope or push permission
  3. Branch protection rules block the push

Fix:

  • Verify the commit-cache job rules: it only runs when CI_COMMIT_BRANCH == "main"
  • Ensure GITLAB_TOKEN has write_repository scope
  • For protected branches, the token user must be allowed to push (Settings > Repository > Protected branches)
  • Check job logs for the specific error message

Permission Denied on Push

Symptom: remote: GitLab: You are not allowed to push code to protected branches

Cause: The token user does not have push access to the protected main branch.

Fix (choose one):

  1. Add the token user to the "Allowed to push" list for the protected branch
  2. Use a Maintainer-role Project Access Token
  3. Unprotect the branch temporarily (not recommended for production)

sha256sum Not Found

Symptom: sha256sum: command not found

Cause: The Docker image does not include sha256sum. The node:20 image (Debian-based) includes it via coreutils.

Fix: If using a minimal image, install coreutils:

before_script:
  - apt-get update && apt-get install -y coreutils

jq Not Found

Symptom: jq: command not found

Cause: The Docker image does not have jq pre-installed.

Fix: The template includes apt-get install -y jq in before_script. If using Alpine-based images, use apk add jq instead.

Score Diffs Not Showing

Symptom: MR comment shows empty Change column.

Causes:

  1. First review (no cached baseline exists)
  2. Cache file is empty or has no entry for this skill path
  3. Skill path changed (rename) so it does not match the cached key

Fix:

  • This is expected on first review after setup
  • Merge an MR to main to populate the cache, then subsequent MRs will show diffs
  • Check .tessl/skill-review-cache.json to see what skill paths are cached

Pipeline Runs in Infinite Loop

Symptom: Cache commit triggers another pipeline run, which commits cache again.

Cause: The [skip ci] marker in the commit message is not being honored.

Fix:

  1. GitLab respects [skip ci] or [ci skip] in commit messages by default
  2. Verify the commit message includes [skip ci]: "chore: update skill review cache [skip ci]"
  3. As a fallback, add a path exclusion to the rules or use workflow: rules: to skip when the commit message contains [skip ci]
  4. Alternatively, exclude the cache file from changes: triggers:
rules:
  - if: $CI_COMMIT_BRANCH == "main"
    changes:
      - "**/SKILL.md"
      - "**/skills/**"
    # The cache file is not in these paths, so cache-only commits won't re-trigger

Duplicate Pipelines (Branch + MR)

Symptom: Two pipelines run for the same commit — one for the branch push, one for the merge request event.

Fix: Add a workflow: rules: block at the top of .gitlab-ci.yml to deduplicate:

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE != "merge_request_event"
    - if: $CI_PIPELINE_SOURCE == "web"

Testing

1. Verify Pipeline Setup (Manual Trigger)

1. Go to your GitLab project > CI/CD > Pipelines
2. Click "Run pipeline"
3. Select branch: main
4. Click "Run pipeline"

Verify:
  - Pipeline runs without errors
  - Review scores appear in the review-skills job logs
  - Artifacts are created (changed_skills.txt, comment.md, skill-review-cache.json)

2. Test MR Comment Flow

# Create a test branch
git checkout -b test/skill-review-setup

# Modify a SKILL.md file
echo "Updated for testing" >> path/to/SKILL.md

# Commit and push
git add path/to/SKILL.md
git commit -m "test: trigger skill review pipeline"
git push -u origin test/skill-review-setup

Then create a Merge Request in GitLab and verify:

  • Pipeline runs on the MR
  • review-skills job completes and produces artifacts
  • post-mr-comment job runs and posts a comment
  • MR comment contains the review results table and collapsible details

3. Test Cache Commit Flow

1. Merge the test MR to main

2. Wait for the pipeline to run on the push to main

3. Verify:
   - commit-cache job runs successfully
   - New commit appears: "chore: update skill review cache [skip ci]"
   - .tessl/skill-review-cache.json contains skill entries
   - Pipeline does NOT trigger again from the cache commit

4. Create another MR modifying the same skill

5. Verify:
   - Score diff indicators appear (🔺 🔻 ➡️)
   - Previous vs Current scores are shown in details

4. Test Comment Update

1. Push another commit to the same MR branch
2. Wait for the pipeline to re-run
3. Verify:
   - The existing MR comment is UPDATED (not duplicated)
   - New scores reflect the latest changes

5. Verify with GitLab API

# List MR notes to verify the comment was posted
curl -s \
  -H "PRIVATE-TOKEN: ${GITLAB_TOKEN}" \
  "${CI_API_V4_URL}/projects/${PROJECT_ID}/merge_requests/${MR_IID}/notes" | \
  jq '.[] | select(.body | test("tessl-skill-review")) | {id, updated_at, body_preview: (.body | .[0:80])}'

Differences from GitHub Actions Implementation

FeatureGitHub ActionsGitLab CI
Config location.github/workflows/*.yml.gitlab-ci.yml
MR/PR triggeron: pull_requestrules: - if: $CI_PIPELINE_SOURCE == "merge_request_event"
Push triggeron: push: branches: [main]rules: - if: $CI_COMMIT_BRANCH == "main"
Manual triggeron: workflow_dispatchrules: - if: $CI_PIPELINE_SOURCE == "web"
Path filteringon: pull_request: paths:rules: - changes:
SecretsGitHub Secrets (repo settings)CI/CD Variables (project/group settings)
MR/PR commentspeter-evans/create-or-update-comment actionGitLab Notes API with PRIVATE-TOKEN header
Build tokenGITHUB_TOKEN (auto-provided)CI_JOB_TOKEN (auto-provided, limited scope)
Artifactsactions/upload-artifact / actions/download-artifactartifacts: paths: / needs: - job: artifacts: true
Step outputs$GITHUB_OUTPUT fileArtifacts passed between jobs
Environment variables$GITHUB_ENV filevariables: block or dotenv artifacts
Warnings/errors::warning:: / ::error::echo "WARNING:" / echo "ERROR:" (visible in logs)
Build summary$GITHUB_STEP_SUMMARYNo built-in equivalent (use MR comment or job log)
Cache file path.github/.tessl/skill-review-cache.json.tessl/skill-review-cache.json
[skip ci]Respected by defaultRespected by default ([skip ci] or [ci skip])
Node.js setupactions/setup-node@v4image: node:20
Checkoutactions/checkout@v4 with fetch-depth: 0Built-in checkout + git fetch for depth
Conditional executionif: on stepsrules: on jobs
Fork securityTwo-workflow with workflow_runProtected variables not exposed to forks by default
Runner environmentGitHub-hosted runnersGitLab shared runners or self-hosted
Permissions modelpermissions: block per workflow/jobToken scopes + project role-based access
TerminologyPull Request (PR)Merge Request (MR)

Changelog

GitLab CI Version (2026-03-05)

  • Initial GitLab CI implementation
  • Mirrors GitHub Actions single-workflow v4 functionality
  • Uses GitLab Notes API for MR comments (create and update via PUT)
  • Supports GITLAB_TOKEN (recommended) and CI_JOB_TOKEN (fallback) authentication
  • Supports CI/CD Variables, group-level variables, and Vault integration for secrets
  • Cache auto-commit on main branch pushes with [skip ci]
  • Score diff tracking with emoji indicators
  • Four-stage pipeline: review, comment, cache, optimize
  • Path-based change detection via rules: changes:
  • Auto-optimize support with conditional MR source branch push
  • Artifact-based data passing between jobs

Install with Tessl CLI

npx tessl i tessl-labs/tessl-skill-review-ci@0.2.0

azure-devops.md

circleci.md

github-actions.md

gitlab-ci.md

jenkins.md

README.md

SKILL.md

TESTING.md

tile.json