CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/regression-suite-selector

Builds a CI workflow that runs only the subset of tests impacted by a PR's changes - combines a per-test → source-file dependency map (built from coverage profiles or, in build-graph projects, queried from the build system itself like Bazel `rdeps`) with the PR's `git diff --name-only`, then selects the union of (impacted by changed files + previously failing + newly added). Always pairs with a periodic full-suite run so a misconfigured map can't silently shrink coverage. Use when the regression suite is large enough that PR-time CI is the bottleneck and a full run is reserved for nightly / pre-release.

69

Quality

87%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Low

Low-risk findings worth noting

Overview
Quality
Evals
Security
Files

instrumentation-and-ci.mdreferences/

Per-test instrumentation and CI

Supporting detail for regression-suite-selector: per-framework recipes for emitting per-test coverage, the Bazel build-graph selection path, and the CI workflow. The core selector (build_map, git diff, select_tests) stays inline in SKILL.md.

Per-test coverage instrumentation (Path A)

Modify the test runner to emit per-test coverage instead of merged coverage, then feed it to build_map (SKILL.md Step 2):

  • Jest: --coverage writes coverage/coverage-final.json with per-test f (function-hit) maps when the runner is configured for it; or use jest-coverage-tracking for per-test data.
  • pytest + coverage.py: coverage run --concurrency=multiprocessing -m pytest --cov-context=test emits per-test contexts.
  • Java + JaCoCo: per-test "session" mode (destfile=...sessionId=<test>.exec).

Persist the built map as test-map.json, checked into the repo or stored as a CI artifact updated on every main run.

Path B - From the build graph (Bazel / Pants / Buck)

In Bazel projects, the dependency graph IS the test-source map:

# What tests depend on changed files?
bazel query 'kind("_test", rdeps(//..., set(<changed-files>)))'

A Bazel target is actually dependent on target Y if Y must be present, built, and up-to-date for X to build correctly; rdeps(<scope>, <target>) reverses the edge and finds targets that depend on <target> (https://bazel.build/concepts/dependencies).

CHANGED=$(git diff --name-only origin/main...HEAD | sed 's|^|//|')
bazel query "kind('_test', rdeps(//..., set(${CHANGED})))" \
  | xargs bazel test

Declared dependencies must comprehensively cover actual dependencies for correct incremental rebuilds, so the build-graph approach is only as good as BUILD-file discipline. Lint via buildozer / gazelle to catch missing declarations.

CI workflow (selected run + shadow full run)

# .github/workflows/regression.yml
jobs:
  selected:
    runs-on: ubuntu-latest
    outputs:
      verdict: ${{ steps.run.outcome }}
    steps:
      - uses: actions/checkout@v5
        with: { fetch-depth: 0 }   # full history for diff
      - name: Compute selection
        id: pick
        run: |
          CHANGED=$(git diff --name-only origin/${{ github.base_ref }}...HEAD)
          python scripts/select_tests.py --changed "$CHANGED" --map test-map.json > selection.txt
          echo "count=$(wc -l < selection.txt)" >> "$GITHUB_OUTPUT"
      - name: Run selected
        id: run
        run: xargs -a selection.txt npm test --

  shadow-full:
    if: github.run_attempt == 1 && (github.event.pull_request.number % 5 == 0)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - run: npm test
      - name: Compare with selected
        run: python scripts/compare_results.py selected.xml shadow.xml

SKILL.md

tile.json