CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/github-actions-toolkit

Complete GitHub Actions toolkit with generation and validation capabilities for workflows, custom actions, and CI/CD configurations

97

Quality

97%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonvalidator/evals/scenario-0/

{
  "context": "Tests that the agent identifies the script injection vulnerability (github.event.pull_request.title and .user.login interpolated directly into shell run steps), explains the attack vector, and fixes it by routing untrusted values through environment variables.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Injection identified",
      "description": "validation-report.md (or equivalent) identifies the direct ${{ }} interpolation in run: steps as a script injection risk",
      "max_score": 15
    },
    {
      "name": "Attack vector explained",
      "description": "The report explains that a PR author could inject shell commands via the title or username values",
      "max_score": 10
    },
    {
      "name": "PR title via env var",
      "description": "In pr-comment.yml, github.event.pull_request.title is no longer interpolated directly in run: — it is passed through an env: block variable",
      "max_score": 20
    },
    {
      "name": "User login via env var",
      "description": "In pr-comment.yml, github.event.pull_request.user.login is no longer interpolated directly in run: — it is passed through an env: block variable",
      "max_score": 15
    },
    {
      "name": "PR number via env var or safe",
      "description": "github.event.pull_request.number is either passed through an env: variable or is used in a context where it is safe (numeric values)",
      "max_score": 10
    },
    {
      "name": "Env vars referenced safely",
      "description": "The shell script references the sanitized values using $ENV_VAR syntax (quoted, e.g., \"$PR_TITLE\") rather than ${{ }} expressions",
      "max_score": 15
    },
    {
      "name": "Issue type labeled",
      "description": "The validation report labels this as a security/injection issue (not just a warning or style issue)",
      "max_score": 10
    },
    {
      "name": "Fix applied in corrected file",
      "description": "The corrected pr-comment.yml has no direct ${{ github.event.* }} interpolation inside run: blocks",
      "max_score": 5
    }
  ]
}

tile.json