CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/github-actions-toolkit

Complete GitHub Actions toolkit with generation and validation capabilities for workflows, custom actions, and CI/CD configurations

97

Quality

97%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonvalidator/evals/scenario-1/

{
  "context": "Tests that the agent identifies all outdated action versions, updates them (ideally with SHA pins), explains the security risk of leaving deprecated versions in place, and produces a report table.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "checkout updated",
      "description": "actions/checkout is updated from @v2 to a newer version (v3, v4, or later)",
      "max_score": 10
    },
    {
      "name": "setup-node updated",
      "description": "actions/setup-node is updated from @v2 to a newer version",
      "max_score": 10
    },
    {
      "name": "upload-artifact updated",
      "description": "actions/upload-artifact is updated from @v2 to a newer version",
      "max_score": 10
    },
    {
      "name": "SHA pinning used",
      "description": "At least one updated action reference uses a full SHA commit hash (40-character hex) rather than a tag version",
      "max_score": 20
    },
    {
      "name": "Deprecation risk explained",
      "description": "modernization-report.md explains that deprecated versions may not receive security patches",
      "max_score": 15
    },
    {
      "name": "Report table present",
      "description": "modernization-report.md includes a table or list with: original version, updated version, and reason for each action",
      "max_score": 15
    },
    {
      "name": "All occurrences updated",
      "description": "Both jobs in ci.yml have their actions updated (checkout and setup-node appear in both jobs and both are updated)",
      "max_score": 10
    },
    {
      "name": "No v2 references remain",
      "description": "No @v2 references remain in ci.yml",
      "max_score": 10
    }
  ]
}

tile.json