CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/github-actions-toolkit

Complete GitHub Actions toolkit with generation and validation capabilities for workflows, custom actions, and CI/CD configurations

97

Quality

97%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonvalidator/evals/scenario-4/

{
  "context": "Tests that the agent explicitly uses lint-only validation (not skipping validation entirely) when Docker is unavailable, documents what checks lint-only covers, and explains what cannot be verified without a container runtime.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Lint-only approach documented",
      "description": "validation-report.md explicitly states that lint-only or static analysis mode was used due to Docker being unavailable",
      "max_score": 20
    },
    {
      "name": "Validation not skipped",
      "description": "The report does NOT say that validation was skipped or deferred entirely because Docker was absent — it documents checks that were performed",
      "max_score": 20
    },
    {
      "name": "Checks performed listed",
      "description": "The report lists what was checked (e.g., YAML syntax, schema compliance, runner labels, expression syntax, security)",
      "max_score": 15
    },
    {
      "name": "Docker limitations acknowledged",
      "description": "The report explicitly identifies at least one category of check that requires Docker/act and cannot be done in lint-only mode",
      "max_score": 15
    },
    {
      "name": "Issues identified",
      "description": "The report identifies at least one actual issue or observation about the workflow (e.g., action version currency, SHA pinning status, or potential concerns)",
      "max_score": 15
    },
    {
      "name": "actionlint referenced",
      "description": "The report mentions actionlint (by name or as the tool used for static analysis)",
      "max_score": 15
    }
  ]
}

tile.json