CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/jenkinsfile-toolkit

Complete jenkinsfile toolkit with generation and validation capabilities

97

Quality

97%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Risky

Do not use without reviewing

Overview
Quality
Evals
Security
Files

criteria.jsonvalidator/evals/scenario-1/

{
  "context": "Tests whether the agent uses the shared library validator script (not the main Jenkinsfile validator), detects the filename naming convention violation (underscores instead of camelCase), flags the missing call() method, and correctly identifies the @NonCPS misuse on a method that calls CPS pipeline steps (sh).",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Uses shared library script",
      "description": "Report mentions invoking or attempting bash scripts/validate_shared_library.sh — not the main validate_jenkinsfile.sh",
      "max_score": 20
    },
    {
      "name": "Naming convention violation flagged",
      "description": "Report identifies that deploy_step.groovy violates camelCase naming conventions for vars/ files (should be deployStep.groovy)",
      "max_score": 20
    },
    {
      "name": "Missing call() method flagged",
      "description": "Report identifies that the file lacks a call() method, which is the required entry point for vars/*.groovy step files",
      "max_score": 20
    },
    {
      "name": "@NonCPS misuse flagged",
      "description": "Report identifies that @NonCPS is incorrectly applied to execute() because the method calls sh steps, which are CPS pipeline steps — @NonCPS cannot call CPS functions",
      "max_score": 25
    },
    {
      "name": "Corrected code provided",
      "description": "Report includes corrected code for at least one of the identified issues (e.g., renamed file, added call() method, or removed @NonCPS)",
      "max_score": 15
    }
  ]
}

tile.json