CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/jenkinsfile-toolkit

Complete jenkinsfile toolkit with generation and validation capabilities

97

Quality

97%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Risky

Do not use without reviewing

Overview
Quality
Evals
Security
Files

criteria.jsonvalidator/evals/scenario-3/

{
  "context": "Tests whether the agent flags the @Library dependency as requiring cross-reference validation, identifies the unpinned library version as a risk, notes the shared library steps used, and warns that validation of the Jenkinsfile in isolation is insufficient.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Main script invoked",
      "description": "Report mentions invoking or attempting bash scripts/validate_jenkinsfile.sh",
      "max_score": 10
    },
    {
      "name": "@Library call noted",
      "description": "Report identifies the @Library('platform-lib') declaration and names the library being loaded",
      "max_score": 20
    },
    {
      "name": "Unpinned version flagged",
      "description": "Report flags that the @Library declaration does not specify a version/tag/commit and uses the default branch — which can cause unexpected breakage when the library updates",
      "max_score": 20
    },
    {
      "name": "Library steps identified",
      "description": "Report identifies buildAndPush and deployApp as shared library steps (not standard Jenkins steps) that require library cross-reference",
      "max_score": 15
    },
    {
      "name": "Isolation warning issued",
      "description": "Report warns that the Jenkinsfile cannot be validated in isolation — the shared library step signatures and current library state must be verified",
      "max_score": 20
    },
    {
      "name": "Deployment caution",
      "description": "Report notes that a clean validation result does not confirm deployment readiness given the unverified library dependency",
      "max_score": 15
    }
  ]
}

tile.json