CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/azure-pipelines-toolkit

Complete azure-pipelines toolkit with generation and validation capabilities

97

Quality

97%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonvalidator/evals/scenario-2/

{
  "context": "Tests that the agent identifies missing displayNames on jobs and steps, the absence of a cache task for node_modules, missing job/stage timeouts, and the :latest Docker tag; then produces a corrected pipeline and a structured best-practices report.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Missing displayName on jobs detected",
      "description": "best-practices-report.md (or equivalent) flags that BuildJob and/or PackageJob lack a displayName property",
      "max_score": 10
    },
    {
      "name": "Missing displayName on steps detected",
      "description": "best-practices-report.md flags that one or more task steps lack a displayName property",
      "max_score": 10
    },
    {
      "name": "Missing cache for npm dependencies detected",
      "description": "best-practices-report.md flags the absence of a Cache@2 task before the npm install step",
      "max_score": 15
    },
    {
      "name": "Missing timeout detected",
      "description": "best-practices-report.md flags that no timeoutInMinutes is set on jobs or stages",
      "max_score": 10
    },
    {
      "name": "latest Docker tag flagged",
      "description": "best-practices-report.md flags the `latest` entry in the Docker tags list as a best practices concern (unpinned image tag)",
      "max_score": 10
    },
    {
      "name": "displayNames added in corrected file",
      "description": "In the corrected azure-pipelines.yml, at least one job and the majority of task steps have a displayName property added",
      "max_score": 15
    },
    {
      "name": "Cache task added in corrected file",
      "description": "In the corrected azure-pipelines.yml, a Cache@2 (or equivalent) task is present before the npm install step, with a key based on package-lock.json or similar",
      "max_score": 15
    },
    {
      "name": "Category assigned to each finding",
      "description": "best-practices-report.md assigns a category (readability, performance, or reliability) to each finding",
      "max_score": 10
    },
    {
      "name": "Fix described for each finding",
      "description": "best-practices-report.md includes a description of the fix applied (not just the problem) for each finding",
      "max_score": 5
    }
  ]
}

tile.json