CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/azure-pipelines-toolkit

Complete azure-pipelines toolkit with generation and validation capabilities

97

Quality

97%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsongenerator/evals/scenario-2/

{
  "context": "Tests whether the agent correctly implements Docker tagging rules (Build.BuildId as primary, also push latest, but deploy using only specific tag, never :latest in production), uses deployment jobs for environment tracking, and uses KubernetesManifest@0 properly.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Build.BuildId tag",
      "description": "Docker images are tagged with `$(Build.BuildId)` (or a variable holding it, like `$(tag)` where tag = $(Build.BuildId))",
      "max_score": 12
    },
    {
      "name": "Also pushes latest tag",
      "description": "The Docker push step includes both the build-specific tag AND `latest` in the tags list",
      "max_score": 8
    },
    {
      "name": "Deploy uses specific tag only",
      "description": "The Kubernetes deployment task references the specific build tag (e.g., $(tag)) and NOT `:latest` as the image tag",
      "max_score": 15
    },
    {
      "name": "Deployment jobs used",
      "description": "Both staging and production deploy stages use `deployment:` jobs (not regular `job:`) with `environment:` specified",
      "max_score": 10
    },
    {
      "name": "KubernetesManifest@0 task",
      "description": "The pipeline uses `KubernetesManifest@0` (pinned at @0) for Kubernetes deployments",
      "max_score": 8
    },
    {
      "name": "Docker@2 task used",
      "description": "The Docker build/push step uses `Docker@2` (pinned at @2)",
      "max_score": 8
    },
    {
      "name": "Pinned vmImage",
      "description": "Pool vmImage uses a specific version (e.g., ubuntu-22.04) not ubuntu-latest",
      "max_score": 7
    },
    {
      "name": "displayName on all tasks",
      "description": "Every task and script step has a displayName property",
      "max_score": 7
    },
    {
      "name": "PascalCase stage names",
      "description": "All stage names use PascalCase",
      "max_score": 7
    },
    {
      "name": "dependsOn between stages",
      "description": "Deploy stages have `dependsOn` referencing prior stages",
      "max_score": 8
    },
    {
      "name": "No @latest task usage",
      "description": "No task in the file uses @latest version",
      "max_score": 10
    }
  ]
}

generator

evals

instructions.json

summary_infeasible.json

summary.json

SKILL.md

tile.json