CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/azure-pipelines-toolkit

Complete azure-pipelines toolkit with generation and validation capabilities

97

Quality

97%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsongenerator/evals/scenario-1/

{
  "context": "Tests whether the agent avoids hardcoding secrets in pipeline YAML and instead uses variable groups, Azure Key Vault references, or service connections. Also tests multi-stage structure with deployment jobs and PascalCase stage/job naming.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "No hardcoded secrets",
      "description": "The pipeline YAML does NOT contain any literal secret values — no passwords, API keys, or connection strings as plain text in variables or task inputs",
      "max_score": 15
    },
    {
      "name": "Variable group or Key Vault reference",
      "description": "The pipeline references a variable group or Azure Key Vault task/link for the payment gateway credentials, rather than inline YAML variables",
      "max_score": 12
    },
    {
      "name": "Service connection for registry",
      "description": "The Docker registry connection uses a service connection reference (not hardcoded registry credentials)",
      "max_score": 10
    },
    {
      "name": "Deployment job in deploy stage",
      "description": "The production deploy stage uses a `deployment:` job (not a regular `job:`) with an environment specified",
      "max_score": 10
    },
    {
      "name": "PascalCase stage names",
      "description": "All stage names use PascalCase (e.g., Build, IntegrationTest, DeployProduction) — NOT snake_case, kebab-case, or ALL_CAPS",
      "max_score": 8
    },
    {
      "name": "PascalCase job names",
      "description": "All job names use PascalCase",
      "max_score": 7
    },
    {
      "name": "displayName on all tasks",
      "description": "Every task and script step has a displayName property",
      "max_score": 8
    },
    {
      "name": "Pinned vmImage",
      "description": "Pool vmImage uses a specific version (e.g., ubuntu-22.04) not ubuntu-latest",
      "max_score": 8
    },
    {
      "name": "Task versions pinned",
      "description": "All tasks include explicit version pins (@N) and none use @latest",
      "max_score": 8
    },
    {
      "name": "Explicit trigger",
      "description": "The pipeline has an explicit trigger with branch includes, not `trigger: none`",
      "max_score": 7
    },
    {
      "name": "dependsOn between stages",
      "description": "Deploy stage has a `dependsOn` referencing the prior stage",
      "max_score": 7
    }
  ]
}

generator

evals

instructions.json

summary_infeasible.json

summary.json

SKILL.md

tile.json