CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/azure-pipelines-toolkit

Complete azure-pipelines toolkit with generation and validation capabilities

97

Quality

97%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsongenerator/evals/scenario-0/

{
  "context": "Tests whether the agent pins task versions, uses a specific vmImage version (not ubuntu-latest), adds displayName to every step, uses correct Sentence case for displayName, and configures Cache@2 for npm.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Pinned vmImage",
      "description": "The pool vmImage is set to a specific version (e.g., ubuntu-22.04) and NOT ubuntu-latest or windows-latest",
      "max_score": 10
    },
    {
      "name": "Task version pinned",
      "description": "All tasks use the @version syntax (e.g., NodeTool@0, PublishTestResults@2) and NOT @latest or bare task names without version",
      "max_score": 10
    },
    {
      "name": "displayName on all tasks",
      "description": "Every task and script step has a displayName property (no step without displayName)",
      "max_score": 12
    },
    {
      "name": "Sentence case displayName",
      "description": "All displayName values use Sentence case (first word capitalized, rest lowercase, e.g., 'Install dependencies', not 'INSTALL DEPENDENCIES' or 'Install Dependencies')",
      "max_score": 8
    },
    {
      "name": "Cache@2 for npm",
      "description": "A Cache@2 task is included to cache npm packages",
      "max_score": 10
    },
    {
      "name": "PublishTestResults task",
      "description": "A PublishTestResults@2 task is included with condition: succeededOrFailed()",
      "max_score": 10
    },
    {
      "name": "Explicit trigger branches",
      "description": "The trigger section specifies explicit branch includes (main and develop) and does NOT use `trigger: none`",
      "max_score": 8
    },
    {
      "name": "NodeTool version specified",
      "description": "NodeTool@0 (or equivalent) is used with an explicit versionSpec (e.g., '20.x', '18.x')",
      "max_score": 8
    },
    {
      "name": "npm ci used",
      "description": "The pipeline uses `npm ci` (not `npm install`) for dependency installation",
      "max_score": 8
    },
    {
      "name": "No @latest usage",
      "description": "No task uses @latest version (zero occurrences of @latest in the file)",
      "max_score": 8
    },
    {
      "name": "camelCase variables",
      "description": "Pipeline variable names use camelCase or snake_case (not mixed case like 'BuildConfiguration' or 'BUILD_CONFIGURATION')",
      "max_score": 8
    }
  ]
}

generator

evals

scenario-0

capability.txt

criteria.json

task.md

instructions.json

summary_infeasible.json

summary.json

SKILL.md

tile.json