CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/ansible-toolkit

Complete ansible toolkit with generation and validation capabilities

97

Quality

97%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsongenerator/evals/scenario-3/

{
  "context": "Tests whether the agent replaces the ignore_errors anti-pattern with block/rescue/always error handling, uses FQCN modules, replaces command module with dedicated modules, and applies correct boolean syntax. Criteria map directly to error handling, FQCN, and module selection instructions.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "No ignore_errors present",
      "description": "The output playbook contains NO instances of `ignore_errors: true` or `ignore_errors: yes`",
      "max_score": 18
    },
    {
      "name": "block/rescue structure used",
      "description": "The deployment steps are wrapped in a `block:` / `rescue:` structure that handles failures and performs rollback",
      "max_score": 18
    },
    {
      "name": "always block for logging",
      "description": "An `always:` section writes deployment status to the log file regardless of success or failure",
      "max_score": 12
    },
    {
      "name": "No command for file copy",
      "description": "Copying or backing up files does NOT use `ansible.builtin.command: cp ...` — uses `ansible.builtin.copy` with `remote_src: true` or `ansible.builtin.file` instead",
      "max_score": 12
    },
    {
      "name": "FQCN modules used",
      "description": "All module references use FQCN (no short names like `copy:`, `service:`, `command:`)",
      "max_score": 12
    },
    {
      "name": "All tasks have name field",
      "description": "Every task in the playbook has a `name:` field",
      "max_score": 8
    },
    {
      "name": "Boolean true/false syntax",
      "description": "All boolean values use `true` or `false`, not `yes` or `no`",
      "max_score": 8
    },
    {
      "name": "Rollback restores backup",
      "description": "The `rescue:` block contains a task that restores the backup JAR and starts the service",
      "max_score": 12
    }
  ]
}

generator

SKILL.md

tile.json