CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/terraform-toolkit

Complete terraform toolkit with generation and validation capabilities

93

Quality

93%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonvalidator/evals/scenario-4/

{
  "context": "Tests whether the agent treats tflint warnings for undeclared variables as non-critical (bad) or treats them seriously by documenting why they matter and fixing all undeclared variable references, while following the complete validation workflow.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "tflint run",
      "description": "The validation_report.md documents running tflint (or noting it was skipped with reason)",
      "max_score": 8
    },
    {
      "name": "Missing vars identified",
      "description": "The report identifies the specific undeclared variables: subnet_id, app_name, environment, owner_email, cost_center",
      "max_score": 12
    },
    {
      "name": "Risk explanation",
      "description": "The report explains why undeclared variables are a problem (silently treated as null, masking misconfiguration, runtime errors at apply time)",
      "max_score": 14
    },
    {
      "name": "Variables declared with types",
      "description": "The fixed/variables.tf file declares all missing variables with a `type` attribute",
      "max_score": 12
    },
    {
      "name": "Variables declared with descriptions",
      "description": "The fixed/variables.tf file declares all missing variables with a `description` attribute",
      "max_score": 10
    },
    {
      "name": "All 5 missing vars fixed",
      "description": "fixed/variables.tf includes declarations for all 5 missing variables: subnet_id, app_name, environment, owner_email, cost_center",
      "max_score": 14
    },
    {
      "name": "terraform validate run",
      "description": "The validation_report.md shows terraform validate was run (after init)",
      "max_score": 8
    },
    {
      "name": "terraform fmt run",
      "description": "The validation_report.md shows terraform fmt was run",
      "max_score": 6
    },
    {
      "name": "extract_tf_info_wrapper run",
      "description": "The validation_report.md documents running the extract_tf_info_wrapper.sh script",
      "max_score": 8
    },
    {
      "name": "Warnings not dismissed",
      "description": "The report does NOT characterize the undeclared variable warnings as safe to ignore or non-critical",
      "max_score": 8
    }
  ]
}

tile.json