CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/makefile-toolkit

Complete makefile toolkit with generation and validation capabilities

97

Quality

97%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonvalidator/evals/scenario-1/

{
  "context": "Tests that the agent identifies hardcoded credentials (DB_PASS, API_KEY), the .EXPORT_ALL_VARIABLES risk, unquoted variable expansion in rm commands, and the bare make call, and fixes all of them.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Hardcoded password removed",
      "description": "The corrected Makefile does NOT contain the literal value 'hunter2' — DB_PASS uses an environment variable reference or is removed",
      "max_score": 15
    },
    {
      "name": "Hardcoded API key removed",
      "description": "The corrected Makefile does NOT contain the literal value 'sk-live-abc123def456' — API_KEY uses an environment variable reference or is removed",
      "max_score": 15
    },
    {
      "name": ".EXPORT_ALL_VARIABLES removed",
      "description": "The .EXPORT_ALL_VARIABLES: directive is removed from the corrected Makefile",
      "max_score": 15
    },
    {
      "name": "EXPORT_ALL risk explained",
      "description": "security-audit.md explains that .EXPORT_ALL_VARIABLES exports all variables (including secrets) to every subprocess",
      "max_score": 10
    },
    {
      "name": "Unquoted rm fixed",
      "description": "The rm -rf $(DEPLOY_DIR) and/or rm -rf $(BUILD_DIR) commands are updated to quote the variable (e.g., \"$(DEPLOY_DIR)\") in the corrected Makefile",
      "max_score": 10
    },
    {
      "name": "Bare make replaced",
      "description": "The bare 'make deploy' in the rollback target is replaced with '$(MAKE) deploy' in the corrected Makefile",
      "max_score": 15
    },
    {
      "name": "Credentials risk explained",
      "description": "security-audit.md explains the risk of hardcoded credentials (committed to source control, visible in config files)",
      "max_score": 10
    },
    {
      "name": "Bare make risk explained",
      "description": "security-audit.md explains why $(MAKE) should be used instead of bare make for recursive calls",
      "max_score": 10
    }
  ]
}

tile.json