CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/gitlab-ci-toolkit

Complete GitLab CI/CD toolkit with generation and validation capabilities for pipelines and configurations

97

Quality

97%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonvalidator/evals/scenario-0/

{
  "context": "Tests that the agent detects hardcoded DEPLOY_TOKEN and DB_PASSWORD in the variables block, the curl-pipe-to-bash pattern in the test job, the SSL bypass (curl -k) in the deploy job, and produces a corrected pipeline and structured security report.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Hardcoded DEPLOY_TOKEN detected",
      "description": "security-report.md (or equivalent) flags the DEPLOY_TOKEN variable value as a hardcoded credential",
      "max_score": 15
    },
    {
      "name": "Hardcoded DB_PASSWORD detected",
      "description": "security-report.md flags the DB_PASSWORD variable value as a hardcoded credential",
      "max_score": 10
    },
    {
      "name": "curl-pipe-to-bash detected",
      "description": "security-report.md flags `curl ... | bash` in the test job as an insecure script pattern",
      "max_score": 15
    },
    {
      "name": "SSL bypass detected",
      "description": "security-report.md flags `curl -k` in the deploy job as a TLS certificate verification bypass",
      "max_score": 10
    },
    {
      "name": "Credentials replaced with CI/CD variable references",
      "description": "In the corrected .gitlab-ci.yml, DEPLOY_TOKEN and DB_PASSWORD no longer contain literal values — they reference CI/CD masked/protected variables (e.g., $DEPLOY_TOKEN remains but its value is removed from the variables block, or a comment explains it must be set in GitLab CI/CD settings)",
      "max_score": 15
    },
    {
      "name": "curl-pipe-to-bash remediated",
      "description": "In the corrected .gitlab-ci.yml, the `curl ... | bash` command is replaced or removed (e.g., script downloaded then verified before execution, or replaced with a package install)",
      "max_score": 15
    },
    {
      "name": "SSL bypass remediated",
      "description": "In the corrected .gitlab-ci.yml, the -k flag is removed from the curl command in the deploy job",
      "max_score": 10
    },
    {
      "name": "Severity levels assigned",
      "description": "security-report.md assigns a severity level (CRITICAL, HIGH, MEDIUM, or INFO) to each finding",
      "max_score": 5
    },
    {
      "name": "Risk explanation provided",
      "description": "security-report.md explains the risk for at least two findings (e.g., credential exposure in logs, arbitrary code execution, MITM)",
      "max_score": 5
    }
  ]
}

tile.json