CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/promql-toolkit

Complete PromQL toolkit with generation and validation capabilities

94

Quality

94%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsongenerator/evals/scenario-0/

{
  "context": "Tests whether the agent correctly matches PromQL functions to metric types: rate() on counters, direct use on gauges. Also checks that aggregations use by() and that label filters are present.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "rate() on counter",
      "description": "The throughput query uses rate() or increase() applied to http_requests_total (a counter), not a raw selector",
      "max_score": 15
    },
    {
      "name": "No rate() on gauge",
      "description": "The memory query does NOT wrap node_memory_MemAvailable_bytes in rate() or increase() — it uses the metric directly or with an *_over_time() function",
      "max_score": 20
    },
    {
      "name": "Label filter on counter queries",
      "description": "At least one counter query includes a job label filter (e.g., {job=\"api-gateway\"})",
      "max_score": 12
    },
    {
      "name": "by() on throughput aggregation",
      "description": "The per-endpoint throughput query uses sum by (endpoint) or an equivalent explicit by() clause",
      "max_score": 12
    },
    {
      "name": "by() on error ratio",
      "description": "Any aggregation in the error ratio query includes an explicit by() or without() clause rather than a bare sum/count",
      "max_score": 10
    },
    {
      "name": "Error ratio uses division",
      "description": "The error ratio query divides 5xx request rate by total request rate (binary / operator between two rate() expressions)",
      "max_score": 12
    },
    {
      "name": "Status code regex for 5xx",
      "description": "The error ratio query uses a regex match on status_code to select 5xx errors (e.g., status_code=~\"5..\")",
      "max_score": 10
    },
    {
      "name": "Memory query preserves instance label",
      "description": "The memory query either has no aggregation (returning all instances) or explicitly keeps the instance label via by(instance)",
      "max_score": 9
    }
  ]
}

generator

evals

scenario-0

capability.txt

criteria.json

task.md

instructions.json

summary_infeasible.json

summary.json

SKILL.md

tile.json