CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/promql-toolkit

Complete PromQL toolkit with generation and validation capabilities

94

Quality

94%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonvalidator/evals/scenario-1/

{
  "context": "Verify that the agent correctly identifies `rate()` misuse on a gauge metric, explains the semantic error, and provides a corrected query.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Metric type misuse is explicitly identified",
      "description": "The report explicitly states that `rate()` is only valid for counters (monotonically increasing metrics) and that `process_resident_memory_bytes` is a gauge.",
      "max_score": 30
    },
    {
      "name": "Intent vs implementation mismatch is explained",
      "description": "The report explains what `rate()` on a gauge actually computes (per-second rate of change) versus what the user wants (current absolute value).",
      "max_score": 25
    },
    {
      "name": "Correct query is provided",
      "description": "The recommended query removes `rate()` and uses a direct gauge query or `avg_over_time()` as appropriate for showing current memory usage.",
      "max_score": 25
    },
    {
      "name": "Syntax validity is stated",
      "description": "The report includes a clear statement that the query is syntactically valid (it parses correctly) before explaining the semantic issue.",
      "max_score": 10
    },
    {
      "name": "Report uses the required section structure",
      "description": "The output file contains all five required sections: Syntax Check, Metric Type Analysis, Intent vs Implementation, Recommended Query, and Cardinality Notes.",
      "max_score": 10
    }
  ]
}

tile.json