CtrlK
BlogDocsLog inGet started
Tessl Logo

ligolnik/flight-weather-watch

Aviation weather briefing tool that fetches WPC prog charts, QPF, and AWC products, overlays flight routes, and generates HTML briefings with Claude analysis.

45

0.00x
Quality

90%

Does it follow best practices?

Impact

0%

0.00x

Average score across 6 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-4/

{
  "context": "Tests whether the agent knows to use --from-cache to bypass the past-departure-time check, and correctly constructs the cache prefix from the route and date.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Uses --from-cache flag",
      "description": "Uses --from-cache to rebuild from cached data, which bypasses the past-departure-time check",
      "max_score": 25
    },
    {
      "name": "Correct cache prefix",
      "description": "Cache prefix is cache_KMQY_KVGT_2026-03-20 (without _charts.json or _llm.json suffix)",
      "max_score": 20
    },
    {
      "name": "Nashville as KMQY",
      "description": "Uses KMQY as the origin airport code",
      "max_score": 10
    },
    {
      "name": "Las Vegas as KVGT",
      "description": "Uses KVGT as the destination airport code (North Las Vegas, the user's preferred Vegas airport)",
      "max_score": 10
    },
    {
      "name": "All positional args present",
      "description": "Includes all positional arguments (KMQY KVGT 2026-03-20 16:00 16000) even with --from-cache",
      "max_score": 10
    },
    {
      "name": "TAS specified",
      "description": "Includes --tas 155",
      "max_score": 5
    },
    {
      "name": "Explains past-time bypass",
      "description": "Notes explain that --from-cache bypasses the past-departure-time check so old briefings can be regenerated",
      "max_score": 15
    },
    {
      "name": "Does not suggest changing the date",
      "description": "Does not suggest faking the departure date to a future date as a workaround",
      "max_score": 5
    }
  ]
}

tile.json