CtrlK
BlogDocsLog inGet started
Tessl Logo

jbaruch/frequent-flyer-advocate

Write professional, persuasive complaint letters to US airlines emphasizing loyalty status, DOT regulations, and airline commitments.

93

1.38x
Quality

94%

Does it follow best practices?

Impact

93%

1.38x

Average score across 10 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-1/

{
  "context": "Tests whether the agent constructs a complaint letter following the required structural format, leads the opening with loyalty credentials, uses the airline's own published words against them, provides precise regulatory citations, calibrates compensation correctly for severity and elite status, and closes with appropriate tone and a specific response deadline.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Subject line: flight details",
      "description": "Subject line includes both the flight number (DL408) and the date of the incident (January 14, 2026 or equivalent)",
      "max_score": 10
    },
    {
      "name": "Subject line: loyalty tier",
      "description": "Subject line references the passenger's loyalty tier (Platinum Medallion, Platinum, or equivalent elite status)",
      "max_score": 10
    },
    {
      "name": "Opening leads with loyalty",
      "description": "The opening paragraph establishes years of loyalty, approximate miles or lifetime status, and/or tier status BEFORE describing the incident — not the incident first",
      "max_score": 10
    },
    {
      "name": "Incident narrative specificity",
      "description": "The incident section includes at least three specific details: flight number, route (LAX to JFK), date, duration of delay (approximately 7 hours), class of service, or ticket price",
      "max_score": 10
    },
    {
      "name": "Airline quote present",
      "description": "The letter contains at least one exact verbatim quote attributed to a named Delta policy document, program document, mission/vision statement, or CEO communication (not paraphrased)",
      "max_score": 10
    },
    {
      "name": "Quote vs. reality contrast",
      "description": "The section containing the airline quote explicitly contrasts Delta's stated promise or commitment with what actually happened during the incident",
      "max_score": 10
    },
    {
      "name": "Specific regulatory citation",
      "description": "The regulatory basis section cites at least one specific regulation, act, or rule by name or number (e.g., FAA Reauthorization Act of 2024, a specific DOT rule number) — NOT just a vague reference to 'federal regulations' or 'consumer protection laws'",
      "max_score": 10
    },
    {
      "name": "Response deadline specified",
      "description": "The requested remedy section asks for a response within a specific timeframe falling between 14 and 21 business days",
      "max_score": 10
    },
    {
      "name": "Compensation matches major disruption",
      "description": "The compensation requested is consistent with a major disruption for an elite member: includes at least one of (50,000+ miles credit, full out-of-pocket expense reimbursement, or future travel voucher) AND the miles/voucher ask is NOT limited to the minor inconvenience range (e.g., not just a $50 voucher or 5,000 miles)",
      "max_score": 10
    },
    {
      "name": "Closing: DOT mention without threat",
      "description": "The closing references the passenger's awareness of or right to file a DOT complaint, but does NOT frame it as a direct threat, ultimatum, or demand ('I will file' as a consequence is acceptable; 'I am threatening you with' is not)",
      "max_score": 10
    }
  ]
}

evals

scenario-1

criteria.json

task.md

tile.json