CtrlK
BlogDocsLog inGet started
Tessl Logo

jbaruch/frequent-flyer-advocate

Write professional, persuasive complaint letters to US airlines emphasizing loyalty status, DOT regulations, and airline commitments.

93

1.38x
Quality

94%

Does it follow best practices?

Impact

93%

1.38x

Average score across 10 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-5/

{
  "context": "Tests whether the agent honestly assesses a weak case (minor delay), calibrates David's compensation correctly based on both denied boarding severity and Executive Platinum status, applies correct DOT complaint timing for a rights violation, provides comprehensive escalation guidance including follow-up compensation logging, and avoids promising outcomes or advising legal threats.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Weak case honesty",
      "description": "assessment-sarah.md advises AGAINST writing a formal complaint letter for a 20-25 minute ATC delay with no material consequences, or explicitly states this is a weak case not worth a formal letter",
      "max_score": 15
    },
    {
      "name": "Statutory denied boarding compensation",
      "description": "letter-david.md requests the maximum statutory compensation for denied boarding (references the specific dollar cap, e.g., up to $1,550 as of 2024, or '400% of one-way fare up to statutory limit')",
      "max_score": 14
    },
    {
      "name": "Status-scaled compensation",
      "description": "letter-david.md requests compensation that is explicitly elevated due to David's Executive Platinum status — mentions that elite members are entitled to proportionally greater recovery or specifically requests additional miles/benefits above what a standard member would receive",
      "max_score": 9
    },
    {
      "name": "Out-of-pocket expense reimbursement",
      "description": "letter-david.md requests reimbursement for the specific out-of-pocket losses (the $380 forfeited hotel and/or the $420 overnight expenses)",
      "max_score": 9
    },
    {
      "name": "DOT complaint: immediate filing",
      "description": "escalation-guide.md recommends filing a DOT complaint IMMEDIATELY (or 'in parallel' with sending the letter, or 'right away') — NOT waiting 30 days — for the denied boarding situation",
      "max_score": 14
    },
    {
      "name": "Executive contact as primary",
      "description": "escalation-guide.md identifies executive customer relations as the PRIMARY place to send the letter (not standard customer service as the first contact)",
      "max_score": 9
    },
    {
      "name": "DOT complaint channel cited",
      "description": "escalation-guide.md includes airconsumer.dot.gov (or air consumer complaints site at DOT) as the DOT complaint filing location",
      "max_score": 9
    },
    {
      "name": "Compensation logging guidance",
      "description": "escalation-guide.md includes guidance to come back and log any compensation received (vouchers, credits, miles) to the credits inventory — reflecting the 'After Completing the Letter' workflow",
      "max_score": 7
    },
    {
      "name": "Timeline expectations provided",
      "description": "escalation-guide.md includes specific timeline expectations: executive-level response window (e.g., 7-14 business days) AND a no-response escalation threshold (e.g., escalate after 30 days with no response)",
      "max_score": 5
    },
    {
      "name": "No outcome promises",
      "description": "Neither letter-david.md nor escalation-guide.md contains language that promises or guarantees David will receive specific compensation (no 'you will definitely get X' or 'you are guaranteed to receive')",
      "max_score": 5
    },
    {
      "name": "No lawsuit threats",
      "description": "letter-david.md does NOT advise or include any reference to filing a lawsuit, taking legal action, or retaining an attorney as leverage in the complaint letter itself",
      "max_score": 4
    }
  ]
}

evals

tile.json