Write professional, persuasive complaint letters to US airlines emphasizing loyalty status, DOT regulations, and airline commitments.
93
94%
Does it follow best practices?
Impact
93%
1.38xAverage score across 10 eval scenarios
Advisory
Suggest reviewing before use
{
"context": "Tests whether the agent honestly assesses a weak case (minor delay), calibrates David's compensation correctly based on both denied boarding severity and Executive Platinum status, applies correct DOT complaint timing for a rights violation, provides comprehensive escalation guidance including follow-up compensation logging, and avoids promising outcomes or advising legal threats.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Weak case honesty",
"description": "assessment-sarah.md advises AGAINST writing a formal complaint letter for a 20-25 minute ATC delay with no material consequences, or explicitly states this is a weak case not worth a formal letter",
"max_score": 15
},
{
"name": "Statutory denied boarding compensation",
"description": "letter-david.md requests the maximum statutory compensation for denied boarding (references the specific dollar cap, e.g., up to $1,550 as of 2024, or '400% of one-way fare up to statutory limit')",
"max_score": 14
},
{
"name": "Status-scaled compensation",
"description": "letter-david.md requests compensation that is explicitly elevated due to David's Executive Platinum status — mentions that elite members are entitled to proportionally greater recovery or specifically requests additional miles/benefits above what a standard member would receive",
"max_score": 9
},
{
"name": "Out-of-pocket expense reimbursement",
"description": "letter-david.md requests reimbursement for the specific out-of-pocket losses (the $380 forfeited hotel and/or the $420 overnight expenses)",
"max_score": 9
},
{
"name": "DOT complaint: immediate filing",
"description": "escalation-guide.md recommends filing a DOT complaint IMMEDIATELY (or 'in parallel' with sending the letter, or 'right away') — NOT waiting 30 days — for the denied boarding situation",
"max_score": 14
},
{
"name": "Executive contact as primary",
"description": "escalation-guide.md identifies executive customer relations as the PRIMARY place to send the letter (not standard customer service as the first contact)",
"max_score": 9
},
{
"name": "DOT complaint channel cited",
"description": "escalation-guide.md includes airconsumer.dot.gov (or air consumer complaints site at DOT) as the DOT complaint filing location",
"max_score": 9
},
{
"name": "Compensation logging guidance",
"description": "escalation-guide.md includes guidance to come back and log any compensation received (vouchers, credits, miles) to the credits inventory — reflecting the 'After Completing the Letter' workflow",
"max_score": 7
},
{
"name": "Timeline expectations provided",
"description": "escalation-guide.md includes specific timeline expectations: executive-level response window (e.g., 7-14 business days) AND a no-response escalation threshold (e.g., escalate after 30 days with no response)",
"max_score": 5
},
{
"name": "No outcome promises",
"description": "Neither letter-david.md nor escalation-guide.md contains language that promises or guarantees David will receive specific compensation (no 'you will definitely get X' or 'you are guaranteed to receive')",
"max_score": 5
},
{
"name": "No lawsuit threats",
"description": "letter-david.md does NOT advise or include any reference to filing a lawsuit, taking legal action, or retaining an attorney as leverage in the complaint letter itself",
"max_score": 4
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10
skills
frequent-flyer-advocate