Write professional, persuasive complaint letters to US airlines emphasizing loyalty status, DOT regulations, and airline commitments.
93
94%
Does it follow best practices?
Impact
93%
1.38xAverage score across 10 eval scenarios
Advisory
Suggest reviewing before use
{
"context": "Tests whether the agent correctly identifies that 42 days have passed (exceeding the 30-day threshold), recommends filing a DOT complaint now with the specific filing channel, directs Priya to resend to executive relations, includes social media tactics, refrains from advising lawsuit threats, avoids promising specific outcomes, recognizes the cancellation itself as a DOT-filing trigger, and includes follow-up compensation logging guidance.",
"type": "weighted_checklist",
"checklist": [
{
"name": "DOT complaint recommended",
"description": "Recommends that Priya file a DOT complaint, given that more than 30 days have passed without a substantive airline response",
"max_score": 11
},
{
"name": "DOT filing channel specified",
"description": "Provides the specific DOT filing channel or URL (airconsumer.dot.gov or similar) rather than just mentioning DOT in the abstract",
"max_score": 7
},
{
"name": "Executive contacts as primary",
"description": "Recommends re-sending or escalating to executive customer relations (not general customer service) as the primary escalation target",
"max_score": 10
},
{
"name": "Resend letter to executive relations",
"description": "Advises Priya to resend her complaint letter (or a revised/strengthened version) specifically to executive customer relations, not just mention executive contacts as an abstract channel",
"max_score": 8
},
{
"name": "Social media tactics included",
"description": "Includes at least one concrete social media tactic (e.g., tagging airline on Twitter/X, posting on LinkedIn, using specific hashtags)",
"max_score": 11
},
{
"name": "Social media channels named",
"description": "Names at least one specific social media platform to use for the escalation",
"max_score": 7
},
{
"name": "No lawsuit threat advised",
"description": "Does NOT advise Priya to threaten a lawsuit or take legal action as a tactic",
"max_score": 11
},
{
"name": "No outcome promised",
"description": "Does NOT promise or guarantee that Priya will receive reimbursement or any specific result",
"max_score": 11
},
{
"name": "Cancellation short-notice DOT basis",
"description": "Recognizes that a cancellation with less than 3 hours notice may independently warrant a DOT filing based on the cancellation itself (refund rights, rebooking obligations), not only because 30 days have passed without response",
"max_score": 9
},
{
"name": "Compensation logging guidance",
"description": "Includes guidance for Priya to come back and log any compensation received (vouchers, credits, miles) to track it — reflecting the 'After Completing the Letter' workflow in the skill",
"max_score": 7
},
{
"name": "Timeline expectations set",
"description": "Provides realistic timeline expectations for the escalated complaint (e.g., expected response timeframe after re-sending or DOT filing)",
"max_score": 8
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10
skills
frequent-flyer-advocate