Write professional, persuasive complaint letters to US airlines emphasizing loyalty status, DOT regulations, and airline commitments.
93
94%
Does it follow best practices?
Impact
93%
1.38xAverage score across 10 eval scenarios
Advisory
Suggest reviewing before use
{
"context": "Tests whether the skill correctly files a completed complaint to the complaint bank with accurate parameters, provides escalation guidance with compensation logging instructions, and prompts the user to return with the resolution.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Complaint filed to bank",
"description": "post-letter-actions.md shows the complaints-bank.py file command (or describes filing) with the complaint details",
"max_score": 15
},
{
"name": "Correct airline and flight",
"description": "The filing command uses airline=UA and flight=UA456",
"max_score": 8
},
{
"name": "Correct category",
"description": "The filing command uses category=DENIED_BOARDING (not DELAY, not SERVICE, not OTHER)",
"max_score": 10
},
{
"name": "Correct severity",
"description": "The filing command uses severity=RIGHTS_VIOLATION (denied boarding is a federal rights violation per the skill's severity framework)",
"max_score": 8
},
{
"name": "Route and date present",
"description": "The filing command includes route=SFO-DEN and flight-date=2026-02-28",
"max_score": 7
},
{
"name": "Summary is concise",
"description": "The filing command includes a summary that is 1-2 sentences describing what happened — not a full paragraph or the entire letter",
"max_score": 7
},
{
"name": "Outcome recorded",
"description": "The filing command includes what was requested in the letter (statutory compensation, $450 reimbursement, miles, upgrade — at least two of these)",
"max_score": 8
},
{
"name": "DOT immediate filing recommended",
"description": "Escalation guidance recommends filing a DOT complaint immediately (in parallel with the letter) since denied boarding is a federal rights violation",
"max_score": 12
},
{
"name": "Follow-up prompt included",
"description": "Tells Tom to come back when he hears from the airline so the resolution can be recorded and any credits/vouchers logged",
"max_score": 10
},
{
"name": "Credits logging mentioned for future",
"description": "Notes that when Tom reports back with compensation, miles or vouchers should be logged to the credits inventory",
"max_score": 8
},
{
"name": "Resolve command shown for future",
"description": "Shows or describes the complaints-bank.py resolve command that will be used when Tom reports the outcome",
"max_score": 7
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10
skills
frequent-flyer-advocate