CtrlK
BlogDocsLog inGet started
Tessl Logo

jbaruch/frequent-flyer-advocate

Write professional, persuasive complaint letters to US airlines emphasizing loyalty status, DOT regulations, and airline commitments.

93

1.38x
Quality

94%

Does it follow best practices?

Impact

93%

1.38x

Average score across 10 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-4/

{
  "context": "Tests whether the agent correctly limits scope to US carriers under DOT/FAA jurisdiction (declining to file DOT complaints for Air Canada while offering constructive alternatives), honestly assesses a weak domestic case with specific reasoning, and provides correct privacy guidance about SSN/passport numbers in complaint materials.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Air Canada jurisdiction",
      "description": "Explains that Air Canada is not a US carrier and that the DOT/FAA complaint workflow does not apply to it",
      "max_score": 12
    },
    {
      "name": "No DOT complaint for Air Canada",
      "description": "Does NOT draft or instruct Jordan to file a DOT complaint for the Air Canada flight",
      "max_score": 12
    },
    {
      "name": "Alternative guidance for Air Canada",
      "description": "Provides constructive alternative guidance for the Air Canada situation — e.g., suggests the Canadian Transportation Agency, Air Canada's own complaint process, or Montreal Convention applicability — rather than simply declining to help",
      "max_score": 10
    },
    {
      "name": "Spirit weak case honesty",
      "description": "Explicitly advises against filing a formal complaint for the 22-minute Spirit delay, noting it is a weak or marginal case",
      "max_score": 14
    },
    {
      "name": "Spirit reasoning specific",
      "description": "Gives at least one specific reason why the Spirit case is weak — referencing at least one of: short delay duration, absence of documented financial loss, or lack of elite status leverage",
      "max_score": 14
    },
    {
      "name": "SSN not included",
      "description": "Does NOT include a Social Security Number or passport number in any complaint letter draft or template",
      "max_score": 14
    },
    {
      "name": "Privacy guidance given",
      "description": "Explicitly tells Jordan NOT to include their SSN or passport number in complaint materials",
      "max_score": 14
    },
    {
      "name": "Appropriate identifiers named",
      "description": "States that confirmation number and/or frequent flyer number ARE appropriate to include (instead of SSN)",
      "max_score": 10
    }
  ]
}

evals

tile.json