CtrlK
BlogDocsLog inGet started
Tessl Logo

jbaruch/speaker-toolkit

Four-skill presentation system: ingest talks into a rhetoric vault, run interactive clarification, generate a speaker profile, then create new presentations that match your documented patterns. Includes an 88-entry Presentation Patterns taxonomy for scoring, brainstorming, and go-live preparation.

96

1.21x
Quality

93%

Does it follow best practices?

Impact

97%

1.21x

Average score across 30 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-7/

{
  "context": "Tests whether the agent correctly audits an illustrated outline using all standard guardrails PLUS the illustration coverage guardrail (#10). The draft has deliberate illustration issues: slide 8 is missing its Format field, slide 6's prompt doesn't use [STYLE ANCHOR], slides 23-28 have a prompt without [STYLE ANCHOR], slide 4 EXCEPTION has no justification, slide 17-22 and 23-28 have prompts that are vague/copy-paste quality, and the closing is rushed (2 min, 3 slides). The audit should catch all of these.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Standard guardrails present",
      "description": "The report includes at least 8 of the 9 standard guardrail checks (slide budget, Act 1 ratio, branding, profanity, data attribution, time-sensitive, closing, cut lines, anti-patterns) — each as a labeled check with PASS/FAIL/WARN",
      "max_score": 10
    },
    {
      "name": "Illustration coverage guardrail runs",
      "description": "The report includes an Illustration Coverage section (guardrail #10) that is NOT [SKIP] — it actually runs checks because the outline has an Illustration Style Anchor. The section reports on coverage ratio, format tags, EXCEPTION justification, style anchor reference, and/or prompt quality",
      "max_score": 12
    },
    {
      "name": "Missing Format field detected",
      "description": "The report flags that slide 8 ('The Tool Sprawl') is missing its Format field — it has Illustration and Image prompt but no '- Format:' line",
      "max_score": 10
    },
    {
      "name": "Missing EXCEPTION justification detected",
      "description": "The report flags that slide 4 ('Shownotes URL') has Format: EXCEPTION but no justification text after it explaining why it uses a real asset (contrast with slide 3 which has 'bio slide with real headshot')",
      "max_score": 10
    },
    {
      "name": "Missing [STYLE ANCHOR] detected",
      "description": "The report flags that at least one image prompt is missing the [STYLE ANCHOR] token — specifically slide 6 and/or slides 23-28 have prompts that don't start with [STYLE ANCHOR]",
      "max_score": 10
    },
    {
      "name": "Low-quality prompts flagged",
      "description": "The report flags prompts that appear to be vague or copy-pasted from the Illustration description — specifically slides 17-22 and/or 23-28 where the prompts are generic one-liners rather than detailed generation instructions",
      "max_score": 8
    },
    {
      "name": "Data attribution issue flagged",
      "description": "The report flags the orphaned statistic on slide 7 — '87% of SREs report alert fatigue' with no source attribution (the speaker note even jokes about it)",
      "max_score": 8
    },
    {
      "name": "Rushed closing detected",
      "description": "The report flags the closing section as having only 2 minutes and 3 slides — violating the recurring_issues guardrail requiring at least 3 min for closing, and/or noting the speaker's 'to wrap up fast' suggests time pressure",
      "max_score": 8
    },
    {
      "name": "Cut lines missing",
      "description": "The report flags that no [CUT LINE] markers are present in the outline despite modular_design being enabled in the profile",
      "max_score": 8
    },
    {
      "name": "Correct PASS/FAIL labeling",
      "description": "Guardrail items that pass (e.g., slide budget: 55/70) are labeled [PASS], items that fail are labeled [FAIL] or [WARN] — the labeling is consistent and accurate for the values in the outline",
      "max_score": 8
    },
    {
      "name": "Illustration checks in guardrail summary",
      "description": "The guardrail summary block includes illustration-related results — either as a single aggregated 'Illustrations:' line with coverage/format/quality metrics, or as individual illustration check lines (coverage ratio, format tags, prompt quality) clearly included alongside the standard guardrail checks. Must not show [SKIP]",
      "max_score": 8
    }
  ]
}

evals

README.md

tile.json