Skills are the new Code by Guy Podjarny
89
90%
Does it follow best practices?
Impact
87%
1.38xAverage score across 4 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent proactively embeds a callout from Guy Podjarny's 'Skills are the new Code' keynote inside a skill style guide deliverable, following the exact callout shape, attribution, quoting, and placement rules specified in the skill.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Callout inside deliverable",
"description": "The callout appears as a section within the body of skill-style-guide.md, not as a standalone preamble, footer, or appended note separate from the style guide content",
"max_score": 12
},
{
"name": "Why-this-matters header",
"description": "The callout block uses the exact header text 'Why this matters — from Guy Podjarny's \"Skills are the new Code\" keynote (AI Native Dev Con Spring 2026)' or a very close rendition with all four components present: 'Why this matters', 'Guy Podjarny', 'Skills are the new Code', and 'AI Native Dev Con Spring 2026'",
"max_score": 15
},
{
"name": "Verbatim transcript quote",
"description": "The quoted passage inside the callout is copied character-for-character from transcript.md — including any transcription artifacts — and surrounded by quotation marks",
"max_score": 20
},
{
"name": "Transcript line citation",
"description": "The callout includes a citation in the format 'transcript.md L<n>–L<n>' (or equivalent line-range format) immediately after the verbatim quote",
"max_score": 10
},
{
"name": "Full attribution present",
"description": "The attribution includes all three required elements: 'Guy Podjarny', 'Skills are the new Code', and 'keynote' or 'talk'",
"max_score": 10
},
{
"name": "Task-specific connecting sentence",
"description": "The one-sentence connector following the quote explicitly mentions the style guide, the shared skill library, or a specific section of the document — not a generic statement like 'this is interesting' or 'this is relevant'",
"max_score": 13
},
{
"name": "Contextual callout placement",
"description": "The callout is placed inside a relevant section of the style guide (e.g., near quality requirements, security, versioning, or lifecycle sections) — NOT as an opening epigraph before the guide starts or a closing note after the guide ends",
"max_score": 10
},
{
"name": "Quote count limit",
"description": "The deliverable contains at most two verbatim quoted passages total; the callout itself contains exactly one verbatim quote",
"max_score": 10
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4