Closing the intent-to-code chasm - specification-driven development with BDD verification chain
86
92%
Does it follow best practices?
Impact
86%
1.82xAverage score across 14 eval scenarios
Advisory
Suggest reviewing before use
{
"context": "Tests whether the agent produces a complete set of planning artifacts from a feature spec: structured technical context, technology decisions with rationale, data model with state transitions, API contracts traceable to requirements, and spec quality assessment before planning.",
"type": "weighted_checklist",
"checklist": [
{
"name": "plan.md exists",
"description": "A plan.md file is created at specs/001-notification-service/plan.md",
"max_score": 5
},
{
"name": "Technical Context fields",
"description": "plan.md contains a Technical Context section with all required fields filled in: Language/Version, Primary Dependencies, Storage, Testing, Target Platform, Project Type, Performance Goals, Constraints, Scale/Scope — none left as empty or placeholder-only",
"max_score": 12
},
{
"name": "No bare Option labels",
"description": "plan.md does NOT contain 'Option 1:', 'Option 2:', 'Option 3:' labels or '[REMOVE IF UNUSED]' markers from the template",
"max_score": 5
},
{
"name": "research.md with rationale",
"description": "research.md exists and contains at least one technology decision with explicit rationale and at least one alternative considered",
"max_score": 8
},
{
"name": "data-model.md entities",
"description": "data-model.md exists and defines at least two entities (e.g., Notification, Channel, RetryPolicy) with their fields listed",
"max_score": 8
},
{
"name": "State transitions in data-model",
"description": "data-model.md includes the notification status state machine or state transitions (queued → sending → delivered/failed)",
"max_score": 8
},
{
"name": "contracts/ directory",
"description": "At least one API contract file exists in specs/001-notification-service/contracts/",
"max_score": 6
},
{
"name": "Contracts reference spec requirements",
"description": "The API contract(s) reference specific functional requirements by identifier (FR-XXX) or clearly trace to individual requirements from the spec",
"max_score": 8
},
{
"name": "quickstart.md exists with scenarios",
"description": "quickstart.md exists and contains specific test scenarios or example commands for exercising the notification feature (not just generic instructions)",
"max_score": 6
},
{
"name": "No governance content in plan",
"description": "plan.md does NOT contain project-wide principles, non-negotiable rules applying to all features, or team workflow/process requirements — governance content belongs in CONSTITUTION.md",
"max_score": 8
},
{
"name": "Spec quality assessment performed",
"description": "The agent assessed the spec quality before planning: checked FR count, flagged any missing measurable criteria, identified unresolved clarifications, and reported a quality score or summary",
"max_score": 12
},
{
"name": "Plan decisions trace to spec FRs",
"description": "plan.md or research.md references specific FR-XXX identifiers from the spec when justifying technical decisions (e.g., choosing a message queue to satisfy FR-003 about retry policies)",
"max_score": 8
},
{
"name": "context.json updated",
"description": ".specify/context.json is updated with plan-related data (e.g., architecture node classifications, eval scores, or other dashboard metadata) without overwriting existing fields",
"max_score": 6
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10
scenario-11
scenario-12
scenario-13
scenario-14
rules
skills
iikit-00-constitution
scripts
dashboard
iikit-01-specify
iikit-02-plan
iikit-03-checklist
scripts
bash
dashboard
iikit-04-testify
iikit-05-tasks
iikit-06-analyze
iikit-07-implement
iikit-08-taskstoissues
iikit-bugfix
scripts
dashboard
iikit-clarify
iikit-core
references
scripts
bash
dashboard
powershell
templates