Reference tile for Themis, a Node.js and TypeScript unit test framework designed for AI coding agents. Covers unit-test authoring, Jest/Vitest migration, agent-readable failure output with repair hints, and first-class integrations for Claude Code, Cursor, and generic agents.
96
94%
Does it follow best practices?
Impact
97%
2.69xAverage score across 10 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent uses the built-in Vitest migration commands (including the --rewrite-imports --convert codemod flags) rather than manual rewrites, and correctly documents the full post-migration failure-handling workflow.",
"type": "weighted_checklist",
"checklist": [
{
"name": "themis migrate vitest in script",
"description": "migrate.sh contains `themis migrate vitest` (or `npx/bunx themis migrate vitest`) as a migration step",
"max_score": 15
},
{
"name": "--rewrite-imports --convert flags used",
"description": "migrate.sh contains `themis migrate vitest --rewrite-imports --convert` (or equivalent with npx/bunx) for the full conversion pass",
"max_score": 15
},
{
"name": "No manual sed/awk rewrite",
"description": "migrate.sh does NOT use sed, awk, perl, or find-replace scripts to manually convert Vitest syntax to Themis syntax across test files",
"max_score": 10
},
{
"name": "--rerun-failed in script",
"description": "migrate.sh includes `themis test --rerun-failed` as a step after fixing failures",
"max_score": 10
},
{
"name": "--update-contracts in script",
"description": "migrate.sh includes `themis test --update-contracts` as a step for handling intentional contract drift",
"max_score": 10
},
{
"name": "Vitest deps removal mentioned",
"description": "migrate.sh or MIGRATION_PLAN.md includes removing Vitest-related devDependencies (vitest, @vitest/globals, etc.) from package.json",
"max_score": 8
},
{
"name": "Three failure states documented",
"description": "MIGRATION_PLAN.md distinguishes between: (1) all passing, (2) failures from migration bugs (use --rerun-failed to fix), (3) intentional contract drift (use --update-contracts)",
"max_score": 12
},
{
"name": "Built-in commands before manual work",
"description": "MIGRATION_PLAN.md describes running Themis migration commands as the primary approach, explicitly before any manual rewriting",
"max_score": 10
},
{
"name": "Final themis test confirms stable",
"description": "migrate.sh ends with `themis test` (no flags) to confirm the suite is stable after all fixes",
"max_score": 10
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10
rules
skills
themis