kumar/tessl-setup
Personal Tessl setup guide — CLI reference for creating plugins/skills, publishing, installing, running evals, and navigating the full plugin lifecycle.
76
95%
Does it follow best practices?
Impact
—
No eval scenarios have been run
Risky
Do not use without reviewing
SKILL.mdskills/tessl-evals/
- name:
- tessl-evals
- description:
- Use when running or generating Tessl evals — measure whether a skill or plugin actually improves agent behavior, generate scenarios from repos, view eval results, iterate on improvements. Triggers on: "run evals", "measure skill quality", "generate scenarios", "eval baseline", "does this skill help", "tessl eval", "tessl scenario".
Tessl Evals
Evals test whether a skill actually improves agent behavior against a baseline.
They require a linked Tessl project (tessl.json pointing at a project in your workspace).
Prerequisite: Link the Project
Evals need a Tessl project linked to the directory:
tessl project create my-project --workspace myworkspace # first time
tessl project repair # fix broken linkWithout a linked project, tessl eval run fails with "No Tessl project found".
Run Evals
# Shorthand: run a plugin's own evals with itself as the context
tessl eval run . # current dir
tessl eval run ./my-plugin # explicit plugin path
# Scenarios dir + separate context
tessl eval run ./evals/ --context ./my-plugin
tessl eval run ./evals/ --context workspace/plugin@1.0.0 # published context
# Variants
tessl eval run --skip-baseline ./my-plugin # only with-context variant
tessl eval run --force ./my-plugin # re-run all including solved
tessl eval run --runs 3 ./my-plugin # repeat each scenario 3x
tessl eval run --label "refactor test" # tag this run
tessl eval run --agent claude:claude-sonnet-4-6 ./my-plugin
tessl eval run --list-agents # show supported agent:model values
tessl eval run --quality-check ./my-plugin # exclude misleading scenarios
tessl eval run --json ./my-plugin # output run IDs immediatelyDefault agent: claude:deepseek-v4-flash. Ctrl-C detaches without cancelling.
How the score works: an eval run solves each scenario twice — once with no skill loaded (baseline), once with the skill loaded — then a judge scores both against the same rubric. The uplift multiplier (e.g. 1.66x) is the ratio.
View / Manage Runs
tessl eval list
tessl eval list --mine --workspace myteam
tessl eval view --last
tessl eval view <run-id>
tessl eval retry --lastGenerate Scenarios from Repo Commits
Auto-generation works only when the task has a checkable output — API calls with concrete arguments pass the feasibility check; conversational workflows don't.
# From repo commits
tessl scenario generate org/repo --commits=abc123,def456 --workspace engteam
tessl scenario generate org/repo --prs=42,43 --workspace engteam
# From a plugin (uses the plugin content itself as the source)
tessl scenario generate ./my-plugin --count=5
# List / view generation runs
tessl scenario list --workspace engteam
tessl scenario view --last
tessl scenario view <id>
# Download to evals/
tessl scenario download --last
tessl scenario download --last --strategy replace --output my-evals/
tessl scenario download <id>Scenarios land at evals/<scenario-name>/ with:
task.md— task brief shown to the agentcriteria.json— weighted checklist rubricscenario.json— optional fixture (repo URL, commit ref, includes)
Hand-Written Scenarios
Auto-generation fails for skills without checkable output. Write scenarios by hand
in the plugin's own evals/ directory:
skills/my-skill/evals/scenario-name/
├── task.md # what the agent should do
├── criteria.json # weighted rubric
└── scenario.json # optional: fixtures, real stateHand-written scenarios often score higher than auto-generated ones because they target specific documented failure modes rather than generic feasibility.
Typical Workflow
# 1. Baseline: how does the agent do without the skill?
tessl eval run ./my-plugin --label "baseline"
# 2. Edit the skill
# 3. Re-run: how does it do with the improved skill?
tessl eval run ./my-plugin --label "after improvement"
# 4. Compare
tessl eval list --workspace myws
tessl eval view <run-id-1>
tessl eval view <run-id-2>Common Failures (evals)
| Error | Cause | Fix |
|---|---|---|
No Tessl project found | Directory not linked | Run tessl project create or tessl project repair |
| Scenarios fail feasibility check | Task has no checkable output | Hand-write scenarios in evals/ or narrow task to a scriptable action |
| Eval scores don't reflect real behavior | scenario.json has no fixtures | Add real state (sandboxed API, fixture data) — structural correctness isn't the same as real correctness |
Registry shows null for evals field | Registry surface lags eval run | Raw eval data via tessl eval view is authoritative; registry may take time to reflect |