CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

autoresearch-agent

Autonomous experiment loop that optimizes any file by a measurable metric. Inspired by Karpathy's autoresearch. The agent edits a target file, runs a fixed evaluation, keeps improvements (git commit), discards failures (git reset), and loops indefinitely. Use when: user wants to optimize code speed, reduce bundle/image size, improve test pass rate, optimize prompts, improve content quality (headlines, copy, CTR), or run any measurable improvement loop. Requires: a target file, an evaluation command that outputs a metric, and a git repo.

68

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

77%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is a well-engineered operating manual: executable commands, a clearly sequenced agent protocol with strong validation and feedback loops, and concrete escalation/self-improvement rules. Its main structural weakness is disclosure — two reference files ship in the bundle but are never linked from the body, while inlined domain detail and dangling references (convert.sh, undefined /ar:* commands) hurt navigation and actionability.

Suggestions

Link references/experiment-domains.md from the Domains section and references/program-template.md from the 'If program.md Already Exists' section, and slim the inlined Domains table to a one-line pointer so the detail lives in the reference file.

Fix dangling references: either add scripts/convert.sh and the /ar:* command definitions to the bundle, or remove them from SKILL.md so the agent isn't pointed at files that don't exist.

Trim agent-irrelevant sections (Installation, Related Skills, subscription-cost bullets, and the sample dashboard output) to reclaim context-window tokens without losing operational guidance.

DimensionReasoningScore

Conciseness

The body is dense and imperative — tables for domains, evaluators, and results.tsv columns; no re-teaching of concepts Claude already knows. Minor padding could be trimmed: the tagline, subscription-cost bullets, sample dashboard output, and the Installation/Related Skills sections, which don't guide the agent's loop. Efficient with minor over-explanation — anchor 4, not 5.

4 / 5

Actionability

Guidance is mostly copy-paste executable: full setup commands with every flag, 'python scripts/run_experiment.py --experiment engineering/api-speed --single', and explicit git commands per iteration. Two gaps keep it from 5: the five '/ar:*' slash commands are listed but no command definitions exist in the bundle, and 'scripts/convert.sh' is referenced but absent from scripts/.

4 / 5

Workflow Clarity

Clear 'Before Starting' and 'Each Iteration' sequences with explicit validation checkpoints and feedback loops: dry-run before starting, 'Test it before starting the loop', timeout kill at 2.5× budget, crash→fix→re-run, 5-consecutive-crash pause-and-alert, and 20-run strategy escalation. The destructive git reset is script-managed with comparisons validated before revert — matches anchor 5.

5 / 5

Progressive Disclosure

The bundle ships references/experiment-domains.md and references/program-template.md, but the body never links or signals either — the Domains table is inlined in SKILL.md instead (duplicating the reference file's role), and program.md guidance never points to the template. Combined with the dangling convert.sh reference, this fits anchor 3 ('references present but not clearly signaled; content that should be separate is inline') better than 4.

3 / 5

Total

16

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it states concrete actions with git mechanics, an explicit 'Use when:' trigger list, and prerequisites, all in third person. Its only weaknesses are a few missing natural trigger phrasings and the broad 'any file by a measurable metric' scope, which creates minor conflict risk with general optimization skills.

DimensionReasoningScore

Specificity

The description lists multiple concrete actions with mechanics — 'edits a target file, runs a fixed evaluation, keeps improvements (git commit), discards failures (git reset), and loops indefinitely' — giving comprehensive, not just minor-gap, coverage. It uses third-person voice ('The agent edits…'), so no voice penalty applies.

5 / 5

Completeness

It clearly answers both what ('Autonomous experiment loop that optimizes any file by a measurable metric… The agent edits a target file, runs a fixed evaluation…') and when ('Use when: user wants to optimize code speed, reduce bundle/image size…'), plus a 'Requires:' clause. This matches the anchor-5 example structure exactly.

5 / 5

Trigger Term Quality

'optimize code speed, reduce bundle/image size, improve test pass rate, optimize prompts, improve content quality (headlines, copy, CTR)' gives good natural keyword coverage, but common phrasings users would actually say — 'make this faster', 'benchmark', 'make it smaller' — are missing. This matches anchor 4 rather than 5 (synonym coverage incomplete) or 3 (coverage well beyond 'some relevant keywords').

4 / 5

Distinctiveness Conflict Risk

'optimizes any file by a measurable metric' is broad and could overlap with general refactoring or improvement skills, but the experiment-loop trigger phrases ('run any measurable improvement loop') carve a mostly distinct niche with minor overlap risk — anchor 4 rather than 5.

4 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

alirezarezvani/claude-skills

Path

engineering/autoresearch-agent/skills/autoresearch-agent/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.