CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

autoresearch-agent

Autonomous experiment loop that optimizes any file by a measurable metric. Inspired by Karpathy's autoresearch. The agent edits a target file, runs a fixed evaluation, keeps improvements (git commit), discards failures (git reset), and loops indefinitely. Use when: user wants to optimize code speed, reduce bundle/image size, improve test pass rate, optimize prompts, improve content quality (headlines, copy, CTR), or run any measurable improvement loop. Requires: a target file, an evaluation command that outputs a metric, and a git repo.

70

Quality

88%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

The canonical home for this skill is autoresearch-agent in alirezarezvani/claude-skills

SKILL.md
Quality
Evals
Security

Quality

Content

85%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

Highly actionable content with a clearly sequenced, well-validated workflow — commands are copy-paste ready and failure modes (crashes, timeouts, strategy stagnation) are explicitly handled. The weaknesses are structural: everything lives inline in one long file with no reference offloading, and the scripts the body depends on are absent from the bundle.

Suggestions

Move the evaluator catalog (free/LLM-judge tables, custom-evaluator template) and the dashboard/export-format details into a references/ file (e.g. EVALUATORS.md) and reference them one level deep, keeping SKILL.md an operational overview.

Trim non-operational sections — the promotional taglines, Installation, and Related Skills — or relocate them to a README, since they consume context without guiding the loop itself.

Ensure the scripts referenced throughout (setup_experiment.py, run_experiment.py, log_results.py, convert.sh) actually ship in the bundle's scripts/ directory, and briefly state each script's interface in SKILL.md so the references are verifiable.

DimensionReasoningScore

Conciseness

The body is dominated by token-efficient tables and copy-paste commands with almost no explanation of concepts Claude already knows. Minor padding remains — the promotional taglines ('You sleep. The agent experiments.', 'Not one guess — fifty measured attempts, compounding.') and the Installation/Related-Skills sections — so it sits at 'efficient; minor instances that could be trimmed' rather than the lean 5 anchor.

4 / 5

Actionability

Commands are fully specified and executable: setup with every flag, 'python scripts/run_experiment.py --experiment engineering/api-speed --single', log_results variants with --format/--output, config.cfg keys, results.tsv column semantics, and named evaluators with metrics and directions. This is the copy-paste-ready, common-cases-covered 5 anchor; the only pseudocode-ish snippet is the explicitly user-authored custom-evaluator template, which the skill frames as an example to adapt.

5 / 5

Workflow Clarity

The Before-Starting checklist and 7-step iteration loop give a clear sequence, and validation is explicit: dry-run before starting, 'Test it before starting the loop. Run once, verify output', KEEP/DISCARD/CRASH output per run, timeout kill at 2.5× budget, and a 5-consecutive-crash pause-and-alert feedback loop. Despite the destructive 'git reset --hard HEAD~1', the guardrails and error-recovery loops match the 5 anchor.

5 / 5

Progressive Disclosure

Sections are well-organized, but the entire skill is one ~300-line file: evaluator catalogs, dashboard/export format details, installation instructions, and related-skills notes are all inline where a SKILL.md overview should point to one-level-deep reference files. Additionally, the referenced scripts/ files (setup_experiment.py, run_experiment.py, log_results.py, convert.sh) are not present in the bundle, so references cannot be verified. This lands on 'some structure but could be better organized' — not 4 because content that should be separate is inline and the referenced paths are unbacked.

3 / 5

Total

17

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it concretely describes the experiment-loop workflow, gives an explicit and enumerated 'Use when' clause, and states preconditions. The only weakness is trigger breadth — 'any file / any measurable metric' invites overlap with generic optimization requests, and a few natural synonyms are missing.

DimensionReasoningScore

Specificity

The description enumerates the complete workflow as concrete actions — 'edits a target file, runs a fixed evaluation, keeps improvements (git commit), discards failures (git reset), and loops indefinitely' — with no vague filler. This matches the comprehensive-coverage anchor; it is not 4 because there are no gaps in the action list.

5 / 5

Completeness

It explicitly answers both what (autonomous experiment loop with commit/reset mechanics) and when ('Use when: user wants to optimize code speed... or run any measurable improvement loop'), plus a 'Requires:' preconditions clause. This is the anchor-5 pattern; not 4 because the when-clause is explicit and enumerated rather than merely present.

5 / 5

Trigger Term Quality

Natural phrases like 'optimize code speed, reduce bundle/image size, improve test pass rate, improve content quality (headlines, copy, CTR)' mirror what users would say. It falls short of the 5 anchor because common variations such as 'benchmark', 'make this faster/smaller', or domain file-type keywords are not covered.

4 / 5

Distinctiveness Conflict Risk

Distinct triggers ('experiment loop', 'autoresearch', 'measurable improvement loop') carve out a clear niche, but 'optimizes any file by a measurable metric' and 'any measurable improvement loop' are broad enough to overlap with general optimization skills. Mostly distinct with minor overlap risk — the 4 anchor.

4 / 5

Total

18

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

referenced_paths_exist

Referenced path issues: 11 missing

Warning

Total

15

/

16

Passed

Repository

alirezarezvani/claude-skills

Path

.gemini/skills/autoresearch-agent/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.