autoresearch-agent

Autonomous experiment loop that optimizes any file by a measurable metric. Inspired by Karpathy's autoresearch. The agent edits a target file, runs a fixed evaluation, keeps improvements (git commit), discards failures (git reset), and loops indefinitely. Use when: user wants to optimize code speed, reduce bundle/image size, improve test pass rate, optimize prompts, improve content quality (headlines, copy, CTR), or run any measurable improvement loop. Requires: a target file, an evaluation command that outputs a metric, and a git repo.

Quality

90%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

—

The risk profile of this skill

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with concrete commands, a clear validated workflow loop, and strong section organization. It loses points only on minor conciseness padding and progressive-disclosure signaling: existing reference files are not linked from the body and one referenced script (convert.sh) is missing from the bundle.

Suggestions

Link the existing reference files from the body — e.g. replace the inlined Domains table with a pointer to references/experiment-domains.md and add a 'See references/program-template.md for ready-made program.md templates' line under Setup.

Remove or fix the scripts/convert.sh reference in the Multi-tool install section since that file is absent from the scripts/ bundle.

Trim the marketing tagline and 'Not one guess — fifty measured attempts, compounding.' lines to reduce token overhead without losing information.

Dimension	Reasoning	Score
Conciseness	The body is mostly efficient and actionable with little explanation of concepts Claude already knows, but carries minor padding (the marketing tagline "You sleep. The agent experiments. You wake up to results.", "Not one guess — fifty measured attempts, compounding.") and somewhat extensive dashboard/export examples that could be trimmed.	4 / 5
Actionability	Fully executable guidance throughout — copy-paste setup commands with exact CLI flags, the per-iteration loop with concrete git/eval commands, named config.cfg fields, results.tsv column definitions, and a complete custom-evaluator code example covering the common cases.	5 / 5
Workflow Clarity	Despite being a batch/destructive loop (git reset --hard, autonomous commits), validation and feedback loops are explicit: the "Before Starting" and "Each Iteration" sequences, KEEP/DISCARD/CRASH eval output, 5-consecutive-crash pause, timeout handling, and "test the eval before starting" checkpoint satisfy the validation requirement rather than missing it.	5 / 5
Progressive Disclosure	Section structure is clear and most content is well placed, but the body never links to the existing reference files (references/experiment-domains.md, references/program-template.md) despite inlining a domains table and program-template material, and it references scripts/convert.sh which does not exist in the bundle — minor organization/signaling gaps that keep it below the 5 anchor.	4 / 5
	Total	18 / 20 Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is concrete, complete, and distinctive, explicitly stating both what the skill does and when to use it with concrete trigger phrases and preconditions. Its only minor weakness is trigger-term breadth, leaning on enumerated use-cases rather than a wider net of natural user phrasings.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions with specific mechanisms — "edits a target file, runs a fixed evaluation, keeps improvements (git commit), discards failures (git reset), and loops indefinitely" — giving comprehensive coverage rather than just naming the domain.	5 / 5
Completeness	Explicitly answers both "what" (autonomous experiment loop: edit, evaluate, keep/discard, loop) and "when" (a dedicated "Use when:" clause with concrete triggers) plus a "Requires:" preconditions list — matching the anchor that demands explicit what and when with concrete trigger phrases.	5 / 5
Trigger Term Quality	Strong natural coverage ("optimize code speed", "reduce bundle/image size", "improve test pass rate", "optimize prompts", "headlines, copy, CTR") with synonyms, but it enumerates use-cases more than casting a wide net of natural trigger phrases a user would spontaneously say, so it sits just below the comprehensive 5 anchor.	4 / 5
Distinctiveness Conflict Risk	Occupies a clear niche — a measurable-metric optimization loop with git keep/discard semantics — with distinct triggers and minimal overlap risk with general coding or analysis skills.	5 / 5
	Total	19 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: alirezarezvani/claude-skills
Path: engineering/autoresearch-agent/skills/autoresearch-agent/SKILL.md
Commit: 19392f7

Reviewed: ~6 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.