autoresearch-loop

Domain-agnostic metric-driven improvement loop, generalizing Karpathy's autoresearch. Use when you want an agent to discover what to measure for a project/goal, then run a keep-or-revert experiment loop that proposes changes, measures them against an objective, keeps wins, discards regressions, and records implemented improvements. Adapts to code perf-auditing, codegen, bug-finding, ad optimization, or any artifact + measurable objective + trial. Trigger: 'autoresearch this', 'find and implement improvements to X', 'discover metrics and optimize'.

Quality

81%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

71%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-sequenced, actionable procedural skill body with strong validation feedback loops, undermined by missing bundle files (DESIGN.md and the adapter set) that break several core references and a couple of verbose anecdotal passages.

Suggestions

Add the missing DESIGN.md and adapters/*.md files referenced by the body (or remove/rewrite the instructions that depend on them) so core Phase 0 and pre-run guidance are reachable.

Tighten the two 'validated the hard way' anecdotes in Phase 1 and Phase 3 to a one-line pitfall note plus a pointer, keeping the lesson without the narrative overhead.

Define 'every N trials' (Phase 3 ADAPT) concretely or delegate the cadence to the adapter contract, so the adaptation checkpoint is executable rather than implicit.

Dimension	Reasoning	Score
Conciseness	The body is dense and procedural and assumes Claude's competence, but two long 'validated the hard way' anecdotal passages (the Phase 1 profiling story and the Phase 3 proxy-degeneracy story) add tokens that could be trimmed without losing the lesson.	4 / 5
Actionability	Provides concrete executable commands (arl init\|run\|log\|status with --metric, --asi, --cost flags and node scripts/arl.mjs), but some loop steps are directional rather than executable ('propose the smallest change most likely to move the primary metric').	4 / 5
Workflow Clarity	The Phase 0→1→2→3 sequence is clear with explicit validation (guardrail checks.sh, MAD confidence scoring, Goodhart/proxy-degeneracy checks, revert-on-discard), satisfying the destructive/batch validation requirement; minor gaps remain such as undefined 'every N trials' and strategy details deferred to adapters.	4 / 5
Progressive Disclosure	Structure is well-organized with a 'Reference files' section and one-level-deep pointers, but 6 of the 11 referenced paths (DESIGN.md and all adapters/.md) do not exist in the bundle, breaking navigation for core instructions like 'Read DESIGN.md once' and 'select a domain adapter (adapters/.md)'.	3 / 5
	Total	15 / 20 Passed

Description

91%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, concrete description with explicit trigger guidance and a clear what/when structure; its only weaknesses are a second-person voice and a somewhat broad improvement-oriented trigger that raises minor overlap risk.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions (discover what to measure, propose changes, measure, keep wins, discard regressions, record improvements), but the second-person 'Use when you want an agent to' triggers the voice penalty, reducing specificity by 1 from a base of 5.	4 / 5
Completeness	Clearly answers what (metric-driven keep-or-revert improvement loop) and when (both a 'Use when' clause and a concrete 'Trigger:' list), matching the top anchor.	5 / 5
Trigger Term Quality	Explicit natural trigger phrases are provided ('autoresearch this', 'find and implement improvements to X', 'discover metrics and optimize') that a user would plausibly say, giving comprehensive keyword coverage.	5 / 5
Distinctiveness Conflict Risk	The autoresearch/metric-loop framing is a distinct niche, but the trigger 'find and implement improvements to X' is broad enough to risk overlap with general review or optimization skills, so it falls short of minimal conflict risk.	4 / 5
	Total	18 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: jdrhyne/agent-skills
Path: skills/autoresearch-loop/SKILL.md
Commit: 439cd3a

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.