CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

ainativedev/aidevcon-2026-nyc

AI Native DevCon 2026 New York — practical agent skills grounded in conference talks

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

Overview
Quality
Evals
Security
Files

application-template.mdskills/talk-carmont-evolving-agent-harnesses/references/

Harness experiment record

This worksheet is Tessl's recommended application of Carmont's talk. Use it to test one explanation for a performance or cost change. It is not a record of a completed experiment until results are supplied.

Problem and competing explanations

Describe the user's task, observed regression, and available evidence. State separate model, harness, and evaluation hypotheses. For each, record one observation that would support it and one that would weaken it. Mark aggregate numbers that lack comparable inputs.

Baseline

Record the model and effort setting, harness version, relevant instruction layers, task set, grader, quality measure, cost, and latency. Identify a small set of cases whose intended answers are understood. Note tasks with missing prerequisites or possibly invalid expected answers.

Removal candidate

Quote the instruction or describe the component to change. Record its purpose and original justification. Explain whether it duplicates model behavior, contradicts another instruction, imposes an unnecessary sequence, or loads irrelevant context. State the behavior that must survive its removal.

Do not substitute “remove 80%” for a hypothesis. Preserve the user's actual policy and acceptance requirements.

Comparison

CaseExisting harnessReduced harnessOutcome evidenceCost and latencyTrace explanation

Use comparable inputs and record repeated-trial uncertainty where available. Hold back additional representative tasks for checking whether the apparent improvement generalizes. These controls are recommendations here, not fixed procedures supplied by the speaker.

Restore and decide

For each regression, identify the smallest targeted guidance that could restore required behavior. Rerun the affected comparison without changing the grader to favor the candidate. Inspect for skipped checks, impossible tasks, and alternate success paths.

End with retain, revise, or reject; the evidence behind that decision; unresolved risks; and the reversible rollout or next experiment. Keep planned results blank and distinguish measured savings from expected savings.

skills

talk-carmont-evolving-agent-harnesses

tile.json