CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

ainativedev/aidevcon-2026-nyc

AI Native DevCon 2026 New York — practical agent skills grounded in conference talks

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

Overview
Quality
Evals
Security
Files

application-template.mdskills/talk-shah-training-agent-skills/references/

Bounded skill-training protocol

This worksheet adapts Shah's training loop. It is a plan until runs and evidence are recorded.

Objective and data

Skill behavior to improve: Known failure: Task sources and permissions: Golden-answer review: Training cases: Evaluation cases hidden from editor: Final test cases hidden from editor and gate: Leakage risks:

Use realistic counterexamples as well as successes. Do not generate a supposedly golden answer solely by repeating the current skill's output.

Candidate record

Baseline version: Motivating training failures: Small proposed edit: Expected effect: Possible regression: Rejected related ideas: Candidate version:

Keep changes scoped to the failure. Record why an idea was rejected and whether that reason still applies after a model or task change.

Scorer and gate

Deterministic scoring available: Judge criteria if needed: Pinned judge configuration: Structured evidence fields: Repeat strategy: Cheap structural checks: Focused training recheck: Paired evaluation design: Predeclared acceptance rule: Critical regression rule:

The editor proposes; the gate decides. A judge score is an input, not a substitute for the acceptance rule.

Optimizer validation

Planted defect and expected recovery: Null test with no learnable signal: Expected null behavior: Observed false acceptances: Final-test access restriction: Consolidation re-evaluation:

A short null test is useful evidence, not proof of immunity to overfitting.

Operation and review

Isolated candidate location: Budget and time bound: Crash recovery: Last accepted version: Candidate and result log: Cost per useful improvement: Final holdout result: Remaining uncertainty:

Discard only changes owned by this experiment. Do not reset unrelated work, publish revisions, or start recurring runs without authorization.

Decision

Accept or reject: Evidence: Reason: Next bounded experiment:

Keep planned tests distinct from executed tests and preserve the final holdout until the protocol calls for it.

skills

talk-shah-training-agent-skills

tile.json