CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

post-training-strategy

为 Agent 模型做后训练或选路线时使用——判断该先补 Mid-training、用 SFT 立协议还是直接上 RL;构造 Mid-training 语料与 SFT 数据、搭建 RL 环境、在 GRPO/PPO/DPO 与蒸馏之间取舍;排查 pass@k 近零、格式不稳就训 RL、组内无奖励差异、训练-推理数值失配、过度训练泛化下降。触发词:后训练、Mid-training、继续预训练、SFT、RL、GRPO、PPO、DPO、拒绝采样、RFT、蒸馏、on-policy、RLVR、LoRA。

Quality

89%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

85%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A high-quality strategy skill: dense, non-obvious operational guidance with explicit decision flows, thresholds, worked examples, and a self-check checklist, at the cost of a few passages re-explaining known training mechanics. Its main structural weakness is that all '配套代码' and '深度阅读' references point to files absent from the skill bundle, leaving the progressive-disclosure layer broken as shipped.

Suggestions

Ship the referenced bundle files (chapter8/ projects and book/chapter8.md) inside the skill, or restructure them under references/ — as delivered, every '配套代码' and '深度阅读' link resolves to a missing path.

Trim passages that re-explain mechanics Claude already knows (SFT-as-next-token math, the PPO value-model explanation) down to their decision-relevant conclusions to reclaim tokens in a dense body.

If the chapter8/book material cannot be bundled, move the per-project descriptions into a single reference file and keep SKILL.md's closing sections to one-line pointers, slimming the overview.

DimensionReasoningScore

Conciseness

The body is exceptionally dense with non-obvious judgment (thresholds, ratios, pitfalls) and contains no filler or library-introduction padding, but a few passages explain mechanics Claude already knows and could be trimmed to their decision-relevant conclusions — e.g. 'SFT 与预训练是同一个数学任务(预测下一个词)...' and the PPO-vs-GRPO value-model explanation 'PPO 训练价值模型逐步估计"通常能做到多好"'. This fits 'efficient; minor instances of over-explanation that could be trimmed'; score 3 would require more substantial unnecessary explanation, and score 5 would require every passage to assume Claude's competence with no such explanations.

4 / 5

Actionability

For an instruction-only skill the guidance is fully concrete: a decision table with an explicit entry threshold column, a numbered ①→④ decision order, a worked GRPO example (16 sandboxes, 4 pass → reward 1, 12 fail → 0, group advantage computation), a formula 'pass@k = 1-(1-p)^k', specific LoRA hyperparameters (rank 64–256 for SFT, 8–32 for RL, ~10x full-FT learning rate), named evaluation gates (LongBench v2, IFEval), and a pre-launch checklist naming exact metrics (概率比均值/分位数/最大值、近似 KL、被裁剪比例、策略 staleness). Specific examples cover the common cases, matching anchor 5.

5 / 5

Workflow Clarity

Multi-step processes are explicitly sequenced with validation checkpoints throughout: the route-decision order ①→④, Mid-training's five-step data construction with '多维门禁决定停止', SFT's '线上数据 → 任务蓝图 → 合成任务 → 多次候选轨迹 → 任务验证与轨迹验证 → SFT 数据' pipeline with train/eval dedup rules, RL's four-step rollout→reward→advantage→update with the 全成功或全失败组 gradient-vanishing caveat, environment engineering order with '先用少量 rollout 检查任务是否可完成', and the 上线前的自检清单 checklist. Validation gates and error-recovery diagnostics (e.g. 'loss 降而 pass@k 不动 = 数据没覆盖所需能力') are pervasive, matching anchor 5.

5 / 5

Progressive Disclosure

The body itself is well organized into 何时使用/核心原则/实践模式/常见陷阱 sections, and deep material is delegated via clearly signaled one-level references with per-item descriptions (nine 'chapter8/...' projects and nine 'book/chapter8.md' sections). However, the actual bundle contains none of these files — no references/, scripts/, assets/, chapter8/, or book/ directories exist alongside SKILL.md — so every deep-dive reference resolves to a missing path and the disclosure layer cannot navigate anywhere. Scored against the actual bundle structure this is a real organizational gap, better than anchor 2 (nothing is wrongly inlined) but below anchor 4's 'minor organization gaps' since the entire reference layer is unshipped.

3 / 5

Total

17

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it states concrete capabilities in third person, gives an explicit use-when clause, and closes with a rich trigger-term list dominated by the domain's natural vocabulary. The only improvements are adding a few missing natural synonyms (微调/finetuning, RLHF) and narrowing very broad triggers like LoRA/RL.

DimensionReasoningScore

Specificity

The description lists multiple concrete, comprehensive actions within the post-training domain: '判断该先补 Mid-training、用 SFT 立协议还是直接上 RL;构造 Mid-training 语料与 SFT 数据、搭建 RL 环境、在 GRPO/PPO/DPO 与蒸馏之间取舍;排查 pass@k 近零、格式不稳就训 RL、组内无奖励差异、训练-推理数值失配、过度训练泛化下降'. These are specific capabilities plus named failure modes, matching the 'multiple specific concrete actions; comprehensive coverage' anchor; score 4 would require notable coverage gaps, which are not present.

5 / 5

Completeness

It explicitly answers 'what' (route decision, corpus/SFT data construction, RL environment building, algorithm trade-offs, failure-mode troubleshooting) and 'when' ('为 Agent 模型做后训练或选路线时使用' plus an explicit 触发词 list). Both are present with concrete trigger phrases, matching the anchor-5 example structure; anchor 4 would require the 'when' clause to be less explicit, which it is not.

5 / 5

Trigger Term Quality

The explicit trigger list '后训练、Mid-training、继续预训练、SFT、RL、GRPO、PPO、DPO、拒绝采样、RFT、蒸馏、on-policy、RLVR、LoRA' covers the natural vocabulary practitioners use, including synonyms (拒绝采样/RFT, 继续预训练). A few common natural terms are still missing (e.g. 微调/finetuning, RLHF, reward model), which keeps it at 'good keyword coverage; a few natural terms missing' rather than the fully comprehensive anchor 5.

4 / 5

Distinctiveness Conflict Risk

The niche is distinct — route diagnosis between Mid-training/SFT/RL keyed to signals like pass@k and 组内奖励差异 is unlikely to trigger for unrelated skills. Minor overlap risk remains with closely related fine-tuning/LoRA or reward-modeling skills, since 'LoRA' and 'RL' are broad triggers, matching 'mostly distinct; minor overlap risk' rather than the minimal-conflict anchor 5.

4 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

bojieli/ai-agent-book

Path

skills/post-training-strategy/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.