constitutional-ai

Anthropic's method for training harmless AI through self-improvement. Two-phase approach - supervised learning with self-critique/revision, then RLAIF (RL from AI Feedback). Use for safety alignment, reducing harmful outputs without human labels. Powers Claude's safety system.

Quality

63%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./backend/cli/skills/llm-tools/constitutional-ai/SKILL.md

Quality

Content

42%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The skill body is well-structured with concrete code and clear phase sequencing, but it is held back by incomplete code (undefined helper placeholders), missing validation/feedback loops for batch training, and references to files that do not exist. Tightening the code and shipping the referenced files would substantially improve it.

Suggestions

Create the referenced references/constitution-design.md, references/rlaif-comparison.md, and references/cot-critique.md files, or remove the broken links, to fix progressive disclosure.

Define or replace the undefined helpers (create_dataset, parse_preferences, create_preference_dataset) and the bare model variable so the workflow code is copy-paste executable.

Add validation/eval checkpoints to the SL and RL workflows (e.g., reward-model accuracy check, KL/eval metrics before proceeding) so the batch training sequence earns a higher workflow-clarity score.

Dimension	Reasoning	Score
Conciseness	Mostly efficient and assumes Claude's competence, but ~280 lines with verbose code and restated conceptual sections ('When to use vs alternatives', 'Principles') could be tightened, matching 'mostly efficient but includes some unnecessary explanation or could be tightened'. Not a 3 because several blocks pad rather than earn their place.	2 / 3
Actionability	Provides concrete transformers/trl code, but several steps rely on undefined placeholders (create_dataset, parse_preferences, create_preference_dataset, the bare model variable), leaving key details incomplete — matching 'some concrete guidance but incomplete; missing key details'. Not a 3 because it is not copy-paste ready.	2 / 3
Workflow Clarity	The SL and RL phases are clearly sequenced with numbered steps, but these batch training operations have no validation checkpoints, eval metrics, or error-recovery feedback loops, which per the rubric scoring notes caps batch-operation workflow clarity at 2. Not a 1 because the sequence is genuinely present.	2 / 3
Progressive Disclosure	The body links to references/constitution-design.md, references/rlaif-comparison.md, and references/cot-critique.md, but the references/ directory does not exist, so these are broken/dangling pointers combined with largely inline content — matching 'poor organization' and worse than the score-2 'references present but not clearly signaled' anchor.	1 / 3
	Total	7 / 12 Passed

Description

85%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is strong overall: it states concrete capabilities, includes an explicit 'Use for...' trigger clause, and is third-person and distinct from other skills. Its main weakness is trigger-term quality, which is slightly jargon-heavy and lacks natural-language variations.

Dimension	Reasoning	Score
Specificity	Names multiple concrete actions — 'training harmless AI through self-improvement', 'supervised learning with self-critique/revision', and 'RLAIF (RL from AI Feedback)' — matching the 'lists multiple specific concrete actions' anchor.	3 / 3
Completeness	Explicitly answers what ('Anthropic's method for training harmless AI through self-improvement... two-phase approach') and when ('Use for safety alignment, reducing harmful outputs without human labels'), satisfying the 'clearly answers both what AND when' anchor.	3 / 3
Trigger Term Quality	Includes relevant phrases like 'safety alignment', 'reducing harmful outputs', and 'without human labels', but leans on technical jargon (RLAIF, self-critique) and misses common natural variations users would say, matching 'some relevant keywords but missing common variations'.	2 / 3
Distinctiveness Conflict Risk	Occupies a clear niche (Constitutional AI / RLAIF safety alignment) with distinct triggers unlikely to overlap with unrelated skills, matching the 'clear niche with distinct triggers' anchor.	3 / 3
	Total	11 / 12 Passed

Validation

81%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning
relative_links	Relative link issues: 3 missing	Warning
referenced_paths_exist	Referenced path issues: 6 missing	Warning

	Total	13 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/llm-tools/constitutional-ai/SKILL.md
Commit: 52845c3

Reviewed: about 10 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.