constitutional-ai

Anthropic's method for training harmless AI through self-improvement. Two-phase approach - supervised learning with self-critique/revision, then RLAIF (RL from AI Feedback). Use for safety alignment, reducing harmful outputs without human labels. Powers Claude's safety system.

Quality

52%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./backend/cli/skills/llm-tools/constitutional-ai/SKILL.md

Quality

Content

46%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body offers genuine, sequenced code for both CAI phases but is undercut by undefined helper functions, absent validation for costly training steps, and broken references to non-existent bundle files.

Suggestions

Add validation/feedback checkpoints to the training workflows (e.g., check loss/reward metrics, validate the preference dataset before RewardTrainer, halt PPO if reward degrades) so destructive/batch steps score above 3.

Define or replace the undefined helpers (create_dataset, parse_preferences, CONSTITUTION, model, tokenizer, reward_model) so the code is copy-paste executable, or mark them clearly as placeholders.

Either create the referenced files under references/ (constitution-design.md, rlaif-comparison.md, cot-critique.md) or remove the dangling links, and move the bulk of the common-issues prose into a reference to tighten the overview.

Dimension	Reasoning	Score
Conciseness	Mostly efficient with concrete code, but padded with conceptual restating Claude already knows (e.g., 'Models learn to critique and revise their own responses using a constitution') and a verbose common-issues section that repeats the workflows.	3 / 5
Actionability	Provides concrete TRL/transformers code for each phase, but several examples are not directly executable — undefined helpers like create_dataset, parse_preferences, CONSTITUTION, model, tokenizer, and reward_model appear without definition.	3 / 5
Workflow Clarity	Workflows 1 and 2 are sequenced as numbered Step 1–4, but they involve expensive/irreversible training operations with no validation checkpoints, error-handling, or validate→fix→retry feedback loops, capping workflow clarity at 3 per the destructive-operation rule.	3 / 5
Progressive Disclosure	The body signals one-level-deep references (constitution-design.md, rlaif-comparison.md, cot-critique.md) but no references/ bundle directory exists, so the referenced paths are broken; structure is otherwise reasonable but the missing targets pull this below the midpoint.	2 / 5
	Total	11 / 20 Passed

Description

58%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific and mostly distinct, clearly conveying both training phases, but its trigger guidance is terse and jargon-heavy rather than using natural user phrases, capping completeness and trigger quality at the mid-band.

Suggestions

Add an explicit 'Use when...' clause with natural trigger phrases (e.g., 'Use when aligning a model to be harmless without human labels, training with AI feedback, or reducing evasive refusals').

Surface user-natural synonyms ('AI safety training', 'safety alignment', 'RL from AI Feedback') alongside the technical RLAIF term to improve trigger quality.

Confirm the description uses third-person voice throughout (it does) and keep it concise — the closing 'Powers Claude's safety system' could be trimmed as it is a claim rather than a trigger.

Dimension	Reasoning	Score
Specificity	Names the domain ('training harmless AI through self-improvement') and lists several concrete actions — supervised learning with self-critique/revision and RLAIF — covering both phases, with only minor gaps in coverage.	4 / 5
Completeness	Has a clear 'what' (two-phase self-critique + RLAIF training) but the 'when' is only a terse 'Use for safety alignment...' clause without concrete trigger phrases, capping completeness at 3 per the missing-trigger guidance.	3 / 5
Trigger Term Quality	Includes relevant terms ('safety alignment', 'harmless', 'RLAIF', 'self-critique') but leans on technical jargon over the natural phrases a user would say, missing common synonyms like 'AI safety' or 'safety training'.	3 / 5
Distinctiveness Conflict Risk	Targets a clear niche (Constitutional AI / RLAIF harmlessness training) with mostly distinct triggers and only minor overlap risk with broader RLHF or general safety skills.	4 / 5
	Total	14 / 20 Passed

Validation

81%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning
relative_links	Relative link issues: 3 missing	Warning
referenced_paths_exist	Referenced path issues: 6 missing	Warning

	Total	13 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/llm-tools/constitutional-ai/SKILL.md
Commit: fcd2e11

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.