agentsociety-paper-review

Use when a completed research manuscript needs a robust internal, venue-calibrated mock peer review. Orchestrates three isolated reviewer subagents in parallel, then a separate MetaReview subagent that verifies evidence, reports agreement and score dispersion, and emits advisory reroutes. Never revises research artifacts, executes experiments, or updates pipeline state.

Quality

92%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Quality

Content

92%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is a well-structured, actionable orchestration spec with explicit validation checkpoints and clean progressive disclosure into real reference files. The only weakness is mild redundancy of integrity constraints across sections.

Dimension	Reasoning	Score
Conciseness	The body is lean and does not explain concepts Claude already knows, but a few integrity constraints are restated across the Workflow, Hard Constraints, and Internal Mock Review Only sections and could be consolidated.	4 / 5
Actionability	Guidance is concrete and executable: exact round paths ('paper/reviews/<venue>-review-r<N>/'), named contracts, a named script ('scripts/prepare_pdf_review.py'), an exact reroute vocabulary, and a precise output schema.	5 / 5
Workflow Clarity	The 8-step workflow is clearly sequenced with explicit validation checkpoints (stop-before-dispatch on failed intake, validate-three-reviews with retry-once, recompute digests before MetaReview) and feedback loops for error recovery.	5 / 5
Progressive Disclosure	The body is an overview that points to well-signaled, one-level-deep real reference files (ensemble-protocol, review/meta-review contracts, reroute-guide, pdf-intake-contract, and six venue templates), with content appropriately split and easy to navigate.	5 / 5
	Total	19 / 20 Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is third-person, concrete, and explicitly pairs a 'Use when' trigger with a precise account of the ensemble review and MetaReview behavior. It is highly specific and distinctive; the only minor gap is a few natural trigger synonyms.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions — 'Orchestrates three isolated reviewer subagents in parallel', 'verifies evidence, reports agreement and score dispersion, and emits advisory reroutes', plus explicit non-actions — giving comprehensive coverage of the skill's capabilities.	5 / 5
Completeness	Explicitly answers both: the 'what' (three reviewers + MetaReview orchestration, evidence verification, dispersion, reroutes) and the 'when' ('Use when a completed research manuscript needs a robust internal, venue-calibrated mock peer review').	5 / 5
Trigger Term Quality	Good natural keyword coverage ('research manuscript', 'mock peer review', 'review', 'score', 'venue-calibrated') that users would plausibly say, though a few common synonyms (feedback, acceptance, rebuttal) are absent.	4 / 5
Distinctiveness Conflict Risk	A clear niche — author-side internal mock peer review via an isolated 3-reviewer ensemble plus MetaReview with advisory reroutes — with distinct triggers and minimal overlap with other skills.	5 / 5
	Total	19 / 20 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning
referenced_paths_exist	Referenced path issues: 6 deeper-than-1-level	Warning

	Total	14 / 16 Passed

Repository: tsinghua-fib-lab/AgentSociety
Path: extension/skills/agentsociety-paper-review/v1.0.0/SKILL.md
Commit: 13e28b5

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.