novelty-check

Verify research idea novelty against recent literature. Use when user says "查新", "novelty check", "有没有人做过", "check novelty", or wants to verify a research idea is novel before implementing.

Quality

83%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Quality

Content

77%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is a well-sequenced, actionable instruction skill with strong error handling and a calibrated-verdict design that resists false ABANDON/PROCEED errors. Its main weaknesses are dangling external references (none of the shared-references files or helper scripts exist in the bundle) and some duplication between the verdict limits and the Important Rules section.

Suggestions

Resolve the dangling references: either include `shared-references/integration-contract.md`, `citation-discipline.md`, `review-tracing.md`, `verify_papers.py`, and `save_trace.sh` in the bundle (e.g., under `references/` and `scripts/`) or inline the essential parts of each so no step depends on an unverifiable file.

De-duplicate the "Two failures waste months equally" guidance and the proximity/calibration rules that appear in both §The verdict limits and §Important Rules — state them once and cross-reference, freeing tokens for a concrete `verify_papers.py` invocation example.

Move the dense anti-hallucination/citation policy paragraph to a clearly signaled reference file (or tighten it to the 2–3 operative rules inline) so the main body stays a scannable overview.

Dimension	Reasoning	Score
Conciseness	The body is efficient and assumes competence — no explanations of what arXiv or novelty means — with tight phase instructions ("Try at least 3 different query formulations per claim"). Minor trimming is possible: the "Important Rules" section restates the verdict limits ("Two failures waste months equally" appears in both §The verdict limits and §Important Rules), and the citation-policy paragraph is dense enough to belong in the referenced reference file.	4 / 5
Actionability	Mostly executable guidance: a concrete MCP invocation block (model, config with `model_reasoning_effort: "xhigh"`, prompt template), a dossier-file pattern with exact contents, specific search targets ("ICLR 2025/2026, NeurIPS 2025", "year filters for 2024-2026"), and a full report template. Minor gaps keep it below 5: the MCP snippet leaves placeholders ("<absolute path to NOVELTY_DOSSIER.md>"), and required helpers (`verify_papers.py`, `save_trace.sh`) are referenced without any invocation example and do not resolve within this bundle.	4 / 5
Workflow Clarity	Phases A–D are clearly sequenced (extract claims → multi-source search → cross-model verification → structured report) with explicit validation and feedback loops: pre-search paper verification, and Policy D1's degraded fallback ("if the helper is unresolved or its invocation fails, tag candidate entries [UNVERIFIED] and surface the uncertainty rather than dropping them"). Error recovery (fallback tagging, permissive-verdict tie-breaking) is spelled out, matching the top anchor.	5 / 5
Progressive Disclosure	The in-file structure is well organized (Constants, Instructions by phase, Important Rules, Review Tracing), but every external reference — `../shared-references/integration-contract.md`, `../shared-references/citation-discipline.md`, `shared-references/review-tracing.md`, `verify_papers.py`, `save_trace.sh` — is dangling: no references/, scripts/, or shared-references directories exist in this bundle, so navigation is unverifiable and protocol detail that should live in those files is inlined in SKILL.md. This matches 'Some structure but could be better organized' rather than the well-placed/one-level-deep anchors.	3 / 5
	Total	16 / 20 Passed

Description

90%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is concise, third-person, and explicitly covers both what the skill does and when to use it, with unusually good bilingual trigger coverage. The only weakness is that the 'what' is a single composite action rather than a list of concrete capabilities.

Dimension	Reasoning	Score
Specificity	Quotes: "Verify research idea novelty against recent literature." This names the domain (research novelty) with one concrete action (verify novelty against literature), matching the anchor 'Names domain and 1-2 concrete actions, but not comprehensive'. It does not list several specific actions (e.g., literature search, prior-work comparison, report generation), so it falls below 4.	3 / 5
Completeness	Quotes: "Verify research idea novelty against recent literature" (explicit what) and "Use when user says \"查新\", \"novelty check\"... or wants to verify a research idea is novel before implementing" (explicit when with concrete trigger phrases). Both questions are clearly and explicitly answered, matching the top anchor.	5 / 5
Trigger Term Quality	Quotes: "\"查新\", \"novelty check\", \"有没有人做过\", \"check novelty\"" — comprehensive natural trigger terms including English synonyms and Chinese phrasings a user would actually say. Coverage of common variations is complete; no anchor below 5 fits better.	5 / 5
Distinctiveness Conflict Risk	Quotes: "verify a research idea is novel before implementing" — a clear niche (pre-implementation novelty verification) with distinctive bilingual triggers unlikely to fire for literature-review or paper-writing skills. Minimal conflict risk.	5 / 5
	Total	18 / 20 Passed

Validation

81%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

Criteria	Description	Result
allowed_tools_field	'allowed-tools' contains unusual tool name(s)	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning
relative_links	Relative link issues: 2 suspicious	Warning

	Total	13 / 16 Passed

Repository: wanshuiyin/Auto-claude-code-research-in-sleep
Path: skills/novelty-check/SKILL.md
Commit: 2132036

Reviewed: 7 minutes ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.