arxiv

Search arXiv papers by keyword, author, category, or ID.

Quality

61%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./skills/research/arxiv/SKILL.md

Quality

Content

65%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is highly actionable with executable examples and a clean supporting script, and it is well sectioned for navigation. It loses points for length/redundant inline parsing, missing error-handling checkpoints in the workflow, and a monolithic single-file structure that could use reference splitting.

Suggestions

Move the large inline XML-parsing and BibTeX snippets into the helper script or a reference file, keeping SKILL.md as a lean overview with short examples.

Add explicit validation/error-handling steps to the workflow (e.g. checking for an empty entry, withdrawn-paper summary, or rate-limit responses) before proceeding to the next step.

Split the Semantic Scholar section into a separate reference file (e.g. references/semantic_scholar.md) and signal it from the overview to improve progressive disclosure.

Dimension	Reasoning	Score
Conciseness	The body is information-dense and assumes Claude's competence without explaining basics, but it is long and carries large inline XML-parsing and BibTeX snippets that duplicate the helper script and could be tightened or offloaded.	2 / 3
Actionability	Executable curl commands, complete Python parsing/BibTeX snippets, and a documented helper script with usage examples make the guidance copy-paste ready, matching the fully-executable anchor.	3 / 3
Workflow Clarity	The 'Complete Research Workflow' gives a clear 7-step sequence, but there are no validation/error-handling checkpoints across the multiple API calls (e.g. handling not-found papers or rate limits), so it sits at the sequence-present-but-checkpoints-missing anchor.	2 / 3
Progressive Disclosure	Sections are well organized and the scripts/search_arxiv.py helper is clearly signaled one level deep, but the SKILL.md is monolithic with the Semantic Scholar reference and BibTeX generation inline rather than split into reference files.	2 / 3
	Total	9 / 12 Passed

Description

57%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific and distinctive, clearly scoping the skill to arXiv search with concrete query dimensions. Its main weakness is the missing explicit 'when to use' guidance, which caps completeness and leaves trigger coverage slightly thin.

Suggestions

Add an explicit 'Use when...' clause, e.g. 'Use when searching arXiv for research papers, preprints, or academic abstracts by topic, author, or category.'

Broaden trigger terms to include natural variations like 'preprints', 'research papers', and 'academic papers'.

Consider mentioning a second concrete action (e.g. retrieving/fetching a specific paper or generating a citation) to raise specificity beyond a single search action.

Dimension	Reasoning	Score
Specificity	Quotes 'Search arXiv papers by keyword, author, category, or ID' name the domain and several concrete search modalities, but it is a single search action parameterized rather than multiple distinct actions like the score-3 anchor (extract, fill forms, merge).	2 / 3
Completeness	It clearly states what the skill does but has no 'Use when...' clause or equivalent explicit trigger guidance, so per the judging guidelines completeness is capped at 2.	2 / 3
Trigger Term Quality	'arXiv papers', 'keyword', 'author', 'category', 'ID' are relevant natural terms, but common variations users would say such as 'preprints', 'research papers', or 'abstracts' are missing, matching the some-keywords-but-gaps anchor.	2 / 3
Distinctiveness Conflict Risk	'arXiv papers' names a specific, well-known service niche with distinct triggers, making it unlikely to fire for the wrong skill.	3 / 3
	Total	9 / 12 Passed

Validation

81%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

Criteria	Description	Result
metadata_version	'metadata.version' is missing	Warning
metadata_field	'metadata' should map string keys to string values	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	13 / 16 Passed

Repository: NousResearch/hermes-agent
Path: skills/research/arxiv/SKILL.md
Commit: d0d116b

Reviewed: about 14 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.