autolab-reporter

Operate the local Trackio reporter for Autolab HF Jobs. Use when a reporter or planner needs to inspect scores, active jobs, worker anomalies, duplicate launches, or the overall experiment board.

Quality

88%

Does it follow best practices?

Impact

—

No eval scenarios have been run

Securityby

Advisory

Suggest reviewing before use

Quality

Content

87%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

This is a well-crafted, concise skill that provides actionable commands and domain-specific operational guidance. Its main weakness is the lack of explicit validation checkpoints in the workflow—particularly after the sync step and before launching new work based on reporter output. The 'What To Watch' and 'Guardrails' sections add genuine value by encoding operational knowledge Claude wouldn't otherwise have.

Suggestions

Add a validation checkpoint after the sync step (e.g., check exit code or expected output) before proceeding to the summary review.

In the Guardrails section, provide a concrete example of an anomaly-fix-verify loop, such as: 'If duplicates found → kill duplicate → re-sync → confirm resolution before proceeding.'

Dimension	Reasoning	Score
Conciseness	The content is lean and efficient. Every section serves a purpose—workflow commands are concrete, 'What To Watch' lists domain-specific knowledge Claude wouldn't inherently know, and guardrails are brief but meaningful. No unnecessary explanations of what Trackio or HF Jobs are.	3 / 3
Actionability	Each workflow step provides a specific, copy-paste-ready command with clear flags and arguments. The commands are fully executable (e.g., `uv run scripts/trackio_reporter.py sync --project autolab`) rather than pseudocode or vague instructions.	3 / 3
Workflow Clarity	Steps are clearly sequenced and numbered, but there are no validation checkpoints or feedback loops. For instance, after syncing, there's no explicit check for sync success/failure, and the guardrails section mentions fixing anomalies but doesn't provide a concrete verify-then-proceed pattern.	2 / 3
Progressive Disclosure	For a skill of this size (~30 lines) with no need for external references, the content is well-organized into clear sections (Workflow, What To Watch, Guardrails) that are easy to scan and navigate. No monolithic walls of text or unnecessary nesting.	3 / 3
	Total	11 / 12 Passed

Description

89%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

This is a solid skill description that clearly identifies a specific tool and domain, provides an explicit 'Use when' clause with relevant trigger terms, and is highly distinctive. The main weakness is that the actions are somewhat homogeneous (all variations of 'inspect/view') rather than listing diverse concrete operations like creating, modifying, or exporting data.

Dimension	Reasoning	Score
Specificity	The description names the domain ('Trackio reporter for Autolab HF Jobs') and lists several actions/areas (inspect scores, active jobs, worker anomalies, duplicate launches, experiment board), but these are more like categories of inspection rather than concrete discrete actions (e.g., 'extract', 'fill', 'merge'). It's between 2 and 3, but closer to 2 since the actions are all variations of 'inspect'.	2 / 3
Completeness	Clearly answers both 'what' (operate the local Trackio reporter for Autolab HF Jobs) and 'when' (explicitly states 'Use when a reporter or planner needs to inspect scores, active jobs, worker anomalies, duplicate launches, or the overall experiment board'). The 'Use when...' clause is explicit and detailed.	3 / 3
Trigger Term Quality	Includes strong natural trigger terms that a user would say: 'scores', 'active jobs', 'worker anomalies', 'duplicate launches', 'experiment board', 'reporter', 'planner', 'Trackio', 'Autolab HF Jobs'. These are domain-specific terms that users in this context would naturally use.	3 / 3
Distinctiveness Conflict Risk	Highly distinctive due to the specific tool name ('Trackio reporter'), platform ('Autolab HF Jobs'), and niche domain terms ('worker anomalies', 'duplicate launches', 'experiment board'). Very unlikely to conflict with other skills.	3 / 3
	Total	11 / 12 Passed

Validation

72%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 8 / 11 Passed

Validation for skill structure

Criteria	Description	Result
metadata_version	'metadata.version' is missing	Warning
metadata_field	'metadata' should map string keys to string values	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	8 / 11 Passed

Repository: huggingface/context-course
Commit: 0448a7c

Reviewed: 26 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.