CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

langsmith-dataset

INVOKE THIS SKILL when creating evaluation datasets, uploading datasets to LangSmith, or managing existing datasets. Covers dataset types (final_response, single_step, trajectory, RAG), CLI management commands, SDK-based creation, and example management. Uses the langsmith CLI tool.

68

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

75%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with exact commands, dual-language executable code, concrete dataset schemas, and a verification-included end-to-end workflow. Its main weakness is redundancy — the CLI command reference is duplicated across two sections and the trace-export step appears twice — plus an implicit rather than inline validation checkpoint in the workflow.

Suggestions

Merge <usage> and <script_usage> into a single CLI reference section to eliminate the duplicated command catalog and reduce token cost.

Add an explicit validation checkpoint before upload (e.g., verify the JSON file is an array of objects with an 'inputs' key, and re-check with 'langsmith example list' after upload) directly in the example workflow rather than only in troubleshooting.

Show the trace-export-to-dataset processing step once (either in <creating_datasets> or <example_workflow>) and cross-reference it instead of repeating the export command in both places.

DimensionReasoningScore

Conciseness

The CLI command catalog is listed twice — once in <usage> ('langsmith dataset list', 'langsmith example create --dataset <name> --inputs <json>', etc.) and again nearly verbatim in <script_usage> — and the trace export command appears in both <creating_datasets> and <example_workflow>, so the content could be meaningfully tightened. Not 4: the duplication is substantial (a whole repeated section), beyond minor trimming; not 2: there is no over-explanation of concepts Claude already knows.

3 / 5

Actionability

Fully executable throughout: exact CLI invocations with flags ('langsmith dataset upload /tmp/dataset.json --name "My Evaluation Dataset" --api-key $LANGSMITH_API_KEY'), copy-paste Python and TypeScript code, concrete JSON example structures per dataset type, and troubleshooting with specific diagnostic commands. Covers the common cases end to end.

5 / 5

Workflow Clarity

The example workflow gives a clear 5-step sequence from trace export to upload to verification ('langsmith dataset list', 'dataset get', 'example list --limit 3') with troubleshooting for upload failures and example count mismatches. Not 5: there is no explicit pre-upload JSON validation step or fix-and-retry loop in the workflow itself — error handling is partly deferred to the troubleshooting section rather than an inline checkpoint.

4 / 5

Progressive Disclosure

The body is well organized into clearly delineated sections (<setup>, <usage>, <creating_datasets>, <dataset_structures>, <script_usage>, <example_workflow>, <troubleshooting>) with no bundle files, so everything appropriately lives in one file. Not 5: minor gaps — non-standard custom tags instead of consistent markdown headers, and duplicated sections (usage vs. script_usage) slightly muddy navigation.

4 / 5

Total

16

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is strong: it states both what the skill covers and when to invoke it using concrete, natural trigger phrases, and it comprehensively enumerates dataset types and creation paths (CLI and SDK). The only weaknesses are a few missing natural synonyms and slight breadth in 'CLI management commands' that could overlap sibling LangSmith skills.

Suggestions

Add common synonyms for dataset-related requests, e.g., 'building test/eval sets or golden data for evaluations', to broaden natural trigger coverage.

Tighten 'CLI management commands' to dataset-scoped operations (list/get/create/delete/export/upload datasets and examples) to reduce overlap with sibling LangSmith CLI skills.

DimensionReasoningScore

Specificity

Lists multiple concrete actions — 'creating evaluation datasets, uploading datasets to LangSmith, or managing existing datasets', 'CLI management commands, SDK-based creation, and example management' — with comprehensive coverage of all dataset types (final_response, single_step, trajectory, RAG). Not 4: coverage spans both CLI and SDK paths and names every dataset type, leaving no meaningful gaps.

5 / 5

Completeness

Explicitly answers 'when' with concrete triggers ('INVOKE THIS SKILL when creating evaluation datasets, uploading datasets to LangSmith, or managing existing datasets') and 'what' ('Covers dataset types..., CLI management commands, SDK-based creation, and example management. Uses the langsmith CLI tool.'). Not 4: both halves are explicit and specific rather than merely implied.

5 / 5

Trigger Term Quality

Natural phrases like 'creating evaluation datasets', 'uploading datasets to LangSmith', and 'managing existing datasets' match what users would actually say. Not 5: common synonyms such as 'test sets', 'golden data', or 'eval examples' are missing, so keyword coverage is good but not comprehensive.

4 / 5

Distinctiveness Conflict Risk

LangSmith dataset management is a clear, distinct niche unlikely to trigger the wrong skill. Not 5: the generic phrase 'CLI management commands' alongside the shared langsmith CLI leaves minor overlap risk with closely related LangSmith skills (e.g., trace or experiment management).

4 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

hefgi/dotfiles

Path

.agents/skills/langsmith-dataset/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.