flash-attention

Speed up long-sequence transformer training and inference.

Quality

68%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./optional-skills/mlops/flash-attention/SKILL.md

Quality

Content

82%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with executable code and checklists, well-organized sections, and two clearly signaled reference files; its main weaknesses are minor conciseness redundancy and three large inlined workflows that keep it from being a lean overview.

Suggestions

Tighten the repeated benchmark/profiling blocks (Workflow 1 Step 3 and Workflow 2 Step 4 overlap) and trim verbose troubleshooting prose to improve conciseness.

Make error-recovery feedback loops explicit inside the workflows (e.g., 'If the max difference is not <1e-3, check dtype/casting and re-run') to push workflow clarity toward 5.

Consider moving the long FA3/FP8 build workflow and detailed benchmark scripts into a reference file so SKILL.md reads as a leaner overview, improving progressive disclosure.

Dimension	Reasoning	Score
Conciseness	The body is mostly lean code with brief one-liner intros ('Flash Attention provides 2-4x speedup...') and assumes Claude's competence with Python/torch, with only minor trimmable redundancy (e.g., repeated benchmark blocks, somewhat verbose 'Common issues'), matching 'efficient; minor instances of over-explanation'.	4 / 5
Actionability	It provides copy-paste-ready, executable code and commands across the common cases (PyTorch SDPA, flash-attn library, FA3 FP8) plus concrete shell commands like 'pip install flash-attn --no-build-isolation' and 'nvidia-smi', matching 'fully executable; copy-paste ready code or commands; specific examples cover the common cases'.	5 / 5
Workflow Clarity	Each of the three workflows has a copyable checklist and explicit verification checkpoints (Step 3 'Verify speedup with profiling', Step 4 'Test accuracy matches baseline', H100/install verification), matching 'clear sequence with most checkpoints present'; not a 5 because embedded 'if validation fails → fix → re-validate' feedback loops are only lightly stated rather than fully spelled out.	4 / 5
Progressive Disclosure	Clear section structure with two real, well-signaled one-level-deep references ('See references/transformers-integration.md', 'See references/benchmarks.md') for overflow topics, matching 'good structure; most content appropriately placed'; not a 5 because the main file inlines three full detailed workflows rather than functioning as a lean overview pointing to detailed materials.	4 / 5
	Total	17 / 20 Passed

Description

53%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description states a clear what (speeding up long-sequence transformer training/inference) in a distinct niche, but uses a single generic action verb and provides no explicit 'Use when' trigger guidance, capping completeness and trigger quality at the mid-level.

Suggestions

Add an explicit 'Use when...' clause naming trigger phrases users would say (e.g., 'Use when speeding up long-context/long-sequence transformer training or inference, or when attention is a memory or throughput bottleneck').

Replace the single generic verb 'Speed up' with several concrete actions (e.g., 'Applies IO-aware Flash Attention to accelerate and reduce memory of long-sequence attention for training and inference').

Include natural synonyms and file/keyword variants (Flash Attention, SDPA, long context, LLM attention) to improve trigger-term coverage.

Dimension	Reasoning	Score
Specificity	Quotes 'Speed up long-sequence transformer training and inference' — names a concrete domain (long-sequence transformer) and two targets (training, inference), but the action itself is a single generic verb ('Speed up') with no list of specific actions, matching the 'names domain and 1-2 concrete actions but not comprehensive' anchor.	3 / 5
Completeness	The 'what' is clear (speed up transformer training/inference), but there is no 'Use when...' clause or equivalent explicit trigger guidance, so per the judging guideline completeness is capped at 3.	3 / 5
Trigger Term Quality	Natural terms like 'transformer training', 'inference', and 'long-sequence' are present, but it omits common variations/synonyms users would say (attention, long context, LLM, Flash Attention), fitting 'some relevant keywords but missing common variations'.	3 / 5
Distinctiveness Conflict Risk	'long-sequence transformer training and inference' carves a fairly distinct niche with only minor overlap risk against general optimization/speedup skills, matching 'mostly distinct; minor overlap risk with closely related skills'; not a 5 because the lack of an explicit trigger weakens distinctiveness.	4 / 5
	Total	13 / 20 Passed

Validation

81%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

Criteria	Description	Result
metadata_version	'metadata.version' is missing	Warning
metadata_field	'metadata' should map string keys to string values	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	13 / 16 Passed

Repository: NousResearch/hermes-agent
Path: optional-skills/mlops/flash-attention/SKILL.md
Commit: 6680afb

Reviewed: about 3 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.