CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

flash-attention

Speed up long-sequence transformer training and inference.

54

Quality

68%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./optional-skills/mlops/flash-attention/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is a strong, execution-oriented skill: three well-sequenced workflows with checklists, explicit validation steps with expected values, accurate version-sensitive guidance (deprecated sdp_kernel, FA2-vs-FA3 packaging), and real one-level-deep references. Weaknesses are minor: a slightly overlong main file, small redundancy between benchmark examples, and one snippet missing its import.

Suggestions

Consolidate the two benchmark code blocks (Workflow 1 Step 3 and Workflow 2 Step 4) into a single profiling recipe to trim length and redundancy.

Make the Workflow 1 Step 3 benchmark snippet self-contained by adding 'import torch.nn.functional as F' inside it.

Move the FA3 source-build instructions and/or the hardware-support matrix into a reference file to keep SKILL.md as a leaner overview.

DimensionReasoningScore

Conciseness

The body is code-forward with no concept tutorials and only brief framing ('Flash Attention provides 2-4x speedup and 10-20x memory reduction...'). Minor over-length keeps it below anchor 5: two separate benchmark code blocks (Workflow 1 Step 3 and Workflow 2 Step 4) overlap, and the transpose boilerplate in Workflow 2 Step 2 is slightly padded. It is above anchor 3 because there is no explanation of concepts Claude already knows.

4 / 5

Actionability

Nearly all guidance is executable: install commands, verification one-liners, copy-paste benchmark and accuracy-check code, and a careful FA3/FP8 section that correctly distinguishes the pip FA2 package from the source-built hopper interface. It falls just short of anchor 5 because the Workflow 1 Step 3 benchmark snippet calls F.scaled_dot_product_attention without importing F in that block, so it is not fully copy-paste ready in isolation.

4 / 5

Workflow Clarity

Each of the three workflows has a copyable checklist, an explicit sequence, and explicit validation steps with expected outcomes (install verification, expected 2-4x speedup, accuracy tolerance 'Should be <1e-3 for float16'), plus an error-recovery 'Common issues' section. This matches anchor 5 ('clear sequence with explicit validation steps; feedback loops for error recovery; checklists for complex processes').

5 / 5

Progressive Disclosure

Both referenced files (references/transformers-integration.md and references/benchmarks.md) exist, are one level deep, and are clearly signaled under 'Advanced topics'. The main body is well-sectioned but at ~370 lines is heavy for an overview - detail such as the FA3 source-build instructions and parts of the troubleshooting section could live in references. Anchor 4 ('most content is appropriately placed; minor organization gaps') rather than 5.

4 / 5

Total

17

/

20

Passed

Description

45%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is concise and states a clear benefit, but it is too thin to do its job: it names no concrete actions, omits all attention/flash-attention trigger terms, and lacks any 'when to use' guidance. It would be hard for a user (or skill router) to select this skill over generic optimization skills.

Suggestions

Add a 'what' with concrete capabilities, e.g. 'Accelerates and reduces memory for transformer attention using Flash Attention (SDPA / flash-attn), including sliding-window, MQA, and H100 FP8 paths.'

Add an explicit 'Use when...' trigger clause covering natural phrases users would say: 'Use when training or running inference on long-context transformers, when attention is slow or OOMing on GPU, or when the user mentions flash attention, SDPA, or attention memory usage.'

Include distinguishing keywords (flash attention, SDPA, KV cache memory, seq length) to reduce conflict risk with quantization or compilation optimization skills.

DimensionReasoningScore

Specificity

The description names the domain ('long-sequence transformer training and inference') but offers only one generic action ('Speed up'), with no concrete capabilities such as attention optimization, memory reduction, or the flash-attention mechanism. It matches anchor 2 ('Names the domain but actions are minimal or generic') rather than 3, which requires 1-2 concrete actions.

2 / 5

Completeness

The 'what' is clear (speeds up long-sequence transformer training and inference) but there is no 'Use when...' clause or equivalent explicit trigger guidance, which caps completeness at 3 per the judging guidelines. It is not 2 because the 'what' is specific rather than vague.

3 / 5

Trigger Term Quality

It includes relevant keywords ('speed up', 'long-sequence', 'transformer', 'training', 'inference') but omits the natural phrases a user would most likely say ('flash attention', 'attention', 'slow training', 'OOM', 'memory'). Matches anchor 3 ('Some relevant keywords but missing common variations or synonyms'); not 4 because the highest-signal terms for this skill are absent.

3 / 5

Distinctiveness Conflict Risk

'Speed up long-sequence transformer training and inference' is somewhat specific but would plausibly trigger for sibling optimization skills (quantization, torch.compile, kernel tuning) since 'flash attention' or 'attention' is never mentioned. Not 4 because overlap risk with closely related optimization skills is material.

3 / 5

Total

11

/

20

Passed

Validation

81%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_version

'metadata.version' is missing

Warning

metadata_field

'metadata' should map string keys to string values

Warning

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

13

/

16

Passed

Repository

NousResearch/hermes-agent

Path

optional-skills/mlops/flash-attention/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.