optimizing-attention-flash

Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Use when training/running transformers with long sequences (>512 tokens), encountering GPU memory issues with attention, or need faster inference. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.

Quality

81%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

78%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable skill body with copy-paste code, clear workflow checklists, and properly split one-level references. The main weakness is redundancy — repeated install commands and benchmark snippets across workflows — that could be consolidated to improve token efficiency.

Suggestions

Consolidate the repeated 'pip install flash-attn --no-build-isolation' instructions into a single install step referenced by each workflow instead of repeating it three times.

Factor the duplicated benchmarking code (Workflow 1 Step 3 and Workflow 2 Step 4) into one reusable benchmark snippet to reduce token cost.

Add explicit in-workflow feedback loops to Workflows 2 and 3 (e.g., 'if no speedup, verify seq length > 512'), rather than relying on the separate Common Issues section.

Dimension	Reasoning	Score
Conciseness	The body is mostly code-driven and avoids conceptual padding, but repeats the same install command ('pip install flash-attn --no-build-isolation') three times and duplicates benchmarking code across workflows; these could be consolidated. It sits between efficient (4) and mostly-efficient (3); the repeated snippets push it to 3.	3 / 5
Actionability	Provides copy-paste ready, executable code covering the common cases (SDPA, flash-attn, MQA, sliding window, benchmarking, accuracy verification), with concrete commands and expected outputs. The only soft spot is the FP8 example which may not run verbatim via flash_attn_func, but the bulk is fully executable.	5 / 5
Workflow Clarity	Each of the three workflows has a numbered checkbox checklist with bold step headers and concrete code, and Workflow 1 includes explicit validation (accuracy match in Step 4, speedup verification in Step 3). However, Workflows 2 and 3 lack in-workflow feedback loops, relying on a separate troubleshooting section, leaving minor validation gaps.	4 / 5
Progressive Disclosure	Clear overview structure with well-signaled, one-level-deep references to references/transformers-integration.md and references/benchmarks.md (both verified to exist), with heavy detailed material split out and practical guidance kept inline. Navigation is easy and references are not nested.	5 / 5
	Total	17 / 20 Passed

Description

83%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, specific description that clearly answers both what the skill does and when to use it, with quantified benefits and concrete supported backends. It could be improved by adding common synonyms (OOM, long context) and tightening the broader trigger phrases that risk overlap with general optimization skills.

Suggestions

Add common synonyms users actually say, such as 'OOM' / 'out of memory', 'long context', and 'context window', to broaden trigger coverage.

Tie the broad 'GPU memory issues' and 'need faster inference' triggers more tightly to attention specifically (e.g., 'GPU memory issues with attention OOMs') to reduce overlap with general optimization skills.

Consider listing distinct actions (e.g., 'drops in via SDPA, swaps to flash-attn kernels, enables FP8 on H100') rather than only naming supported backends.

Dimension	Reasoning	Score
Specificity	Names the core action ('Optimizes transformer attention with Flash Attention') with quantified outcomes (2-4x speedup, 10-20x memory reduction) and lists several concrete supported backends (SDPA, flash-attn, H100 FP8, sliding window), but these are features rather than many distinct actions, leaving minor coverage gaps.	4 / 5
Completeness	Explicitly states both what it does ('Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction') and when to use it via a concrete 'Use when...' clause with multiple specific triggers.	5 / 5
Trigger Term Quality	Includes natural triggers users would say ('training/running transformers with long sequences', 'GPU memory issues with attention', 'need faster inference'), but misses common synonyms like OOM, long context, or context window.	4 / 5
Distinctiveness Conflict Risk	Flash Attention / transformer attention optimization is a clear niche with distinct triggers, but broader phrases like 'GPU memory issues' and 'need faster inference' create minor overlap risk with general memory- or inference-optimization skills.	4 / 5
	Total	17 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/flash-attention/SKILL.md
Commit: 1d182e9

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.