CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

quantizing-models-bitsandbytes

Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss. Use when GPU memory is limited, need to fit larger models, or want faster inference. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers. Works with HuggingFace Transformers. Also covers AWQ, GGUF, GPTQ and HQQ (guides in references/).

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

75%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly executable skill: three checklist-driven workflows, concrete code, a clear when-to-use-vs-alternatives decision table, and advanced material genuinely delegated to real bundle files. It lands at 'good with minor gaps' on every dimension — duplicated config blocks, a few non-self-contained snippets, a missing validation step in the QLoRA workflow, and a second nesting level in the merged tool guides.

Suggestions

Deduplicate the 4-bit NF4 BitsAndBytesConfig: define the full config once (e.g., in Workflow 1 Step 3) and have the quick start, Workflow 2, and the 'Lower accuracy than expected' issue reference it instead of repeating it three times.

Make workflow snippets self-contained so they are copy-paste executable: show how train_dataset/tokenizer are built in Workflow 2, and define model in Workflow 3's optimizer example.

Add an explicit validation step to the QLoRA workflow (e.g., confirm the print_trainable_parameters output looks sane and evaluate the saved adapter on a test prompt before declaring success).

DimensionReasoningScore

Conciseness

Mostly lean — code blocks, tables, and checklists with no explanation of concepts Claude already knows — but the 4-bit NF4 BitsAndBytesConfig block appears three times (quick start, Workflow 1 Step 3, 'Lower accuracy than expected') and the quick start substantially duplicates Workflow 1. Anchor 4 ('minor instances... that could be trimmed'), not 5, because that duplication means not every token earns its place; not 3 because there is no genuinely unnecessary explanation.

4 / 5

Actionability

Concrete, near copy-paste guidance throughout ('pip install bitsandbytes transformers accelerate', full BitsAndBytesConfig objects, the VRAM/model-size recommendation table, executable troubleshooting commands), but several snippets reference undefined variables ('train_dataset=train_dataset', 'tokenizer=tokenizer' in Workflow 2; bare 'model' in Workflow 3). Anchor 4's 'concrete code with minor gaps', not 5's fully copy-paste ready.

4 / 5

Workflow Clarity

Each workflow opens with a copyable checklist and sequenced steps, and Workflow 1 ends in explicit verification ('Test inference', 'Check memory'); Workflow 3 includes a memory-monitoring step. Anchor 4, not 5, because Workflow 2 (QLoRA) has no post-training validation checkpoint and error-recovery guidance lives in a separate 'Common issues' section rather than as feedback loops inside the workflows.

4 / 5

Progressive Disclosure

Scored against the actual bundle: SKILL.md is a genuine overview with well-signaled one-level references ('See [references/qlora-training.md]...') and a descriptive merged-tools table, and every referenced path exists on disk. Anchor 4, not 5, because the four tool sub-guides each add a second reference level (references/gguf/overview.md → references/gguf/references/advanced-usage.md), breaking the one-level-deep ideal; not lower because the nesting is clearly labeled with descriptive links and each overview contains substantive content itself.

4 / 5

Total

16

/

20

Passed

Description

96%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: concrete third-person capabilities, an explicit 'Use when' trigger clause with multiple concrete conditions, and unusually broad keyword coverage including the alternative tool names the skill now bundles. The only soft spot is the generic 'faster inference' trigger, which slightly raises overlap risk with dedicated inference-serving skills.

DimensionReasoningScore

Specificity

Lists multiple concrete actions — 'Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction', 'QLoRA training, and 8-bit optimizers', 'Works with HuggingFace Transformers', plus coverage of AWQ/GGUF/GPTQ/HQQ — with no identifiable coverage gaps, matching the comprehensive anchor. Not 4, because that anchor requires 'minor gaps in coverage' and none are evident.

5 / 5

Completeness

Explicitly answers both: what ('Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss... 8-bit optimizers') and when ('Use when GPU memory is limited, need to fit larger models, or want faster inference') with concrete trigger conditions. Not 4, because the 'when' clause is already explicit and specific rather than needing more detail.

5 / 5

Trigger Term Quality

Comprehensive natural-term coverage: 'GPU memory is limited', 'fit larger models', 'faster inference', '8-bit', '4-bit', 'INT8, NF4, FP4', 'QLoRA', and the alternative tool names (AWQ, GGUF, GPTQ, HQQ) users actually say — GGUF doubling as the file-format name. Not 4, because only isolated synonyms (e.g., 'VRAM') are missing rather than 'a few natural terms'.

5 / 5

Distinctiveness Conflict Risk

The bitsandbytes/HuggingFace niche is clear and the named alternatives (AWQ, GGUF, GPTQ, HQQ) are bundled into this same skill, so they don't create cross-skill conflict. Not 5, because the generic 'want faster inference' trigger and GGUF's CPU-inference scope create minor overlap risk with dedicated inference-serving or llama.cpp skills.

4 / 5

Total

19

/

20

Passed

Validation

81%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

relative_links

Relative link issues: 4 deeper-than-1-level

Warning

referenced_paths_exist

Referenced path issues: 8 deeper-than-1-level

Warning

Total

13

/

16

Passed

Repository

OpenLAIR/dr-claw

Path

skills/optimization/bitsandbytes/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.