CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

quantizing-models-bitsandbytes

Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss. Use when GPU memory is limited, need to fit larger models, or want faster inference. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers. Works with HuggingFace Transformers.

69

Quality

87%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

82%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

Highly actionable content with executable code, checklists, decision tables, and well-signaled one-level-deep references. The main improvements are de-duplicating the repeated 4-bit config blocks, moving the inline QLoRA workflow detail into the existing qlora-training.md reference, and adding post-training validation checkpoints.

Suggestions

Deduplicate the 4-bit NF4 config (appears in Quick start, Workflow 1 Step 3, and Workflow 2 Step 2) — show it once and reference it elsewhere.

Trim Workflow 2 to the essentials and point to references/qlora-training.md, which already covers the full fine-tuning procedure.

Add an explicit validation step after training in Workflows 2 and 3 (e.g. run a generation/eval check on saved adapters) to complete the feedback loop.

DimensionReasoningScore

Conciseness

The body is dense with code, tables, and checklists with almost no conceptual padding, but the same 4-bit NF4 BitsAndBytesConfig block appears three times (Quick start, Workflow 1 Step 3, Workflow 2 Step 2) and the memory formula/table is restated across sections — minor trimming opportunities. Not 5 because of that repeated content; not 3 because there is no over-explanation of concepts Claude already knows.

4 / 5

Actionability

Every section is copy-paste-ready: executable Python (full BitsAndBytesConfig, from_pretrained with device_map, PEFT LoRA setup, Trainer loop), install commands, concrete VRAM-to-model-size tables, and printed memory-verification snippets. Not 4 because examples cover the common cases (8-bit, 4-bit, QLoRA, 8-bit optimizer) completely with no gaps in key details.

5 / 5

Workflow Clarity

Each workflow has an explicit copyable checklist with numbered steps, and Workflow 1 includes a test-inference/memory verification in Step 4 plus a "Common issues" section for error recovery. Not 5 because Workflows 2 and 3 end at train/monitor with no post-run validation checkpoint, so feedback loops are only implicit in parts of the skill.

4 / 5

Progressive Disclosure

The three references (qlora-training.md, quantization-formats.md, memory-optimization.md) all exist, are substantial (447-521 lines), one level deep, and are clearly signaled under "Advanced topics" with what each covers. Not 5 because the ~420-line body inlines a full QLoRA workflow and troubleshooting that partially duplicates the dedicated reference files — content that belongs in the bundle sits in SKILL.md.

4 / 5

Total

17

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: quantified what, explicit use-when triggers, and named formats/framework. Its main weaknesses are the absence of the keyword "bitsandbytes" itself (weakest distinctiveness discriminator) and slightly thin synonym coverage for trigger terms.

Suggestions

Include the keyword "bitsandbytes" in the description itself (not just the name field) to sharpen distinctiveness against GPTQ/GGUF/llama.cpp quantization skills.

Add one or two natural user phrasings as trigger terms, e.g. "reduce VRAM usage" or "shrink/quantize a HuggingFace model".

DimensionReasoningScore

Specificity

"Quantizes LLMs to 8-bit or 4-bit", "Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers" — multiple concrete actions (quantize to 8/4-bit, QLoRA training, 8-bit optimizers) with named formats and quantified outcomes, matching the comprehensive-coverage anchor. Not 4 because coverage of the skill's core capabilities is complete rather than having minor gaps.

5 / 5

Completeness

Explicitly answers what ("Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss") and when ("Use when GPU memory is limited, need to fit larger models, or want faster inference") with concrete trigger phrases. Not 4 because the when-clause is already explicit and specific rather than improvable-but-present.

5 / 5

Trigger Term Quality

"GPU memory is limited", "fit larger models", "QLoRA", "8-bit", "quantize" are natural phrases users would say when needing this skill, but the keyword "bitsandbytes" itself and common variations like "quantization"/"reduce VRAM" phrasing are thin. Not 5 because a few natural terms a user would plausibly say are missing; not 3 because coverage is good, not partial.

4 / 5

Distinctiveness Conflict Risk

"Works with HuggingFace Transformers" plus the INT8/NF4/FP4/QLoRA focus carves a clear niche, but the description never names bitsandbytes — the single strongest discriminator — so it could overlap with sibling GPTQ/GGUF quantization skills. Not 5 because of that minor overlap risk; not 3 because the format and framework terms already distinguish it from most adjacent skills.

4 / 5

Total

18

/

20

Passed

Validation

87%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_version

'metadata.version' is missing

Warning

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

14

/

16

Passed

Repository

synthetic-sciences/openscience

Path

backend/cli/skills/ml-training/bitsandbytes/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.