quantizing-models-bitsandbytes

Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss. Use when GPU memory is limited, need to fit larger models, or want faster inference. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers. Works with HuggingFace Transformers.

Quality

92%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

85%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with executable code, clear sequenced checklists, and well-organized progressive disclosure into real reference files. Its only weakness is moderate redundancy where the Quick start configs are repeated in the workflow sections.

Suggestions

Remove the duplicated 8-bit and 4-bit BitsAndBytesConfig blocks in Workflow 1 Step 3, or replace them with a one-line pointer back to the Quick start config and only show the additional options (llm_int8_threshold, bnb_4bit_use_double_quant).

Trim explanatory inline comments like '# Outlier threshold' and '# Compute in FP16' where the parameter name is already self-explanatory, to tighten token efficiency toward the level-3 conciseness bar.

Dimension	Reasoning	Score
Conciseness	The body is mostly lean code, formulas, and tables, but the 8-bit and 4-bit BitsAndBytesConfig blocks from the Quick start are repeated again in Workflow 1 Step 3, and some explanatory comments could be trimmed. It earns level 2 (mostly efficient but with some redundancy) rather than level 3's every-token-earns-its-place bar.	2 / 3
Actionability	Provides fully executable, copy-paste-ready Python and bash snippets plus concrete memory formulas and a VRAM/model decision table, matching the level-3 anchor of specific executable examples.	3 / 3
Workflow Clarity	Each workflow ships a copyable checklist with clearly sequenced steps and an explicit verification step (Workflow 1 Step 4 tests inference and checks allocated memory; Workflow 2 prints trainable params), satisfying the level-3 clear-sequence-with-validation anchor. These are non-destructive operations, so the destructive-operations cap does not apply.	3 / 3
Progressive Disclosure	Keeps quick-start and workflows inline and delegates depth to three well-signaled, one-level-deep references (qlora-training.md, quantization-formats.md, memory-optimization.md), all of which exist as real files, matching the level-3 clear-overview-with-one-level-references anchor.	3 / 3
	Total	11 / 12 Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, third-person, and cleanly answers both what the skill does and when to use it with natural trigger terms and a distinct niche. It is a strong, concise description with no notable weaknesses.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions: 'Quantizes LLMs to 8-bit or 4-bit' and 'Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers', matching the level-3 anchor of several specific concrete actions rather than a single named domain.	3 / 3
Completeness	Explicitly answers both what ('Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss') and when ('Use when GPU memory is limited, need to fit larger models, or want faster inference'), matching the level-3 anchor with an explicit 'Use when' clause.	3 / 3
Trigger Term Quality	Uses natural terms users would say ('GPU memory is limited', 'fit larger models', 'faster inference', '8-bit', '4-bit', 'QLoRA'), giving good coverage rather than only technical jargon.	3 / 3
Distinctiveness Conflict Risk	The bitsandbytes LLM quantization niche with INT8/NF4/FP4/QLoRA triggers is clearly distinguishable and unlikely to fire for unrelated skills, matching the level-3 clear-niche anchor.	3 / 3
	Total	12 / 12 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: Orchestra-Research/AI-Research-SKILLs
Path: 10-optimization/bitsandbytes/SKILL.md
Commit: 773a529

Reviewed: about 5 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.