gptq

Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-tuning.

Quality

82%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

76%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with comprehensive executable examples and good progressive-disclosure structure pointing to real reference files. Its main weakness is the absence of a validation/feedback loop in the quantization workflow before persisting or deploying a model.

Suggestions

Add an explicit validation checkpoint after model.quantize() — e.g., evaluate perplexity on a held-out set and compare to FP16 before calling save_quantized()/push_to_hub().

Deduplicate the BaseQuantizeConfig example: the 'Standard 4-bit' config block repeats the earlier quantize-your-own block; reference it or show only the differing parameters.

Trim marketing-style prose such as 'Excellent quality preservation - less than 2% degradation!' since the table already conveys this.

Dimension	Reasoning	Score
Conciseness	Largely lean and code-forward with little explanation of known concepts, but the standard 4-bit config block repeats the earlier BaseQuantizeConfig example and a few lines of marketing prose ('Excellent quality preservation') could be trimmed, keeping it just below the lean anchor.	4 / 5
Actionability	Provides copy-paste-ready, fully executable code for loading, quantizing, all kernel backends, transformers/QLoRA integration, multi-GPU offload, and batch inference, covering the common cases comprehensively.	5 / 5
Workflow Clarity	The quantize-your-own flow is sequenced (load → config → calibrate → quantize → save → push) but has no validation checkpoint (e.g., eval perplexity vs FP16 before saving/pushing), which the rubric caps at 3 for quality-sensitive batch/destructive operations.	3 / 5
Progressive Disclosure	Well-organized sections with clearly signaled, one-level-deep references to verified files (calibration.md, integration.md, troubleshooting.md); the body is fairly long with benchmarks/kernel details inlined that could arguably live in references, so it is not a clean 5.	4 / 5
	Total	16 / 20 Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is strong: it concisely states what GPTQ does and gives concrete, quantified 'use when' triggers covering memory, accuracy, and speed. It is slightly technical in vocabulary and has minor overlap with related quantization skills.

Dimension	Reasoning	Score
Specificity	Names the domain plus multiple concrete, quantified actions ('Post-training 4-bit quantization', '4× memory reduction with <2% perplexity degradation', '3-4× speedup vs FP16', 'QLoRA fine-tuning'), matching the comprehensive-coverage anchor.	5 / 5
Completeness	Explicitly answers both what ('Post-training 4-bit quantization for LLMs with minimal accuracy loss') and when ('Use for deploying large models... when you need 4× memory reduction... or for faster inference'), with concrete quantified trigger phrases.	5 / 5
Trigger Term Quality	Strong domain keywords a practitioner would say ('quantization', '4-bit', 'QLoRA fine-tuning', 'FP16', 'perplexity') but skews technical and omits natural synonyms like 'compress'/'shrink'/'reduce memory', so it sits just below the comprehensive anchor.	4 / 5
Distinctiveness Conflict Risk	GPTQ-style post-training 4-bit quantization with QLoRA is a clear niche, but overlaps with sibling quantization skills (AWQ, bitsandbytes) referenced in the body, leaving minor conflict risk.	4 / 5
	Total	18 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: OpenLAIR/dr-claw
Path: skills/optimization/gptq/SKILL.md
Commit: 4b35756

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.