eagle3-new-model
Add a new model to the EAGLE3 offline pipeline. Generates an hf_offline_eagle3.yaml launcher config for a new model checkpoint, choosing the right hidden state dump backend (TRT-LLM / HF / vLLM) and GPU configuration. Use when user wants to run EAGLE3 on a model that does not yet have a YAML in tools/launcher/examples/ or asks how to configure the pipeline for a new checkpoint.
75
93%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
View guide
Passed
No findings from the security scan
EAGLE3 New Model Configuration
Create tools/launcher/examples/<Org>/<Model>/hf_offline_eagle3.yaml by copying the
closest existing example and adapting it. Pick a reference with the same shape as the
target (dense vs MoE, similar size) from tools/launcher/examples/ — e.g. the Qwen3-8B
config for a dense model.
The pipeline is a 4-task config (task_0 data synthesis → task_1 hidden-state dump →
task_2 train → task_3 benchmark). The task structure, args, containers, and GPU/node
sizing are all visible in the existing examples — infer them from a reference rather than
hand-rolling. This file documents only the two things that are not obvious from the
examples: which dump backend to pick, and the model-specific gotchas.
Choosing the task_1 hidden-state dump backend
| Backend | Script | When to use |
|---|---|---|
| vLLM | common/eagle3/dump_offline_data_vllm.sh | Default. Broad coverage via vLLM's native hidden-state extractor. |
| HF | common/eagle3/dump_offline_data_hf.sh | VLMs / multimodal, custom-code models, sliding-window attention (TRT-LLM can't serve these). |
| TRT-LLM | common/eagle3/dump_offline_data.sh | Pure-text models with TRT-LLM support; pass --tp <TP> and --moe-ep <EP>. |
Rule of thumb: HF if the model is a VLM or uses sliding-window attention; vLLM otherwise. TRT-LLM only when you specifically want its kernels for a supported plain-text model.
Model-specific adjustments
These are the non-obvious knobs that vary per model:
| Situation | What to change |
|---|---|
Requires --trust-remote-code | Add to task_0 vLLM args (before the -- separator) and to task_3 benchmark args |
| MoE with large expert hidden dim | Increase intermediate_size in eagle_config.json to match moe_intermediate_size |
| Custom tokenizer (e.g. tiktoken) | Set TIKTOKEN_RS_CACHE_DIR env var in task_0 and task_1 |
After adapting the config, preview it with --dryrun before submitting.
- Repository
- NVIDIA/Model-Optimizer
- Commit
33d05b0
- Last updated
- First committed
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.