Skip to content

Commit 06342ff

Browse files
authored
[feat] Implement model-agnostic one-engine eagle3 (#4778)
Signed-off-by: Yilin Fan <[email protected]>
1 parent 25aa388 commit 06342ff

File tree

4 files changed

+435
-441
lines changed

4 files changed

+435
-441
lines changed

tensorrt_llm/_torch/models/modeling_auto.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ def from_config(
1616
# Hack to detect eagle3 checkpoints. TODO: should we provide
1717
# our own checkpoints with the correct arch? It would let us
1818
# avoid nasty stuff like this.
19+
model_arch = model_arch.replace("Eagle3",
20+
"") # Strip the appended EAGLE3
1921
if hasattr(config.pretrained_config, "draft_vocab_size"):
2022
model_arch = "EAGLE3" + model_arch
2123

0 commit comments

Comments
 (0)