Skip to content

Commit 9d2df04

Browse files
authored
[None][doc] fix mtp.py typo (#9307)
Signed-off-by: liugaoji <[email protected]>
1 parent a92af27 commit 9d2df04

File tree

1 file changed

+3
-3
lines changed
  • tensorrt_llm/_torch/speculative

1 file changed

+3
-3
lines changed

tensorrt_llm/_torch/speculative/mtp.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -407,20 +407,20 @@ def forward(
407407
- KV cache: (ABCD) + EFGH (H's KV cache is invalid)
408408
- hidden states: H_E, H_F, H_G, H_H (H_H is invalid)
409409
Draft model:
410-
MPT1:
410+
MTP1:
411411
# For generation request, `mtp_num_modules` of tokens will be used as input.
412412
- input tokens: FGX
413413
- input hidden states: H_E, H_F, H_G
414414
- KV cache: (BCDE) + FGX
415415
- output hidden states: h_F, h_G, h_X
416416
- output next draft token: N
417-
MPT2:
417+
MTP2:
418418
- input tokens: GXN
419419
- input hidden states: H_F, H_G, h_X
420420
- KV cache: (CDEF) + GXN
421421
- output hidden states: h_G, h_X, h_N
422422
- output next draft token: O
423-
MPT3:
423+
MTP3:
424424
- input tokens: XNO
425425
- input hidden states: H_G, H_X, h_N
426426
- KV cache: (DEFG) + XNO

0 commit comments

Comments
 (0)