File tree Expand file tree Collapse file tree 1 file changed +3
-3
lines changed
tensorrt_llm/_torch/speculative Expand file tree Collapse file tree 1 file changed +3
-3
lines changed Original file line number Diff line number Diff line change @@ -407,20 +407,20 @@ def forward(
407407 - KV cache: (ABCD) + EFGH (H's KV cache is invalid)
408408 - hidden states: H_E, H_F, H_G, H_H (H_H is invalid)
409409 Draft model:
410- MPT1 :
410+ MTP1 :
411411 # For generation request, `mtp_num_modules` of tokens will be used as input.
412412 - input tokens: FGX
413413 - input hidden states: H_E, H_F, H_G
414414 - KV cache: (BCDE) + FGX
415415 - output hidden states: h_F, h_G, h_X
416416 - output next draft token: N
417- MPT2 :
417+ MTP2 :
418418 - input tokens: GXN
419419 - input hidden states: H_F, H_G, h_X
420420 - KV cache: (CDEF) + GXN
421421 - output hidden states: h_G, h_X, h_N
422422 - output next draft token: O
423- MPT3 :
423+ MTP3 :
424424 - input tokens: XNO
425425 - input hidden states: H_G, H_X, h_N
426426 - KV cache: (DEFG) + XNO
You can’t perform that action at this time.
0 commit comments