Skip to content

Commit 97cd38e

Browse files
authored
Skip weight loading in deepgemm compilation (#11312)
1 parent 3c06b67 commit 97cd38e

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

python/sglang/compile_deep_gemm.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,9 @@ def refine_server_args(server_args: ServerArgs, compile_args: CompileArgs):
141141
server_args.enable_torch_compile = False
142142
print(f"Disable CUDA Graph and Torch Compile to save time...")
143143

144+
server_args.load_format = "dummy"
145+
print(f"Set load format to dummy to save time...")
146+
144147
# Set watchdog timeout to compile_args.timeout because compilation will take a long time
145148
server_args.watchdog_timeout = compile_args.timeout
146149
server_args.warmups = "compile-deep-gemm"

0 commit comments

Comments
 (0)