本地部署

[HAOYON-666]

请问8卡4090，单卡24G显存能在本地部署吗？我好像没有找到本地部署的demo

[shua-chen]

绰绰有余，本地部署得先把模型文件下载下来

[HAOYON-666]

嗯，已经用vllm部署成功了，测了一下agent能力，好像也没有特别惊艳，可能比同尺寸的稍微好些 Shua ***@***.***> 在 2025年9月26日 周五 15:03 写道： shua-chen left a comment (Alibaba-NLP/DeepResearch#168) 绰绰有余，本地部署得先把模型文件下载下来 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

[PeterXiaTian]

嗯，已经用vllm部署成功了，测了一下agent能力，好像也没有特别惊艳，可能比同尺寸的稍微好些

                Shua ***@***.***> 在 2025年9月26日 周五 15:03 写道：

shua-chen left a comment (Alibaba-NLP/DeepResearch#168)

绰绰有余，本地部署得先把模型文件下载下来

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

请教下,我本地vllm部署失败,原因如下:
ValueError: The checkpoint you are trying to load has model type qwen3_moe but Transformers does not recognize this architecture

部署命令:
CUDA_VISIBLE_DEVICES=7 nohup python -m vllm.entrypoints.openai.api_server
--model &{modelpath}
--served-model-name alibaba/tongyi-deepresearch-30b-a3b
--gpu-memory-utilization 0.8
--max-model-len 8192
--trust-remote-code
--port 10086 >tongyi_ds_agent.log 2>&1 &

[HAOYON-666]

你是什么机器?  我用24G的4090至少要4张卡，vllm版本是0.10.1 ***@***.***> 在 2025年9月30日 周二 14:20 写道： PeterXiaTian left a comment (Alibaba-NLP/DeepResearch#168) 嗯，已经用vllm部署成功了，测了一下agent能力，好像也没有特别惊艳，可能比同尺寸的稍微好些 Shua ***@***.***> 在 2025年9月26日 周五 15:03 写道： shua-chen left a comment (Alibaba-NLP/DeepResearch#168) 绰绰有余，本地部署得先把模型文件下载下来 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread. 请教下,我本地vllm部署失败,原因如下: ValueError: The checkpoint you are trying to load has model type qwen3_moe but Transformers does not recognize this architecture 部署命令: CUDA_VISIBLE_DEVICES=7 nohup python -m vllm.entrypoints.openai.api_server --model &{modelpath} --served-model-name alibaba/tongyi-deepresearch-30b-a3b --gpu-memory-utilization 0.8 --max-model-len 8192 --trust-remote-code --port 10086 >tongyi_ds_agent.log 2>&1 & — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.