AVX support?

[a-51-50]

I have no clue if older Intel Xeon E5-2680 v2 is causing these errors or its something else, setup is:
Dell Precision T7610:
2x Intel Xeon E5-2680 v2
512GB DDR3 ECC
1x1050ti - for display
3x3060 (12gb) - for llm

vllm_server_pp2.log

[devnen]

AVX is not the problem. Your Xeon E5-2680 v2 has AVX1, that is enough. The CUDA work runs on the 3060s.

The real error is at the bottom of your log:

multiproc_executor.py, line 736, in wait_for_ready
    assert isinstance(pipe, Connection)
AssertionError

That is a Windows multi-GPU bug. v1.3.4 just shipped today and fixes it (new Ampere wheel +devnen.3). Run update.bat and try again.

One caveat: the shipped 2-GPU snapshot is PP=2, not PP=3. v1.3.4 was verified on 2x 3090. Three 3060s at PP=3 is untested. If it still crashes after the update, post the new log.

https://github.com/devnen/qwen3.6-windows-server/releases/tag/v1.3.4

[a-51-50]

Will do, thanks.

[a-51-50]

(APIServer pid=9232) INFO 05-11 00:01:06 [utils.py:299]

(APIServer pid=9232) INFO 05-11 00:01:06 [utils.py:299] █ █ █▄ ▄█

(APIServer pid=9232) INFO 05-11 00:01:06 [utils.py:299] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 0.19.0

(APIServer pid=9232) INFO 05-11 00:01:06 [utils.py:299] █▄█▀ █ █ █ █ model D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\models\Qwen3.6-27B-int4-AutoRound

(APIServer pid=9232) INFO 05-11 00:01:06 [utils.py:299] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀

(APIServer pid=9232) INFO 05-11 00:01:06 [utils.py:299]
(APIServer pid=9232) INFO 05-11 00:01:06 [utils.py:233] non-default args: {'model_tag': 'D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\models\Qwen3.6-27B-int4-AutoRound', 'chat_template': 'D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\templates\qwen3.5-enhanced.jinja', 'default_chat_template_kwargs': {'preserve_thinking': False}, 'enable_auto_tool_choice': True, 'tool_call_parser': 'qwen3_coder', 'host': '0.0.0.0', 'port': 8084, 'model': 'D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\models\Qwen3.6-27B-int4-AutoRound', 'trust_remote_code': True, 'max_model_len': 100000, 'quantization': 'auto-round', 'served_model_name': ['qwen3.6-27b-autoround'], 'use_tqdm_on_load': False, 'attention_backend': 'TRITON_ATTN', 'reasoning_parser': 'qwen3', 'distributed_executor_backend': 'mp', 'pipeline_parallel_size': 3, 'block_size': 32, 'gpu_memory_utilization': 0.72, 'kv_cache_dtype': 'fp8_e4m3', 'enable_prefix_caching': True, 'limit_mm_per_prompt': {'image': 0, 'video': 0}, 'max_num_batched_tokens': 4128, 'max_num_seqs': 1, 'enable_chunked_prefill': True, 'scheduler_reserve_full_isl': False}
(APIServer pid=9232) INFO 05-11 00:01:06 [system_utils.py:279] Windows detected, skipping ulimit adjustment.
(APIServer pid=9232) WARNING 05-11 00:01:06 [envs.py:1786] Unknown vLLM environment variable detected: VLLM_ATTENTION_BACKEND
(APIServer pid=9232) WARNING 05-11 00:01:06 [envs.py:1786] Unknown vLLM environment variable detected: VLLM_SLEEP_WHEN_IDLE
(APIServer pid=9232) INFO 05-11 00:01:06 [model.py:554] Resolved architecture: Qwen3_5ForConditionalGeneration
(APIServer pid=9232) WARNING 05-11 00:01:06 [model.py:1970] Your device 'NVIDIA GeForce GTX 1050 Ti' (with compute capability 6.1) doesn't support torch.bfloat16. Falling back to torch.float16 for compatibility.
(APIServer pid=9232) WARNING 05-11 00:01:06 [model.py:2023] Casting torch.bfloat16 to torch.float16.
(APIServer pid=9232) INFO 05-11 00:01:06 [model.py:1685] Using max model len 100000
(APIServer pid=9232) INFO 05-11 00:01:07 [cache.py:253] Using fp8_e4m3 data type to store kv cache. It reduces the GPU memory footprint and boosts the performance. Meanwhile, it may cause accuracy drop without a proper scaling factor
(APIServer pid=9232) INFO 05-11 00:01:07 [scheduler.py:238] Chunked prefill is enabled with max_num_batched_tokens=4128.
(APIServer pid=9232) WARNING 05-11 00:01:07 [config.py:306] Mamba cache mode is set to 'align' for Qwen3_5ForConditionalGeneration by default when prefix caching is enabled
(APIServer pid=9232) INFO 05-11 00:01:07 [config.py:326] Warning: Prefix caching in Mamba cache 'align' mode is currently enabled. Its support for Mamba layers is experimental. Please report any issues you may observe.
(APIServer pid=9232) INFO 05-11 00:01:07 [vllm.py:799] Asynchronous scheduling is enabled.
(APIServer pid=9232) INFO 05-11 00:01:07 [kernel.py:199] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'])
(APIServer pid=9232) INFO 05-11 00:01:09 [registry.py:126] All limits of multimodal modalities supported by the model are set to 0, running in text-only mode.
(EngineCore pid=19300) INFO 05-11 00:01:26 [core.py:105] Initializing a V1 LLM engine (v0.19.0) with config: model='D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\models\Qwen3.6-27B-int4-AutoRound', speculative_config=None, tokenizer='D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\models\Qwen3.6-27B-int4-AutoRound', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=100000, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=3, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=inc, quantization_config=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=fp8_e4m3, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='qwen3', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=0, served_model_name=qwen3.6-27b-autoround, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::kda_attention', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_images_per_batch': 0, 'compile_sizes': [], 'compile_ranges_endpoints': [4128], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 2, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native']), enable_flashinfer_autotune=True, moe_backend='auto')
(EngineCore pid=19300) INFO 05-11 00:01:26 [multiproc_executor.py:137] DP group leader: node_rank=0, node_rank_within_dp=0, master_addr=127.0.0.1, mq_connect_ip=192.168.1.11 (local), world_size=3, local_world_size=3
INFO 05-11 00:01:45 [registry.py:126] All limits of multimodal modalities supported by the model are set to 0, running in text-only mode.
(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] WorkerProc failed to start.

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] Traceback (most recent call last):

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\multiproc_executor.py", line 840, in worker_main

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] worker = WorkerProc(*args, **kwargs)

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] ^^^^^^^^^^^^^^^^^^^^^^^^^^^

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\tracing\otel.py", line 178, in sync_wrapper

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] return func(*args, **kwargs)

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] ^^^^^^^^^^^^^^^^^^^^^

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\multiproc_executor.py", line 619, in init

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] self.worker.init_device()

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\worker\worker_base.py", line 312, in init_device

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] self.worker.init_device() # type: ignore

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] ^^^^^^^^^^^^^^^^^^^^^^^^^

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\tracing\otel.py", line 178, in sync_wrapper

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] return func(*args, **kwargs)

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] ^^^^^^^^^^^^^^^^^^^^^

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\worker\gpu_worker.py", line 249, in init_device

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] assert self.parallel_config.local_world_size <= visible_device_count, (

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

(Worker pid=16428) ERROR 05-11 00:01:45 [multiproc_executor.py:871] AssertionError: local_world_size (3) must be less than or equal to the number of visible devices (2).
INFO 05-11 00:01:45 [registry.py:126] All limits of multimodal modalities supported by the model are set to 0, running in text-only mode.
(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] WorkerProc failed to start.

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] Traceback (most recent call last):

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\multiproc_executor.py", line 840, in worker_main

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] worker = WorkerProc(*args, **kwargs)

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] ^^^^^^^^^^^^^^^^^^^^^^^^^^^

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\tracing\otel.py", line 178, in sync_wrapper

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] return func(*args, **kwargs)

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] ^^^^^^^^^^^^^^^^^^^^^

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\multiproc_executor.py", line 619, in init

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] self.worker.init_device()

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\worker\worker_base.py", line 312, in init_device

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] self.worker.init_device() # type: ignore

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] ^^^^^^^^^^^^^^^^^^^^^^^^^

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\tracing\otel.py", line 178, in sync_wrapper

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] return func(*args, **kwargs)

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] ^^^^^^^^^^^^^^^^^^^^^

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\worker\gpu_worker.py", line 249, in init_device

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] assert self.parallel_config.local_world_size <= visible_device_count, (

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

(Worker pid=19320) ERROR 05-11 00:01:45 [multiproc_executor.py:871] AssertionError: local_world_size (3) must be less than or equal to the number of visible devices (2).
INFO 05-11 00:01:45 [registry.py:126] All limits of multimodal modalities supported by the model are set to 0, running in text-only mode.
(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] WorkerProc failed to start.

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] Traceback (most recent call last):

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\multiproc_executor.py", line 840, in worker_main

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] worker = WorkerProc(*args, **kwargs)

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] ^^^^^^^^^^^^^^^^^^^^^^^^^^^

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\tracing\otel.py", line 178, in sync_wrapper

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] return func(*args, **kwargs)

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] ^^^^^^^^^^^^^^^^^^^^^

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\multiproc_executor.py", line 619, in init

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] self.worker.init_device()

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\worker\worker_base.py", line 312, in init_device

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] self.worker.init_device() # type: ignore

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] ^^^^^^^^^^^^^^^^^^^^^^^^^

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\tracing\otel.py", line 178, in sync_wrapper

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] return func(*args, **kwargs)

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] ^^^^^^^^^^^^^^^^^^^^^

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\worker\gpu_worker.py", line 243, in init_device

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] assert self.local_rank < torch.accelerator.device_count(), (

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

(Worker pid=15528) ERROR 05-11 00:01:45 [multiproc_executor.py:871] AssertionError: DP adjusted local rank 2 is out of bounds.
(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] EngineCore failed to start.

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] Traceback (most recent call last):

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\engine\core.py", line 1082, in run_engine_core

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\tracing\otel.py", line 178, in sync_wrapper

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] return func(*args, **kwargs)

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\engine\core.py", line 848, in init

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] super().init(

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\engine\core.py", line 114, in init

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] self.model_executor = executor_class(vllm_config)

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\multiproc_executor.py", line 105, in init

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] super().init(vllm_config)

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\tracing\otel.py", line 178, in sync_wrapper

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] return func(*args, **kwargs)

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\abstract.py", line 109, in init

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] self._init_executor()

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\multiproc_executor.py", line 208, in _init_executor

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] self.workers = WorkerProc.wait_for_ready(unready_workers)

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\multiproc_executor.py", line 736, in wait_for_ready

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] assert isinstance(pipe, Connection)

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

(EngineCore pid=19300) ERROR 05-11 00:01:47 [core.py:1108] AssertionError
(EngineCore pid=19300) Process EngineCore:
(EngineCore pid=19300) Traceback (most recent call last):
(EngineCore pid=19300) File "multiprocessing\process.py", line 314, in _bootstrap
(EngineCore pid=19300) File "multiprocessing\process.py", line 108, in run
(EngineCore pid=19300) File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\engine\core.py", line 1112, in run_engine_core
(EngineCore pid=19300) raise e
(EngineCore pid=19300) File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\engine\core.py", line 1082, in run_engine_core
(EngineCore pid=19300) engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=19300) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=19300) File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\tracing\otel.py", line 178, in sync_wrapper
(EngineCore pid=19300) return func(*args, **kwargs)
(EngineCore pid=19300) ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=19300) File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\engine\core.py", line 848, in init
(EngineCore pid=19300) super().init(
(EngineCore pid=19300) File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\engine\core.py", line 114, in init
(EngineCore pid=19300) self.model_executor = executor_class(vllm_config)
(EngineCore pid=19300) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=19300) File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\multiproc_executor.py", line 105, in init
(EngineCore pid=19300) super().init(vllm_config)
(EngineCore pid=19300) File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\tracing\otel.py", line 178, in sync_wrapper
(EngineCore pid=19300) return func(*args, **kwargs)
(EngineCore pid=19300) ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=19300) File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\abstract.py", line 109, in init
(EngineCore pid=19300) self._init_executor()
(EngineCore pid=19300) File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\multiproc_executor.py", line 208, in _init_executor
(EngineCore pid=19300) self.workers = WorkerProc.wait_for_ready(unready_workers)
(EngineCore pid=19300) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=19300) File "D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Lib\site-packages\vllm\v1\executor\multiproc_executor.py", line 736, in wait_for_ready
(EngineCore pid=19300) assert isinstance(pipe, Connection)
(EngineCore pid=19300) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=19300) AssertionError
forrtl: error (200): program aborting due to window-CLOSE event
Image PC Routine Line Source
KERNELBASE.dll 00007FFE4E62C213 Unknown Unknown Unknown
KERNEL32.DLL 00007FFE50897374 Unknown Unknown Unknown
ntdll.dll 00007FFE50DBCC91 Unknown Unknown Unknown

[devnen]

The new log is a different problem.

The GTX 1050 Ti is visible to CUDA and vLLM is picking it up for dtype detection (Pascal, compute 6.1). It can't run fp8 KV cache or TRITON_ATTN, so it has to be hidden from vLLM even though you only use it for display.

PyTorch only sees 2 CUDA devices, not 4. That's why local_world_size (3) must be <= visible_device_count (2) fires. One of the 3060s isn't being initialised.

can you post the output of:

  nvidia-smi -L
  python -c "import torch; print(torch.cuda.device_count())"

If torch reports fewer than 4, it's a driver / slot issue on one of the 3060s and PP=3 is impossible until that's fixed. If it reports 4, then setting CUDA_VISIBLE_DEVICES to the three 3060 indices before launching should fix this.

[a-51-50]

C:\Windows\System32>nvidia-smi -L
GPU 0: NVIDIA GeForce GTX 1050 Ti (UUID: GPU-36297fc8-7d12-54d9-c93d-6ffcec2c3eeb)
GPU 1: NVIDIA GeForce RTX 3060 (UUID: GPU-efdb67aa-1de9-6944-b578-51930327c1c4)
GPU 2: NVIDIA GeForce RTX 3060 (UUID: GPU-d2e22b1a-953b-4c53-9564-e370e65de213)
GPU 3: NVIDIA GeForce RTX 3060 (UUID: GPU-453c311a-58b0-77b1-2e82-68da352af0a2)

C:\Windows\System32> python -c "import torch; print(torch.cuda.device_count())"
4

I did set CUDA_VISIBLE_DEVICES to 3060's only from the start, yet 1050ti is still being visible for some reason.

@echo off
set CUDA_DEVICE_ORDER=PCI_BUS_ID
set CUDA_VISIBLE_DEVICES=1,2,3
set OMP_NUM_THREADS=4
set VLLM_HOST_IP=127.0.0.1
set MASTER_ADDR=127.0.0.1
set GL_LOG_LEVEL=ERROR

D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\python\Scripts\vllm.exe serve ^
D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\models\Qwen3.6-27B-int4-AutoRound ^
--served-model-name=qwen3.6-27b-autoround ^
--quantization=auto-round ^
--max-model-len=100000 ^
--max-num-seqs=1 ^
--max-num-batched-tokens=4128 ^
--block-size=32 ^
--enable-prefix-caching ^
--enable-chunked-prefill ^
--no-scheduler-reserve-full-isl ^
--enable-auto-tool-choice ^
--tool-call-parser=qwen3_coder ^
--reasoning-parser=qwen3 ^
--chat-template=D:\qwen3.6-windows-server-portable-x64\qwen3.6-windows-server\templates\qwen3.5-enhanced.jinja ^
--default-chat-template-kwargs="{"preserve_thinking":false}" ^
--kv-cache-dtype=fp8_e4m3 ^
--tensor-parallel-size=1 ^
--pipeline-parallel-size=3 ^
--gpu-memory-utilization=0.72 ^
--trust-remote-code ^
--attention-backend=TRITON_ATTN ^
--distributed-executor-backend=mp ^
--no-use-tqdm-on-load ^
--host=0.0.0.0 ^
--port=8084

pause