feat(dflash): support Qwen3.6-27B-DFlash draft (SWA layers) — 106 t/s on RTX 4090

[Quitetall]

Summary

Adds support for z-lab/Qwen3.6-27B-DFlash which uses sliding window attention layers. The 3.6 matched draft gives 57% speedup over the mismatched 3.5 draft.

Results (RTX 4090, Qwen3.6-27B Q4_K_M target)

Draft	Speed	Tok/step	Acceptance
3.5 (mismatch)	67.4 t/s	3.76	23.5%
3.6 (matched)	106.1 t/s	5.12	32.0%

Budget sweep (3.6 matched):

budget=6:   93.9 t/s, 4.27 tok/step
budget=10: 106.1 t/s, 5.12 tok/step  ← sweet spot
budget=14:  96.5 t/s, 4.74 tok/step
budget=18: 102.7 t/s, 5.82 tok/step
budget=22: 103.5 t/s, 5.82 tok/step

What changed

internal.h: Added is_swa per-layer flag and swa_window to DraftWeights.

safetensors_draft.cpp: Reads sibling config.json for layer_types and sliding_window. Sets is_swa=true for "sliding_attention" layers. Prints diagnostic:

[draft] SWA layers: 4/5 (window=2048)

qwen3_dflash_graph.cpp: Plumbs SWA mask through ggml_flash_attn_ext. Currently a no-op for short contexts (window=2048 > typical DFlash ctx_len), but the flag is ready for when longer contexts are used.

How the 3.6 draft differs from 3.5

3.5: layer_types = [full, full, full, full, full]
3.6: layer_types = [swa,  swa,  swa,  swa,  full]  sliding_window=2048

Same tensor names, same shapes, same architecture otherwise. The key difference is the weights are trained with sliding window attention patterns, giving better acceptance on the Qwen3.6 target.

Test

# Download matched 3.6 draft
huggingface-cli download z-lab/Qwen3.6-27B-DFlash model.safetensors config.json --local-dir models/draft-36

# Run (copy config.json next to model.safetensors for SWA detection)
DFLASH27B_KV_K=q4_0 DFLASH27B_KV_V=q4_0 DFLASH27B_FA_WINDOW=0 \
  python scripts/run.py \
  --prompt "Write Python quicksort" \
  --n-gen 256 --budget 10 --max-ctx 4096 \
  --target models/Qwen3.6-27B-Q4_K_M.gguf \
  --draft models/draft-36

Builds on #89 (conv_input_cache fix, already merged).

🤖 Generated with Claude Code

[Quitetall]

I noticed the llama.cpp qwen 3.6 draft implementation was complete, so I simply ported that over to lucebox with the help of claude opus. Code should be clean, reproducible.

[Quitetall]

Note: This was benchmarked on Qwen-3.6-27B Heretic, not standard weights. Tok/s may be even higher on the standard model that the draft model was trained on.

[cubic-dev-ai]

1 issue found across 3 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="dflash/src/safetensors_draft.cpp">

<violation number="1" location="dflash/src/safetensors_draft.cpp:442">
P2: Config path derivation fails for bare filenames, so sibling `config.json` is never found and SWA metadata silently falls back to non-SWA defaults.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[tomByrer]

Heretic

Thanks for testing that version; I heard it may be better at coding, so was looking at testing it myself on 3090.

[howard0su]

Please double check the code.

[howard0su]

allocate something then immediately null it. I don't understand how this helps the performance?