Whether I fine-tune the Qwen2.5VL multimodal model or Gemma3 when i convert it to GGUF with mmproj files, the LoRA fine-tuning effects are lost.

[YuBin8]

this is the command i use : python.exe convert_hf_to_gguf2.py D:\work\AI\Model\TestModel --mmproj --outfile D:\work\AI\Model\GGUF\TestModel.f16.gguf --outtype f16

There is no problem with the text part, the problem lies in the image recognition part (mmproj file)