Is finetune learning possible using the 'bnb-4bit' models?

[poilly54]

I am finetuning the qwen3-vl model using unsloth.
General base models such as Qwen3-VL-4B-Instruct can be finetuned well.
However, lightweight models such as Qwen3-VL-8B-Instruct-bnb-4bit or *-unsloth-bnb-4bit cannot be finetune.

Am I making a mistake? Or is finetune learning not supported for lightweight models?

[poilly54]

@Parveshiiii Thank you very much for your kind reply.
I will keep the discussion open in the hope that a successful case may emerge.

[mmathew23]

You can absolutely train the bnb-4bit models with a training method called QLoRA. @poilly54 check out https://docs.unsloth.ai/get-started/fine-tuning-llms-guide

[mmathew23]

I think @Parveshiiii is referring to GGUF style quantized models. Those are made for inference with llama.cpp

[OE-Void]

You can absolutely train the bnb-4bit models with a training method called QLoRA. @poilly54 check out https://docs.unsloth.ai/get-started/fine-tuning-llms-guide

actually i was saying the same i also told about QLoRA the only thing is not all quantized models can't be used for Qlora that's why unsloth quantize models and uploads them to their hf org

[mmathew23]

Yes but you said, "You're not making a mistake — this is a known limitation with 4-bit quantized models like Qwen3-VL-8B-Instruct-bnb-4bit." This statement is not true.

I want to make clear that you can absolutely finetune with bnb-4bit models. The unsloth models that you cannot finetune are the GGUF models.

[poilly54]

@mmathew23
I mark your answer as answered. Thank you so much. I will try it!