lora微调后的Qwen2.5-VL-7B模型量化

[gameboy777x]

背景：
用llama factory对qwen2.5-vl-7B进行了lora微调，并将adapter合并导出。打算量化qwen2.5-vl-7B-finetuning
（单卡5090）

问题：
swift指令如下：
CUDA_VISIBLE_DEVICES=0 swift export
--model ~/LLaMA-Factory/merged_model/qwen25vl_7b_finetuning/
--quant_bits 4
--quant_method awq
--output_dir ~/ms-swift/awq_model/
--dataset ~/ms-swift/dataset
--model_type qwen2_5_vl

报错：
[INFO:swift] Successfully registered ~/ms-swift1/.venv/lib/python3.10/site-packages/swift/llm/dataset/data/dataset_info.json.
[INFO:swift] rank: -1, local_rank: -1, world_size: 1, local_world_size: 1
[INFO:swift] Loading the model using model_dir: /llama-factory/LLaMA-Factory/merged_model/qwen25vl_7b_lab_16/
[WARNING:swift] Please install the package: pip install "qwen_vl_utils>=0.0.6" "decord" -U.
[INFO:swift] Setting args.lazy_tokenize: True
[INFO:swift] args.output_dir: ~/ms-swift/awq_model
[INFO:swift] Global seed set to 42
[INFO:swift] args: ExportArguments(model='/llama-factory/LLaMA-Factory/merged_model/qwen25vl_7b_lab_16/', model_type='qwen2_5_vl', model_revision=None, task_type='causal_lm', torch_dtype=torch.float16, attn_impl=None, new_special_tokens=[], num_labels=None, problem_type=None, rope_scaling=None, device_map=None, max_memory={}, max_model_len=None, local_repo_path=None, init_strategy=None, template='qwen2_5_vl', system=None, max_length=2048, truncation_strategy='delete', max_pixels=None, agent_template=None, norm_bbox=None, use_chat_template=True, padding_free=False, padding_side='right', loss_scale='default', sequence_parallel_size=1, response_prefix=None, template_backend='swift', dataset=['/ms-swift/dataset'], val_dataset=[], split_dataset_ratio=0.0, data_seed=42, dataset_num_proc=1, load_from_cache_file=True, dataset_shuffle=True, val_dataset_shuffle=False, streaming=False, interleave_prob=None, stopping_strategy='first_exhausted', shuffle_buffer_size=1000, download_mode='reuse_dataset_if_exists', columns={}, strict=False, remove_unused_columns=True, model_name=None, model_author=None, custom_dataset_info=[], quant_method='awq', quant_bits=4, hqq_axis=None, bnb_4bit_compute_dtype=torch.float32, bnb_4bit_quant_type='nf4', bnb_4bit_use_double_quant=True, bnb_4bit_quant_storage=None, max_new_tokens=None, temperature=None, top_k=None, top_p=None, repetition_penalty=None, num_beams=1, stream=False, stop_words=[], logprobs=False, top_logprobs=None, ckpt_dir=None, lora_modules=[], tuner_backend='peft', train_type='lora', adapters=[], external_plugins=[], seed=42, model_kwargs={}, load_args=True, load_data_args=False, packing=False, packing_length=None, lazy_tokenize=True, cached_dataset=[], custom_register_path=[], use_hf=False, hub_token=None, ddp_timeout=18000000, ddp_backend=None, ignore_args_error=False, use_swift_lora=False, merge_lora=False, safe_serialization=True, max_shard_size='5GB', output_dir='/ms-swift/awq_model', quant_n_samples=256, quant_batch_size=1, group_size=128, to_cached_dataset=False, to_ollama=False, to_mcore=False, to_hf=False, mcore_model=None, mcore_adapters=[], thread_count=None, test_convert_precision=False, test_convert_dtype=torch.float32, push_to_hub=False, hub_model_id=None, hub_private_repo=False, commit_message='update files', to_peft_format=False, exist_ok=False)
[INFO:swift] Start time of running main: 2025-09-18 20:24:31.082479
[INFO:swift] swift.version: 3.8.1
~/ms-swift1/.venv/lib/python3.10/site-packages/awq/init.py:21: DeprecationWarning:
I have left this message as the final dev message to help you transition.

Important Notice:

• AutoAWQ is officially deprecated and will no longer be maintained.
• The last tested configuration used Torch 2.6.0 and Transformers 4.51.3.
• If future versions of Transformers break AutoAWQ compatibility, please report the issue to the Transformers project.

Alternative:

• AutoAWQ has been adopted by the vLLM Project: https://github.com/vllm-project/llm-compressor

For further inquiries, feel free to reach out:

• X: https://x.com/casper_hansen_

• LinkedIn: https://www.linkedin.com/in/casper-hansen-804005170/

  warnings.warn(_FINAL_DEV_MESSAGE, category=DeprecationWarning, stacklevel=1)
  [INFO:swift] Loading the model using model_dir: /llama-factory/LLaMA-Factory/merged_model/qwen25vl_7b_lab_16/
  [WARNING:swift] Please install the package: pip install "qwen_vl_utils>=0.0.6" "decord" -U.
  Using use_fast=True but torchvision is not available. Falling back to the slow image processor.
  [INFO:swift] model_kwargs: {'device_map': 'cuda:0'}
  Traceback (most recent call last):
  File "/ms-swift1/.venv/lib/python3.10/site-packages/swift/cli/export.py", line 5, in
  export_main()
  File "/ms-swift1/.venv/lib/python3.10/site-packages/swift/llm/export/export.py", line 53, in export_main
  return SwiftExport(args).main()
  File "/ms-swift1/.venv/lib/python3.10/site-packages/swift/llm/base.py", line 49, in main
  result = self.run()
  File "/ms-swift1/.venv/lib/python3.10/site-packages/swift/llm/export/export.py", line 30, in run
  quantize_model(args)
  File "/ms-swift1/.venv/lib/python3.10/site-packages/swift/llm/export/quant.py", line 255, in quantize_model
  QuantEngine(args).quantize()
  File "/ms-swift1/.venv/lib/python3.10/site-packages/swift/llm/export/quant.py", line 25, in init
  self.model, self.template = prepare_model_template(args, **kwargs)
  File "/ms-swift1/.venv/lib/python3.10/site-packages/swift/llm/infer/utils.py", line 145, in prepare_model_template
  model, processor = args.get_model_processor(**kwargs)
  File "/ms-swift1/.venv/lib/python3.10/site-packages/swift/llm/argument/base_args/base_args.py", line 317, in get_model_processor
  return get_model_tokenizer(**kwargs)
  File "/ms-swift1/.venv/lib/python3.10/site-packages/swift/llm/model/register.py", line 660, in get_model_tokenizer
  model, processor = get_function(model_dir, model_info, model_kwargs, load_model, **kwargs)
  File "/ms-swift1/.venv/lib/python3.10/site-packages/swift/llm/model/model/qwen.py", line 763, in get_model_tokenizer_qwen2_5_vl
  return get_model_tokenizer_qwen2_vl(*args, **kwargs)
  File "/ms-swift1/.venv/lib/python3.10/site-packages/swift/llm/model/model/qwen.py", line 689, in get_model_tokenizer_qwen2_vl
  model, tokenizer = get_model_tokenizer_multimodal(*args, **kwargs)
  File "/ms-swift1/.venv/lib/python3.10/site-packages/swift/llm/model/register.py", line 346, in get_model_tokenizer_multimodal
  model, _ = get_model_tokenizer_with_flash_attn(model_dir, *args, **kwargs)
  File "/ms-swift1/.venv/lib/python3.10/site-packages/swift/llm/model/register.py", line 339, in get_model_tokenizer_with_flash_attn
  return get_model_tokenizer_from_local(model_dir, model_info, model_kwargs, load_model, **kwargs)
  File "/ms-swift1/.venv/lib/python3.10/site-packages/swift/llm/model/register.py", line 293, in get_model_tokenizer_from_local
  model = automodel_class.from_pretrained(
  File "/ms-swift1/.venv/lib/python3.10/site-packages/awq/models/auto.py", line 83, in from_pretrained
  return AWQ_CAUSAL_LM_MODEL_MAP[model_type].from_pretrained(
  File "/ms-swift1/.venv/lib/python3.10/site-packages/awq/models/base.py", line 389, in from_pretrained
  model = target_cls.from_pretrained(
  File "/ms-swift1/.venv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 571, in from_pretrained
  return model_class.from_pretrained(
  File "/ms-swift1/.venv/lib/python3.10/site-packages/swift/llm/model/patcher.py", line 302, in _new_from_pretrained
  model = from_pretrained(cls, *args, **kwargs)
  File "/ms-swift1/.venv/lib/python3.10/site-packages/transformers/modeling_utils.py", line 279, in _wrapper
  return func(*args, **kwargs)
  File "/ms-swift1/.venv/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4342, in from_pretrained
  model = cls(config, *model_args, **model_kwargs)
  File "/ms-swift1/.venv/lib/python3.10/site-packages/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py", line 1481, in init
  super().init(config)
  File "/ms-swift1/.venv/lib/python3.10/site-packages/transformers/modeling_utils.py", line 1884, in init
  self.generation_config = GenerationConfig.from_model_config(config) if self.can_generate() else None
  File "/ms-swift1/.venv/lib/python3.10/site-packages/transformers/generation/configuration_utils.py", line 1290, in from_model_config
  decoder_config_dict = decoder_config.to_dict()
  AttributeError: 'dict' object has no attribute 'to_dict'

[gameboy777x]

环境依赖包如下：
absl-py==2.3.1
accelerate==1.10.1
addict==2.4.0
aiofiles==24.1.0
aiohappyeyeballs==2.6.1
aiohttp==3.12.15
aiosignal==1.4.0
aliyun-python-sdk-core==2.16.0
aliyun-python-sdk-kms==2.16.5
annotated-types==0.7.0
anyio==4.10.0
async-timeout==5.0.1
attrdict==2.0.1
attrs==25.3.0
autoawq==0.2.9
binpacking==1.5.2
Brotli==1.1.0
certifi==2025.8.3
cffi==2.0.0
charset-normalizer==3.4.3
click==8.2.1
contourpy==1.3.2
cpm-kernels==1.0.11
crcmod==1.7
cryptography==46.0.1
cycler==0.12.1
dacite==1.9.2
datasets==3.6.0
dill==0.3.8
distro==1.9.0
einops==0.8.1
exceptiongroup==1.3.0
fastapi==0.116.2
ffmpy==0.6.1
filelock==3.19.1
fonttools==4.60.0
frozenlist==1.7.0
fsspec==2025.3.0
future==1.0.0
gradio==5.46.0
gradio_client==1.13.0
groovy==0.1.2
grpcio==1.75.0
h11==0.16.0
hf-xet==1.1.10
httpcore==1.0.9
httpx==0.28.1
huggingface-hub==0.35.0
idna==3.10
importlib_metadata==8.7.0
jieba==0.42.1
Jinja2==3.1.6
jiter==0.11.0
jmespath==0.10.0
joblib==1.5.2
kiwisolver==1.4.9
Markdown==3.9
markdown-it-py==4.0.0
MarkupSafe==3.0.2
matplotlib==3.10.6
mdurl==0.1.2
modelscope==1.30.0
mpmath==1.3.0
ms_swift==3.8.1
multidict==6.6.4
multiprocess==0.70.16
networkx==3.4.2
nltk==3.9.1
numpy==2.2.6
nvidia-cublas-cu12==12.8.4.1
nvidia-cuda-cupti-cu12==12.8.90
nvidia-cuda-nvrtc-cu12==12.8.93
nvidia-cuda-runtime-cu12==12.8.90
nvidia-cudnn-cu12==9.10.2.21
nvidia-cufft-cu12==11.3.3.83
nvidia-cufile-cu12==1.13.1.3
nvidia-curand-cu12==10.3.9.90
nvidia-cusolver-cu12==11.7.3.90
nvidia-cusparse-cu12==12.5.8.93
nvidia-cusparselt-cu12==0.7.1
nvidia-nccl-cu12==2.27.3
nvidia-nvjitlink-cu12==12.8.93
nvidia-nvtx-cu12==12.8.90
openai==1.108.0
orjson==3.11.3
oss2==2.19.1
packaging==25.0
pandas==2.3.2
peft==0.17.1
pillow==11.3.0
propcache==0.3.2
protobuf==6.32.1
psutil==7.1.0
pyarrow==21.0.0
pycparser==2.23
pycryptodome==3.23.0
pydantic==2.11.9
pydantic_core==2.33.2
pydub==0.25.1
Pygments==2.19.2
pyparsing==3.2.4
python-dateutil==2.9.0.post0
python-multipart==0.0.20
pytz==2025.2
PyYAML==6.0.2
regex==2025.9.1
requests==2.32.5
rich==14.1.0
rouge==1.0.1
ruff==0.13.0
safehttpx==0.1.6
safetensors==0.6.2
scipy==1.15.3
semantic-version==2.10.0
sentencepiece==0.2.1
shellingham==1.5.4
simplejson==3.20.1
six==1.17.0
sniffio==1.3.1
sortedcontainers==2.4.0
starlette==0.48.0
sympy==1.14.0
tensorboard==2.20.0
tensorboard-data-server==0.7.2
tiktoken==0.11.0
tokenizers==0.21.4
tomlkit==0.13.3
torch==2.8.0
tqdm==4.67.1
transformers==4.51.3
transformers-stream-generator==0.0.5
triton==3.4.0
trl==0.15.0
typer==0.17.4
typing-inspection==0.4.1
typing_extensions==4.15.0
tzdata==2025.2
urllib3==2.5.0
uvicorn==0.35.0
websockets==15.0.1
Werkzeug==3.1.3
xxhash==3.5.0
yarl==1.20.1
zipp==3.23.0
zstandard==0.25.0

[gameboy777x]

数据集：
alpaca_gpt4_data_zh.json
https://huggingface.co/datasets/llamafactory/alpaca_gpt4_zh/tree/main

[Jintao-Huang]

没有训练过的qwen2.5-vl 能量化嘛