模型错误

[Zhoushanshen]

model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "unsloth/DeepSeek-R1-Distill-Llama-8B",
cache_dir="/root/shared-nvme/huggingface1",
max_seq_length = 2048,
load_in_4bit=False,
load_in_8bit = True, # 关键修改点
fast_inference = True,
max_lora_rank = 16,
gpu_memory_utilization = 0.5,
)
基于上述代码导入了基础模型，
然后测试问答时，
inputs = tokenizer([prompt_style.format(question, "")], return_tensors="pt").to("cuda")
outputs = base_model.generate(
input_ids=inputs.input_ids,
attention_mask=inputs.attention_mask,
max_new_tokens=1200,
use_cache=True,
)
response = tokenizer.batch_decode(outputs)
print(response[0])
发现基础模型的回答不正常：
<｜begin▁of▁sentence｜>
问题：“省略”

回答：
根据条件，so the correct me.etc.
ButWait, no, the first, I think>
Wait，but I'm not sure.
Wait, no, but that's
Wait, I'm not sure, let meeds on'tion.
Wait, let's the first.
Wait, so if the problem.
Wait, but I thinko Wait, but I'm.
Wait, so the result.
Wait, so maybe this could be or wait, let’s.
...
Wait, but the problem. 有大佬知道这是什么问题的吗，是模型导入错误还是什么。