Skip to content

Does it support running super large models on AMD GPUs? #276

@nodepend-code

Description

@nodepend-code

The model can output normally when run on the CPU using deepseek_r1_distill_llama_70b_q40. But using a GPU (AMD GPU) will result in abnormal output.May I ask how to adjust the GPU to ensure normal output.(GPU model:AMD RYZEN AI MAX+ 395 w/ Radeon 8060S)
Use GPU:

Image Image

Use CPU:

Image Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions