Skip to content

Commit e6d8ad8

Browse files
toncaocpatonnbrian-dellabetta
authored
Resolve mappings for Cohere2ForCausalLM, Glm4MoeForCausalLM, SeedOssForCausalLM, and Ernie4_5_MoeForCausalLM (#1926)
This repo is to resolve mappings for Cohere2ForCausalLM, Glm4MoeForCausalLM, SeedOssForCausalLM, and Ernie4_5_MoeForCausalLM models. TEST PLAN: Local make test results: ``` ======================================================================== short test summary info ========================================================================= FAILED tests/llmcompressor/modeling/test_calib_deepseek_v3.py::test_calib_deepseekv3_module - torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 56.00 MiB. GPU 0 has a total capacity of 23.57 GiB of which 14.12 MiB is free. Including non-PyTorch mem... FAILED tests/llmcompressor/utils/test_helpers.py::test_disable_cache[MllamaForConditionalGeneration-meta-llama/Llama-3.2-11B-Vision-Instruct] - torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 64.00 MiB. GPU 0 has a total capacity of 23.57 GiB of which 54.12 MiB is free. Including non-PyTorch mem... FAILED tests/lmeval/test_lmeval.py::TestLMEval::test_lm_eval[None] - TypeError: argument should be a str or an os.PathLike object where __fspath__ returns a str, not 'NoneType' =========================================================== 3 failed, 242 passed, 4 skipped in 80.83s (0:01:20) =========================================================== ``` Co-authored-by: toncao <[email protected]> Co-authored-by: Brian Dellabetta <[email protected]>
1 parent 7194d4b commit e6d8ad8

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

src/llmcompressor/modifiers/awq/mappings.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,11 +147,13 @@ class AWQMapping:
147147
"BloomForCausalLM": _bloom_mappings,
148148
"CohereForCausalLM": _cohere_mappings,
149149
"Cohere2ForCausalLM": _cohere_mappings,
150+
"Cohere2VisionForConditionalGeneration": _cohere_mappings,
150151
"DeepseekV3ForCausalLM": _deepseek_mappings,
151152
"Gemma2ForCausalLM": _gemma_mappings,
152153
"Gemma3ForCausalLM": _gemma_mappings,
153154
"Gemma3ForConditionalGeneration": _gemma_mappings,
154155
"LlamaForCausalLM": _default_mappings,
156+
"Llama4ForConditionalGeneration": _default_mappings,
155157
"Mistral3ForConditionalGeneration": _default_mappings,
156158
"MistralForCausalLM": _default_mappings,
157159
"Phi3ForCausalLM": _phi_mappings,
@@ -161,6 +163,9 @@ class AWQMapping:
161163
"Qwen2MoeForCausalLM": _moe_default_mappings,
162164
"Qwen3ForCausalLM": _default_mappings,
163165
"Qwen3MoeForCausalLM": _moe_default_mappings,
166+
"Glm4MoeForCausalLM": _default_mappings,
167+
"SeedOssForCausalLM": _default_mappings,
168+
"Ernie4_5_MoeForCausalLM": _default_mappings,
164169
}
165170

166171

0 commit comments

Comments
 (0)