Why extract multiple zero_point / scale values from the same quantization group?

[frankxyy]

Hi, I read the code of gptq reconstruct and find this line:

It seems that 4 values of zero_point are read out from the matrix for the same group. From the theory of gptq quantization, isn't it that there is only one particular zero_point and scale value for one specific group?

[turboderp]

It does four columns in each thread, and each column has its own scale and offset. It's faster this way, loading four offsets as one uint16, four scales as two half2s and 4 x 8 weights an int4.

[frankxyy]

Got it! Thanks a lot