Skip to content

Docs snippets for AdaScale/SpinQuant call _prefill_inputs and _set_tensors_to_output_n_bit_symmmetric with non-existent keyword arguments #4112

Description

@Anai-Guo

Summary

The Python snippets that are literalinclude-d into the AdaScale and SpinQuant
documentation pass keyword arguments that do not exist on the functions they
call. Every one of these calls raises TypeError the moment it is reached, so
the copy-paste flow the docs describe cannot run.

Affected functions

Function Actual signature
GenAILab/bench/onnx/quant_recipes.py:87 _prefill_inputs(quantsim, generator, dataloader, num_iterations=None)
GenAILab/bench/torch/quant_recipes.py:63 _prefill_inputs(generator, dataloader, num_iterations=None, device=None)
GenAILab/qai_hub_lm/backends/onnx/quantsim_utils.py:104 _set_tensors_to_output_n_bit_symmmetric(quantsim_model, n=8)

Affected call sites

Snippet Call Rendered into
Docs/snippets/onnx/apply_adascale.py:72 _set_tensors_to_output_n_bit_symmmetric(quantsim, kv_bits=8) Docs/ptq_techniques/adascale.rst
Docs/snippets/onnx/apply_adascale.py:92 _prefill_inputs(..., num_batches=ADASCALE_NUM_BATCHES) Docs/ptq_techniques/adascale.rst
Docs/snippets/onnx/apply_adascale.py:107 _prefill_inputs(..., num_batches=20) Docs/ptq_techniques/adascale.rst
Docs/snippets/onnx/apply_spinquant.py:71 _set_tensors_to_output_n_bit_symmmetric(quantsim, kv_bits=8) Docs/ptq_techniques/spinquant.rst
Docs/snippets/onnx/apply_spinquant.py:92 _prefill_inputs(..., num_batches=20) Docs/ptq_techniques/spinquant.rst
Docs/snippets/torch/apply_adascale.py:61 _prefill_inputs(..., num_batches=ADASCALE_NUM_BATCHES, ...) Docs/ptq_techniques/adascale.rst

kv_bits / num_batches appear to have been copied from the local variable
names in GenAILab/bench/onnx/quant_recipes.py:314 (AdaScale.apply) and
Examples/onnx/quantize.py:199, where they are passed positionally.

Sibling call sites that are correct

  • Examples/onnx/quantize.py:199, Examples/onnx/evaluate.py:126 — positional n
  • Examples/onnx/quantize.py:222, Examples/torch/quantize.py:230 — positional num_iterations
  • GenAILab/bench/onnx/quant_recipes.py:274,314, GenAILab/bench/torch/quant_recipes.py:243,270 — positional
  • TrainingExtensions/onnx/test/python/test_adascale.py:1114, test_model_converter.py:110,252 — explicit num_iterations=

So the docs snippets are the only six call sites in the repo that get this wrong.

Reproduction

inspect.Signature.bind replay against the signatures parsed out of the three
defining modules (no AIMET runtime needed):

BROKEN Docs/snippets/onnx/apply_adascale.py:72   -> TypeError: got an unexpected keyword argument 'kv_bits'
BROKEN Docs/snippets/onnx/apply_spinquant.py:71  -> TypeError: got an unexpected keyword argument 'kv_bits'
BROKEN Docs/snippets/onnx/apply_adascale.py:92   -> TypeError: got an unexpected keyword argument 'num_batches'
BROKEN Docs/snippets/onnx/apply_adascale.py:107  -> TypeError: got an unexpected keyword argument 'num_batches'
BROKEN Docs/snippets/onnx/apply_spinquant.py:92  -> TypeError: got an unexpected keyword argument 'num_batches'
BROKEN Docs/snippets/torch/apply_adascale.py:61  -> TypeError: got an unexpected keyword argument 'num_batches'
OK     Examples/onnx/quantize.py:199 (sibling)    -> binds
OK     Examples/onnx/evaluate.py:126 (sibling)    -> binds
OK     Examples/onnx/quantize.py:222 (sibling)    -> binds
OK     test_adascale.py:1114 (sibling)            -> binds

Suggested fix

Pass the value positionally in all six places, matching the repo's own callers.
This keeps the num_batches wording that Docs/ptq_techniques/adascale.rst
uses in its prose and its recipe table, so no .rst change is needed.

I have a patch ready and will open a PR against develop.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions