Support custom MAXTEXT_DIR and --quantization option #1619
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR updates
test-maxtext.shwith two features.MAXTEXT_DIRin their environment, this dir will be used instead of/opt/maxtext. This makes it easier for development. Default behavior of using /opt/maxtext is unchanged unless that environment variable is specified.quantizationconfig via a new--quantizationarg.Before this PR, this script uses
--dtype=fp8to specify MaxText FP8 DelayedScaling. However, when developing new quantization recipes, support for different types of quantization beyond "fp8" is useful. For backwards compatibility, I've kept the behavior of--dtype=fp8unchanged. But you can now specify--quantization="..."with an arbitrary string and it will be passed to MaxText'squantizationconfig unfiltered. An error is raised when both--dtype=fp8and--quantization="..."are specified at the same time.