Skip to content

Conversation

@jberchtold-nvidia
Copy link
Contributor

This PR updates test-maxtext.sh with two features.

  1. If a user specifies MAXTEXT_DIR in their environment, this dir will be used instead of /opt/maxtext. This makes it easier for development. Default behavior of using /opt/maxtext is unchanged unless that environment variable is specified.
  2. Support for more quantization recipes via MaxText's quantization config via a new --quantization arg.

Before this PR, this script uses --dtype=fp8 to specify MaxText FP8 DelayedScaling. However, when developing new quantization recipes, support for different types of quantization beyond "fp8" is useful. For backwards compatibility, I've kept the behavior of --dtype=fp8 unchanged. But you can now specify --quantization="..." with an arbitrary string and it will be passed to MaxText's quantization config unfiltered. An error is raised when both --dtype=fp8 and --quantization="..." are specified at the same time.

@jberchtold-nvidia
Copy link
Contributor Author

Closed and replaced with #1622 which uses a branch on the JAX-Toolbox repo instead of the fork

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant