Skip to content

Commit 91b2ed4

Browse files
authored
Update quantization.md
1 parent 76ebe7c commit 91b2ed4

File tree

1 file changed

+1
-2
lines changed

1 file changed

+1
-2
lines changed

docs/explanations/quantization.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -143,7 +143,6 @@ Note that `use_qwix_quantization` is not set to `True`.
143143

144144
For further reading, please refer to the [Qwix Read the Docs website](https://qwix.readthedocs.io/en/latest/get_started.html#).
145145

146-
147146
## DeepSeek V3 Fine-tuning FP8 Recipe
148147
To improve the performance of DeepSeek V3 fine-tuning, we developed a custom recipe optimized for FP8 throughput. The method prioritizes specific compute-intensive and bandwidth-heavy components while preserving training stability through a fine-grained scaling strategy.
149148

@@ -160,7 +159,7 @@ To realize these gains, the recipe employs a w8a8g8 (8-bit weights, activations
160159
* Rounding: rounding to nearest even
161160
* Precision
162161
* Activations and weights: e4m3fn
163-
* Gradients:e5m2
162+
* Gradients: e5m2
164163
* Scaling granularity: per-axis
165164
* Scaling mode:
166165
* static for weights and activations

0 commit comments

Comments
 (0)