Skip to content

Commit ef07a40

Browse files
ggml webgpu: add support for soft_max, optimize rms_norm (ggml-org#16357)
* Add inplace softmax * Move rms_norm to split row approach * Update debug for supports_op * clean up debug statements * Update tests/test-backend-ops.cpp Co-authored-by: Georgi Gerganov <[email protected]> --------- Co-authored-by: Georgi Gerganov <[email protected]>
1 parent 34fcc5a commit ef07a40

File tree

6 files changed

+565
-47
lines changed

6 files changed

+565
-47
lines changed

ggml/include/ggml.h

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1630,6 +1630,13 @@ extern "C" {
16301630
float scale,
16311631
float max_bias);
16321632

1633+
GGML_API struct ggml_tensor * ggml_soft_max_ext_inplace(
1634+
struct ggml_context * ctx,
1635+
struct ggml_tensor * a,
1636+
struct ggml_tensor * mask,
1637+
float scale,
1638+
float max_bias);
1639+
16331640
GGML_API void ggml_soft_max_add_sinks(
16341641
struct ggml_tensor * a,
16351642
struct ggml_tensor * sinks);

0 commit comments

Comments
 (0)