Fix: Remove redundant epsilon in loss mask weight calculation

[Abhinavexists]

This PR fixes incorrect use of epsilon in mask weight calculation that caused numerical instability when all mask values are False.

In keras/src/losses/loss.py, the mask weight calculation used:

valid = ops.sum(mask)  # May be 0!
mask *= total/ valid + backend.epsilon()

This is problematic because:

• Adding backend.epsilon() i feel defeats this purpose by ensuring b is never actually zero
• When valid = 0 (all mask values are False), this computes total / epsilon ≈ 10^7, causing mask weights to explode to huge values instead of becoming 0

[gemini-code-assist]

Summary of Changes

Hello @Abhinavexists, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical numerical instability issue within the Keras loss calculation by refining the apply_mask function. It specifically targets an incorrect use of backend.epsilon() during mask weight computation, which previously led to an explosion of values when the mask was entirely false. By removing this redundant epsilon and relying solely on ops.divide_no_nan, the change ensures more robust and stable loss calculations, preventing unexpected numerical behavior.

Highlights

• Numerical Stability Fix: Removed a redundant backend.epsilon() from the mask weight calculation in apply_mask function, which was causing numerical instability and exploding mask weights when all mask values were false.
• Corrected Division Logic: The mask weight calculation now correctly uses ops.divide_no_nan(total, valid), leveraging its built-in zero-division handling without interference from epsilon.
• New Test Case: Added a new test test_divide_no_nan to loss_test.py to explicitly verify the behavior of ops.divide_no_nan when dividing by zero.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request correctly identifies and resolves a numerical instability issue in the loss mask weight calculation by removing a redundant epsilon. The fix is sound, and the use of ops.divide_no_nan is appropriate for handling division by zero gracefully. The addition of a new test case, test_divide_no_nan, is a valuable contribution as it ensures the behavior of the underlying operation is verified. Overall, the changes are well-executed and improve the robustness of the loss calculation.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.59%. Comparing base (f0a48a6) to head (b4a224d).
⚠️ Report is 7 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #21908      +/-   ##
==========================================
+ Coverage   76.30%   82.59%   +6.29%     
==========================================
  Files         580      580              
  Lines       60029    60226     +197     
  Branches     9432     9444      +12     
==========================================
+ Hits        45803    49742    +3939     
+ Misses      11750     8050    -3700     
+ Partials     2476     2434      -42     

Flag	Coverage Δ
keras	82.41% <100.00%> (+6.24%)	⬆️
keras-jax	61.92% <100.00%> (-0.20%)	⬇️
keras-numpy	57.13% <100.00%> (-0.19%)	⬇️
keras-openvino	35.66% <0.00%> (+1.36%)	⬆️
keras-tensorflow	64.11% <100.00%> (?)
keras-torch	63.03% <100.00%> (-0.19%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[hertschuh]

Thanks for the PR!

[hertschuh]

This is already tested: https://github.com/keras-team/keras/blob/master/keras/src/ops/numpy_test.py#L2681-L2690

So no need to test this here.

Additionally, it looks like this case of all zeros mask is already covered:
https://github.com/keras-team/keras/blob/master/keras/src/losses/loss_test.py#L125-L129

[Abhinavexists]

The test have been removed