Skip to content

Fix KL divergence for zero true labels#14608

Open
Kcstring wants to merge 1 commit intoTheAlgorithms:masterfrom
Kcstring:fix-kl-divergence-zero-true
Open

Fix KL divergence for zero true labels#14608
Kcstring wants to merge 1 commit intoTheAlgorithms:masterfrom
Kcstring:fix-kl-divergence-zero-true

Conversation

@Kcstring
Copy link
Copy Markdown

Describe your change:

Fixes #12233.

kullback_leibler_divergence() now ignores positions where y_true is zero before applying np.log, matching the KL convention that zero-probability true labels contribute nothing to the sum. This avoids returning nan for inputs containing zero true labels without adding an epsilon or changing the behavior for non-zero labels.

  • Add an algorithm?
  • Fix a bug or typo in an existing algorithm?
  • Add or change doctests? -- Note: Please avoid changing both code and tests in a single pull request.
  • Documentation change?

Checklist:

  • I have read CONTRIBUTING.md.
  • This pull request is all my own work -- I have not plagiarized.
  • I know that pull requests will not be merged if they fail the automated tests.
  • This PR only changes one algorithm file. To ease review, please open separate PRs for separate algorithms.
  • All new Python files are placed inside an existing directory.
  • All filenames are in all lowercase characters with no spaces or dashes.
  • All functions and variable names follow Python naming conventions.
  • All function parameters and return values are annotated with Python type hints.
  • All functions have doctests that pass the automated testing.
  • All new algorithms include at least one URL that points to Wikipedia or another similar explanation.
  • If this pull request resolves one or more open issues then the description above includes the issue number(s) with a closing keyword: "Fixes #ISSUE-NUMBER".

Local verification:

  • python3 -m pytest machine_learning/loss_functions.py
  • ruff check machine_learning/loss_functions.py
  • ruff format --check machine_learning/loss_functions.py

@algorithms-keeper algorithms-keeper Bot added enhancement This PR modified some existing files awaiting reviews This PR is ready to be reviewed labels Apr 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting reviews This PR is ready to be reviewed enhancement This PR modified some existing files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avoid log(0) in KL divergence

2 participants