loky: remove misleading set_start_method docs and refresh Task & results serialization section#631
Open
mvanhorn wants to merge 1 commit into
Conversation
…lts serialization section Fixes joblib#404
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Single-file edit to
docs/API.rst.Remove the "Processes start methods in
loky" section header and its body. This is the entire block beginningProcesses start methods in :mod:\loky`and ending before "Protection against memory leaks". The section currently lives between theautofunction:: loky.wrap_non_picklable_objects` directive and the "Protection against memory leaks" heading.In the "Task & results serialization" section, rewrite the paragraph that currently reads:
New wording (keep the same overall flow, just correct the two outdated claims):
Keep the existing list of three ways to control serialization (
job_reducers/result_reducers,LOKY_PICKLER,wrap_non_picklable_objects) verbatim. Do not change them in this PR; deprecation ofwrap_non_picklable_objectsis out of scope per the deferral above.Build the docs locally to verify no Sphinx warnings or broken cross-references are introduced (the removed section did not contain any
.. _xxx:labels that other pages link to; a quickgrep -r "set_start_method" docs/after the edit confirms no leftover references).PR title:
DOC Remove misleading set_start_method section and refresh cloudpickle wordingPR body (concise, factual):
Why this matters
Issue #404 reports that the loky docs imply
loky.set_start_methodis public API, when the actual import path isloky.backend.context.set_start_methodand the function is not exposed at the top-level module. Maintainer @ogrisel responded on 2025-03-06 stating he does not see value in changing the start method at the loky level, and proposed three concrete docs cleanups in the same comment:loky" section indocs/API.rstso the docs stop implyingloky.set_start_methodis public API.__main__), as demonstrated by the small reproducer in the maintainer comment.loky.wrap_non_picklable_objectsand theloky_init_mainstart method. These touch runtime code and the deprecation cycle is a larger conversation; this PR will keep the autofunction directive forwrap_non_picklable_objectsso the deprecation can be staged in a follow-up.Testing
cd docs && make htmlcompletes without new Sphinx warnings or errors compared to a cleanmasterbuild (capture both before and after stderr to diff).grep -rn "set_start_method" docs/returns no matches after the edit. (Before the edit, it matches the section being removed.)grep -rn "slower than the :mod:\pickle`" docs/` returns no matches after the edit.docs/_build/html/API.htmllocally and visually confirm that the API page renders, the autofunction directive forwrap_non_picklable_objectsstill appears, and the Protection against memory leaks section now follows the Task & results serialization section directly.Fixes #404
AI was used for assistance.