Off-by-one error in _winapi.WaitForMultipleObjects

[randomascii]

Bug report

_winapi_WaitForMultipleObjects_impl in _winapi.c creates an array of MAXIMUM_WAIT_OBJECTS handles but then arbitrarily enforces a cap of MAXIMUM_WAIT_OBJECTS - 1 handles. This is incorrect. WaitForMultipleObjects is clearly documented as being able to handle MAXIMUM_WAIT_OBJECTS (64) handles. This simple C++ sample code demonstrates this:

#include <stdio.h>
#include <windows.h>

#include

int main(int argc, char* argv[]) {
for (DWORD count = 60; count < 66; ++count) {
printf("About to wait on %d handles.\n", count);
std::vector handles(count);
for (int i = 0; i < count; ++i)
handles[i] = CreateEventW(nullptr, FALSE, TRUE, nullptr);
DWORD wait_result = WaitForMultipleObjects(count, handles.data(), TRUE, INFINITE);
if (wait_result == WAIT_OBJECT_0)
printf(" Wait satisfied by object 0.\n");
else if (wait_result == WAIT_FAILED)
printf(" Wait failed.\n");
else
printf(" Result was %lu.\n", wait_result);
for (int i = 0; i < count; ++i)
CloseHandle(handles[i]);
}
}

This error also shows up here:
3988986#diff-eb5cd13d65d2f215d8dc0d95eb1a1341ff69b226ac56ec6099e39582edecc8cfR219
and here:
3988986#diff-84a955ea642d41d9f164e9f94499d8eb33c357ec7f4362a341951f6f67ab21a1R117

Note that this is in addition to the off-by-one error that was tracked by this issue:
#84444

That is, there used to be an off-by-two error (maximum of 60 processes could be waited on) and now there is an off-by-one error (maximum of 61 processes can be waited on). In Python 3.8.10 the limit is still 60, but in Python 3.9 the limit is (as far as I can tell) 61. It should be 62.

Your environment

Python 3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Windows 10

Linked PRs

• gh-94242: Clarify comments for _MAX_WINDOWS_WORKERS in concurrent.futures.process #135537

[eryksun]

If the wait is for any kernel object to be signaled, instead of waiting for all objects to be signaled, and it's the main thread, then the limit must be MAXIMUM_WAIT_OBJECTS - 1 (63) objects because the last index is reserved for _PyOS_SigintEvent(), the kernel event object that gets set by Python's SIGINT handler:

    /* If this is the main thread then make the wait interruptible
       by Ctrl-C unless we are waiting for *all* handles */
    if (!wait_flag && _PyOS_IsMainThread()) {
        sigint_event = _PyOS_SigintEvent();
        assert(sigint_event != NULL);
        handles[nhandles++] = sigint_event;
    }

I assume the same limit is applied for all threads for the sake of simplicity. Technically, it could be extended to 64 handles if the condition !wait_flag && _PyOS_IsMainThread() is false.

[randomascii]

Gotcha. The comments are incorrect still since they say that the Windows limit is 63 and that Python uses two handles, but in fact the Windows limit is 64 and Python uses three handles.

I think having the limit as 61 instead of 62 is fine. I don't know whether the misleading comments matter or not.

[eryksun]

Did you want to submit a PR to update the comment in Lib/concurrent/futures/process.py? Otherwise this can be closed.

[randomascii]

Did you want to submit a PR to update the comment in Lib/concurrent/futures/process.py? Otherwise this can be closed.

Sure. In progress.