Skip to content

Commit e44a0aa

Browse files
authored
Collapse the dispatch paths and split the large modules (#82)
One dispatcher for every context request in C, one NIF table entry list per area, one process body module per mode on the Erlang side, and a py facade that delegates streaming, venvs and shared dicts to their own modules. The thread functions and flags are named for what they are rather than for the worker mode they came from. No public API changes.
1 parent 1fb8714 commit e44a0aa

26 files changed

Lines changed: 2153 additions & 3312 deletions

‎CHANGELOG.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,22 @@
4949
storms, loop churn, 60 s mixed workload with resource counters checked.
5050
- Guide: `docs/isolated.md`, with what each of the three modes guarantees.
5151

52+
### Changed
53+
54+
- The NIF side of every context request goes through one dispatcher
55+
(`ctx_dispatch`, `ctx_dispatch_async` in `c_src/py_nif.c`) instead of a
56+
per-request copy of the enqueue-and-wait loop; the execute functions are
57+
`ctx_execute_*` and the thread functions `ctx_thread_main_*`, since both
58+
serve worker and owngil contexts. Creating a process-local env and
59+
applying imports or paths run on the context thread in `worker` mode too;
60+
the scheduler-side copies of those paths are gone.
61+
- The NIF function table is assembled from one `PY_*_NIFS` macro per area,
62+
defined at the end of the file that owns the NIFs.
63+
- `py_context` keeps the API and the reply protocol; the process body for
64+
embedded modes moved to `py_context_embedded`. `py` delegates streaming,
65+
virtual environments and shared dicts to `py_stream`, `py_venv` and
66+
`py_shared_dict`. The public API is unchanged.
67+
5268
### Removed
5369

5470
- The legacy worker API (`py_nif:worker_new/0,1`, `worker_call`, `worker_eval`,

‎c_src/README.md‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ where things are.
1414
| File | What it owns | Notes |
1515
|---|---|---|
1616
| `py_nif.h` | All shared types: `py_context_t` and its request queue, request types, callback and suspension state, runtime state machine, atoms, globals, declarations | 2.4k lines. The struct comments carry the locking rules; read `py_context_t` before touching threads |
17-
| `py_nif.c` | Runtime init and finalize, resource types, context create/destroy, the request queue, `worker_context_thread_main` and `owngil_context_thread_main`, `owngil_execute_*` (used by both thread kinds), the `nif_context_*` NIFs, process-local envs, `py_ref`, the NIF function table at the end | Sections are banner-separated; `grep -n '^ \* ===\|^/\* ==='` lists them |
17+
| `py_nif.c` | Runtime init and finalize, resource types, context create/destroy, the request queue, `ctx_thread_main_worker` and `ctx_thread_main_owngil`, `ctx_execute_*` (one set for both thread kinds), `ctx_dispatch` / `ctx_dispatch_async` (the only way a NIF reaches a context thread), the `nif_context_*` NIFs, process-local envs, `py_ref`, the NIF function table at the end, assembled from the `PY_*_NIFS` macros of the other files | Sections are banner-separated; `grep -n '^ \* ===\|^/\* ==='` lists them |
1818
| `py_convert.c` | `py_to_term`, `term_to_py`, depth limits, tagged tuples (`{bytes, B}`, `{'$py_shm', ...}`), error tuples `{error, {Type, Msg}}` | The type mapping tables in the comments are the reference for `_etf.py` |
1919
| `py_exec.c` | Execution mode detection (free-threaded or GIL build) | |
2020
| `py_callback.c` | The `erlang` Python module: `call`, `send`, `whereis`, `schedule*`, `Atom`/`Pid`/`Ref` types, callback delivery paths (suspension, blocking pipe, async pipe), channel and shared-dict methods, callback name registry | `erlang_call_impl` documents the path precedence |
@@ -29,8 +29,8 @@ where things are.
2929
## Where the live paths are
3030

3131
- `py:call/3` in worker or owngil mode: `nif_context_call_async` (`py_nif.c`)
32-
enqueues; `worker_context_thread_main` or `owngil_context_thread_main`
33-
dequeues and calls `owngil_execute_request`; the reply goes out as
32+
enqueues; `ctx_thread_main_worker` or `ctx_thread_main_owngil`
33+
dequeues and calls `ctx_execute_request`; the reply goes out as
3434
`{py_result, Ref, Result}`.
3535
- `erlang.call` from Python: `erlang_call_impl` (`py_callback.c`).
3636
- Interrupt: `nif_context_interrupt` (`py_nif.c`), `interrupt_mutex` rules on
@@ -57,7 +57,9 @@ where things are.
5757

5858
1. Implement `static ERL_NIF_TERM nif_x(ErlNifEnv*, int, const ERL_NIF_TERM[])`
5959
next to related code.
60-
2. Add `{"x", Arity, nif_x, Flags}` to `nif_funcs[]` at the end of `py_nif.c`.
60+
2. Add `{"x", Arity, nif_x, Flags}` to the `PY_*_NIFS` macro at the end of
61+
that file (or to the `py_nif.c` block of `nif_funcs[]` for NIFs that
62+
live there).
6163
3. Add the stub and its `-spec` and doc to `src/py_nif.erl`.
6264
4. Cover it in a suite; `rebar3 dialyzer` and `rebar3 xref` must stay clean.
6365

‎c_src/py_buffer.c‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1103,3 +1103,10 @@ static ERL_NIF_TERM nif_py_buffer_close(ErlNifEnv *env, int argc,
11031103

11041104
return ATOM_OK;
11051105
}
1106+
1107+
/* NIF table entries of this file; py_nif.c concatenates them into nif_funcs[].
1108+
* Flags: ERL_NIF_DIRTY_JOB_* for anything that can block or run Python. */
1109+
#define PY_BUFFER_NIFS \
1110+
{"py_buffer_create", 1, nif_py_buffer_create, 0}, \
1111+
{"py_buffer_write", 2, nif_py_buffer_write, 0}, \
1112+
{"py_buffer_close", 1, nif_py_buffer_close, 0}

‎c_src/py_callback.c‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4053,3 +4053,9 @@ static ERL_NIF_TERM nif_unregister_callback_name(ErlNifEnv *env, int argc, const
40534053

40544054
return ATOM_OK;
40554055
}
4056+
4057+
/* NIF table entries of this file; py_nif.c concatenates them into nif_funcs[].
4058+
* Flags: ERL_NIF_DIRTY_JOB_* for anything that can block or run Python. */
4059+
#define PY_CALLBACK_NIFS \
4060+
{"register_callback_name", 1, nif_register_callback_name, 0}, \
4061+
{"unregister_callback_name", 1, nif_unregister_callback_name, 0}

‎c_src/py_channel.c‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1033,3 +1033,21 @@ ERL_NIF_TERM nif_byte_channel_wait_bytes(ErlNifEnv *env, int argc, const ERL_NIF
10331033
/* Return ok - Python will await Future */
10341034
return ATOM_OK;
10351035
}
1036+
1037+
/* NIF table entries of this file; py_nif.c concatenates them into nif_funcs[].
1038+
* Flags: ERL_NIF_DIRTY_JOB_* for anything that can block or run Python. */
1039+
#define PY_CHANNEL_NIFS \
1040+
{"channel_create", 0, nif_channel_create, 0}, \
1041+
{"channel_create", 1, nif_channel_create, 0}, \
1042+
{"channel_send", 2, nif_channel_send, 0}, \
1043+
{"channel_receive", 2, nif_channel_receive, 0}, \
1044+
{"channel_try_receive", 1, nif_channel_try_receive, 0}, \
1045+
{"channel_reply", 3, nif_channel_reply, 0}, \
1046+
{"channel_close", 1, nif_channel_close, 0}, \
1047+
{"channel_info", 1, nif_channel_info, 0}, \
1048+
{"channel_wait", 3, nif_channel_wait, 0}, \
1049+
{"channel_cancel_wait", 2, nif_channel_cancel_wait, 0}, \
1050+
{"channel_register_sync_waiter", 1, nif_channel_register_sync_waiter, 0}, \
1051+
{"byte_channel_send_bytes", 2, nif_byte_channel_send_bytes, 0}, \
1052+
{"byte_channel_try_receive_bytes", 1, nif_byte_channel_try_receive_bytes, 0}, \
1053+
{"byte_channel_wait_bytes", 3, nif_byte_channel_wait_bytes, 0}

‎c_src/py_event_loop.c‎

Lines changed: 74 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5735,7 +5735,7 @@ ERL_NIF_TERM nif_reactor_on_read_ready(ErlNifEnv *env, int argc,
57355735
#ifdef HAVE_SUBINTERPRETERS
57365736
/* OWN_GIL mode: dispatch to dedicated thread */
57375737
if (ctx->uses_own_gil) {
5738-
return dispatch_reactor_read_to_owngil(env, ctx, fd, buffer);
5738+
return dispatch_reactor_read(env, ctx, fd, buffer);
57395739
}
57405740
#endif
57415741

@@ -5831,7 +5831,7 @@ ERL_NIF_TERM nif_reactor_on_write_ready(ErlNifEnv *env, int argc,
58315831
#ifdef HAVE_SUBINTERPRETERS
58325832
/* OWN_GIL mode: dispatch to dedicated thread */
58335833
if (ctx->uses_own_gil) {
5834-
return dispatch_reactor_write_to_owngil(env, ctx, fd);
5834+
return dispatch_reactor_write(env, ctx, fd);
58355835
}
58365836
#endif
58375837

@@ -5917,7 +5917,7 @@ ERL_NIF_TERM nif_reactor_init_connection(ErlNifEnv *env, int argc,
59175917
#ifdef HAVE_SUBINTERPRETERS
59185918
/* OWN_GIL mode: dispatch to dedicated thread */
59195919
if (ctx->uses_own_gil) {
5920-
return dispatch_reactor_init_to_owngil(env, ctx, fd, argv[2]);
5920+
return dispatch_reactor_init(env, ctx, fd, argv[2]);
59215921
}
59225922
#endif
59235923

@@ -8528,3 +8528,74 @@ int init_subinterpreter_event_loop(ErlNifEnv *env) {
85288528
}
85298529
return 0;
85308530
}
8531+
8532+
/* NIF table entries of this file; py_nif.c concatenates them into nif_funcs[].
8533+
* Flags: ERL_NIF_DIRTY_JOB_* for anything that can block or run Python. */
8534+
#define PY_EVENT_LOOP_NIFS \
8535+
{"set_event_loop_priv_dir", 1, nif_set_event_loop_priv_dir, 0}, \
8536+
{"event_loop_new", 0, nif_event_loop_new, 0}, \
8537+
{"event_loop_destroy", 1, nif_event_loop_destroy, 0}, \
8538+
{"event_loop_set_router", 2, nif_event_loop_set_router, 0}, \
8539+
{"event_loop_set_worker", 2, nif_event_loop_set_worker, 0}, \
8540+
{"event_loop_set_id", 2, nif_event_loop_set_id, 0}, \
8541+
{"event_loop_wakeup", 1, nif_event_loop_wakeup, 0}, \
8542+
{"event_loop_run_async", 7, nif_event_loop_run_async, ERL_NIF_DIRTY_JOB_IO_BOUND}, \
8543+
{"submit_task", 7, nif_submit_task, 0}, \
8544+
{"submit_task_with_env", 8, nif_submit_task_with_env, 0}, \
8545+
{"process_ready_tasks", 1, nif_process_ready_tasks, ERL_NIF_DIRTY_JOB_CPU_BOUND}, \
8546+
{"event_loop_set_py_loop", 2, nif_event_loop_set_py_loop, 0}, \
8547+
{"event_loop_exec", 2, nif_event_loop_exec, ERL_NIF_DIRTY_JOB_IO_BOUND}, \
8548+
{"event_loop_eval", 2, nif_event_loop_eval, ERL_NIF_DIRTY_JOB_IO_BOUND}, \
8549+
{"add_reader", 3, nif_add_reader, 0}, \
8550+
{"remove_reader", 2, nif_remove_reader, 0}, \
8551+
{"add_writer", 3, nif_add_writer, 0}, \
8552+
{"remove_writer", 2, nif_remove_writer, 0}, \
8553+
{"call_later", 3, nif_call_later, 0}, \
8554+
{"cancel_timer", 2, nif_cancel_timer, 0}, \
8555+
{"poll_events", 2, nif_poll_events, ERL_NIF_DIRTY_JOB_IO_BOUND}, \
8556+
{"get_pending", 1, nif_get_pending, 0}, \
8557+
{"dispatch_callback", 3, nif_dispatch_callback, 0}, \
8558+
{"dispatch_timer", 2, nif_dispatch_timer, 0}, \
8559+
{"get_fd_callback_id", 2, nif_get_fd_callback_id, 0}, \
8560+
{"reselect_reader", 2, nif_reselect_reader, 0}, \
8561+
{"reselect_writer", 2, nif_reselect_writer, 0}, \
8562+
{"reselect_reader_fd", 1, nif_reselect_reader_fd, 0}, \
8563+
{"reselect_writer_fd", 1, nif_reselect_writer_fd, 0}, \
8564+
{"handle_fd_event", 2, nif_handle_fd_event, 0}, \
8565+
{"handle_fd_event_and_reselect", 2, nif_handle_fd_event_and_reselect, 0}, \
8566+
{"fd_arm", 2, nif_fd_arm, 0}, \
8567+
{"stop_reader", 1, nif_stop_reader, 0}, \
8568+
{"start_reader", 1, nif_start_reader, 0}, \
8569+
{"stop_writer", 1, nif_stop_writer, 0}, \
8570+
{"start_writer", 1, nif_start_writer, 0}, \
8571+
{"close_fd", 1, nif_close_fd, 0}, \
8572+
{"create_test_pipe", 0, nif_create_test_pipe, 0}, \
8573+
{"close_test_fd", 1, nif_close_test_fd, 0}, \
8574+
{"dup_fd", 1, nif_dup_fd, 0}, \
8575+
{"write_test_fd", 2, nif_write_test_fd, 0}, \
8576+
{"read_test_fd", 2, nif_read_test_fd, 0}, \
8577+
{"create_test_tcp_listener", 1, nif_create_test_tcp_listener, 0}, \
8578+
{"accept_test_tcp", 1, nif_accept_test_tcp, 0}, \
8579+
{"connect_test_tcp", 2, nif_connect_test_tcp, 0}, \
8580+
{"create_test_udp_socket", 1, nif_create_test_udp_socket, 0}, \
8581+
{"recvfrom_test_udp", 2, nif_recvfrom_test_udp, 0}, \
8582+
{"sendto_test_udp", 4, nif_sendto_test_udp, 0}, \
8583+
{"set_udp_broadcast", 2, nif_set_udp_broadcast, 0}, \
8584+
{"set_python_event_loop", 1, nif_set_python_event_loop, 0}, \
8585+
{"set_isolation_mode", 1, nif_set_isolation_mode, 0}, \
8586+
{"set_shared_worker", 1, nif_set_shared_worker, 0}, \
8587+
{"context_get_event_loop", 1, nif_context_get_event_loop, 0}, \
8588+
{"reactor_register_fd", 3, nif_reactor_register_fd, 0}, \
8589+
{"reactor_reselect_read", 1, nif_reactor_reselect_read, 0}, \
8590+
{"reactor_select_write", 1, nif_reactor_select_write, 0}, \
8591+
{"get_fd_from_resource", 1, nif_get_fd_from_resource, 0}, \
8592+
{"reactor_on_read_ready", 2, nif_reactor_on_read_ready, ERL_NIF_DIRTY_JOB_CPU_BOUND}, \
8593+
{"reactor_on_write_ready", 2, nif_reactor_on_write_ready, ERL_NIF_DIRTY_JOB_CPU_BOUND}, \
8594+
{"reactor_init_connection", 3, nif_reactor_init_connection, ERL_NIF_DIRTY_JOB_CPU_BOUND}, \
8595+
{"reactor_close_fd", 2, nif_reactor_close_fd, 0}, \
8596+
{"fd_read", 2, nif_fd_read, ERL_NIF_DIRTY_JOB_IO_BOUND}, \
8597+
{"fd_write", 2, nif_fd_write, ERL_NIF_DIRTY_JOB_IO_BOUND}, \
8598+
{"fd_select_read", 1, nif_fd_select_read, 0}, \
8599+
{"fd_select_write", 1, nif_fd_select_write, 0}, \
8600+
{"fd_close", 1, nif_fd_close, 0}, \
8601+
{"socketpair", 0, nif_socketpair, 0}

‎c_src/py_logging.c‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -453,3 +453,11 @@ static ERL_NIF_TERM nif_clear_trace_receiver(ErlNifEnv *env, int argc, const ERL
453453

454454
return ATOM_OK;
455455
}
456+
457+
/* NIF table entries of this file; py_nif.c concatenates them into nif_funcs[].
458+
* Flags: ERL_NIF_DIRTY_JOB_* for anything that can block or run Python. */
459+
#define PY_LOGGING_NIFS \
460+
{"set_log_receiver", 2, nif_set_log_receiver, 0}, \
461+
{"clear_log_receiver", 0, nif_clear_log_receiver, 0}, \
462+
{"set_trace_receiver", 1, nif_set_trace_receiver, 0}, \
463+
{"clear_trace_receiver", 0, nif_clear_trace_receiver, 0}

0 commit comments

Comments
 (0)