Skip to content

Reuse managed CPU execution coordination - #1389

Closed
Ryo-wtnb11 wants to merge 6 commits into
tensor4all:mainfrom
Ryo-wtnb11:perf/cpu-managed-entry-reuse
Closed

Reuse managed CPU execution coordination#1389
Ryo-wtnb11 wants to merge 6 commits into
tensor4all:mainfrom
Ryo-wtnb11:perf/cpu-managed-entry-reuse

Conversation

@Ryo-wtnb11

Copy link
Copy Markdown
Collaborator

Why

A persistent managed CPU context still rebuilt execution coordination on every entry: immutable CPU domains were cloned, active arbiter requests used allocating tree nodes, and execution-owner state was broadcast to every worker on entry and exit. Faer policy prepared outside the owned pool could also capture an unrelated ambient Rayon degree.

This made empty warm entry pay costs unrelated to the numerical operation and made provider planning depend on where it was constructed.

Closes #1388.

What changed

  • Store immutable CpuSet contents in shared Arc<[CpuId]> storage.
  • Retain active-request capacity in the resource arbiter while preserving the fair waiter queue, conflict rules, reentry, and RAII release behavior.
  • Register a shared worker execution scope once when a managed Rayon pool is constructed, then update its owner/depth without per-entry broadcasts.
  • Derive Faer parallelism explicitly from the configured CpuContext degree, including policies created outside the managed pool.
  • Add allocation, panic/reentry, worker-registration, provider-policy, source-contract, and benchmark coverage.
  • Keep the TBLIS selection/fallback and provider-owned threading contract introduced by Add optional TBLIS CPU contraction provider #1344 unchanged.

Allocation boundary

The regression contract removes mandatory tenferro-owned allocation from compatible warm managed entry. It does not promise that Rayon never grows injector storage or that third-party providers never allocate. The sustained diagnostic observed occasional scheduler allocations after warmup, so the CI gate verifies that warm entry can complete without a mandatory allocation for one, two, and four configured workers rather than claiming unbounded global zero allocation.

Verification

  • cpu-faer lib tests: 288 passed on macOS, including the portable cross-pool reentry fixture.
  • Release install_allocation_tests: 1 passed.
  • cargo check --no-default-features --features cpu-blas: passed.
  • cargo check --no-default-features --features cpu-faer,cpu-tblis: passed with the dynamic-loading TBLIS feature. This is a compile check; an external TBLIS link/runtime was not exercised.
  • TBLIS provider source-contract tests: 5 passed in independent review.
  • cargo fmt --all -- --check and git diff --check: passed.
  • Independent review of the final range found no P0-P2 issues.

Platform note

An earlier broader macOS CPU run on the pre-#1344 base reported five placement/NUMA failures caused by unsupported managed-affinity environment assumptions, not by numerical or execution-scope results. #1344 made those platform requirements explicit and cfg-gated the affected fixtures. On the current base, the macOS cpu-faer lib suite passes 288/288; the newly portable cross-pool fixture runs on macOS rather than being hidden by the placement gate.

Repeated backend entry is the ownership boundary exercised by tensor workloads. Count a sustained warm window so Rayon scheduler tail work cannot hide a per-execution allocation.
Empty backend entry should not allocate in proportion to worker count. Share immutable CPU domains, retain active-request capacity, and register worker execution scopes once so warm execution avoids Rayon broadcasts and tree-node allocation without weakening exclusion or reentry semantics.
Rayon injector blocks grow periodically even after backend-owned entry storage is warm. Gate the mandatory per-entry costs directly, retain sustained allocation evidence, and compare one-, two-, and four-worker latency without making an unbounded zero-allocation claim.
Prepared plans and sessions create their Faer policy before entering the managed Rayon pool. Pin the policy to the configured CPU context degree so an ambient global or custom pool cannot silently change task partitioning.
Prepared policies can be constructed before entering the managed Rayon pool, where rayon(0) captures an unrelated ambient degree. Pass the configured context count explicitly so planning, sessions, and ordinary Faer kernels obey one stable worker contract.
Persistent execution-scope reentry no longer depends on platform affinity, so gate the regression only on Faer and keep the repository's normative worker policy synchronized with the configured CpuContext degree.
@Ryo-wtnb11

Copy link
Copy Markdown
Collaborator Author

Superseded by a same-repository PR because the repository-rules workflow intentionally rejects external-head PRs. The failure was the IS_EXTERNAL_PR=true policy gate, not a code or test failure. The replacement uses the identical df4e6d46 HEAD/tree.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Performance]: Remove mandatory allocations from warm managed CPU entry

1 participant