Add optional TBLIS CPU contraction provider - #1344
Conversation
Repository rules reviewRepository rules review (c76f260b595c292aa5fbedfc87de80b06e6a0a8f...c9a91158a1f89eadc8ce7d413ac9c6e7dd283c0f) Verdict: pass No findings. |
9ab6b04 to
22d20a0
Compare
|
Clarifying the TBLIS coverage contract for this PR.
Supported TBLIS attempt in this PR:
Intentional fallback cases include:
Reviewers should read results in three categories: TBLIS executed, TBLIS was intentionally not applicable and faer/BLAS fallback ran, or the request is an unsupported provider/placement combination and should error rather than fallback. |
shinaoka
left a comment
shinaoka
left a comment
There was a problem hiding this comment.
Blocking: CpuBackendKind::Tblis should not be introduced in this PR.
TBLIS only implements an optional fast path for dot_general; it is not a complete CPU backend. Making it a backend kind changes the scope of the provider selection and makes the fallback policy implicit. In particular, when both cpu-faer and cpu-blas are compiled, the current implementation silently chooses BLAS as the TBLIS fallback, rather than preserving the user-selected base provider.
Please keep CpuBackendKind for the complete base providers (Faer / Blas) and add TBLIS as a dot_general-specific dispatch option, e.g. DotGeneralProvider::TblisIfAvailable. Unsupported layouts/shapes and unavailable TBLIS should fall back to that backend's selected base provider.
Also separate TBLIS availability from execution policy: support mutually exclusive linked and runtime-loaded build modes, with runtime lazy loading as the normal path. The dynamic loader should retain the library and validated symbol table, report its search paths clearly, and permit an explicit Required mode for callers who do not want fallback.
This keeps the existing CPU backend model intact, avoids a hard-coded BLAS-over-faer fallback, and matches tenferro's runtime-loading direction.
|
Follow-up with a concrete direction for the requested redesign:
pub enum DotGeneralProvider {
Native,
PreferTblis,
RequireTblis,
}
pub struct CpuBackendOptions {
pub base_kind: CpuBackendKind,
pub dot_general_provider: DotGeneralProvider,
}
The normal path should be the dynamic one, matching tenferro's runtime-loading direction. It should follow the cuTENSOR/cuBLAS pattern: try
Minimum acceptance tests should cover:
This is intentionally a redesign, but it removes the hidden provider priority, avoids treating a partial accelerator as a full backend, and keeps dynamic loading as the default deployment model. |
|
Updated the PR for the review feedback. What changed:
Local checks run:
|
|
Follow-up pushed after CI feedback:
|
…lis-cpu-internal # Conflicts: # crates/tenferro-cpu/Cargo.toml # crates/tenferro-cpu/src/backend.rs # crates/tenferro-cpu/src/backend/tests.rs # crates/tenferro-cpu/src/lib.rs
|
CI 復旧と最終設計意図を記録します。 この修正の意図
最終設計は issue #1332 にも追記しました: #1332 (comment) CI 復旧
したがって、CI security policy や required check を回避・緩和せずに、PR head と最新 main の組み合わせを実GPUで検証しています。 |
2 participants
Closes #1332.
Summary
dot_general.cpu-faerorcpu-blasremains the fallback and linalg provider, and default features are unchanged.cpu-tblis-runtime(cpu-tblisremains its compatibility alias) usingtblis-ffidynamic loading, andcpu-tblis-linkedusing a statically linked source build.t4a-tblis-srcunderthird_party/, excluded from the tenferro workspace and consumed through the neutraltblis-srcdependency alias.f32,f64,Complex32, andComplex64, including conjugated and accumulated paths, with fallback for unsupported shapes, layouts, devices, and grouped GEMM.eb719e718976572e0ab53975f4e0c799faeb35f2, uses bundled BLIS, and links the source-backed route statically by default.Loader and package boundary
tblis-ffi 0.2.6panics when runtime discovery fails. The runtime route temporarily catches only the one-time pre-FFI loader probe, does not replace the process-global panic hook, and marks the compatibility bridge as an explicit audit exception. The linked route does not use this bridge.try_dyload_lib()while retaining the existing panic wrapper. The tenferro exception can be removed after a release containing that API is adopted.t4a-tblis-srccarries the Apache-2.0 source-glue license and attribution plus the pinned native projects license files and redistribution notes.t4a-tblis-srcrequires separate explicit user approval.AGENTS.mdnow forbids automatic publication of any new package and requires metadata and package-content validation before every publish.Verification
cargo test --workspace --releasecargo llvm-cov --workspace --release --json --output-path coverage.jsonpython3 scripts/check-coverage.py coverage.json— 159/159 checked files passedcargo doc --workspace --no-depspython3.11 scripts/check-docs-site.py— 13 workspace library crates and four guide dependency snippets passedcargo clippy --workspace --all-targets -- -D warningscargo clippy --manifest-path ext/tropical/Cargo.toml --all-targets -- -D warningscpu-tblis-runtimeandcpu-tblis-linkedcargo package --list, andcargo packageforthird_party/t4a-tblis-srcpython3 scripts/repository-rules-review.py --base origin/main --head HEAD ...— pass, no findingscargo fmt --all --check, standalone package formatting, andgit diff --checkBenchmarks
Quick local release benchmarks. Threads were pinned with
TBLIS_NUM_THREADS=1,RAYON_NUM_THREADS=1,OMP_NUM_THREADS=1, and for BLAS runsOPENBLAS_NUM_THREADS=1.Command:
Nis the square matrix size.nis the size of each tensor axis in the higher-rank cases. The rank-5 case uses fixed batch size4.f64_matrix_square_gemmC[i,j] = sum_k A[i,k] B[k,j]A,B: f64[N,N][1], rhs[0]C: f64[N,N]c64_matrix_square_gemm_lhs_conjC[i,j] = sum_k conj(A[i,k]) B[k,j]A,B: c64[N,N][1], rhs[0]C: c64[N,N]f64_rank4_packed_contract_axesC[a,b,c,d] = sum_{x,y} A[a,b,x,y] B[x,y,c,d]A,B: f64[n,n,n,n][2,3], rhs[0,1]C: f64[n,n,n,n]f64_rank4_mixed_contract_axesC[a,b,c,d] = sum_{x,y} A[a,x,b,y] B[c,y,x,d]A,B: f64[n,n,n,n][1,3], rhs[2,1]C: f64[n,n,n,n]f64_rank5_batched_mixed_contract_axesC[b,c,d,e,a] = sum_{x,y} A[a,b,x,c,y] B[a,d,y,x,e]A,B: f64[4,n,n,n,n][0]; contract lhs[2,4], rhs[3,2]C: f64[n,n,n,n,4]f64_rank4_row_major_view_mixed_contract_axesC[a,b,c,d] = sum_{x,y} A[a,x,b,y] B[c,y,x,d]A,B: f64[n,n,n,n], row-major strides[n^3,n^2,n,1][1,3], rhs[2,1]C: f64[n,n,n,n]Default/Small Sample
f64_matrix_square_gemmN=32f64_matrix_square_gemmN=64f64_matrix_square_gemmN=128c64_matrix_square_gemm_lhs_conjN=32c64_matrix_square_gemm_lhs_conjN=64c64_matrix_square_gemm_lhs_conjN=128f64_rank4_packed_contract_axesn=4f64_rank4_packed_contract_axesn=8f64_rank4_mixed_contract_axesn=4f64_rank4_mixed_contract_axesn=8f64_rank5_batched_mixed_contract_axesn=4, batch4f64_rank5_batched_mixed_contract_axesn=8, batch4f64_rank4_row_major_view_mixed_contract_axesn=4f64_rank4_row_major_view_mixed_contract_axesn=8Large Sample
Run with
TENFERRO_TBLIS_BENCH_MATMUL_SIZES=256,512 TENFERRO_TBLIS_BENCH_HIGHER_RANK_NS=16.f64_matrix_square_gemmN=256f64_matrix_square_gemmN=512c64_matrix_square_gemm_lhs_conjN=256c64_matrix_square_gemm_lhs_conjN=512f64_rank4_packed_contract_axesn=16f64_rank4_mixed_contract_axesn=16f64_rank5_batched_mixed_contract_axesn=16, batch4f64_rank4_row_major_view_mixed_contract_axesn=16Large Sample, 4 Threads
Run with
TENFERRO_TBLIS_BENCH_MATMUL_SIZES=256,512 TENFERRO_TBLIS_BENCH_HIGHER_RANK_NS=16andTBLIS_NUM_THREADS=4 RAYON_NUM_THREADS=4 OMP_NUM_THREADS=4 OPENBLAS_NUM_THREADS=4.f64_matrix_square_gemmN=256f64_matrix_square_gemmN=512c64_matrix_square_gemm_lhs_conjN=256c64_matrix_square_gemm_lhs_conjN=512f64_rank4_packed_contract_axesn=16f64_rank4_mixed_contract_axesn=16f64_rank5_batched_mixed_contract_axesn=16, batch4f64_rank4_row_major_view_mixed_contract_axesn=16Interpretation: TBLIS is not a broad small-size win. Larger complex, mixed-axis, batched, and view-heavy cases are more favorable, including at 4 threads, so this PR presents TBLIS as an optional provider rather than a universal replacement.