Stripe transfers across compatible NIC pairs

[samlurye]

Summary:
Introduce StripeId as the shared identity carried by QueuePairOp, PostedOpEntry, and StripeResult. Queue-pair operations contain already-sliced local and remote registration views, leaving stripe planning to their callers.

QueuePairRouter::pair_registrations returns a disjoint list of registration pairings allowed by the affinity policy. QueuePairRouter::plan_ops computes the stripe count and contiguous ranges directly, shuffles registration pairs only when it must select a subset, and slices each registration pair for its stripe. A transfer smaller than the configured minimum remains whole, and splitting never creates an undersized remainder. Interior stripe boundaries are aligned to 64 bytes, with any residual bytes assigned to the final stripe, to avoid the severe unaligned host-memory RDMA throughput penalty measured on GB200.

Both warm direct submission and cold manager submission use the same planner. Their dispatch loops remain separate because the warm path must verify that every cached route exists before sending any stripe, while the manager path may register memory and create missing queue pairs.

The following results measure median warm initiator throughput in GB/s. Every cell is CPU-to-CPU p2p across 2 hosts with 1 proc per host, same lane pairing, 4 RDMA runtime threads, unlimited NICs per buffer, 64 QPs per CQ, per-device polling, shuffled compatible-pair selection, 3 fresh-buffer runs, 3 warmup iterations, and 10 measured iterations. Single stripe is the 32 MiB minimum configuration. 512 KiB vs single is (512 KiB throughput / single-stripe throughput) - 1.

H100 read:

Pattern	Ops	Payload	512 KiB	8 MiB	Single stripe	512 KiB vs single
p2p	1	1 MB	5.28	3.27	3.53	+49.6%
p2p	1	16 MB	65.38	22.15	25.68	+154.6%
p2p	1	32 MB	100.03	65.08	25.70	+289.2%
p2p	4	1 MB	17.19	14.87	14.57	+18.0%
p2p	4	16 MB	112.47	56.30	52.12	+115.8%
p2p	4	32 MB	134.11	104.58	60.73	+120.8%
p2p	128	1 MB	87.08	100.71	90.13	-3.4%
p2p	128	16 MB	176.79	128.19	126.70	+39.5%
p2p	128	32 MB	179.09	125.06	113.97	+57.1%

H100 write:

Pattern	Ops	Payload	512 KiB	8 MiB	Single stripe	512 KiB vs single
p2p	1	1 MB	5.51	5.86	3.90	+41.3%
p2p	1	16 MB	68.34	28.78	33.83	+102.0%
p2p	1	32 MB	100.56	72.53	36.28	+177.2%
p2p	4	1 MB	18.57	16.25	16.01	+16.0%
p2p	4	16 MB	123.42	77.90	70.12	+76.0%
p2p	4	32 MB	138.30	121.95	83.15	+66.3%
p2p	128	1 MB	87.02	83.34	76.04	+14.4%
p2p	128	16 MB	175.88	162.40	161.14	+9.1%
p2p	128	32 MB	180.43	170.50	161.08	+12.0%

GB200 read:

Pattern	Ops	Payload	512 KiB	8 MiB	Single stripe	512 KiB vs single
p2p	1	1 MB	5.55	3.59	3.48	+59.5%
p2p	1	16 MB	62.37	26.04	26.33	+136.9%
p2p	1	32 MB	59.35	57.50	32.48	+82.7%
p2p	4	1 MB	10.63	10.20	14.05	-24.3%
p2p	4	16 MB	78.24	58.86	61.70	+26.8%
p2p	4	32 MB	102.84	88.91	75.86	+35.6%
p2p	128	1 MB	74.60	57.47	81.62	-8.6%
p2p	128	16 MB	148.56	140.81	139.45	+6.5%
p2p	128	32 MB	143.27	148.91	142.81	+0.3%

GB200 write:

Pattern	Ops	Payload	512 KiB	8 MiB	Single stripe	512 KiB vs single
p2p	1	1 MB	2.35	2.46	3.69	-36.3%
p2p	1	16 MB	31.32	21.59	22.04	+42.1%
p2p	1	32 MB	55.47	52.10	29.25	+89.6%
p2p	4	1 MB	9.37	12.67	13.29	-29.5%
p2p	4	16 MB	77.72	72.60	61.45	+26.5%
p2p	4	32 MB	103.23	110.13	81.62	+26.5%
p2p	128	1 MB	95.95	84.78	85.68	+12.0%
p2p	128	16 MB	155.18	144.60	144.25	+7.6%
p2p	128	32 MB	147.10	162.53	148.03	-0.6%

Differential Revision: D119713607

[meta-codesync]

@samlurye has exported this pull request. If you are a Meta employee, you can view the originating Diff in D119713607.