issue: 4728517 Fix credit prediction mismatch in hw_queue_tx

Fix assertion failure in debug builds and Send Queue waste caused by
mismatch between credit prediction (credits_calculate) and actual WQE
size calculation (fill_wqe).

The bug manifested with multi-SGE packets where total data length was
small enough to be inlined (<= 204 bytes). For example, a packet with
num_sge=2 and total_data_len=142 bytes would:

- credits_calculate(): See num_sge != 1, predict 2 credits (scatter-gather)
- fill_wqe(): See data_len <= 204, use inline path, need 3 wqebbs
- Result: Assertion failure (wqebbs=3 > credits=2)

Additionally, the initial fix using a conservative upper bound (max of 4)
caused severe performance degradation: 2-SGE UDP packets >204 bytes would
reserve 4 WQEBBs but only use 1 WQEBB, wasting 75% of Send Queue capacity
and increasing packet drops in high-PPS scenarios.

The root cause was different branching logic:
- credits_calculate() checked: num_sge == 1 && length <= 204
- fill_wqe() checked: total_data_len <= 204 (regardless of num_sge)

Fix by making credits_calculate() match fill_wqe() logic exactly:

1. Single-SGE fast path: Check length directly, no loop
   - If length <= 204: use inline formula
   - Else: return 1 credit (non-inline single SGE)

2. Multi-SGE path: Calculate total length to distinguish inline vs
   scatter-gather, ensuring accurate predictions
   - If total <= 204: use inline formula (up to 4 WQEBBs)
   - Else: use scatter-gather formula (typically 1 WQEBB for 2-SGE UDP)

Also fix test synchronization issues:
- Add SO_RCVTIMEO to prevent indefinite blocking
- Add wait_fork() to properly wait for child process

Signed-off-by: Tomer Cabouly <[email protected]>

Author: tomerdbz

src/core/dev/hw_queue_tx.h

@@ -129,23 +129,40 @@ class hw_queue_tx : public xlio_ti_owner {
          *  3. TSO packet with inline headers
          *
          * Formulas details:
-         *  1. WQEBB is 64 bytes, the 1st WQEBB contains ctrl segment, eth segment and 18 bytes of
-         *     inline data. So, we take the 1st WQEBB and number of WQEBBs for the packet minus 18
-         *     bytes.
-         *  2. Data segment for each scatter-gather element is 16 bytes. Therefore, WQEBB can hold
-         *     up to 4 data segments. The 1st element fits into the 1st WQEBB after the eth segment.
-         *     So, we take the 1st WQEBB and number of WQEBBs for scatter-gather elements minus 1.
-         *  3. Inline header starts from offset 46 in WQE (2 bytes before 16 bytes alignment).
-         *     Decrease inline header size by 2 to align it to 16 bytes boundary at the right edge.
-         *     This compensates data segments alignment. Add the 2 bytes back and length of
-         *     scatter-gather elements. Take into account that 18 bytes goes to the 1st WQEBB and
-         *     add the 1st WQEBB to the result.
+         *  1. Inline path (≤204 bytes): 18 bytes of ETH header are inlined into the eth segment.
+         *     WQEBB is 64 bytes, the 1st WQEBB contains ctrl segment (16B), eth segment (32B with
+         *     inline header), and the remaining packet data goes into inline data segment.
+         *     Formula (length + 63 - 18) / 64 + 1 accounts for the 18 bytes already inlined.
+         *
+         *  2. Scatter-gather path (>204 bytes): NO inline header (inline_hdr_sz = 0).
+         *     ETH segment is just 16 bytes (1 octoword). Data segment for each scatter-gather
+         *     element is 16 bytes. Therefore, WQEBB can hold up to 4 data segments.
+         *     The 1st data segment fits into the 1st WQEBB after ctrl (1 octoword) + eth (1
+         * octoword). Formula: (num_sge + 3 - 1) / 4 + 1 accounts for ctrl + eth + N data segments.
          */
         if (xlio_send_wr_opcode(*p_send_wqe) != XLIO_IBV_WR_TSO) {
-            if (p_send_wqe->num_sge == 1 && p_send_wqe->sg_list->length <= 204) {
-                return (p_send_wqe->sg_list->length + 63U - 18U) / 64U + 1U;
+            if (p_send_wqe->num_sge == 1) {
+                if (p_send_wqe->sg_list->length <= 204) {
+                    // Single-SGE inline path (fast path for small packets)
+                    return (p_send_wqe->sg_list->length + 63U - 18U) / 64U + 1U;
+                } else {
+                    // 1 data segment fits into 1 WQEBB
+                    return 1U;
+                }
             } else {
-                return (p_send_wqe->num_sge + 3U - 1U) / 4U + 1U;
+                // Multi-SGE: Calculate total to differentiate inline vs scatter-gather
+                uint32_t total_len = 0;
+                for (int i = 0; i < p_send_wqe->num_sge; i++) {
+                    total_len += p_send_wqe->sg_list[i].length;
+                }
+
+                if (total_len <= 204) {
+                    // Multi-SGE inline path
+                    return (total_len + 63U - 18U) / 64U + 1U;
+                } else {
+                    // Multi-SGE scatter-gather: 1 ctrl + 1 eth + N data segments
+                    return (p_send_wqe->num_sge + 3U - 1U) / 4U + 1U;
+                }
             }
         } else {
             return (((p_send_wqe->tso.hdr_sz + 15U - 2U) & ~15U) + 2U + p_send_wqe->num_sge * 16U -

tests/gtest/udp/udp_bind.cc

@@ -470,6 +470,13 @@ class pktinfo : public udp_base {
         int fd = socket(AF_INET6, SOCK_DGRAM, IPPROTO_IP);
         EXPECT_GT(fd, 0) << "Socket failed to open";
 
+        /* Set receive timeout to prevent indefinite blocking */
+        struct timeval timeout;
+        timeout.tv_sec = 10;
+        timeout.tv_usec = 0;
+        EXPECT_EQ(setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &timeout, sizeof(timeout)), 0)
+            << "Failed to set receive timeout";
+
         sockaddr_store_t server_any_sockaddr = {
             .addr6 = {
                 .sin6_family = AF_INET6,
@@ -551,5 +558,8 @@ TEST_F(pktinfo, check_recvmsg_returns_expected_pktinfo)
         };
 
         server_func(pid, server_code_under_test);
+
+        /* Wait for child process to complete and validate exit status */
+        ASSERT_EQ(0, wait_fork(pid));
     }
 }