file mode + Buffer API, payloads above certain size not arriving over LTE, but works over DSL

[oviano]

I've been tracking a really odd issue where sometimes under file mode, the client connection would break due to a timeout.

I set up a simple test scenario where the client connects to the server and they play ping-pong; the client sends the first message, the server sends a reply, the client replies and so on. The server replies send a message of increasing payload size starting at 1000 bytes and increasing by 100 bytes each time, stopping at 2400 bytes.

When the client connects to the server out through my DSL connection, this works as expected.

However, when it connects out through a LTE connection, this breaks when the server sends a payload of 1500 bytes. What happens is the server sends it, but the client never receives it, and eventually the client times-out.

I have put together heavy logging for both scenarios. All I can see is it seems to relate to when the packet has to be split.

Client log (over DSL):
https://ovcollyer.synology.me:5001/d/f/504866555218108425

Server log (over DSL):
https://ovcollyer.synology.me:5001/d/f/504866419712729093

Client log (over LTE):
https://ovcollyer.synology.me:5001/d/f/504866562782535691

Server log (over LTE):
https://ovcollyer.synology.me:5001/d/f/504866419853238279

[oviano]

So this was caused by the default value of SRTO_MSS of 1500 being wrong for my server internet connection, which is BT Fibre and has an MTU of 1492.

When I change SRTO_MSS to 1492 the problem goes away. I didn't see the problem with Live mode because the packets are never big enough (7 * 188 + headers).

However, since SRTO File mode + Buffer API could be described as analogous to TCP, I wonder if this ought to be handled automatically somehow. It's not something that needs setting in TCP, or does TCP just have a lower "safe" value or something?

[oviano]

...although, it seems from reading up a bit, that larger UDP packets should be simply fragmented in any case, so I'm still not too sure why they are being dropped in this case.

[oviano]

So looking into at the code, I can see that SRT/UDT splits packets up according to MSS (addBuffer) so my question then is does that mean it is also disabling UDP's own fragmenting/reassembling capability? I cannot see where...

From the trace, for the problem packet I can see that PB_FIRST is never received on the client, only PB_LAST, which suggests that packets that are too big for the path MTU are being dropped, instead of split and then reassembled.

Is this expected behaviour, a bug, or something about my configuration (maybe my router is preventing fragmenting or something)?

[maxsharabayko]

There are two variables: MTU size (referred as CUDT::m_iMSS) and payload size (CUDT::m_iMaxSRTPayloadSize).
A payload will be split on two packets, when the size exceeds the payload. So if you set MTU 1500 and Payload to 1456, then SRT will actually try to send that data.
If you reduce MTU to 1492, the maximum payload size will also be reduced.

[oviano]

Right, thanks.

What I'm taking from this is that using MSS 1500 when my server is on a DSL connection with MTU 1492 means that even though SRT is not splitting packets, the router/path is fragmenting it.

Usually this is not a problem, other than being less efficient, but I assume in the case of the LTE connection, something is preventing the fragmented packets from all arriving and being reassembled. I read up a bit yesterday and although not that common there are a few scenarios where fragments of a UDP packet do not all arrive at the destination (one example I read about was where load balancing treated fragments as unrelated to each other, i.e. sending them off to different places).

I don't think there is a problem with SRT as firstly in live mode, the packets are much less likely to hit the MTU limit (frequently 7 * 188 + header) so it's more a potential issue for file mode.

I guess this is one scenario where SRT in file mode + buffer API does differ a little from what I'm used to with TCP. With TCP the MSS would be figured out as part of the protocol, so this would never be an issue. With SRT clearly it's something that could catch you out in file mode if you set/leave MSS larger than the path MTU and somewhere along the path fragmented packets are not handled correctly.