srt-live-transmit use of file://con only receives first 13160 bytes on windows builds

[dnastrain]

When piping data in via stdin to srt-live-transmit, it works as expected on linux, but not on windows builds, where is seems to consistently only send (or receive) the 1st 13160 bytes.

Linux Source-side:
ffmpeg -re -i ./data/big_buck_bunny_480p_surround-fix.ts -codec copy -f mpegts pipe:1 | ./srt-live-transmit -v file://con srt://:5555

Linux Target-side:
./srt-live-transmit -v srt://127.0.0.1:5555 file://con > ./data2/output.ts
-or-
./srt-live-transmit -v srt://127.0.0.1:5555 file://con | ffplay -i pipe:0

The above receives the 'entire transmission' successfully on the target-side output.ts file.

Repeating the test using windows builds...

Windows Source-side:
ffmpeg.exe -re -i ./data/big_buck_bunny_480p_surround-fix.ts -codec copy -f mpegts pipe:1 | srt-live-transmit.exe -v file://con srt://:5555

Windows Target-side:
srt-live-transmit.exe -v srt://127.0.0.1:5555 file://con > ./data2/output-win.ts
-or-
srt-live-transmit.exe -v srt://127.0.0.1:5555 file://con | ffplay.exe -i pipe:0

The above only receives the 1st 13160 bytes of the output stream file.

[dnastrain]

I have a "windows-only" workaround/fix that am testing -- I plan to send a PR tomorrow.

[maxsharabayko]

Looks related to #407

[maxsharabayko]

@dnastrain Could you please describe the issue in more details? Do you receive only the first 13160 bytes from file://con and nothing else then, or do you get errors after the first 13160 bytes received?

According to your fix #651, the issue is related to the stdin buffer size, which is 4096 bytes on Windows. In that case reading more data from stdin should also be used on Linux as well, because there is still a change to loose it on high bitrates.
What do you think about using the setbuf instead or in addition?

[dnastrain]

@maxlovic, what I observed building the srt master-branch code as-is, is that the Windows Source-Side successfully reads (and sends) 10*1316 bytes from stdin, and then all subsequent calls to srt_epoll_wait return -1

[maxsharabayko]

@dnastrain In case of file://con the following is done:

case UriParser::FILE:
	if (srt_epoll_add_ssock(pollid,
		src->GetSysSocket(), &events))

where

int ConsoleSource::GetSysSocket() { return 0; };

On unix (POSIX), stdin is a valid FD with value 0. But not on Windows. This means that the first time epoll hits on Windows is when the connection is done for SRT target. After that you will not receive any other events from the poll. Furthermore, if you have connected and there is nothing in the stdin buffer waiting to be read, you will not read anything at all, because the polling event will never happen.
So I guess in case of UriParser::FILE == src->uri.type() no polling should be done on Windows. Just after the connection is established you should go inside the loop every time.

[dnastrain]

@maxlovic, ok your findings make sense!

I would think that ConsoleSource should use this then? :

int ConsoleSource::GetSysSocket() { return STDIN_FILENO; };

And ConsoleTarget use this then? :

int ConsoleTarget::GetSysSocket() { return STDOUT_FILENO; };

Regarding my PR #651, I agree it is a work-around -- which your explanation highlights actually only works when the stdin buffer is being filled faster than srt-live-transmit can read the 10*1316 bytes during each loop iteration.

Would you like me to abandon the PR, so someone can implement the proper fix in the near future?

[maxsharabayko]

@dnastrain Maybe for windows __fileno should be used. Still it will return 0 for stdin.

Please do not close your PR untill the solution is found.

I am thinking the best way might be to move source polling into the Source class and call GetNumberOfConsoleInputEvents on Windows instead of polling on the stdin file descriptor. The function does not have a timeout value though. The same for std::cin.peek(). But some sleep can be used to reduce the loop freqency. 🤔
Also the tricky part is that at the start of srt-live-transmit, the target is also being polled to catch the connection event. Right now both source and target are in the same poll.

Some time is required to find a nice solution. Do you have ideas?

[dnastrain]

Hi @maxlovic, I don't have better ideas at this point. I experimented with both WaitForSingleObjectEx and GetNumberOfConsoleInputEvents, using the handle from GetStdHandle(STD_INPUT_HANDLE), but the behavior is about the same at this point.

[maxsharabayko]

Hi @dnastrain.
Roman (@rndi) and I discussed this issue. Might be the only way to solve it on Windows is to have a separate thread that will check stdin fullness and report back to main thread, maybe via a conditionable variable or atomic_bool. Something close to what you've done in #651, but when srcReadMore is false, that thread should do the job, keep checking stdin for more input data. When srcReadMore the checking thread should be idle.

[dnastrain]

Hi @maxlovic, I actually updated the PR a few minutes ago, just slightly, to poll stdin if the target is connected, rather than assume that more data is ready in stdin when 10*1316 was read. It works, while still feeling like a workaround