[mythtv-users] ExternalRecorder - Errno 11

[Stephen Worthington]

On Wed, 22 Nov 2017 10:28:39 -0800, you wrote:

>On Tue, Nov 21, 2017 at 11:11 PM, Stephen Worthington <
>stephen_agent at jsw.gen.nz> wrote:
>
>> On Wed, 22 Nov 2017 06:03:31 +0000, you wrote:
>>
>> >Hi Everyone,
>> >
>> >I've been working on an external recorder for a while. It works most of
>> the
>> >time, but recently i've been getting new errors.
>> >
>> <snip>
>> >My problem below is that I am getting a "Resource temporarily unavailable
>> >(11)" when mythtv is trying to read from the stdin (the stdout of my
>> >program with the video stream)
>> >
>> >This seems to happen from this call:
>> >https://github.com/MythTV/mythtv/blob/master/mythtv/
>> libs/libmythtv/recorders/ExternalStreamHandler.cpp#L125
>> >
>> >The STDIN stream is configured to be non blocking, so the errno 11 isn't
>> >unexpected:
>> >https://github.com/MythTV/mythtv/blob/master/mythtv/
>> libs/libmythtv/recorders/ExternalStreamHandler.cpp#L343
>> >
>> >I am not sure what to change to keep this from happening.... it fails
>> about
>> >50% of the time, which is annoying.
>> >
>> >I am looking for advice on a fix, or how to troubleshoot. I am running
>> 0.29
>> >on Ubuntu 16.04.
>> >
>> >Thanks!
>> >Marc
>> >
>> <snip>
>>
>> If I am reading the ExternalStreamHandler.cpp.ExternIO::Read function
>> correctly, then the only way that can happen is if your stdout stream
>> indicates that it has data buffered and available to be read (the
>> poll() call from the Ready() call says there is data there), but then
>> your stdout fails to provide that data when it is called from read().
>> The most obvious reason I can think of for that would be if your code
>> was slow in responding to the read(), due to some unforeseen
>> circumstance such as garbage collection or being swapped out at the
>> time.  But I would have thought that your stdout buffer would have
>> been there waiting to be read, rather than any higher code needing to
>> be run to provide the data.  Are you using buffered I/O on your
>> stdout?
>>
>>
>Thanks for the quick reply Stephen. I am going to very quickly expose my
>ignorance on how these FDs work. (just setting expectations)
>
>Yes, I am using buffered output, specifically a BufferedWriter in Python (
>https://docs.python.org/2/library/io.html#io.BufferedWriter). I added
>explicit "flush" commands after the 'write' command, but it didn't help.
>
>I'll try dropping the buffered output tonight and see what happens. I
>didn't think that using a buffered writer would affect how data is present
>in the stream.
>
>Thanks,
>Marc

I would have hoped that using a BufferedWriter would have provided
more buffering rather than less, but it would be useful to try a
normal file instead, using the normal buffering, to see if it makes a
difference.  The problem with this situation is that the
ExternalStreamHandler.cpp code needs to be able to read the data from
the pipe buffer instantly, as it has been told that there is data
waiting.  So if the buffer the data is coming from is not locked in
RAM, it may not be instantly available.  The normal file I/O buffers
in Python are handled in the underlying C library and should work the
way ExternalStreamHandler.cpp expects them to, but it is vaguely
possible that BufferedWriter changes that and accessing its buffers
requires running the Python interpreter, which could cause this
problem.  The documentation of BufferedWriter is not specific enough
about how it works to know exactly how its low level buffering works,
but it is suggestive that the data may be lying in a higher level
buffer.