<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Wed, Nov 22, 2017 at 4:41 PM, Stephen Worthington <span dir="ltr"><<a href="mailto:stephen_agent@jsw.gen.nz" target="_blank">stephen_agent@jsw.gen.nz</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail-HOEnZb"><div class="gmail-h5">On Wed, 22 Nov 2017 10:28:39 -0800, you wrote:<br>
<br>
>On Tue, Nov 21, 2017 at 11:11 PM, Stephen Worthington <<br>
><a href="mailto:stephen_agent@jsw.gen.nz">stephen_agent@jsw.gen.nz</a>> wrote:<br>
><br>
>> On Wed, 22 Nov 2017 06:03:31 +0000, you wrote:<br>
>><br>
>> >Hi Everyone,<br>
>> ><br>
>> >I've been working on an external recorder for a while. It works most of<br>
>> the<br>
>> >time, but recently i've been getting new errors.<br>
>> ><br>
>> <snip><br>
>> >My problem below is that I am getting a "Resource temporarily unavailable<br>
>> >(11)" when mythtv is trying to read from the stdin (the stdout of my<br>
>> >program with the video stream)<br>
>> ><br>
>> >This seems to happen from this call:<br>
>> ><a href="https://github.com/MythTV/mythtv/blob/master/mythtv/" rel="noreferrer" target="_blank">https://github.com/MythTV/<wbr>mythtv/blob/master/mythtv/</a><br>
>> libs/libmythtv/recorders/<wbr>ExternalStreamHandler.cpp#L125<br>
>> ><br>
>> >The STDIN stream is configured to be non blocking, so the errno 11 isn't<br>
>> >unexpected:<br>
>> ><a href="https://github.com/MythTV/mythtv/blob/master/mythtv/" rel="noreferrer" target="_blank">https://github.com/MythTV/<wbr>mythtv/blob/master/mythtv/</a><br>
>> libs/libmythtv/recorders/<wbr>ExternalStreamHandler.cpp#L343<br>
>> ><br>
>> >I am not sure what to change to keep this from happening.... it fails<br>
>> about<br>
>> >50% of the time, which is annoying.<br>
>> ><br>
>> >I am looking for advice on a fix, or how to troubleshoot. I am running<br>
>> 0.29<br>
>> >on Ubuntu 16.04.<br>
>> ><br>
>> >Thanks!<br>
>> >Marc<br>
>> ><br>
>> <snip><br>
>><br>
>> If I am reading the ExternalStreamHandler.cpp.<wbr>ExternIO::Read function<br>
>> correctly, then the only way that can happen is if your stdout stream<br>
>> indicates that it has data buffered and available to be read (the<br>
>> poll() call from the Ready() call says there is data there), but then<br>
>> your stdout fails to provide that data when it is called from read().<br>
>> The most obvious reason I can think of for that would be if your code<br>
>> was slow in responding to the read(), due to some unforeseen<br>
>> circumstance such as garbage collection or being swapped out at the<br>
>> time. But I would have thought that your stdout buffer would have<br>
>> been there waiting to be read, rather than any higher code needing to<br>
>> be run to provide the data. Are you using buffered I/O on your<br>
>> stdout?<br>
>><br>
>><br>
>Thanks for the quick reply Stephen. I am going to very quickly expose my<br>
>ignorance on how these FDs work. (just setting expectations)<br>
><br>
>Yes, I am using buffered output, specifically a BufferedWriter in Python (<br>
><a href="https://docs.python.org/2/library/io.html#io.BufferedWriter" rel="noreferrer" target="_blank">https://docs.python.org/2/<wbr>library/io.html#io.<wbr>BufferedWriter</a>). I added<br>
>explicit "flush" commands after the 'write' command, but it didn't help.<br>
><br>
>I'll try dropping the buffered output tonight and see what happens. I<br>
>didn't think that using a buffered writer would affect how data is present<br>
>in the stream.<br>
><br>
>Thanks,<br>
>Marc<br>
<br>
</div></div>I would have hoped that using a BufferedWriter would have provided<br>
more buffering rather than less, but it would be useful to try a<br>
normal file instead, using the normal buffering, to see if it makes a<br>
difference. The problem with this situation is that the<br>
ExternalStreamHandler.cpp code needs to be able to read the data from<br>
the pipe buffer instantly, as it has been told that there is data<br>
waiting. So if the buffer the data is coming from is not locked in<br>
RAM, it may not be instantly available. The normal file I/O buffers<br>
in Python are handled in the underlying C library and should work the<br>
way ExternalStreamHandler.cpp expects them to, but it is vaguely<br>
possible that BufferedWriter changes that and accessing its buffers<br>
requires running the Python interpreter, which could cause this<br>
problem. The documentation of BufferedWriter is not specific enough<br>
about how it works to know exactly how its low level buffering works,<br>
but it is suggestive that the data may be lying in a higher level<br>
buffer.<br>
<div class="gmail-HOEnZb"><div class="gmail-h5"><br></div></div></blockquote><div><br></div><div>Just gave it a go and no joy, same behavior as before...</div><div><br></div><div>In my Python code, the salient part is:</div><div>data = fd.read(256)<br></div><div><div>if data:</div><div> if (self.last_write_time is None):</div><div> self.logger.debug("First bytes written to stdout")</div><div> self.__bin_stdout.write(data)</div><div> self.__bin_stdout.flush()</div><div> self.last_write_time = time.time()</div><div>self.writer_control.wait(0.5)</div></div><div><br></div><div>and self.__bin_stdout is equal to sys.stdout.</div><div><br></div><div>Would it have to do with:</div><div><a href="http://pubs.opengroup.org/onlinepubs/9699919799/functions/poll.html">http://pubs.opengroup.org/onlinepubs/9699919799/functions/poll.html</a><br></div><div><br></div><div>It indicates that for streams, the POLLIN is set even if the message is of zero length.</div><div><br></div><div>Thanks,</div><div>Marc</div><div> </div></div></div></div>