<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p><br>
</p>
<br>
<div class="moz-cite-prefix">On 11/23/2017 12:50 AM, Marc Rawji
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAJ14JXpCzR3D7pPaOGEWw+o-GNY5BnBbz4--abUo6jm9iFXs4A@mail.gmail.com">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Wed, Nov 22, 2017 at 4:41 PM,
Stephen Worthington <span dir="ltr"><<a
href="mailto:stephen_agent@jsw.gen.nz" target="_blank"
moz-do-not-send="true">stephen_agent@jsw.gen.nz</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div class="gmail-HOEnZb">
<div class="gmail-h5">On Wed, 22 Nov 2017 10:28:39
-0800, you wrote:<br>
<br>
>On Tue, Nov 21, 2017 at 11:11 PM, Stephen
Worthington <<br>
><a href="mailto:stephen_agent@jsw.gen.nz"
moz-do-not-send="true">stephen_agent@jsw.gen.nz</a>>
wrote:<br>
><br>
>> On Wed, 22 Nov 2017 06:03:31 +0000, you
wrote:<br>
>><br>
>> >Hi Everyone,<br>
>> ><br>
>> >I've been working on an external recorder
for a while. It works most of<br>
>> the<br>
>> >time, but recently i've been getting new
errors.<br>
>> ><br>
>> <snip><br>
>> >My problem below is that I am getting a
"Resource temporarily unavailable<br>
>> >(11)" when mythtv is trying to read from
the stdin (the stdout of my<br>
>> >program with the video stream)<br>
>> ><br>
>> >This seems to happen from this call:<br>
>> ><a
href="https://github.com/MythTV/mythtv/blob/master/mythtv/"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://github.com/MythTV/<wbr>mythtv/blob/master/mythtv/</a><br>
>> libs/libmythtv/recorders/<wbr>ExternalStreamHandler.cpp#L125<br>
>> ><br>
>> >The STDIN stream is configured to be non
blocking, so the errno 11 isn't<br>
>> >unexpected:<br>
>> ><a
href="https://github.com/MythTV/mythtv/blob/master/mythtv/"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://github.com/MythTV/<wbr>mythtv/blob/master/mythtv/</a><br>
>> libs/libmythtv/recorders/<wbr>ExternalStreamHandler.cpp#L343<br>
>> ><br>
>> >I am not sure what to change to keep this
from happening.... it fails<br>
>> about<br>
>> >50% of the time, which is annoying.<br>
>> ><br>
>> >I am looking for advice on a fix, or how
to troubleshoot. I am running<br>
>> 0.29<br>
>> >on Ubuntu 16.04.<br>
>> ><br>
>> >Thanks!<br>
>> >Marc<br>
>> ><br>
>> <snip><br>
>><br>
>> If I am reading the
ExternalStreamHandler.cpp.<wbr>ExternIO::Read function<br>
>> correctly, then the only way that can happen
is if your stdout stream<br>
>> indicates that it has data buffered and
available to be read (the<br>
>> poll() call from the Ready() call says there
is data there), but then<br>
>> your stdout fails to provide that data when
it is called from read().<br>
>> The most obvious reason I can think of for
that would be if your code<br>
>> was slow in responding to the read(), due to
some unforeseen<br>
>> circumstance such as garbage collection or
being swapped out at the<br>
>> time. But I would have thought that your
stdout buffer would have<br>
>> been there waiting to be read, rather than
any higher code needing to<br>
>> be run to provide the data. Are you using
buffered I/O on your<br>
>> stdout?<br>
>><br>
>><br>
>Thanks for the quick reply Stephen. I am going to
very quickly expose my<br>
>ignorance on how these FDs work. (just setting
expectations)<br>
><br>
>Yes, I am using buffered output, specifically a
BufferedWriter in Python (<br>
><a
href="https://docs.python.org/2/library/io.html#io.BufferedWriter"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://docs.python.org/2/<wbr>library/io.html#io.<wbr>BufferedWriter</a>).
I added<br>
>explicit "flush" commands after the 'write'
command, but it didn't help.<br>
><br>
>I'll try dropping the buffered output tonight and
see what happens. I<br>
>didn't think that using a buffered writer would
affect how data is present<br>
>in the stream.<br>
><br>
>Thanks,<br>
>Marc<br>
<br>
</div>
</div>
I would have hoped that using a BufferedWriter would have
provided<br>
more buffering rather than less, but it would be useful to
try a<br>
normal file instead, using the normal buffering, to see if
it makes a<br>
difference. The problem with this situation is that the<br>
ExternalStreamHandler.cpp code needs to be able to read
the data from<br>
the pipe buffer instantly, as it has been told that there
is data<br>
waiting. So if the buffer the data is coming from is not
locked in<br>
RAM, it may not be instantly available. The normal file
I/O buffers<br>
in Python are handled in the underlying C library and
should work the<br>
way ExternalStreamHandler.cpp expects them to, but it is
vaguely<br>
possible that BufferedWriter changes that and accessing
its buffers<br>
requires running the Python interpreter, which could cause
this<br>
problem. The documentation of BufferedWriter is not
specific enough<br>
about how it works to know exactly how its low level
buffering works,<br>
but it is suggestive that the data may be lying in a
higher level<br>
buffer.<br>
<div class="gmail-HOEnZb">
<div class="gmail-h5"><br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Just gave it a go and no joy, same behavior as
before...</div>
<div><br>
</div>
<div>In my Python code, the salient part is:</div>
<div>data = fd.read(256)<br>
</div>
<div>
<div>if data:</div>
<div> if (self.last_write_time is None):</div>
<div> self.logger.debug("First bytes written to
stdout")</div>
<div> self.__bin_stdout.write(data)</div>
<div> self.__bin_stdout.flush()</div>
<div> self.last_write_time = time.time()</div>
<div>self.writer_control.wait(0.5)</div>
</div>
<div><br>
</div>
<div>and self.__bin_stdout is equal to sys.stdout.</div>
<div><br>
</div>
<div>Would it have to do with:</div>
<div><a
href="http://pubs.opengroup.org/onlinepubs/9699919799/functions/poll.html"
moz-do-not-send="true">http://pubs.opengroup.org/onlinepubs/9699919799/functions/poll.html</a><br>
</div>
<div><br>
</div>
<div>It indicates that for streams, the POLLIN is set even
if the message is of zero length.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Marc</div>
<div> </div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
</blockquote>
I suggest creating a ticket for this. <br>
<br>
You may want to look at the import recorder. I recently committed a
fix that allows it to be used for recordings. See the fix at
<a class="moz-txt-link-freetext" href="https://code.mythtv.org/trac/ticket/13114">https://code.mythtv.org/trac/ticket/13114</a> and documentation at
<a class="moz-txt-link-freetext" href="https://www.mythtv.org/wiki/Import_recorder">https://www.mythtv.org/wiki/Import_recorder</a> . Currently the fix is
only available in master, scheduled for v30.<br>
<br>
Peter<br>
</body>
</html>