[mythtv] Playback next steps

[Peter Bennett]

On 12/17/18 6:21 AM, Mark Kendall wrote:
>
>     --- On Fri, 14/12/18, David Engel <david at istwok.net
>     <mailto:david at istwok.net>> wrote:
>
>     > I didn't
>     > realize mythavtest was rotting.  I thought it was a
>     > "simpler"
>     > wrapper for
>     > tv_play/mythplayer without the rest of the frontend.  Is
>     > that not the case?
>
>
> No - it isn't rotting, I just extended it a little - for double rate 
> deinterlacing and gpu decoder usage.
> I wrote mythavtest to just play video at the fastest possible rate 
> (using the current display profile) with no audio and no a/v sync. You 
> need to turn off any sync to vblank that the driver may be using. When 
> using opengl, there are also a number of environment variables that 
> you can set to disable various functionality to test different code paths.
>
>     > There are some platforms like the second
>     > generation firetv stick and
>     > new, mi box,
>     > that are limited in some way such that uyvy doesn't
>     > work
>     > acceptably.  YV12 currently works on
>     > them for some definitions of
>     > work.
>
>
> I suspect it is either the extra CPU processing or the extra 
> framebuffer stage that is currently needed for UYVY.
>
>     On Fri, 14/12/18, Peter Bennett <pb.mythtv at gmail.com
>     <mailto:pb.mythtv at gmail.com>> wrote:
>
>     Linear blend looks fine to
>      me with YV12. That is what I use. Maybe I am
>      missing something.
>      One thing to
>      note - using kernel on some android devices has a
>      performance impact, it starts dropping frames -
>      that does not happen
>      with linear blend.
>
>
> YV12 linear blend is currently not double rate at all - so presumably 
> you are only running single rate?
Hmm - I selected Linear blend 2X and it seemed to be better than plain 
linear blend. Maybe not?
> As discussed later, kernel is much more GPU intensive - so not 
> surprising that it starts to drop frames.
>
>
>     It sounds useful to me. At one point I
>      temporarily added some debug code
>      to print
>      the OpenGL shader code used. The MythTV source applies many
>
>      dynamic changes to the shader code before
>      sending it to OpenGL, and it
>      was difficult
>      for me to know what the code that actually ran looked
>      like.
>
>
> The full and final shader code is already dumped to the logs with 
> debug level logging.
>
>     His code originally was for full screen OpenGL ES
>      and required a customized QT
>      build. Some
>      people did not like the softblend so I did some strange
>      stuff with the OpenGL ES OSD. What we have now
>      is x11 based QT
>      displaying the GUI in a X11
>      window, OpenMAX video displaying the
>      playback using the full-screen OpenMAX API, and
>      OpenGL ES displaying the
>      OSD using the
>      OpenGL ES full-screen API. For OpenMAX and OpenGL it does
>
>      some calculations to put the video and the
>      OSD into the correct place to
>      position it
>      on the QT window, to give the illusion that it is actually
>
>      all in a window.
>
>      In versions of Raspberry Pi Raspbian starting
>      from 2018, using the
>      OpenGL ES OSD causes
>      severe slowdowns in the video playback, even if
>      nothing is visible in the OSD, so it has become
>      useless and we reverted
>      to the softblend
>      OSD.
>
>      Piotr O has his own
>      build of Raspberry Pi Mythfrontend which uses
>      full-screen QT specially built for the purpose.
>      I don't know how that
>      all works.
>
>
> I'm currently compiling master with an up to date raspbian stretch 
> lite with slight modifications to enable ES2.0 and QT5 opengl (both 
> are currently disabled if not an android build).
>
> I noticed that the rasbpian qt5 is built with eglfs/egl support - will 
> see whether that actually means broadcom specific eglfs - I don't 
> think it does - in which case I'll cross compile with the correct 
> configuration.
> Qt5 eglfs should just work without issue (it works on my debian/mesa 
> desktop). There is then work to be done to use the OpenMax egl code 
> when using eglfs and use the 'regular' opengl osd. For x11, we should 
> stick with the current approach and disable the EGL OSD if running 
> with eglfs.
>
> I think there are also some fixes needed to the configure script. At 
> the moment I think it looks for gles2.0 support in the qt spec to 
> enable gles2.0 - but gles2.0 support appears to be available when egl 
> is available. Not sure of the correct permutations here - and the 
> configure script makes my brain melt.
>
> I suspect that the best performance (and memory consumption etc) will 
> only be achieved if using straight eglfs - i.e. no X11. It's a shame 
> no-one provides a PPA for the right QT5.
>
>
>     > software fallback - why bother unless you
>      have a modern CPU and a 15 year old GPU.
>      Some reasons for using software decode
>      - VDPAU has a bug with decoding MPEG2 that
>      results in pixellation on
>      many USA
>      stations.
>      - fire stick 2g mediacodec has a
>      bug where deinterlaced content causes
>      the
>      decoder to hang.
>      - Subtitles are not working
>      for some decoders. They work with software
>      decoding, for those people who need them.
>
>
> To be clear, this is in reference to software fallback for YUV to RGB 
> conversion - this already assumes software decode and is not related 
> to hardware accelerated decode.
>
>      Note there is an android problem with UYVY. In
>      some devices (e.g. fire
>      stick g2), OpenGL
>      ES does not support float precision highp and defaults
>      to mediump. The OpenGL code that applies the
>      color suffers a rounding
>      error and instead
>      of each pixel getting its correct color, each
>      alternate pixel gets the color for its neighbor
>      instead, on the right
>      hand half of the
>      screen. See https://imgur.com/dLoMUau and
>     https://imgur.com/lbfyEWQ . I
>      don't know why YV12 does not suffer from
>      that problem.
>
>
> Yes - definitely a precision issue - but is it always exactly half way 
> across the screen or at different positions? If always exactly half 
> way regardless of source width, it sounds more like an 'off by one' error.
>
I think it is actually about two thirds of the way across the screen 
that it starts. Other details -
The jaggies occur with both interlaced and progressive video.
In file mythrender_opengl2es.h there is the line
m_qualifiers = "precision highp float;\n";
This was previously set as mediump - and the jaggies occurred on the 
fire TV  and fire TV 4k.
After I changed this to highp -
On fire TV there was an error message that highp is not supported and it 
is reverting to mediump, and the jaggies remain
On fire TV 4K the jaggies went away and the picture is perfect
> But the issue may be moot...
>
> Since I wrote that summary, I've realised that both YV12 and and UYVY 
> are subject to the 'interlaced chroma upsampling' bug. I fixed this 
> with the default, pre-UYVY code and then obviously forgot about it:)
>
> The fix for UYVY is pretty straightforward. Don't use FFmpeg for 
> upsampling (known issue with FFmpeg), use the Mythtv packing code and 
> don't try and pack two pixels into one sample. Actually simplifies the 
> code and removes the need for the extra filter stage. Would benefit 
> from some NEON specific code - then most platforms will have SIMD 
> optimisations (I doubt there are many platforms not covered by either 
> MMX or NEON).
>
> Not so sure about YV12. I think the shaders are trying to deal with 
> it. But it doesn't work and it is also resampling progressive content 
> as well. If the frame is sampled for interlaced chroma, it needs 
> proper repacking regardless of whether it is deinterlaced or not - and 
> progressive content needs 'regular' resampling/packing.
>
> With the UYVY change in mind, I'd actually propose the following:-
>
> Drop support for the software YUV to RGB case.
>  - it only actually works with OpenGL1 as the only requirement for 
> GL2/ES2 to work is shaders - which are mandatory per the spec.
>
> Drop the opengl-lite option
>   - if the UYVY code is changed, I can't see any benefit to the lite 
> code. The shaders are simple and using the apple/arb extensions for 
> lite just means you lose all colourspace, colour standard control.
>
> This then leaves you with UYVY and YV12 - which I would change to 
> profiles(OpenGL-YV12 and OpenGL-UYVY) rather than settings per se.
Regarding the settings - the code was looking at environment variables 
to decide if either of these was to be disabled. I added the settings 
recently because there seemed no easy way to pass in environment 
variables with android.
> Makes the interface cleaner (no settings) and you essentially have one 
> profile that is more CPU/memory transfer costly (UYVY) and one that is 
> more GPU costly (YV12). Users can then decide which works best for 
> their system.
>
Sounds good - at some point we need to add support for 10-bit color, 
which maybe would be a third option?
> The 'extra filter stage' could no doubt go (if still needed) after 
> UYVY changes.
>
I added a setting to extra filter stage because on fire tv 4k, randomly, 
playback would just show a black screen. Adding that extra stage solved 
it, but I do not understand what was happening. If I pulled up an OSD 
display, I could see the video playing through the transparent parts of 
the OSD, but the rest of the screen remained black.
> thoughts?
>

This is mostly way over my head, but it sounds good.

My recommendation is that we should only commit this into master after 
we have created the fixes/30 branch in January. The changes sound major 
and they could potentially cause some unexpected issues with some 
configurations. I would not want the users who install from Ubuntu 19.04 
to encounter problems.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mythtv.org/pipermail/mythtv-dev/attachments/20181217/5eb269a8/attachment.html>