[mythtv] OpenGL, Android and Fire Stick

Thu Oct 11 08:57:44 UTC 2018

On 11/10/2018 00:08, Mark Spieth wrote:
> 
> On 11/10/18 07:11, Peter Bennett wrote:
>>
>>
>> On 10/08/2018 11:12 PM, Peter Bennett wrote:
>>>
>>>
>>> On 10/08/2018 06:20 PM, Mark Spieth wrote:
>>>>
>>>>
>>>> On 09/10/18 02:07, Peter Bennett wrote:
>>>>> With the fire stick I find that video decoding is fast enough for
>>>>> regular 1x playback of MPEG2, H264 and H265 up to 1080. However
>>>>> OpenGL presents a variety of problems.
>>>>>
>>>> This is a S905 I seem to remember.
>>>>> There are three sets of shaders for video playback:
>>>>> yv12: On the fire stick playback is restricted to 15-20 fps, giving
>>>>> a rather unpleasant viewing experience. Also there is a fat green
>>>>> line along the bottom (as we observed also with shield) and a green
>>>>> line at the right.
>>>> yv12 should be better because it has less data per frame! WTF
>>>> Fat green line could be 1088 lines or upscale for a lower res. Green
>>>> is 0 data and beyond the frame range.
>>>>> uyvu: On the fire stick playback speed is much better, but the
>>>>> right hand side of the picture has low resolution. Starting from
>>>>> the center of the picture, moving to the right the resolution
>>>>> gradually reduces, resulting in jagged edges and loss of detail
>>>>> towards the right of the picture.
>>>>> rgb:   On the fire stick playback speed is good, as it is for uyvu.
>>>>> There is no picture resolution problem on the right or anywhere,
>>>>> there is no green line at the bottom, but the colors are wrong. I
>>>>> thing the red color is missing entirely.
>>>> This resolution issue is very wierd. Normally this is yuvu or yuv2.
>>>> not sure it uyvu is a typo.
>>> The code has a macro called MYTHTV_UYVY and a field called uyvy. It
>>> is related to definition "AV_PIX_FMT_UYVY422 packed YUV 4:2:2, 16bpp,
>>> Cb Y0 Cr Y1" in ffmpeg
>>>>>
>>>>> The fire stick is capable of reasonable playback speed and of good
>>>>> quality picture, but the OpenGL needs some changes. I think part of
>>>>> it may be the differences between OpenGL and OpenGL ES.
>>>> I think the implementation of GL/accel HW is not to standard. Not a
>>>> GLES issue per se.
>>>>>
>>>>> I have little understanding of OpenGL, so I am going through the
>>>>> OpenGL ES book and examples, to try to get an understanding of what
>>>>> goes on in those shaders, so that I can create shaders that works
>>>>> with good performance and quality in this situation. At the same
>>>>> time I hope to eliminate the green line problem so that yv12 can be
>>>>> used when needed.
>>>> See my greenline patches from a while back. I still run with them
>>>> and it fixes the green lines on my shield and h96 pro (S912). I can
>>>> resend i that helps. May not fix the thick green line though.
>>>>
>>> Is that the patch
>>> https://code.mythtv.org/trac/attachment/ticket/13230/20180402-yuv-greenline-interlaced.patch
>>> ? I will look at at that also and see if it fits in with what I do.
>>>> There may also be a scaling issue with sone GL implementations in
>>>> that the iterate over lines/pixels 0..N instead of 0..N-1 and this
>>>> causes the green lines bottom and right.
>>>>
>>>> The other critical patch may be the float resolution. I changed this
>>>> from medium to high and that fixed quite a few issues too. Give that
>>>> a try. From my reading, float res is not specified, some
>>>> hardware/impl is better than others for mediump and lowp.
>>>>
>>>> libmythui/mythrender_opengl2es.h
>>>> change "precision mediump float" to "precision highp float"
>>>>
>>>> I also think the deinterlace gets the line ordering for fields wrong
>>>> too. This may be an amlogic specific issue.
>>>>
>>> Thanks - I will try these and see how they work on the fire tv. There
>>> may be differences in level of support, so it may be helpful to add
>>> settings to select different ways of doing things. For example the
>>> precision highp may not be supported in all cases. The settings may
>>> be a bit opaque to users but we can document in the wiki the best
>>> combinations to use for different situations.
>>>
>>> Peter
>>
>>
>> This is the unhelpful result of using highp:
>> I/mfe     (19351): 2018-10-10 14:03:42.789211 E  0:2: P0004: High
>> precision not supported, instead compiling high precision as medium
>> precision
>>
>> My issue seems to be with this opengl es code:
>>
>> void main(void)
>> {
>>     vec4 yuva    = texture2D(s_texture0, v_texcoord0);
>>     if (fract(v_texcoord0.x * 1024.00000000) < 0.5)
>>         yuva = yuva.rabg;
>>     gl_FragColor = vec4(yuva.arb, 1.0) * m_colourMatrix;
>> }
>>
>> The shader uses a different formula for even and odd numbered pixels
>> because two pixels' color information is packed into 1 byte.
>> The fragment shader gets the pixel location (texcoord0.x) as a float
>> between 0 and 1. It seems to not have enough precision to cater for
>> 2048 pixels which is the texture size being used. Once you get past
>> pixel 1024 in the x axis it cannot tell the difference between odd
>> numbered and even numbered pixels, so the right hand half of the
>> picture gets messy.
>>
> Peter,
> 
> There are minimum resolutions defined in the GL/GLES standard for
> low,medium and high when I looked at them. Some medium implementations
> were better than others.
> 
> see
> https://stackoverflow.com/questions/4414041/what-is-the-precision-of-highp-floats-in-glsl-es-2-0-for-iphone-ipod-touch-ipad.
> 
> and http://www.khronos.org/files/opengles_shading_language.pdf section
> 4.5.2
> 
> GLES2 defines float and int min low 8 bits, medium 10 bits (our
> problem), high optional and min 16 bits.
> This restricts the iterator v_texcoord0 to 10 bits since high does not
> work (optional not supported) and this is inadequate for FHD+.
> 
> Looks like this device implementation is adhering to the minimum
> standards for resolution. And since high does not work, this hardware
> should be designated as too old to work at FHD res.
> 
> Even converting this to integers the precision is still probably the
> same so you don't gain anything. The only thing you could do is split
> the textures in 2 (or more) with a base x,y vec, but that will get
> cumbersome fast.
> 
> Calls to the glGetShaderPrecisionFormat() for all 3 will show the
> constraints of the system.
> May be a good check to determine minimum GL support for various
> resolutions of framebuffers/tvs.

Would the OpenGL(ES) profile levels give us sufficient information
or would we still need to query each shaders precision format?

Just thinking it may be an easier way of drawing a line for
underpowered devices (and throw the relevant warnings)

Regards
Stuart