[mythtv] OpenGL, Android and Fire Stick

Wed Oct 10 23:08:23 UTC 2018

On 11/10/18 07:11, Peter Bennett wrote:
>
>
> On 10/08/2018 11:12 PM, Peter Bennett wrote:
>>
>>
>> On 10/08/2018 06:20 PM, Mark Spieth wrote:
>>>
>>>
>>> On 09/10/18 02:07, Peter Bennett wrote:
>>>> With the fire stick I find that video decoding is fast enough for 
>>>> regular 1x playback of MPEG2, H264 and H265 up to 1080. However 
>>>> OpenGL presents a variety of problems.
>>>>
>>> This is a S905 I seem to remember.
>>>> There are three sets of shaders for video playback:
>>>> yv12: On the fire stick playback is restricted to 15-20 fps, giving 
>>>> a rather unpleasant viewing experience. Also there is a fat green 
>>>> line along the bottom (as we observed also with shield) and a green 
>>>> line at the right.
>>> yv12 should be better because it has less data per frame! WTF
>>> Fat green line could be 1088 lines or upscale for a lower res. Green 
>>> is 0 data and beyond the frame range.
>>>> uyvu: On the fire stick playback speed is much better, but the 
>>>> right hand side of the picture has low resolution. Starting from 
>>>> the center of the picture, moving to the right the resolution 
>>>> gradually reduces, resulting in jagged edges and loss of detail 
>>>> towards the right of the picture.
>>>> rgb:   On the fire stick playback speed is good, as it is for uyvu. 
>>>> There is no picture resolution problem on the right or anywhere, 
>>>> there is no green line at the bottom, but the colors are wrong. I 
>>>> thing the red color is missing entirely.
>>> This resolution issue is very wierd. Normally this is yuvu or yuv2. 
>>> not sure it uyvu is a typo.
>> The code has a macro called MYTHTV_UYVY and a field called uyvy. It 
>> is related to definition "AV_PIX_FMT_UYVY422 packed YUV 4:2:2, 16bpp, 
>> Cb Y0 Cr Y1" in ffmpeg
>>>>
>>>> The fire stick is capable of reasonable playback speed and of good 
>>>> quality picture, but the OpenGL needs some changes. I think part of 
>>>> it may be the differences between OpenGL and OpenGL ES.
>>> I think the implementation of GL/accel HW is not to standard. Not a 
>>> GLES issue per se.
>>>>
>>>> I have little understanding of OpenGL, so I am going through the 
>>>> OpenGL ES book and examples, to try to get an understanding of what 
>>>> goes on in those shaders, so that I can create shaders that works 
>>>> with good performance and quality in this situation. At the same 
>>>> time I hope to eliminate the green line problem so that yv12 can be 
>>>> used when needed.
>>> See my greenline patches from a while back. I still run with them 
>>> and it fixes the green lines on my shield and h96 pro (S912). I can 
>>> resend i that helps. May not fix the thick green line though.
>>>
>> Is that the patch 
>> https://code.mythtv.org/trac/attachment/ticket/13230/20180402-yuv-greenline-interlaced.patch 
>> ? I will look at at that also and see if it fits in with what I do.
>>> There may also be a scaling issue with sone GL implementations in 
>>> that the iterate over lines/pixels 0..N instead of 0..N-1 and this 
>>> causes the green lines bottom and right.
>>>
>>> The other critical patch may be the float resolution. I changed this 
>>> from medium to high and that fixed quite a few issues too. Give that 
>>> a try. From my reading, float res is not specified, some 
>>> hardware/impl is better than others for mediump and lowp.
>>>
>>> libmythui/mythrender_opengl2es.h
>>> change "precision mediump float" to "precision highp float"
>>>
>>> I also think the deinterlace gets the line ordering for fields wrong 
>>> too. This may be an amlogic specific issue.
>>>
>> Thanks - I will try these and see how they work on the fire tv. There 
>> may be differences in level of support, so it may be helpful to add 
>> settings to select different ways of doing things. For example the 
>> precision highp may not be supported in all cases. The settings may 
>> be a bit opaque to users but we can document in the wiki the best 
>> combinations to use for different situations.
>>
>> Peter
>
>
> This is the unhelpful result of using highp:
> I/mfe     (19351): 2018-10-10 14:03:42.789211 E  0:2: P0004: High 
> precision not supported, instead compiling high precision as medium 
> precision
>
> My issue seems to be with this opengl es code:
>
> void main(void)
> {
>     vec4 yuva    = texture2D(s_texture0, v_texcoord0);
>     if (fract(v_texcoord0.x * 1024.00000000) < 0.5)
>         yuva = yuva.rabg;
>     gl_FragColor = vec4(yuva.arb, 1.0) * m_colourMatrix;
> }
>
> The shader uses a different formula for even and odd numbered pixels 
> because two pixels' color information is packed into 1 byte.
> The fragment shader gets the pixel location (texcoord0.x) as a float 
> between 0 and 1. It seems to not have enough precision to cater for 
> 2048 pixels which is the texture size being used. Once you get past 
> pixel 1024 in the x axis it cannot tell the difference between odd 
> numbered and even numbered pixels, so the right hand half of the 
> picture gets messy.
>
Peter,

There are minimum resolutions defined in the GL/GLES standard for 
low,medium and high when I looked at them. Some medium implementations 
were better than others.

see 
https://stackoverflow.com/questions/4414041/what-is-the-precision-of-highp-floats-in-glsl-es-2-0-for-iphone-ipod-touch-ipad.
and http://www.khronos.org/files/opengles_shading_language.pdf section 4.5.2

GLES2 defines float and int min low 8 bits, medium 10 bits (our 
problem), high optional and min 16 bits.
This restricts the iterator v_texcoord0 to 10 bits since high does not 
work (optional not supported) and this is inadequate for FHD+.

Looks like this device implementation is adhering to the minimum 
standards for resolution. And since high does not work, this hardware 
should be designated as too old to work at FHD res.

Even converting this to integers the precision is still probably the 
same so you don't gain anything. The only thing you could do is split 
the textures in 2 (or more) with a base x,y vec, but that will get 
cumbersome fast.

Calls to the glGetShaderPrecisionFormat() for all 3 will show the 
constraints of the system.
May be a good check to determine minimum GL support for various 
resolutions of framebuffers/tvs.

I would suggest bin (trash) it. Too many cheap more powerful devices on 
the market.

HTH
Mark