[mythtv] Android mediacodec woes

[David Engel]

On Tue, Jun 26, 2018 at 06:35:52PM -0400, Peter Bennett wrote:
> On 06/26/2018 05:32 PM, David Engel wrote:
> > I don't believe dropping every other frame would be acceptable.  I
> > know I wouldn't like it.
> > 
> > Are the timestamps in each frame correct?
> Yes - they are accurate, for the example I looked at. The frame times are
> pased through as equally spaced at 59.94 per second. (1501/90000 of a second
> alternating with 1502/90000 of a second) When playing the same file with
> software decoding I see 3003/90000 of a second on each frame.

That's good.

> > Some years ago, someone (I
> > forget who) reworked the seek table (and possibly markup table too),
> > to properly handle recordings which contained frame rate changes (*).
> > Some of that logic might need to be used here.  So when we want to
> > skip ahead one minute, instead of asking what frame am I on, ask what
> > is the time of the frame I'm on.
> There are no frame rate changes as such in this case, and I have to bear in
> mind that one frontend may be running on Linux and see a 29.97 frame rate
> and another frontend on Shield will see the same recording with a 59.94
> frame rate.

I think you misunderstood my idea.  I'll try to elaborate with some
pseudo code.

When we had a constant frame rate, we did something like this:

    When a new frame is played:

        cur_frame_num += 1

    To get or display the current time:

        cur_time = cur_frame_num * frame_rate

    To seek by some relative amount of time:

        new_frame_num = cur_frame_num + desired_skip_time * frame_rate
        file_offset = lookup_from_recordedseek(new_frame_num)

To support changing frame rates, we now do something like this:

    When a new frame is played:

        cur_frame_num += 1

    To get or display the current time:

        cur_time = compute_time_from_frame_num(cur_frame_num)

     To seek by some relative amount of time:

        cur_time = compute_time_from_frame_num(cur_frame_num)
        desired_time = cur_time + desired_skip_time
        file_offset, new_frame_num =
            lookup_from_recordedseek(desired_time)

With my idea, we could do something like this:

    When a new frame is played:

        cur_time += 1 / cur_frame_rate

    To get or display the current time:

        cur_time (we already have it)

    To seek by some relative amount of time:

        desired_time = cur_time + desired_skip_time
        file_offset, new_time =
            lookup_from_recordedseek(desired_time)

In other words, we keep direct track of time as we play frames instead
of computing it as needed from the frame number.  Ideally, we wouldn't
even need to know the current frame number anymore.  I think the
recordedmarkup table uses frame number, though, so we'd likely still
have to keep some notion of it, possibly by computing it from the
time.

Note that this is just an idea I'm throwing out there.  I don't know
if it's a good idea or not, so don't take this as any advocacy for it.

> > Not to pile on, but here's a couple more issues.  When we ff/rew, we
> > want to decode a single frame, skip ahead, decode a single frame, skip
> > ahead, repeat.  How much data do we need to feed the decoder to get
> > the desired frame out and can we detect and discard any undesired
> > frames?
> I have not got to that yet, I will look at it when I have finished with the
> frame rate problem, but see my comment below.
> 
> >   We might have to resort to some sort of hybrid solution where
> > we use software decoding for ff/rew and mediacodec for normal
> > decoding.
> > 
> This had occurred to me, but I did not see any easy way to achieve it.
> 
> Have you tried FF and REW with a progressive recording? I tried that and it
> seems to work correctly (but I don't use FF and REW normally so I am not
> familiar with what is normal). So I think that maybe once I have sorted out
> the timing, FF and REW may work without any more changes.

I haven't tried the latest patch yet.  With the earlier ones, though,
ff/rew was equally bad whether the content was progressive or
interlaced.  I'l try the latest patch, but it probably won't be until
tomorrow.

David
-- 
David Engel
david at istwok.net