[mythtv-users] schedule howto

Stephen Worthington stephen_agent at jsw.gen.nz
Sat Dec 9 03:47:53 UTC 2023

On Sat, 9 Dec 2023 05:18:34 +0800, you wrote:

>> On Dec 9, 2023, at 00:27, Mike Perkins <mikep at randomtraveller.org.uk> wrote:
>> On 08/12/2023 15:56, Stephen Worthington wrote:
>>> On Fri, 8 Dec 2023 20:55:31 +0800, you wrote:
>>>> G’day all
>>>> playing with the rules I can solve once, but everytime is a pain 

>>>> Aprogram (news on Australian ABC) scheduled to record from 1900 to 2000 (ish)
>>>> Bprogram scheduled 1930 to 2230
>>>> Aprogram (daily) can be on any combination of 2, 20, 21, 22, 23
>>>> My hardware causes some virtual channels to sometimes record broken recordings (badly pixelated) as a result I’ve set <no> virtual channels.
>>>> As suggested I put “prefer HD”
>>>> Now Aprogram records on tuner one, two, three, four
>>>> One recording of News is [red] conflict
>>>> One recording of Bprogram is [red] conflict
>>>> Wunce-upon-a-time I recall a rule “record 1 showing only”. That with “Prefer HD” would completely solve my problem. What is the solution? Better tuners, they are Haupauge USB dual tuners.
>>>> James
>>>> PS I can't give prority to any 2, 20-23, that channel may not carry Aprogram
>>> How sure are you that the pixelation is caused by virtual tuners? Most
>>> people do not have that problem, and there are plenty of users of
>>> Hauppauge USB tuners.  The way that virtual tuners work is just to
>>> allow more than one set of streams to be recorded at the same time
>>> from the same tuner.  So the tuner hardware gets told to send streams
>>> a,b, and c to the PC for one channel and streams f, g, and h for
>>> another channel.  It sends all those streams at once to the CPU.  The
>>> CPU sorts out which streams are for which channel and writes them to
>>> separate recording files.  The only problem you could get from getting
>>> the tuner to send more streams is if the total bit rate exceeded the
>>> maximum for USB 2.0, which is not possible for a DVB-T transmitter,
>>> especially an Australian one that operates on 7 MHz bandwidth where
>>> the rest of the world mostly uses 8 MHz.
>>> When you set virtual tuners to 0, all you are doing is telling
>>> mythbackend to record from effectively one virtual tuner, in other
>>> words, record all the streams for one channel only from the tuner. But
>>> the same code is used as when multiple virtual tuners is specified.
>>> So your problem is more likely to be at the other end of things - if
>>> you write too many recordings at once to a hard drive, you will
>>> eventually get to the point where the heads will not be able to move
>>> between the file locations fast enough to keep up, and mythbackend's
>>> recording buffers will overflow before all the data can be written to
>>> disk.  Which causes gaps in the data, which you see as pixellation and
>>> missing bits, and it also causes glitches in the sound.  If this is
>>> happening, you should be getting bad recordings for all the recordings
>>> going to the overloaded hard disk during the time it was overloaded. I
>>> have been there and done that.
>>> So how many recordings are you writing to disk when you are getting
>>> pixilated recordings?  How many disks are they going to?  How fast are
>>> the disks?
>>> If this is what your problem is, then the solution is normally just to
>>> had another hard drive to the PC.  I have 7 recording drives in order
>>> to prevent this problem.  To be completely sure not to get overloaded
>>> hard drives, I work on the basis that there should be only 2
>>> recordings at once going to one hard drive.  That then allows for
>>> playback of another recording from the same drive while those two
>>> recordings are in progress.  In theory, the faster more recent hard
>>> drives should be able to cope with 3 recordings at once plus playback,
>>> or maybe more, but I have never tested that.  I still have 2 old hard
>>> drives that probably can not do that, but I keep them as they are very
>>> quiet compared to more modern ones, so I can use them for the
>>> relatively few overnight recordings and not have the hard drive noises
>>> keep me awake.
>>> Along with recordings and playback, there can be other hard disk
>>> activity for commercial skip processing.  Modern CPUs can normally
>>> cope with doing commercial skip processing in real-time from the
>>> in-memory recording buffers on the basis of one core per recording.
>>> Such processing does not cause any extra hard disk activity.  If you
>>> are making more recordings at once than you have CPU cores, you
>>> normally set the number of commercial skip processes to the number of
>>> cores, and the extra recordings get scheduled for later processing. In
>>> which case, the commercial skip processes will be reading back those
>>> recordings from hard disk at a speed significantly greater than for
>>> playback or recording.  And that can interfere with any recordings
>>> going on at the time as the heads will be moving back and forth quite
>>> a bit.  I do not have this problem as I do not do any commercial skip
>>> processing - it does not work usefully on any New Zealand channels
>>> that I know of.
>> There is a possible confusion here between virtual /tuners/ and virtual /channels/. I agree that whether a tuner is 'virtual' or not is going to make any difference, it is all the same code.
>> However, virtual /channels/ could be a different problem, depending on specific language. In the US I know they call those 'subchannels' and it is likely that some or most have lower bitrates to cram more subchannels in. I don't know if the same applies in AU, but that could be the problem.
>> Alternatively, it is simply possible that the signal is not strong enough and sometimes drops out. Perhaps a tweak to the antenna direction, check all the joints for soundness and/or water, etc? That might go a long way to solving the problem.
>How I wish sane logic would prevail.
>Here be what happens:
>With Virtual Tuners about 1 recording a week is ‘damaged’ with bad pixelation
>With my bizare setting I’ve seen NO damaged recordings in 5 months.
>All the storage is SSD
>OS and all the associated stuff is on a samsung 980 M2 (2T)
>All myth including Videos is on a samsung QVO SATA (4T)
>CPU is Ryzen 7 5700U (8 core) with 24G ram (I use this box as as workstation during the day)
>The Oz TV (successfully !) channels defeat commercial skipping by changing the algorithm even for a single show  so I disable all commercial skipping.
>Please wax lyrical on virtual channels. How could I tell a virtual channel from a mux?

There is no such thing as a virtual channel.  All channels on digital
TV are exactly the same technically - they are simply a collections of
streams of data.  The streams are normally 1 video, 1 or more audio
(eg AAC stereo and AC3 5.1, more for different languages) and 0 or
more for subtitles.  Each stream consists of packets of data with the
same PID (packet ID) value at the start of the packet.  There are
occasional packets that are repeated regularly that give the data
about what PIDs are part of each channel that is being broadcast, and
some other streams that are not part of channels such as for EIT
and/or MHEG5 EPG data.

In ATSC regions, there is political nomenclature that confuses this
simple technical setup, where a mux will typically be owned by a TV
station and will have one major channel in HD and high bit rate, and
then one or more other channels, typically SD and lower bit rate.
These two types of channels are differentiated by the language used to
describe them, and possibly also by legislation, but technically they
are no different to each other.

>Definitely there is some fiddling:
>The usual bitrate on 1080 is cica 1000 Kb/s
>Last year ABC had a live documentary of coral spawning where they raised the bitrate to over 2mb/s
>I did not check what they did on other channels to compensate
>[diversion] coral spawning is special: on 1 moonlit night per year at the right time ALL the coral spawn. On the barrier reef that means simultainous spawning over 1000s km.
>There is no logic on damaged channels. It could be the sole recording being made.
>I have tested 8 simultainous recordings all without any problem.
>I use an external antenna some 10 km line-of-sight to the transmitter farm. We have various ‘filler’ (low power) transmitters round Perth. I carefully pruned them from my list.

The Samsung 870 QVO 4 Tbyte specifications say it has a 4 Gbyte DRAM
cache.  What the specs carefully do not say is the actual speed of the
flash memory (especially the write and erase speeds).  So whenever you
fill the DRAM cache, I would expect a very significant slow down in
the write speed.  When you get bad recordings, is it always when you
have many simultaneous recordings happening?  Is it possible that you
have exceeded the cache size as a result?

How full is the QVO drive normally?  When a flash drive is fairly
full, it has few spare blocks available that can be pre-erased and put
on the writeable queue.  If all the blocks on the writeable queue have
been used up, then a no longer in use block will need to be erased
before it can be written to and the erase times are horrendously
slower than the write times - they can be seconds, not milliseconds.
The recommendation for formatting any flash SSD is that you leave 10%
of the space as non-partitioned, so that 10% of the flash blocks are
available to be erased and put on the writeable queue, even when the
rest of the SSD is full.  Did you do that?

When you tested with 8 simultaneous recordings, did you leave them
running for the typical recording time of 1 hour + pre- and post-roll
times?  If not, then you do not really know if that situation could
cause problems.  I have found in my testing of spinning rust that the
internal buffering by mythbackend is sufficient to cope with my
typical pre- and post-roll settings causing the number of simultaneous
recordings to double during the pre/post roll overlaps, as long as the
overlaps are not more than about 4-5 minutes.  So I always do test
recordings that are longer than that, about 15 minutes.  To fill a 4
Gbyte cache if the data was not being written to flash at all would
take about 1.3 hours of one of the NZ HD channels.

When you have had a sole recording that was damaged, was it possible
that it was made following a very busy set of multiple recordings that
might have overflowed the DRAM cache or the available erased blocks?
Or was it completely on its own?

One other thing to think about with USB tuners is their connections
and cables.  USB plugs, cables and sockets vary in their tolerances,
and do not always fit very well.  When they are not a close fit,
movement of the cable can cause disconnections of pins, which can
cause the same sort of bad recordings you are having.  I used to have
this problem when I was using multiple USB tuners, and typically it
was one or two of the USB tuners out of the three I was using that had
the problem.  Changing the cables on the bad tuners to better fitting
ones and switching the USB ports they were plugged into greatly
reduced the incidence of bad recordings, as did laying out the tuners
and cables carefully.  My ultimate fix for this was to change to using
PCIe tuners.  I now keep the old USB tuners as emergency backups and
for use on my laptop when I am away from home.

More information about the mythtv-users mailing list