[mythtv-users] schedule howto

[James Linder]

> On Dec 9, 2023, at 11:47, Stephen Worthington <stephen_agent at jsw.gen.nz> wrote:
> 
> On Sat, 9 Dec 2023 05:18:34 +0800, you wrote:
> 
>> 
>> 
>>> On Dec 9, 2023, at 00:27, Mike Perkins <mikep at randomtraveller.org.uk> wrote:
>>> 
>>> On 08/12/2023 15:56, Stephen Worthington wrote:
>>>> On Fri, 8 Dec 2023 20:55:31 +0800, you wrote:
>>>>> G’day all
>>>>> playing with the rules I can solve once, but everytime is a pain 
> 
>>>>> 
>>>>> Aprogram (news on Australian ABC) scheduled to record from 1900 to 2000 (ish)
>>>>> Bprogram scheduled 1930 to 2230
>>>>> 
>>>>> Aprogram (daily) can be on any combination of 2, 20, 21, 22, 23
>>>>> My hardware causes some virtual channels to sometimes record broken recordings (badly pixelated) as a result I’ve set <no> virtual channels.
>>>>> As suggested I put “prefer HD”
>>>>> 
>>>>> Now Aprogram records on tuner one, two, three, four
>>>>> One recording of News is [red] conflict
>>>>> One recording of Bprogram is [red] conflict
>>>>> 
>>>>> Wunce-upon-a-time I recall a rule “record 1 showing only”. That with “Prefer HD” would completely solve my problem. What is the solution? Better tuners, they are Haupauge USB dual tuners.
>>>>> 
>>>>> James
>>>>> 
>>>>> PS I can't give prority to any 2, 20-23, that channel may not carry Aprogram
>>>> How sure are you that the pixelation is caused by virtual tuners? Most
>>>> people do not have that problem, and there are plenty of users of
>>>> Hauppauge USB tuners.  The way that virtual tuners work is just to
>>>> allow more than one set of streams to be recorded at the same time
>>>> from the same tuner.  So the tuner hardware gets told to send streams
>>>> a,b, and c to the PC for one channel and streams f, g, and h for
>>>> another channel.  It sends all those streams at once to the CPU.  The
>>>> CPU sorts out which streams are for which channel and writes them to
>>>> separate recording files.  The only problem you could get from getting
>>>> the tuner to send more streams is if the total bit rate exceeded the
>>>> maximum for USB 2.0, which is not possible for a DVB-T transmitter,
>>>> especially an Australian one that operates on 7 MHz bandwidth where
>>>> the rest of the world mostly uses 8 MHz.
>>>> When you set virtual tuners to 0, all you are doing is telling
>>>> mythbackend to record from effectively one virtual tuner, in other
>>>> words, record all the streams for one channel only from the tuner. But
>>>> the same code is used as when multiple virtual tuners is specified.
>>>> So your problem is more likely to be at the other end of things - if
>>>> you write too many recordings at once to a hard drive, you will
>>>> eventually get to the point where the heads will not be able to move
>>>> between the file locations fast enough to keep up, and mythbackend's
>>>> recording buffers will overflow before all the data can be written to
>>>> disk.  Which causes gaps in the data, which you see as pixellation and
>>>> missing bits, and it also causes glitches in the sound.  If this is
>>>> happening, you should be getting bad recordings for all the recordings
>>>> going to the overloaded hard disk during the time it was overloaded. I
>>>> have been there and done that.
>>>> So how many recordings are you writing to disk when you are getting
>>>> pixilated recordings?  How many disks are they going to?  How fast are
>>>> the disks?
>>>> If this is what your problem is, then the solution is normally just to
>>>> had another hard drive to the PC.  I have 7 recording drives in order
>>>> to prevent this problem.  To be completely sure not to get overloaded
>>>> hard drives, I work on the basis that there should be only 2
>>>> recordings at once going to one hard drive.  That then allows for
>>>> playback of another recording from the same drive while those two
>>>> recordings are in progress.  In theory, the faster more recent hard
>>>> drives should be able to cope with 3 recordings at once plus playback,
>>>> or maybe more, but I have never tested that.  I still have 2 old hard
>>>> drives that probably can not do that, but I keep them as they are very
>>>> quiet compared to more modern ones, so I can use them for the
>>>> relatively few overnight recordings and not have the hard drive noises
>>>> keep me awake.
>>>> Along with recordings and playback, there can be other hard disk
>>>> activity for commercial skip processing.  Modern CPUs can normally
>>>> cope with doing commercial skip processing in real-time from the
>>>> in-memory recording buffers on the basis of one core per recording.
>>>> Such processing does not cause any extra hard disk activity.  If you
>>>> are making more recordings at once than you have CPU cores, you
>>>> normally set the number of commercial skip processes to the number of
>>>> cores, and the extra recordings get scheduled for later processing. In
>>>> which case, the commercial skip processes will be reading back those
>>>> recordings from hard disk at a speed significantly greater than for
>>>> playback or recording.  And that can interfere with any recordings
>>>> going on at the time as the heads will be moving back and forth quite
>>>> a bit.  I do not have this problem as I do not do any commercial skip
>>>> processing - it does not work usefully on any New Zealand channels
>>>> that I know of.
>>> There is a possible confusion here between virtual /tuners/ and virtual /channels/. I agree that whether a tuner is 'virtual' or not is going to make any difference, it is all the same code.
>>> 
>>> However, virtual /channels/ could be a different problem, depending on specific language. In the US I know they call those 'subchannels' and it is likely that some or most have lower bitrates to cram more subchannels in. I don't know if the same applies in AU, but that could be the problem.
>>> 
>>> Alternatively, it is simply possible that the signal is not strong enough and sometimes drops out. Perhaps a tweak to the antenna direction, check all the joints for soundness and/or water, etc? That might go a long way to solving the problem.
>>> 
>> 
>> How I wish sane logic would prevail.
>> 
>> Here be what happens:
>> 
>> With Virtual Tuners about 1 recording a week is ‘damaged’ with bad pixelation
>> With my bizare setting I’ve seen NO damaged recordings in 5 months.
>> All the storage is SSD
>> OS and all the associated stuff is on a samsung 980 M2 (2T)
>> All myth including Videos is on a samsung QVO SATA (4T)
>> CPU is Ryzen 7 5700U (8 core) with 24G ram (I use this box as as workstation during the day)
>> 
>> The Oz TV (successfully !) channels defeat commercial skipping by changing the algorithm even for a single show  so I disable all commercial skipping.
>> 
>> Please wax lyrical on virtual channels. How could I tell a virtual channel from a mux?
> 
> There is no such thing as a virtual channel.

This was posted and I did not understand

>>> However, virtual /channels/ could be a different problem, depending on specific language. In the US I know they call those 'subchannels' and it is likely that some or most have lower bitrates to cram more subchannels in. I don't know if the same applies in AU, but that could be the problem.
>>> 
>>> Alternatively, it is simply possible that the signal is not strong enough and sometimes drops out. Perhaps a tweak to the antenna direction, check all the joints for soundness and/or water, etc? That might go a long way to solving the problem.

>  All channels on digital
> TV are exactly the same technically - they are simply a collections of
> streams of data.  The streams are normally 1 video, 1 or more audio
> (eg AAC stereo and AC3 5.1, more for different languages) and 0 or
> more for subtitles.  Each stream consists of packets of data with the
> same PID (packet ID) value at the start of the packet.  There are
> occasional packets that are repeated regularly that give the data
> about what PIDs are part of each channel that is being broadcast, and
> some other streams that are not part of channels such as for EIT
> and/or MHEG5 EPG data.
> 
> In ATSC regions, there is political nomenclature that confuses this
> simple technical setup, where a mux will typically be owned by a TV
> station and will have one major channel in HD and high bit rate, and
> then one or more other channels, typically SD and lower bit rate.
> These two types of channels are differentiated by the language used to
> describe them, and possibly also by legislation, but technically they
> are no different to each other.
> 
>> Definitely there is some fiddling:
>> The usual bitrate on 1080 is cica 1000 Kb/s
>> Last year ABC had a live documentary of coral spawning where they raised the bitrate to over 2mb/s
>> I did not check what they did on other channels to compensate
>> 
>> [diversion] coral spawning is special: on 1 moonlit night per year at the right time ALL the coral spawn. On the barrier reef that means simultainous spawning over 1000s km.
>> 
>> There is no logic on damaged channels. It could be the sole recording being made.
>> I have tested 8 simultainous recordings all without any problem.
>> 
>> I use an external antenna some 10 km line-of-sight to the transmitter farm. We have various ‘filler’ (low power) transmitters round Perth. I carefully pruned them from my list.
>> 
>> James
> 
> The Samsung 870 QVO 4 Tbyte specifications say it has a 4 Gbyte DRAM
> cache.  What the specs carefully do not say is the actual speed of the
> flash memory (especially the write and erase speeds).  So whenever you
> fill the DRAM cache, I would expect a very significant slow down in
> the write speed.  When you get bad recordings, is it always when you
> have many simultaneous recordings happening?  Is it possible that you
> have exceeded the cache size as a result?
> 
> How full is the QVO drive normally?

About 60%, 5% unallocated, trim daily cron

>  When a flash drive is fairly
> full, it has few spare blocks available that can be pre-erased and put
> on the writeable queue.  If all the blocks on the writeable queue have
> been used up, then a no longer in use block will need to be erased
> before it can be written to and the erase times are horrendously
> slower than the write times - they can be seconds, not milliseconds.
> The recommendation for formatting any flash SSD is that you leave 10%
> of the space as non-partitioned, so that 10% of the flash blocks are
> available to be erased and put on the writeable queue, even when the
> rest of the SSD is full.  Did you do that?
> 
> When you tested with 8 simultaneous recordings, did you leave them
> running for the typical recording time of 1 hour + pre- and post-roll
> times?

Almost certainly not. probably 15 minutes (6 months ago, memory is sus’)

>  If not, then you do not really know if that situation could
> cause problems.  I have found in my testing of spinning rust that the
> internal buffering by mythbackend is sufficient to cope with my
> typical pre- and post-roll settings causing the number of simultaneous
> recordings to double during the pre/post roll overlaps, as long as the
> overlaps are not more than about 4-5 minutes.  So I always do test
> recordings that are longer than that, about 15 minutes.  To fill a 4
> Gbyte cache if the data was not being written to flash at all would
> take about 1.3 hours of one of the NZ HD channels.
> 
> When you have had a sole recording that was damaged, was it possible
> that it was made following a very busy set of multiple recordings that
> might have overflowed the DRAM cache or the available erased blocks?
> Or was it completely on its own?

Again 6 months ago!

> 
> One other thing to think about with USB tuners is their connections
> and cables.  USB plugs, cables and sockets vary in their tolerances,
> and do not always fit very well.  When they are not a close fit,
> movement of the cable can cause disconnections of pins, which can
> cause the same sort of bad recordings you are having.  I used to have
> this problem when I was using multiple USB tuners, and typically it
> was one or two of the USB tuners out of the three I was using that had
> the problem.  Changing the cables on the bad tuners to better fitting
> ones and switching the USB ports they were plugged into greatly
> reduced the incidence of bad recordings, as did laying out the tuners
> and cables carefully.  My ultimate fix for this was to change to using
> PCIe tuners.  I now keep the old USB tuners as emergency backups and
> for use on my laptop when I am away from home.
> 

The dog might have nested in the wires, a zillion things, so I need to carefully rinse and repeat.
Will do in the next week.
What about the original question, specially if cache size is the issue. I have one record rule that may record 1 to 5 copies, typically 2 to 4. How to impliment ‘record 1 showing’ ‘prefer HD’ ? 

James