[mythtv] Re: Proposed alternate behaviour for show delete

[Daniel Manjarres]

Hi guys,

Well just to give you a data point:

I have a 700GB LVM dedicated to mythtv and only mythtv. My music and
video files are stored elsewhere. I have autoexpire turned on by
default, but several recording settings have it turned back off, the
example I will use here is "The Daily Show" which I have been
recording every episode of since around July 2003. Between then and
now Myth has gone through some unstable times when the backend would
crash, and my cable service sometimes goes out, and sometimes the same
guest is on twice meaning the episode title is the same, and it would
be falsely marked as a duplicate. So I have TDS set to always record,
no matter what, with priority over all other shows. I get lots of
copies of re-runs, just in case something bad happened when one of
them was being recorded. I delete the extra copies whenever I get
around to it. I have sometimes wanted to undelete a duplicate when I
realized the begining or end of the show was cut-off in the copy I
chose to save, but that's not a big deal. So undelete is nice for me,
since having extra copies taking up disk space doesn't interfere with
any other programs (and if these cheap IDE drives fail extra copies
means more data security actually).

BUT I will repeat for you the tale of woe regarding my database
evolution failure from a few weeks ago. I noticed queries were slow,
investigated why, and saw some differences between my database schema
and the default. I tried to speed things up by changing the schema
manually, and  somehow the recording storage settings wound up all
mixed-up, not matching the recorded shows, and the backend proceded to
DELETE 200GB OF EPISODES of TDS, spaning a historically important 8
month period of time, before I 'kill -9'ed it. It did this under the
"max episodes" feature from some mixed-up profile, not the autoexpire
feature. Anyway there was around 50GB of free space on the LVM
filesystem at the time, so there was no need to hard-delete anything.

I understand that some people need to maintain a certain level of free
disk space, for other data. So I have some crude suggestions. I think
these cover every possible situation (yeah right) from an
INFRASTRUCTURE point of view, not necessarily a policy POV.

1) Don't run autoexpire every n minutes. Run a simple disk space check
every 10 seconds, If the disk space is too low delete a single show
from the "about_to_delete queue". If there are no entries in the
about_to_delete queue, THEN run autoexpire (& possibly some
other_auto_delete_methods) to try to move shows into the
delete_candidate queue, and then run some method of choosing shows
from the delete_candidate queue to move into the about_to_delete
queue. Finally, if disk space miraculously comes back, push shows from
the about_to_delete queue back into the delete_candidate queue.

2) The metric for "disk space is too low" above could be selectable from
a) filesystem is near full (same as current autoexpire)
b) delete queue is larger than n GBs (make this zero for hard deletes)
c) Mythtv is using more than n GBs for all recordings (usefull for
managing filesystems shared with other data, rather than just plopping
all files together with no forethought)
d) logical ANDs of the above

3) Profit



3) Profit

4) add a (possibly hidden) panic feature. If more than n shows are in
the delete queue for any reason, refuse to delete them. This saves
people in case of bugs or unexpected behavior changes. If the panic is
in error and the user really wants to delete that much crap see (5).

5) This is really important. Add a group to the watch/delete
recordings screen with the contents of the delete queues (when
non-empty). Maybe make it potentially visually annoying, as in: the
more crap in there the more it catches your eye. Allow users to delete
everything in there in one-shot with a "Are you sure?" confirmation
This gives notification and manual confirmation to every auto delete
method, and together with (4) would have saved my ass, not just the
asses of people with autoexpire complaints.

Rationale: (my carpal tunnel is hurting so this will not be good
english.  my apologies)

a) splitting the delete queue in half: max episodes will allways put
the oldest episode in the delete_candidate queue. This way the user
can see IN ADVANCE what will be deleted in the watch/delete recording
screen. Otherwise this would always be empty in the GUI, as it would
only get filled in right before it was about to be cleared out.

b) splitting the delete queue in half: The method of moving
recorrdings from delete_candidate to about_to_delete can be NULL,
meaning mythtv does all the thinking about what to delete, but needs
user confirmation. This saves time scrolling through my 700+
recordings looking for what to nix. Think of this as deltetion
suggestions, like recording suggestions.

c) using a periodically serviced queue at all: you never know; disk
space might be temporarily used, but quickly freed up by those other
programs mythtv is coexisting with, say while ripping a bunch of CDs
or DVDs. And if that happens we don't want to keep deleting shows once
the disk usage of the other program has dropped. Deleting 1 show every
10 seconds is surely freeing up disk space faster than a DVD can get
ripped. This makes it a soft threshold, not a hard threshold, be
aware.

d) profit is good

e) panic feature: works for me. Switches from auto-delete to deletion
suggestions mode. Easy to implement. This description of panic is
incomplete, and broken, but a better way is easy to figure out.

f) various "disk space is too low criteria": these allow you to manage
disk space a lot better. Say you want to shrink the LVM volume:
instead of manually deleteing stuff randomly  you can just set a max
disk space limit, which puts shows in the delete_candidate queue,
possiblly setting of the panic feature. You confirm, and disk space is
released. Then you can resize the LVM volume with no problem. Delete
suggestions mode saves lots of keypresses.

g) displaying the delete_candidate queue: hell if you don't like it
make it optional.

h) having undelete in the first place: because we can adapt the
program to the needs of all kinds of users in all kinds of scenarios
without annoying the "hard delete" advocates.

i) many others: can't type anymore, lots more in mind tho