[mythtv-users] Database issue? Recordings not showing up in frontend

Tue Sep 15 17:01:56 UTC 2015

References: <02310C79-E563-4DF4-B4EA-7D673E801055 at wackywombats.com> <op.x4trbdrfb49nse at localhost> <97FC1249-36FB-42FC-8F2D-3883D8B90387 at wackywombats.com> <op.x4ueamz0b49nse at study> <2FF6D4A9-6A16-46A9-A8F0-79E076FEB717 at wackywombats.com> <op.x4u8llbcb49nse at localhost> <EB0DE880-8EB8-4F70-A19F-A592766450C7 at wackywombats.com> <op.x4u945izb49nse at localhost> <A137FEBD-0E12-4D98-9218-375238BCDB54 at wackywombats.com> <CAOOwNt+cWDBeaYA1bFhU9W2jXOoftuctm9fJ+tp3Z6J7kj-Tnw at mail.gmail.com> <5899A26C-67EA-4E4A-8131-2ACC17E26FAB at wackywombats.com> <277617979.20150914060250 at gmail.com> <50A83A3C-672E-4866-A5D0-0D63A0F2D19B at wackywombats.com> <op.x4y1w5pjb49nse at localhost> <77776338-69CD-44C7-A4AB-914A9D2CB04A at wackywombats.com> <op.x4y8prhlb49nse at localhost> <55F77865.20204 at thirdcontact.com> <CAKfgc2J4FXDmHThxK6f=c8-Lr7k=1i5A3RQXSQ8XVAt-2DFiyA at mail.gmail.com> <55F81C94.9000302 at thirdcontact.com> <55F81D80.90103 at thirdcontact.co m> <A6B4E51F-E45B-4991-9F75-A48C46684708 at wackywombats.com> <55F848FC.5040100 at thirdcon
  tact.com> <D28DC99F-2936-4DDF-9547-153E116A98DE at wackywombats.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Return-Path: hikavdh at gmail.com
X-OriginalArrivalTime: 15 Sep 2015 17:04:22.0135 (UTC) FILETIME=[912CC870:01D0EFD8]

Hoi Paul,

Tuesday, September 15, 2015, 6:46:55 PM, you wrote:

>> On Sep 15, 2015, at 9:36 AM, Michael T. Dean <mtdean at thirdcontact.com> w=
rote:
>>=20
>> On 09/15/2015 11:21 AM, Paul Stillwell wrote:
>>> On Sep 15, 2015, at 6:30 AM, Michael T. Dean wrote:
>>>> On 09/15/2015 09:26 AM, Michael T. Dean wrote:
>>>>> On 09/14/2015 10:44 PM, Mike Holden wrote:
>>>>>> But deletepending appears to be 1 for the records posted earlier? No=
t sure I understand the comments about corrupted databases?
>>>>> Ah, you're right. I misread it because of the bad wrapping and though=
t the filesize values were the deletepending values.
>>>>>=20
>>>>> So, a 1 means that the backend was in the process of deleting them. B=
ecause they're not actually deleting, that means that something very bad ha=
ppened to the backend when it was in the process and they got stuck in a li=
mbo state, where they're not deleted, but they're not available.
>>>> BTW, something very bad like a very ugly shutdown (power outage/kill -=
KILL/...) at a very bad time (the tiny amount of time when deletion is in a=
 dangerous state) right after the recordings were deleted.
>>> OK, looks like this is resolved for me. It looks like the recordings we=
re stuck somehow and I was able to recover them by cleaning up the deletepe=
nding flag.
>>>=20
>>> The very bad thing that happened to my database
>>=20
>> Actually, I didn't explain it well, but the very bad thing has to happen=
 to mythbackend--the database itself can't cause the problem. It could poss=
ibly even be cause by a very badly-timed backend crash, so maybe that happe=
ned and something automatically restarted it for you, so you didn't realize?
>>=20
>>>  was find_orphans.py. I=E2=80=99m not sure what went wrong with it, but=
 everything was fine until I ran that and then everything wasn=E2=80=99t fi=
ne.
>>=20
>> find_orphans.py just asked mythbackend to delete all your recordings (me=
tadata).  It did this because your recording files were not there--likely b=
ecause you had misconfigured (or hadn't yet configured/updated) your Storag=
e Groups or your file systems referenced by your Storage Group directory li=
sts weren't mounted/accessible (i.e. permissions or whatever) by the backen=
d(s) when find_orphans.py was run.
>>=20
>>>  Now that I think about it, wasn=E2=80=99t there some scripts that we w=
ere not supposed to use anymore when we went to 0.27? I have some nagging t=
hought in the back of my head about something changing and some scripts not=
 working anymore, but I can=E2=80=99t remember.
>>>=20
>>> Thanks to everyone for their help!!
>>=20
>> find_orphans.py works just fine with 0.27 (and likely even development, =
though it's not guaranteed to work with development). The only reason it fa=
iled for you is because a) your backends were unable to find the recording =
files you had and b) you told it to delete the long list of recordings, whi=
ch turned out to be all your recordings.
>>=20
>> find_orphans.py simply talks to the backend, asking it for the list of a=
ll the recording metadata, the list of all the files in the Storage Group d=
irectory lists, and--if you tell it to do so--to delete the metadata and/or=
 files that are lacking matching files/metadata.  It's primarily scripts th=
at do things themselves--especially manipulating and changing data directly=
--that are dangerous.
>>=20
>> Anyway, I'm glad you got things working again.
>>=20
>> Mike

> Mike,

> Thanks for the explanation! Maybe this is what caused the issue:

> I had an internal hard drive fail that had some recordings on it. I
> was able to rsync most of that drive to an external hard drive. I
> have a second internal drive that also has some recordings on it.
> When I would look at the front end there would be a bunch of
> recordings that were greyed out and had Xs by them. I knew they were
> on the failed drive (that wasn=E2=80=99t being mounted) and I knew at some
> point I would get another hard drive and replace the failed one. I
> finally replaced the failed drive last week and rsynced all the data
> from the external drive to the internal drive (that was mounted with
> the same name as the old failed drive). When the sync was done, all
> my recordings that had Xs next to them were back with the exception
> of a few (they were not recoverable from the failed drive). Could
> that have caused the issue? I thought that Myth didn=E2=80=99t care where
> the files were as long as they were on a drive that was mounted as part o=
f the storage group.

> Paul

But that is exactly what find_orphans.py is supposed to do, "throw
away DB records for files that at that moment do not exist!"  and/or
the other way around. So you should never run it if such records
exist that should get fixed when you come to replacing that drive!

Tot mails,
  Hika                            mailto:hikavdh at gmail.com

"Zonder hoop kun je niet leven
Zonder leven is er geen hoop
Het eeuwige dilemma
Zeker als je hoop moet vernietigen om te kunnen overleven!"

De lerende Mens