[mythtv-users] media-automount not calling the unmount.script , and hence not purging videometada of records from the removable drive

Fri Nov 4 10:40:05 UTC 2022

> Wiadomość napisana przez Jay Harbeston <jharbestonus at gmail.com> w dniu 31.10.2022, o godz. 15:41:
> 
> Piotr, 
> I have been ‘exercising’ the mythadder and media-automount and I am running into some troubles when unmounting/disconnecting.
> 
> I think the dev rule needs to trigger on something else besides remove… say maybe unmount(if available) … 

Well – here we touching purpose I develop media-automount:
It is assistant for:
1.       automounting at external mass storage connect + execute something at connect (after fs mount)
2.       auto-cleanup + post removal executions triggered _after_ external mass storage removal  

Foreseen (and exploited by me) use-cases:
-mythadder (when video archive drive is connected/disconnected)
-auto backup (when backup drive is connected)
-managing adding / remove recordings on ext.hdd with TV shows archive (when tv archive drive is connected/disconnected)

Doing fs unmount in media-automount imho is pointless – as media-automount is driven by udev and udev is driven by user hw manipulations (connecting/disconnecting)
In this context fs unmount by media-automount is useless as hw mass storage with fs in question is already disconnected (media-automount is executed by udev AFTER drive disconnect)
By this I’m saying: media-automount does auto-cleanup + post removal actions triggered at external mass storage removal  

Generally speaking, imho proper procedure (valid for any apps using connected/disconnected HW should be):
1.connect hw
2.do what you want
3.manually do unmount (+ any other actions you want to do)
4.dissconnect hw

In my use cases (for which I developed media-automount) I do following:

-mythadder: ext.hdd is mounted r/o – so I can safe remove drive usb cable any time (fs will be clean after removal)

-auto backup: my backup script auto unmounts fs after backup + sends OSD notify to user about user can now safe remove ext.drive

-auto adding / remove TV shows archive: fs is mounted r/w and before disconnection user _must_ explicitly call option in UI telling to system: “I’m ending watching TV shows from ext. ach. drive. So: move new marked to archive shows to ext.drive, add in all archived tv shows titles (archived) marker; send OSD when you finish so user can safe disconnect drive”

> OR,
> The logic in media-auto mount needs to be tweaked. What I am seeing in it 
> Is since the partition has already been unmounted, the section in media-automount needs to be tweaked to only make sure that the partition has been unmounted. 
> 

The whole thing with media-automount is rather not about user helper for mount/unmout but automating some actions to relieve user from them

unmount has nothing with in media-automount because media-automount is run at system state when unmount don’t makes sense as drive is already disconnected.

All is about actions user wants to be auto-executed _after drive disconnection_.

Perfect example is mythadder. 
Mounting fs with r/o effectively removes user requirement for unmount.
So all can be fully automated.
Fully automated operation is like this:
1. user connects drive. mythadder is called.
2. few sec later user gets OSD “Your movies from driveX are ready to watch”
3. movies started to appear in video gallery
4. user watches videos
5. user decides to stop watching and disconnect ext.hdd (to save power)
6. user removes usb plug of ext.drive
7. media-automount triggers mythadder to remove relevant videos from video gallery
8. media-automount sends OSD “All good. Thx for using ext.drive”)

> I get the message in logs indicating that the device doesn’t exist anymore or is not a block device, and so myth adder.py doesn’t get called.
> 

mythadder.py is designed to work in following way (at disconnect):

-is sholud be called with env set to devname, action and mythdb config

see https://github.com/warpme/mythadder/blob/b974d5e7999f520e9e335731cd3fdd125603df4f/USB-Movies1.unmount.script#L14

-mythadder reads this 
see https://github.com/warpme/mythadder/blob/b974d5e7999f520e9e335731cd3fdd125603df4f/mythadder.py#L150 

You can play and see what mythadder.py is doing at connection/dissconnection by calling mythadder from sh script - but remember properly set env. vars in this sh script…
see https://github.com/warpme/mythadder/blob/b974d5e7999f520e9e335731cd3fdd125603df4f/mythadder.py#L13

Here are exemplary logs from m y system:

at hdd connection

2022-09-17 20:45:04.504791 I [3501827] mythadder.py.
    mythadder.py v1.2.0 by Wagnerrp, Piotr Oniszczuk
....
     -Device      : /dev/sde1
     -Action      : add
     -DeviceLabel : USB-Movies1
     -DeviceUUID  : 97101397-ed8f-492f-9b2c-74e57113710d
....
2022-09-17 20:45:04.504892 I [3501827] mythadder.py.
    -->Connecting to MythTV DB
....
2022-09-17 20:45:04.521592 I [3501827] mythadder.py.
    -->REMOVABLEVIDEO table schema 1030 is valid
....
2022-09-17 20:45:09.535032 I [3501827] mythadder.py Disk mounted at /myth/video/USB-Movies1
2022-09-17 20:45:11.283365 I [3501827] mythadder.py.
    Inserting into REMOVABLEVIDEOS table record with:
     -Disk UUID   : 97101397-ed8f-492f-9b2c-74e57113710d
     -Disk Label  : USB-Movies1
     -File Inode  : 51
     -Movie Title : Step Up 3
     -File Path   : /myth/video/USB-Movies1/Step Up 3.m2ts
     -File(in SG) : USB-Movies1/Step Up 3.m2ts
     -VideoSG path: /myth/video/
     -File Hash   : d3c75aa29fb3e607
…….

……..
…….
....
2022-09-17 20:46:21.744670 I [3501827] mythadder.py.
    -->Removing from REMOVABLEVIDEOS all records for files deleted on USB-Movies1 disk
....
2022-09-17 20:46:21.751765 I [3501827] mythadder.py.
    -->Insert all records from REMOVABLEVIDEO table into VIDEOMETADATA table
....
2022-09-17 20:46:21.761674 I [3501827] mythadder.py.
    -->Getting all movie records from REMOVABLEVIDEO table not yet seen by myth
....
2022-09-17 20:46:21.762560 I [3501827] mythadder.py.
    -->All done with success!
     Exiting …

at drive disconnection:

2022-09-18 15:24:13.185288 I [3648272] mythadder.py.
    mythadder.py v1.2.0 by Wagnerrp, Piotr Oniszczuk
....
     -Device      : /dev/sde1
     -Action      : remove
     -DeviceLabel : False
     -DeviceUUID  : False
....
2022-09-18 15:24:13.185396 I [3648272] mythadder.py.
    -->Connecting to MythTV DB
....
2022-09-18 15:24:13.201700 I [3648272] mythadder.py.
    -->REMOVABLEVIDEO table schema 1030 is valid
....
2022-09-18 15:24:13.201801 I [3648272] mythadder.py.
    -->Updating all metadata fields from VIDEOMETADATA in REMOVABLEVIDEO table
....
2022-09-18 15:24:13.209102 I [3648272] mythadder.py.
    -->Remove all relevant movie records from VIDEOMETADAT table
....
2022-09-18 15:24:13.214373 I [3648272] mythadder.py.
    -->All done with success!
     Exiting ...

> Do you see any issues changing that logic so that it will go ahead and remove the records in the video metadata table?
> 
> Otherwise, with this current operation mode, mythadder.py never gets called to remove them.
> 
> Regards,
> 
> Jay
> 
> 
> On Jul 22, 2022, at 5:28 AM, Piotr Oniszczuk <piotr.oniszczuk at gmail.com> wrote:
>> 
>> 
>> 
>>> Wiadomość napisana przez Jay Harbeston <jharbestonus at gmail.com> w dniu 21.07.2022, o godz. 14:19:
>>> 
>>> --I see that the Storage group needs to be added, and my plan is to just add /media/mythtv as the directory on the frontend in 
>>> Settings -> Video settings-> General Settings -> Directories that Hold videos so I can plug the external drive with video into it.
>>> 
>> 
>> 
>> IIRC mythadder concept not requires dedicated SG.
>> 
>> mythadder simply moves metadata from mythtv movies metadata DB table to aux/cache table (when ext. drive is disconnected) or vice versa (when you connect ext. drive).
>> 
>> Movie files from ext. drive can be in subdirectory of dir where permanent movies are stored or can be in dedicated additional dir defined in video SG.
>> 
>> In my home system:
>> 
>> - I have 5 usb attached drives labeled USB-Movies[1..5]
>> - Backend has video SG with /myth/video dir. 
>> - When I’m connecting USB drive, media-automout mounts USB drive content to /myth/video/USB-Movies[1..5] and calls mythadder
>> - Mythadder moves metadata of all USB movies to mythtv movies table - so all my 4 disk-less remote frontends started to offer movies from USB drive.
>> - When user asks for watching movie from USB drive - backend starts stream (by myth protocol) movie from /myth/video/USB-Movies[1..5]/<movie> to remote disk-less frontend
>> - Disconnecting USB drive moves all relevant metadata from movies DB table to aux/cache table (so metadata are not lost) - but becomes not visible to my remote frontends.     
>> 
>> 
>>> --Does that make sense, or does it need to be plugged into the backend and add /media/mythtv to the storage groups?
>>> 
>> 
>> It is best to connect ext. USB drive to backed - as backend is place where streaming starts when you watch movie.
>> If you want to have setup where user connects ext. USB to one from remote frontends, then i see 2 options:
>> 
>> 1. at remote frontend host - where you are connecting ext.drive:
>> - scripting mounts USB drive remotely to backed (i.e. via NFS or CIFS or whatever (you can do this i.e. in media-automount script)
>> - scripting calls locally mythadder to update with mythtv movies DB with just attached movies from ext.USB.
>> 
>> 2.remote frontend host - where you are connecting ext.drive - is also slave backend with defined its local storage (including ext. USB storage ) in mythtv video SG.
>> 
>> Comparing, Option2 is better because there is no traffic tromboning between remote fe and be when you play movie on the same host where you have connected ext. USB. (traffic tromboning in Option1 will be like this: ExUSB->remoteFE->(NFS)->BE->(mythproto)->remoteFE->Display)
>> 
>>> 
>>> Will the updated myth adder and auto mounter work on either the frontend or the backend, or ONLY on the back end?
>> 
>> Should work ok from mythadder perspective - as long as ext.USB drive block device is local to mythadder (in other words: all is n the same host)
>> 
>>> 
>>> I’m not sure how videometadata works regarding frontend vs back end directories, but it would be nice if it would work on the frontend as I keep the backend in a different part of the house. :-)
>> 
>> As i mention - you can go with 2 options here.
>> 
>> I would go with option2.
>> 
>> First thing i’ll do will be test: is mythtv capable to offer videos hosted on multiple discrete remote hosts (backend and remote frontend with connected ext.USB)
>> 
>> You can do this i.e. by:
>> -on your remote frontend make sure ext.USB is mounted in persistent mount point
>> -configure/start slave backend on remote frontend
>> -on slave backend config add video SG with dir where ext.USB is mounted
>> -connect ext.USB, make sure it is mounted ok (in video SG defined dir)
>> -scan videos
>> -check: do you see videos from ext.USB on remote frontend / other  frontends and test can you watch
>> -if yes - ask to download metadata and see is it seen by others frontends
>> 
>> if this will work - then we are in home :-)
>> (you can play i.e. with media-automounter to automatise all this :-) )
>> 
>> good luck!
>> 
>> 
>> _______________________________________________
>> mythtv-users mailing list
>> mythtv-users at mythtv.org
>> http://lists.mythtv.org/mailman/listinfo/mythtv-users
>> http://wiki.mythtv.org/Mailing_List_etiquette
>> MythTV Forums: https://forum.mythtv.org
> 
> _______________________________________________
> mythtv-users mailing list
> mythtv-users at mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org