[mythtv-users] Inconsistent file deletions from the backend.

Thu Mar 22 18:04:50 UTC 2012

On Thu, Mar 22, 2012 at 12:32 PM, Michael T. Dean
<mtdean at thirdcontact.com>wrote:

> On 03/22/2012 12:51 PM, Zarthan South wrote:
> > On Wed, Mar 21, 2012 at 6:13 PM, Zarthan South wrote:
> >> I am baffled by what appears to be a totally inconsistent behavior.
> After
> >> a recording I use a user job to convert recordings to mp4 and then add
> them
> >> back into the database. As part of the conversion I use mythlink to
> create
> >> a directory structure for myth upnp. When I delete the file from
> mythweb I
> >> use a system event to find and remove the broken links that were created
> >> with mythlink. Perhaps up to 20% of the file deletions fail and even
> though
> >> the database is updated and I no longer show the program in mythweb the
> >> actual mp4 files are still on the disk.  I set the logs to only show
> >> file,extra to see if I could see what is happening.
> >> Looking at the logs it seems like in spite of the fact that the mp4 file
> >> is found, the delete function is against the mpg file.
> >>
> >> Can someone help me understand this?
> >>
> >>
> >> MythTV Version   : v0.24.1-80-g1de0431
> >> MythTV Branch    : fixes/0.24
> >> Network Protocol : 63
> >> Library API      : 0.24.20110505-1
> >> One that didn't work:
> >>
> >> 2012-03-21 17:58:50.692 MainServer::ANN Monitor
> >> 2012-03-21 17:58:50.716 adding: dgsws as a client (events: 2)
> >> 2012-03-21 17:58:50.761 MainServer::ANN Monitor
> >> 2012-03-21 17:58:50.791 adding: dgsws as a client (events: 2)
> >> 2012-03-21 17:58:50.841 MythSystemEventHandler: SubstituteMatches:
> BEFORE:
> >> /usr/sbin/stale_links.sh
> >> 2012-03-21 17:58:50.879 SG(Default): FindRecordingFile: Searching for
> >> '1041_20120321165100.mp4'
> >> 2012-03-21 17:58:50.908 SG(Default): FindRecordingDir: Checking '/tv'
> for
> >> '/tv/1041_20120321165100.mp4'
> >> 2012-03-21 17:58:50.933 SG(Default): FindRecordingFile: Found
> >> '/tv/1041_20120321165100.mp4'
> >> 2012-03-21 17:58:50.958 ProgramInfo(1041_20120321165100.mp4):
> >> GetPlaybackURL: File is local: '/tv/1041_20120321165100.mp4'
> >> 2012-03-21 17:58:50.983 MythSystemEventHandler: SubstituteMatches:
> AFTER :
> >> /usr/sbin/stale_links.sh
> >> 2012-03-21 17:58:51.008 Launching: /usr/sbin/stale_links.sh
> >> 2012-03-21 17:58:51.038 PID 22564: launched
> >> 2012-03-21 17:58:53.848 About to delete file:
> /tv/1041_20120321165100.mpg
> >> 2012-03-21 17:58:56.077 About to delete file:
> >> /tv/1041_20120321165100.mpg.-1.100x56.png
> >> 2012-03-21 17:58:56.107 About to delete file:
> >> /tv/1041_20120321165100.mpg.png
> >>
> >>
> >> And One that did:
> >>
> >> 2012-03-21 18:07:12.644 MythSystemEventHandler: SubstituteMatches:
> BEFORE:
> >> /usr/sbin/stale_links.sh
> >> 2012-03-21 18:07:12.671 SG(Default): FindRecordingFile: Searching for
> >> '1041_20120304080000.mp4'
> >> 2012-03-21 18:07:12.716 SG(Default): FindRecordingDir: Checking '/tv'
> for
> >> '/tv/1041_20120304080000.mp4'
> >> 2012-03-21 18:07:12.741 SG(Default): FindRecordingDir: Checking
> >> '/tv/Shows/_Series_' for '/tv/Shows/_Series_/1041_20120304080000.mp4'
> >> 2012-03-21 18:07:12.766 SG(Default): FindRecordingFile: Found
> >> '/tv/Shows/_Series_/1041_20120304080000.mp4'
> >> 2012-03-21 18:07:12.792 ProgramInfo(1041_20120304080000.mp4):
> >> GetPlaybackURL: File is local:
> '/tv/Shows/_Series_/1041_20120304080000.mp4'
> >> 2012-03-21 18:07:12.825 MythSystemEventHandler: SubstituteMatches:
> AFTER :
> >> /usr/sbin/stale_links.sh
> >> 2012-03-21 18:07:12.850 Launching: /usr/sbin/stale_links.sh
> >> 2012-03-21 18:07:12.886 PID 22697: launched
> >> 2012-03-21 18:07:15.652 About to delete file:
> >> /tv/Shows/_Series_/1041_20120304080000.mp4
> >> 2012-03-21 18:07:18.486 About to delete file:
> >> /tv/Shows/_Series_/1041_20120304080000.mp4.-1.100x56.png
> >> 2012-03-21 18:07:18.517 About to delete file:
> >> /tv/Shows/_Series_/1041_20120304080000.mp4.png
> >>
> > While this might not affect many, this does appear to be a bug or at
> least
> > an internal mythtv process I don't understand. In the first example both
> > the MPG and MP4 files existed and it appears that mythtv deleted the MP4
> > reference from the database but deleted the MPG files and ignored the MP4
> > files. I had been keeping the MPG files for a day in case the transcode
> > user job had issues and used a cronjob to remove older MPG files. The
> > apparent inconsistency was for freshly recorded programs since both the
> MPG
> > and MP4 files existed side by side. Once my cronjob ran and deleted the
> > older MPG files, mythtv correctly removed the database reference to the
> MP4
> > files and deleted the MP4 files from disk.
> >
> > I have added a a step in my transcode job that moves the MPG file to a
> new
> > location which takes care of the problem but I do think it is a bug of
> > sorts.
> >
> > Can someone please comment.
> >
> > Many thanks
>
> I don't understand what exactly you're doing with the user job.
>
> Are you saying that you end up with the recording file being a .mp4 file
> (like 1041_20120304080000.mp4 ), but that you're using a link that ends
> with .mpg (like 1041_20120321165100.mpg) whose name is identical to the
> pre-transcode file name (so that you can "add them back into the
> database")?
>
> If so, did you enable:
>
> Follow symbolic links when deleting files
> If enabled, MythTV will follow symlinks when recordings and related
> files are deleted, instead  of deleting the symlink and leaving the
> actual file.
>
> in mythtv-setup?
>
> If that's not what you're doing, you're probably going to have to let us
> see what exactly your user job is doing.
>
> Mike
> _______________________________________________
> mythtv-users mailing list
> mythtv-users at mythtv.org
> http://www.mythtv.org/mailman/listinfo/mythtv-users
>
I take the MPG file as recorded by MBE and transcode to an MP4 file and
replace the MPG file in the database with the MP4 file. If the MPG and MP4
files are both in the original location (a real  1041_20120304080000.mp4
and a real  1041_20120304080000.mpg) deleting from mythweb removes the
database reference to the  1041_20120304080000.mp4 file but removes the
real  1041_20120304080000.mpg file ignoring the  1041_20120304080000.mp4
file. When I remove the MPG file or move it to a different location the MP4
file is deleted from the database and from the disk.

I probably shouldn't have mentioned mythlink.pl since it doesn't affect
anything other than when the MP4 file isn't deleted the links aren't
removed with the system event either. Initially I wasn't sure that it
wasn't involved in my issue.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.mythtv.org/pipermail/mythtv-users/attachments/20120322/9f423722/attachment.html 