[mythtv-users] advice on recording direct to NAS

[Mark Perkins]

> -----Original Message-----
> From: mythtv-users [mailto:mythtv-users-bounces at mythtv.org] On Behalf
> Of Stephen Worthington
> Sent: Saturday, 23 May 2015 10:56 PM
> To: Discussion about MythTV
> Subject: Re: [mythtv-users] advice on recording direct to NAS
> 
> On Sat, 23 May 2015 04:56:58 +0000, you wrote:
> 
> >On Fri, May 22, 2015 at 9:51 PM Mark Perkins <perkins1724 at hotmail.com>
> >wrote:
> >
> >>  Hi all, looking for advice:
> >>
> >> Background: I've just lost a second HDD in 6months and its getting
> >> very, very, very annoying.
> >>
> >> I have a NAS that would have capacity to record to, but have not done
> >> so because of concern for failed recordings due to potential
> >> inconsistencies in network and / or NAS performance. I run 4 HDD in
> >> my mythbackend supporting 30tuners (nominal worst case 8 recordings
> >> at a time due to overlap of pre/post roll).
> >>
> >> Does anyone record directly to a NAS folder? How did you share the
> >> folder with your server? Does it work well? Do temporary spikes in
> >> network load (for example someone starting a large download or
> >> streaming a HD movie
> >> elsewhere) cause problems?
> >>
> >> Or am I better recording to local drives and using some sort of cron
> >> job to move stuff to the NAS periodically?
> >>
> >> Or (thinking on the fly) should I abandon the 4 stand alone drives
> >> and set them up in a RAID array locally?
> >>
> >> Any advice gratefully received.
> >>  _______________________________________________
> >> mythtv-users mailing list
> >> mythtv-users at mythtv.org
> >> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> >> http://wiki.mythtv.org/Mailing_List_etiquette
> >> MythTV Forums: https://forum.mythtv.org
> >
> >
> >I've previously recorded to my syncology NAS, and just recently last
> >Friday moved back to using it (I moved away from it not for performance
> >reasons, but space reasons. I moved back after I upgraded my NAS drives
> >and moved my backend to a Raspberry Pi 2). I share the folder off my
> >NAS via NFS and have set MythTV to keep 100GB free space (so I can move
> >files to the NAS without worrying about free space).
> >
> >I've not seen any performance issues recording to the NAS. I've got a
> >HD Homerun Prime.
> 
> I would recommend doing some performance testing on your NAS, then
> calculating the Ethernet traffic also before using it for recording.
> There are plenty of NAS boxes out there that perform quite badly, and I
have
> seen performance issues with recording directly to hard drives when I only
> had two, so I would be very careful of using something slower than hard
> drives.  Generally, a home made NAS box using an old PC will perform
better
> than a commercial one intended for the home market - NAS boxes often
> have quite cheap small CPUs and have lots of performance bottlenecks, and
> sometimes pretty bad software.  An old PC tends to have a CPU and other
> hardware that matches its drives performance capabilities rather better
than
> a cheap NAS does.  The downsides of using an old PC as a NAS are higher
> 24/7 power use (older PCS use more power than newer ones), and maybe
> fan noise.
> 
> A good way to do a real life test would be to set up a special recording
group
> pointing to the NAS drive(s) and then just create some one off recording
> rules set to record to that storage group.  See how many recordings at
once
> can be done safely, and try some other use of the NAS at the same time.
> Make sure to try starting a number of recordings at exactly the same time,
as
> the workload of a filesystem can be substantially higher on file creation
(or
> deletion), with the heads have to move to the various parts of the drive
> where it keeps directory and file information, as well as to the actual
> recording files themselves.  And remember that HD recordings use lots more
> data than SD ones, often four times as much, so they generate more disk
> activity and more Ethernet traffic.
> 
> A NAS should be fine for moving recordings to after they finish recording.
> Just set up a separate storage group for the NAS drive(s).
> MythTV will be able to play back files moved to that storage group, but
will
> only record to the Default storage group unless you tell it to use another
> storage group in the recording rule.  Then a job run automatically from
cron
> or maybe at the end of a recording can move files off the recording
drive(s)
> to the NAS.  The only gotcha I can see is if someone tries to play a file
that is
> currently being moved or the script tries to move a file that is being
played or
> recorded, so a script that ran in the middle of the night might be a good
idea,
> if your household is free of night owls.  Once the script has checked that
the
> file is not a current recording or playback file, a good safeguard would
be to
> have it rename the file it is currently moving, so MythTV can not find it.
Then
> once it has completed the move to the NAS drive, rename it back again.
> Then there would only be a very short window of opportunity for a clash
for
> use of the file.  Once it is renamed, it will show up in mythfrontend
greyed
> out with an X icon (or whatever your theme does for missing files).  If a
file is
> shown like that, you normally need to page up and page down or something
> like that to get mythfrontend to refresh its status for the file so it can
be
> played.
> _______________________________________________


Thanks for the pointers Stephen. I'm going to try moving recordings over by
bash script and cron job in the first instance. I'm not much of a programmer
but I've put together something that I think will do the job, including
checking whether a job is in progress first before moving the file. I'm
still yet to test it but if anyone spots any obvious errors don't hesitate
to let me know. I have left 'echo' in front of the 'mv' commands for posting
to the list, just to be on the safe side, and the TEST_COUNTER is just used
to limit executions through the loop while testing. I'm on Ubuntu 14.04 and
MythTV 0.27.

#!/bin/bash

MYTHDB="mythconverg"
MYTHUSER="mythtv"
MYTHPWD="mythtv"
BASE_DIR1="/media/fs3/recordings/"
BASE_DIR2="/media/fs4/recordings/"
NAS_DIR="/media/servertwo/Myth_Recordings/"
COUNTER=0
TEST_COUNTER=0

for FILE in $(find ${BASE_DIR1} ${BASE_DIR2} -type f | grep "\.mpg$"); do

    let TEST_COUNTER=TEST_COUNTER+1
    COUNTER=0

    read -ra vars <<< $(mysql -D$MYTHDB -u$MYTHUSER -p$MYTHPWD -se "SELECT
CONCAT(chanid, '_', DATE_FORMAT(starttime,'%Y%m%d%H%i%s'), '.mpg') As
BASENAME FROM inuseprograms")

    for JOB in "${vars[@]}"; do
        if [[ "$FILE" == *"$JOB" ]]
        then
            let COUNTER=COUNTER+1
        fi
    done

    if [ "$COUNTER" -eq "0" ]
    then

        #first move the file to a new name on the same directory so that it
is no longer available to mythtv
        echo "mv $FILE ${FILE}_block"

        #need to extract just the original file name
        FILENAME=$(echo "$FILE" | sed 's/.*\///')

        #need to move the file to the NAS keeping it blocked from mythtv
        echo "mv ${FILE}_block ${NAS_DIR}${FILENAME}_block"

        #now to return the file in the new location back to its original
name
        echo "mv ${NAS_DIR}${FILENAME}_block ${NAS_DIR}${FILENAME}"

        #reset the counter each time
        COUNTER=0
    else
        echo "$FILE is in use, skipping"
        COUNTER=0
    fi

    if [ "$TEST_COUNTER" -gt "0" ]
    then
        break
    fi

done