[mythtv-users] advice on recording direct to NAS

Michael Watson michael at thewatsonfamily.id.au
Tue May 26 00:57:14 UTC 2015


On 25/05/2015 11:37 PM, Mark Perkins wrote:
>
>> -----Original Message-----
>> From: mythtv-users [mailto:mythtv-users-bounces at mythtv.org] On Behalf
>> Of Stephen Worthington
>> Sent: Saturday, 23 May 2015 10:56 PM
>> To: Discussion about MythTV
>> Subject: Re: [mythtv-users] advice on recording direct to NAS
>>
>> On Sat, 23 May 2015 04:56:58 +0000, you wrote:
>>
>>> On Fri, May 22, 2015 at 9:51 PM Mark Perkins <perkins1724 at hotmail.com>
>>> wrote:
>>>
>>>>   Hi all, looking for advice:
>>>>
>>>> Background: I've just lost a second HDD in 6months and its getting
>>>> very, very, very annoying.
>>>>
>>>> I have a NAS that would have capacity to record to, but have not done
>>>> so because of concern for failed recordings due to potential
>>>> inconsistencies in network and / or NAS performance. I run 4 HDD in
>>>> my mythbackend supporting 30tuners (nominal worst case 8 recordings
>>>> at a time due to overlap of pre/post roll).
>>>>
>>>> Does anyone record directly to a NAS folder? How did you share the
>>>> folder with your server? Does it work well? Do temporary spikes in
>>>> network load (for example someone starting a large download or
>>>> streaming a HD movie
>>>> elsewhere) cause problems?
>>>>
>>>> Or am I better recording to local drives and using some sort of cron
>>>> job to move stuff to the NAS periodically?
>>>>
>>>> Or (thinking on the fly) should I abandon the 4 stand alone drives
>>>> and set them up in a RAID array locally?
>>>>
>>>> Any advice gratefully received.
>>>>   _______________________________________________
>>>> mythtv-users mailing list
>>>> mythtv-users at mythtv.org
>>>> http://lists.mythtv.org/mailman/listinfo/mythtv-users
>>>> http://wiki.mythtv.org/Mailing_List_etiquette
>>>> MythTV Forums: https://forum.mythtv.org
>>>
>>> I've previously recorded to my syncology NAS, and just recently last
>>> Friday moved back to using it (I moved away from it not for performance
>>> reasons, but space reasons. I moved back after I upgraded my NAS drives
>>> and moved my backend to a Raspberry Pi 2). I share the folder off my
>>> NAS via NFS and have set MythTV to keep 100GB free space (so I can move
>>> files to the NAS without worrying about free space).
>>>
>>> I've not seen any performance issues recording to the NAS. I've got a
>>> HD Homerun Prime.
>> I would recommend doing some performance testing on your NAS, then
>> calculating the Ethernet traffic also before using it for recording.
>> There are plenty of NAS boxes out there that perform quite badly, and I
> have
>> seen performance issues with recording directly to hard drives when I only
>> had two, so I would be very careful of using something slower than hard
>> drives.  Generally, a home made NAS box using an old PC will perform
> better
>> than a commercial one intended for the home market - NAS boxes often
>> have quite cheap small CPUs and have lots of performance bottlenecks, and
>> sometimes pretty bad software.  An old PC tends to have a CPU and other
>> hardware that matches its drives performance capabilities rather better
> than
>> a cheap NAS does.  The downsides of using an old PC as a NAS are higher
>> 24/7 power use (older PCS use more power than newer ones), and maybe
>> fan noise.
>>
>> A good way to do a real life test would be to set up a special recording
> group
>> pointing to the NAS drive(s) and then just create some one off recording
>> rules set to record to that storage group.  See how many recordings at
> once
>> can be done safely, and try some other use of the NAS at the same time.
>> Make sure to try starting a number of recordings at exactly the same time,
> as
>> the workload of a filesystem can be substantially higher on file creation
> (or
>> deletion), with the heads have to move to the various parts of the drive
>> where it keeps directory and file information, as well as to the actual
>> recording files themselves.  And remember that HD recordings use lots more
>> data than SD ones, often four times as much, so they generate more disk
>> activity and more Ethernet traffic.
>>
>> A NAS should be fine for moving recordings to after they finish recording.
>> Just set up a separate storage group for the NAS drive(s).
>> MythTV will be able to play back files moved to that storage group, but
> will
>> only record to the Default storage group unless you tell it to use another
>> storage group in the recording rule.  Then a job run automatically from
> cron
>> or maybe at the end of a recording can move files off the recording
> drive(s)
>> to the NAS.  The only gotcha I can see is if someone tries to play a file
> that is
>> currently being moved or the script tries to move a file that is being
> played or
>> recorded, so a script that ran in the middle of the night might be a good
> idea,
>> if your household is free of night owls.  Once the script has checked that
> the
>> file is not a current recording or playback file, a good safeguard would
> be to
>> have it rename the file it is currently moving, so MythTV can not find it.
> Then
>> once it has completed the move to the NAS drive, rename it back again.
>> Then there would only be a very short window of opportunity for a clash
> for
>> use of the file.  Once it is renamed, it will show up in mythfrontend
> greyed
>> out with an X icon (or whatever your theme does for missing files).  If a
> file is
>> shown like that, you normally need to page up and page down or something
>> like that to get mythfrontend to refresh its status for the file so it can
> be
>> played.
>> _______________________________________________
>
> Thanks for the pointers Stephen. I'm going to try moving recordings over by
> bash script and cron job in the first instance. I'm not much of a programmer
> but I've put together something that I think will do the job, including
> checking whether a job is in progress first before moving the file. I'm
> still yet to test it but if anyone spots any obvious errors don't hesitate
> to let me know. I have left 'echo' in front of the 'mv' commands for posting
> to the list, just to be on the safe side, and the TEST_COUNTER is just used
> to limit executions through the loop while testing. I'm on Ubuntu 14.04 and
> MythTV 0.27.
>
> #!/bin/bash
>
> MYTHDB="mythconverg"
> MYTHUSER="mythtv"
> MYTHPWD="mythtv"
> BASE_DIR1="/media/fs3/recordings/"
> BASE_DIR2="/media/fs4/recordings/"
> NAS_DIR="/media/servertwo/Myth_Recordings/"
> COUNTER=0
> TEST_COUNTER=0
>
> for FILE in $(find ${BASE_DIR1} ${BASE_DIR2} -type f | grep "\.mpg$"); do
>
>      let TEST_COUNTER=TEST_COUNTER+1
>      COUNTER=0
>
>      read -ra vars <<< $(mysql -D$MYTHDB -u$MYTHUSER -p$MYTHPWD -se "SELECT
> CONCAT(chanid, '_', DATE_FORMAT(starttime,'%Y%m%d%H%i%s'), '.mpg') As
> BASENAME FROM inuseprograms")
>
>      for JOB in "${vars[@]}"; do
>          if [[ "$FILE" == *"$JOB" ]]
>          then
>              let COUNTER=COUNTER+1
>          fi
>      done
>
>      if [ "$COUNTER" -eq "0" ]
>      then
>
>          #first move the file to a new name on the same directory so that it
> is no longer available to mythtv
>          echo "mv $FILE ${FILE}_block"
>
>          #need to extract just the original file name
>          FILENAME=$(echo "$FILE" | sed 's/.*\///')
>
>          #need to move the file to the NAS keeping it blocked from mythtv
>          echo "mv ${FILE}_block ${NAS_DIR}${FILENAME}_block"
>
>          #now to return the file in the new location back to its original
> name
>          echo "mv ${NAS_DIR}${FILENAME}_block ${NAS_DIR}${FILENAME}"
>
>          #reset the counter each time
>          COUNTER=0
>      else
>          echo "$FILE is in use, skipping"
>          COUNTER=0
>      fi
>
>      if [ "$TEST_COUNTER" -gt "0" ]
>      then
>          break
>      fi
>
> done
>
>
>
> _______________________________________________
> mythtv-users mailing list
> mythtv-users at mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org

Dont know that I would go to this much trouble.  Its TV - Not mission 
critical stuff.

If you add a new storage group, move the existing local folders to the 
new storage group, allowing myth to still find the files.
Add your NAS Drive(s) to the existing Default storage group - So new 
recordings get written directly to the NAS.

Start copying files.
If you use rsync, you can limit bandwidth used, thus not flooding the 
network and possibly causing recordings to fail.



More information about the mythtv-users mailing list