[mythtv-users] BE3 - Issues with installation and database

[Stephen Worthington]

On Thu, 6 Jun 2019 14:49:52 -0500, you wrote:

>
>Hi Stephen!
>
>OK, that looks like I have all the information I need. BE2 has 2.5 
>Tbytes of recordings that you want moved to BE3. There is only one 
>recording partition each on BE2 and BE3. So the setup is quite simple.
>
>    Good! You’re doing a lot of work for me which I greatly appreciate
>    and would feel guilty if gets too complicated.
>
>I can now write a version of mythexport that will create a .sql file 
>with all the data for each of the recordings on BE3 and store it in the 
>same directory as the recording files. Then we need to get that 
>directory copied or moved to BE3 and run mythimport from it on BE3. My 
>plan is to have the special mythexport version run in the recording 
>directory on BE2, and it will simply take a list of all the recording 
>files in that directory (*.mpg, *.ts and *.nuv) and for each of those 
>files, it will run mysqldump with the right commands to export all the
>database entries matching that recording (from these tables: recorded, 
>recordedseek, recordedrating, recordedprogram, recordedmarkup and 
>recordedcredits).
>
>    Probably does not make any difference but I only have *.ts
>    recordings in (BE2’s) /data3/mythtv/recordings., along with the
>    associated *.png files.
>
>The only thing really left to decide is that 2.5 Tbytes will take a very 
>long time to move to a new drive, either directly between drives or over 
>Ethernet. The easiest way to get all those recordings onto BE3 is simply 
>to take that drive out of BE2 and install it on BE3, then import the 
>data for the recordings into BE3's database. The files are never moved, 
>and BE3 then has a lot more space for future recordings.
>
>    Agree would be a lot faster using SATA (6 Gbps?) but I have a
>    mechanical issue swapping hard drives and would need to remove the
>    motherboard. Will just have to transfer via the network and be patient.
>
>If you want to keep using BE2 to record, then the recording files will 
>need to be copied or moved to BE3's drive, which has enough space for 
>them all plus what is already there. Given how much data there is to
>be moved, it is going to take a day or more. So you will likely want to 
>have BE3 (and maybe also BE2) doing recordings at the same time. That is 
>a bit more complicated, but I do have code that can detect when the next 
>recording time is going to be, and stop a copy or move that is in 
>progress before that time. So it should be possible to expand that code 
>a bit so that it will check both BE2 and BE3 for the next recording time 
>and will stop if either of them need to record.
>
>    You are one smart cookie y’know! :)
>
>
>    Probably already considered and accounted for but will ask anyway.
>    What happens if there is a power failure? Both BE2 and BE3 and the
>    network hardware between are on UPSs so a short term outage is
>    resolved here, just considering the one the middle of last month
>    where didn’t have power for 2½ hours. Very rare situation, but
>    always a possibility.

The code I use for copying or moving recordings is done so that until
the files are all at the new location, the old files are untouched. So
if the operation fails in the middle for any reason, it can just be
started from the beginning again.  All the files for one recording are
treated as a group for this purpose.  So all you should need to do
after a power failure is restart the copying / moving program.  Of
course, you will need to have run full fsck checks on all partitions
first, to ensure the filesystems are undamaged, which you should be
doing anyway after any situation where a PC was not shut down
properly.

>All of the recording files on BE2 that were being moved would be put in 
>a special new directory that was in a new storage group on BE2, not 
>"Default". That prevents new BE2 recordings from being put into that 
>directory, and would allow the copy/move software to work without having 
>to handle the problem of new files appearing. The copying or moving 
>would use a network connection to copy or move the BE2 files to BE3, and 
>as it completed the copy or move of a file, it would move that recording 
>out of the current BE2 directory to yet another one that you would 
>create that was also in BE2's new storage group, where BE2 could see it 
>but the copy/move program would not see it any more.
>
>    Wow! :) I almost have the feeling my power failure question is
>    resolved by that.

Yes.

>    Thanks for all the work you’ve put into this!
>
>    Barry