[mythtv-users] Migrating existing recordings to new Storage Groups

[Bruce Markey]

Matt wrote:
...
> To me it sounded like he had 2 physical discs which he partitioned 7
> times between the two, and that many of the partions are 50GB XFS
> partions, each configured to be a storage group.  This is how I'm
> interpretting what he said... but let's go with that as I'm trying to

Yeah, sorry, I didn't want to go there. I don't like this as
it's counter to the advantage of SG. I wanted to point out
that you may still want more than one group on the same disk...

> get an understanding of the optimization of multiple tuners recording
> at the same time with multiple storage groups.
> 
> For this example, lets say we have two 200GB HD and 4 tuners.  Can you
> give us a comparison of which of the following scenarios would be best
> if all 4 tuners are recording at the same time.  I'm also assuming
> that these disks are only used for MythTV and not for any other OS or
> application functions.

Okay.

> 1) No storage groups - 1 directory on a single partition - obviously,
> this is what a lot of people see now, and is worst case from a disk
> I/O performance standpoint

Right. This is the problem. Four recordings, playback of another
and four commflag at the same time. Though realtime commflag
often can read from buffered memory, this could be nine positions.
To make things worse, lets imagine striping several disks (great
idea for a usenet news server, terrible for video files). So now
rather than streaming continuous video, it writes a stripe then
seeks on another disk for the next bit. This is true for each
file so each disk is now seeking between nine positions. I wouldn't
want to sit too close and risk getting hit by shrapnel.

> 2) Storage groups - 1 partition on disk 1, 1 partition on disk 2.  The
> 4 tuners will be recording, most likely, two to a partition.

For it. FYI, there is an internal scoring system to decide
which is best by deciding things like, it's better to write
two files to the same local disk than one to a remote but
better to write one remote than three to the same local disk.
Transcode and commflag jobs count too but author Chris Pinkham
knows more about this than I do.

> 3) Storage groups - 2 partitions on disk 1, 2 paritions on disk 2.
> The 4 tuners will be recording, most likely, one to a partition.

Right. Here's where you have to go beyond the black box and pictrue
what is really happening inside the disk drive. Imagine that the
drive is opened up and you can see the platters spinning and the
head arms swinging back and forth. If there are two recordings
in one partition, they are being written close together. The
drive should have no problem flopping between these two positions.
If there was one recording, it could tick off track to track seeks
and it would be a slam dunk.

If there were two partitions on the same spindle, there would
still be no problem moving but now it has to traverse ~50% of
the tracks on the disk to get to the other partition. So, it
will work but there is no advantage to doing so and it is actually
less efficient.

> Is there any benefit to having the head thrashing 2 different places
> on a single disk with partitions versus without partitions?  I hope I
> clearly stated what I was trying to get at.

You did.

Reality check is that a 3.6GB/hr recording only needs to write
1MB/sec and disks can stream tens of MB/sec. The problem is that
they only write when the heads are positioned over the target
blocks. When they seek, no data is being read or written.

A 7200rpm disk spins 120 times per second and so the disk could
write a max of 120 times the total size of an average cylinder.
Seek times are on an order of 10ms so the disk can seek about
100 times per second. If I had a news server with an average
file size of 5k and the articles in all different directories,
it couldn't seek more than 100 times and would write less than
500kb/s (actually, it has to seek, wait for the disk to rotate
to the target block, write then seek again so <200k is a better
estimate.

For most apps, if the disk writes are slow, the app just runs
slower while it waits. However, for recording video, there are
going to be 30 frames per second whether the disks are ready or
not (see Lucy in the chocolate factory).

So, the problem isn't if the disks are "fast" enough (whatever
that means), there's speed to burn. The problem is that if the
disks spend >90% of there potential throughput moving from place
to place before getting back to the same file and not being able
to write anything that whole time, then there will be problems
that result in dropped frames and other badness. The best case
solution is one spindle, one partition, one recording in progress.

-- bjm