[mythtv] [mythtv-commits] Ticket #7949: Multiple identical partitions on identical drives are listed as only one Mythtv drive

[Chris Pinkham]

* On Wed Feb 03, 2010 at 02:58:47AM -0500, f-myth-users at media.mit.edu wrote:
>     > Changes (by cpinkham):
> 
>     > Comment:
> 
>     >  This is a known issue that automatically corrects itself once you have the
>     >  first recording on each drive.  We have not come up with a determinate,
>     >  easy, replicatable solution to determine whether
>     >  filesystems are indeed the same filesystem, links to the same filesystem,
>     >  bind mounts, nfs mounts, cifs mounts, etc. blah to the same filesystem.

Note I use the term filesystem, not directory above.

> I've never been clear about why this is true.  Maybe this was
> discussed in the past and I just missed it somehow.

Yeah, it's been discussed, not quite as many times as what commercial
flagging enhancements could be made, but just about as many. :)

> Wouldn't it be possible to write a different nonce to every filesystem
> and then see which nonces have appeared in multiple places?  After
> all, Myth -does- have permission to write (if it didn't, we wouldn't
> care if they were the same because we're not trying to load-balance
> to them), so why not gensym a name for each, touch it, check which
> have appeared where, and then delete the nonces again?  It seems like
> this would be much simpler logic (and simpler to prove correct) than
> the current heuristic.

I've thought about this method before and to me it sounds a lot more
complicated than the current method.

Currently, we do something like this:

1) master backend asks each slave about their configured directories
2) master compares sizes to see if they match within a given threshold
3) master compares free space to see if it matches within a given threshold
4) if 2 & 3 are true for two entries, then these must be the same filesystem

With a temp file written to each Storage Group Directory, we're
talking more along the lines of this:

1) master backend tells slaves to write out a temp file to all their
   Storage Group dirctories
2) each backend writes out a temp file to each directory it has configured
   and that it can write to (remember, not all Storage Group directories are
   for recordings and may not be writeable by the backend)
3) master asks backends which files they can see in their SG directories
   that are less than X seconds old
4) master gets list of all files/hosts and runs a bunch of loops to see
   who can see what.
4b) master tells slaves to delete all their temp files
5) master now has a list that says:
   host1:/nas/video1 == host2:/nas/video1 and host3:/nas/video1
   host2:/nas/video2 == host3:/nas/video4 and host4:/nas/video3
   host1:/nas/video3 == host3:/nas/video6 and host4:/nas/video8
6) now the master has to determine what filesystem these directories are
   on.  remember, we want to schedule recordings across filesystems/spindles,
   not across different directories which may be on the same filesystem.
7) master gives up and goes and executes the old code above which checks
   filesystem sizes and free space and determines that the user inadvertently
   create /nas on their host1 root filesystem and created subdirs of
   /nas/video1 and /nas/video3 and these are actually the same filesystem.

It's not enough to know that two hosts can see the same directory,
we want to know if directories 1 and 2 are on the same filesystem.
If we restricted Storage Groups somehow so that you could not create
multiple SG directories (whether in the same group or not) on the same
filesystem, then it might be easier, but one of the biggest advantages
of Storage Groups and the disk scheduling is the fact that you can
spread out the I/O load and minimize seeking, fragmentation, etc..

We also need to handle the situation where all storage is remote.
These filesystems could all be on one or more NFS or CIFS servers on
the network, so we can't query the OS to ask if directory A and
directory B are on the same filesystem.  We can't see the fsid of
the NFS share we've mounted twice on the same server or that
/nas/video1 and /nas/video2 are on the same /nas filesystem on
the fileserver.

Long-story-short, the current method is what will be used until
someone makes a good case for a new method.  And 'good case' in
this context includes a working patch that provides a worthwhile
enhancement over the existing method.  The current method is
implemented in about 25 lines of code.

--
Chris