[mythtv-users] WAY OT: Storage designs

[Greg Oliver]

On Mon, Aug 30, 2010 at 10:15 AM, Simon Hobson <linux at thehobsons.co.uk> wrote:
> Greg Oliver wrote:
>
>>  > If you mean what I think you mean then - no you can't. iSCSI is a block
>>>
>>>  protocol and you can't mount it more than once at a time unless you use
>>> a
>>>  clustered filesystem - if you try then you'll get instant corruption.
>>
>> You can with SCST  :)  That's the only reason I would consider it..
>> It allows multiple accesses to the same LUNs simultaneously as well.
>> I had never heard of it before, but it popped up on lkml, so I started
>> reading up on it..  And of course immediately wanted to try it..
>
> I suggest you read up some more then, because I don't think you've
> understood some of the details. Yes, it is **standard iSCSI functionality**
> that more than one initiator can connect to a single target, but ...
>
> You need to do one of three things :
>
> 1) Ensure that you never, ever (and I mean ever) mount the same filesystem
> on more than one system at a time - ie you always unmount it cleanly on one
> system before mounting it on another.
>
> 2) Use a clustered filesystem, or clustered LVM.
>
> 3) Mount it read-only on every machine - and no I don't mean you can mount
> it r/w on one system and r/o on the others, that won't work either.
>
>
> If you do try and mount a volume r/w in more than one place, with a standard
> filesystem, then it is guaranteed to corrupt the file system - big time.
> Each OS where the volume is mounted will assume that it's the only one
> running the filesystem. Each OS will cache parts of the filesystem data it's
> read from disk, and it will cache changes made to the filesystem. The result
> is that each OS will have an inconsistent view of what is on the disk, and
> multiple uncordinated changes will be made. Eg, machine A makes a change,
> but it isn't all written immediately to disk. Machine B makes changes
> independently of machine A. Depending on timing, what ends up on disk is
> some random mish-mash of changes from both machines. Even if machine A wrote
> all changes immediately (ie no caching), then machine B would not be aware
> of them because at least some of the data would be in it's read cache.
>
> Even if A is the only machine to mount the filesystem r/w, B would still not
> see a consistent filesystem due to write caching on A and read caching on B.
>
>
> There's discussion of iSCSI etc over on the Xen mailing lists. I don't
> follow the details as I'm not up to that level yet - there are some
> interesting things you can do with Xen guests and iSCSI storage.

I'm fully aware of the ramifications of double mounting file stores -
and I'm not trying to argue - I use iSCSI every day at work..  Like I
said - I was intrigued by the SCST project because it *does* allow you
to do the unthinkable.  I do not know if it add a new layer between
the kernel's VFS and the iSCSI stack or what?!?!  I just know it opens
up possibilities that were never there before.  Like I said originally
I have never used it, so ust got excited when I saw it posted.  I can
assure you I will not be *testing* on a live filesystem  :)

-Greg