[mythtv-users] Is there a distributed filesystem available?

[Brian Wood]

On Jun 20, 2006, at 4:24 AM, Steve Hodge wrote:

> On 6/20/06, Brian Wood <beww at beww.org> wrote:
>> On Jun 19, 2006, at 10:45 PM, Steve Hodge wrote:
>>> RAID 5 gives you a high level of fault tolerance at the cost of
>>> complexity and capacity. What Debabrata wants is a partially  
>>> tolerant
>>> setup, basically a file system that can span X disks, and when a  
>>> disk
>>> fails it only loses the files on the failed disk (i.e. 1/X of the  
>>> data
>>> if it were full). It's sort of in between RAID 0 and RAID 5: RAID 5
>>> can have a single disk fail without any data loss, RAID 0 will lose
>>> all data when a single disk fails, this hypothetical filesystem  
>>> would
>>> lose some but not all of the data if a disk fails.
>>
>> Wouldn't simple aggregation (sometimes known as "linear RAID)
>> accomplish this? While it would take some specialized measures to
>> recover the data on the remaining operational disks the data is not
>> "interleaved" the way RAID0 data is (which makes it impossible to
>> recover any files larger than a single block if it is partially on a
>> failed disk). Also, some files would be non-contiguous, but if you
>> used the filesystem just for writing and reading large video
>> information files, and perhaps de-fragged often, you could probably
>> recover a lot of the information if a single disk failed.
>
> Yup, that's what I was thinking. But it could be much more robust if
> the filesystem was aware of what was going on. Ideally I was thinking
> of a "meta" filesystem that would use other filesystems for the actual
> storage. I.e. you mount your various underlying drives, tell the meta
> filesystem where they can be found and then mount that filesystem
> wherever you want it. If one of the underlying filesystems vanishes
> then the files stored on that particular filesystem would just
> disappear as well.

So what you really want is the benefit of a multiple device  
filesystem but with as little "spanning" as possible, in other words  
you want the system to try and keep individual files on single  
devices if possible.

To an extent you would be working at cross purposes, systems like LVM  
are designed to hide the individuality of the devices, not be aware  
of them when making storage decisions.

In any case the situation is temporary. As soon as multi-terrabyte  
single drives are available cheaply we will all wonder why we had  
this discussion :-)

If we need a reminder of how fast hard drive costs and capacities  
change:

http://www.littletechshoppe.com/ns1625/winche02.html