[mythtv-users] Storage Group Disk Scheduler

[Raymond Wagner]

On 7/9/2011 19:39, Dave Stuart wrote:
> The device that I am using to run the backend is a goflex net using 
> the plugapps instructions.  It has 128mb of ram. I have a 1tb drive 
> and a 1.5tb drive that I am using to store shows.

While there is no official minimum requirement for a master backend, any 
of the developers will tell you a pogoplug and its ilk will undercut the 
minimums by a wide berth.  The database requires power and memory. The 
backend requires power and memory.  The scheduler and guide data 
processor requires a lot of power and memory for short durations.  
Commflagging and transcoding take a lot of power if you wish to do 
those.  Even file serving takes memory.  RAID cards come with 512MB or 
more dedicated these days, and ZFS recommends 2GB of memory per 1TB of 
storage.  Just because something can run Linux, doesn't mean you can put 
whatever application you want on it.

> I am using ext3 filesystem (journaling), but I want to be able to use 
> fsck as a backup.
>
> I am not uncleanly unmounting partitions.

If you don't uncleanly mount, then you don't have to fsck.  EXT3 just 
has a nasty carry over from EXT2, in that it doesn't trust itself, and 
runs an fsck periodically anyway.

> More memory is not an option for this machine, and this is the machine 
> that I will be using.

You should really reconsider that.

> Can you recommend a filesystem that would be better to use for this 
> application?  Should I specify format parameters to make the fsck use 
> less memory?

XFS doesn't even have an fsck mechanism, instead relying on replaying 
the journal during mount for all situations.  However XFS caches 
aggressively, and will probably not behave well on a machine with only 
128MB of memory.  JFS has a fsck application, but for most situations, 
that fsck will simply replay the journal.  It is rare it will need to do 
a full fsck.

How does the unit natively handle a damaged filesystem?

> In any case, it seems to me that the mythtv scheduler should be using 
> the device rather than the filesystem to determine the 'weight'.  I 
> can imagine a number of scenarios when using storage groups where this 
> would be a good idea.

I can't.  There is no good reason to have multiple physical partitions 
on a bulk storage disk.  You have one big partition with one big 
filesystem, and then you use different folders for different storage 
groups if you want to break stuff up for organization purposes.  The 
current matching algorithm, based off total and free space, works 
universally on any OS, FS, and even when multiple machines have a shared 
NFS mount.  A setup to do what you want would require different code to 
manage polling the mount tables on different OSs, and even different 
filesystem types on a single OS.  It would be a horrid mess of code, and 
nigh unmaintainable.

The single scenario where I could see this being a problem is users with 
multiple logical partitions on ZFS or BTRFS.  While the logical 
partitions pull from a shared pool of disk space, they report the total 
space as the sum of what they're using and what is left in the pool, 
meaning each would report a different total space.