[mythtv-users] Contemplating a major change

[Stephen P. Villano]

On 4/6/14, 3:31 PM, Raymond Wagner wrote:
> On 4/6/2014 1:46 PM, Stephen P. Villano wrote:
>> On 4/6/14, 2:24 PM, Gary Buhrmaster wrote:
>>> On Sun, Apr 6, 2014 at 6:05 PM, Stephen P. Villano
>>> <stephen.p.villano at gmail.com> wrote:
>>>> On 4/6/14, 12:49 PM, Raymond Wagner wrote:
>>>>> The "hardware" part was key.  The only thing hardware RAID10 gives
>>>>> you
>>>>> over software is ease of boot.  Hardware RAID is only beneficial when
>>>>> you have to perform parity calculations, and even then it's largely
>>>>> inconsequential on a modern CPU.
>>>> For RAID10, perhaps, perhaps not. It's used in enterprise for a
>>>> good reason.
>>> And many of those enterprise controllers also have various
>>> battery backed caches.  Which *can* have benefit for
>>> certain workloads (often databases), although various
>>> SSD devices are now the usual accelerator since the
>>> logs tend to be small enough (even a ZIL on a large
>>> ZFS installation can often fit in a reasonably priced
>>> (for enterprise quality) SSDs), and the L2ARC is also
>>> moving to SSDs for more performant target installations.
>> I don't know. Logs are *usually* small, but when something goes wrong,
>> well, my Mythbuntu 12.04 ended with five gigs of errors and climbing
>> once, with a backup of 5 gigs of compressed log.
>> Logs are typically small until a system error occurs, then the sky's the
>> limit!
>
> Different log.  The ZIL or ZFS Intent Log is what ZFS "intends" to
> write to disk.  It fulfills the same purpose as the battery-backed
> write cache on hardware RAID cards.  You write to the non-volatile ZIL
> nearly instantly, and then ZFS takes care of pushing that data to
> rotating storage when convenient.  The general rule of thumb for a ZIL
> is roughly 10 seconds at your maximum write throughput, so a few GB
> for a large RAID array.  When using ZFS with a ZIL, you typically
> partition just a small amount of it for ZIL, and leave the rest for
> L2ARC, which is a second level read cache, stacked on top of the
> typical in-memory disk cache.
> _______________________________________________
Ah, the joy of calculating what was once called buffers, latency and in
general overhead. Even rotational latency was a factor, years ago.
When properly done, makes one's data throughout scream. When improperly
done, creates nightmares figuring out what was done wrong to make high
end hardware behave like it's low end.  :)
I was recalling how we compensated for rotational latency, even in an
array. It is going by the wayside, only to be replaced by minor delays,
but still a vast improvement.*

Still, the intent log could end up jamming things up in the case of a
malfunction that triggers massive writes, such as the error chain that I
experienced. In the error I had recently experienced, I was unable to
shut down, as it appeared to be still trying to log and holding process
time from shutting down all processes. I ended up killing processes
manually end waiting for the logging to finally end its storm.
Then, I shut down cold.

*Yes, I go back far enough that all those nice automagic things were
done manually on initial configuration. And back when the SCSI bus would
hang while awaiting an ill tempered device to announce itself...