[mythtv-users] Hitachi 3T drives - reliable?

Rod Smith mythtv at rodsbooks.com
Thu Jan 20 16:09:32 UTC 2011

On 01/20/2011 07:49 AM, Simon Hobson wrote:
> Jean-Yves Avenard wrote:
>> Sorry, but this doesn't make any sense. Why would a 64 bits OS got
>> anything to do with the size of a hard drive.
> Directly, not a lot. But ...
> For whatever reason, 2TB seems to be a common limit (I know it is in
> older versions of Windows). 2TB is 2^31 bytes, so the maximum value for
> a signed 32 bit integer.

I've missed a few posts in this thread; but....

The limit works out like this:

2^32 sectors * 512 bytes/sector = 2^41 bytes = 2 TiB

The elaborate: Various addressing systems for hard disks, including the 
Master Boot Record (MBR) partitioning system and internal data 
structures on various OSes, rely on 32-bit (unsigned) sector pointers. 
Given the near-universal (until recently) practice of using 512-byte 
sectors, this becomes 2 TiB (note that's tebibytes, not terabytes, when 
being precise with the units).

As others have pointed out, when using disks with larger sectors, the 
size limit goes up; however, to the best of my knowledge, no internal 
hard disks today use anything but 512-byte logical sectors. ("Advanced 
Format" drives use 4096-byte physical sectors but present 512-byte 
logical sectors to the OS, so they don't help on this score. There are 
also external drives that do the opposite -- they use 512-byte physical 
sectors but present larger logical sectors to the OS.) Part of the 
problem is that there are a lot of assumptions about sector size written 
into BIOSes, OSes, disk partitioners, etc. (Try running GNU Parted on a 
disk with something other than 512-byte sectors sometime!) These 
assumptions mean that if a disk manufacturer were to release a disk with 
4096-byte sectors that presented itself as such, they'd probably get 
tons of returns because the disk wouldn't work with a lot of computers.

> I suspect it's more a case of "it's fixed in OS versions that happen to
> be 64bit", while 32bit OSs have been 'left behind'.

Whatever is used internally by the OS, the MBR limit is a hard one; you 
just can't get around it and still use MBR. (The MBR data structures are 
explicitly 32-bit, no matter what bit size your CPU uses.) Thus, one 
step in overcoming the limit is to ditch MBR in favor of something else, 
like the GUID Partition Table (GPT), which uses 64-bit pointers. If an 
OS supports GPT, it MUST use 64-bit pointers, at least when addressing 
the GPT data structures, no matter what its CPU's native bit depth. An 
OS could of course convert this to a 32-bit structure at other points in 
its internal processing, thus eliminating one of the benefits of GPT -- 
namely its ability to handle huge disks. My understanding is that some 
drivers for Windows XP and earlier did just this, which means that even 
add-on GPT drivers for Windows XP only support disks of up to 2^32 sectors.

I did some tests once in a virtual machine, and 32-bit Linux (I think I 
used Ubuntu 9.04) was able to handle a 4 TiB virtual disk (with GPT) 
just fine. Windows XP is limited by MBR, since it doesn't support GPT. 
Windows Vista and above support GPT, but not for boot disks unless the 
computer uses EFI rather than BIOS. I'm not sure offhand if 32-bit Vista 
or 7 can handle a disk with more than 2^32 sectors that's partitioned 
with GPT (I couldn't get these OSes installed under QEMU, which I used 
for my tests).

> As for disks (of the computer storage rather than brake variety), I
> suspect there's nothing fundamental about 32bit OSs to stop them
> supporting >2TB drives, more likely that <something> is a 32bit signed
> integer and that sets a limit. For obvious reasons, if the same software
> is compiled for 64bit, then the 32bit value may become a 64bit value and
> the limitation disappears.

That's basically correct, except that you've got a whole range of data 
structures -- in disk partitioning software, in the kernel's partition 
table handling, in low-level disk drivers, in the computer's firmware, etc.

FWIW, I'm familiar with the partition table aspect of it because I'm the 
author of GPT fdisk (aka gdisk; http://www.rodsbooks.com/gdisk/). I'm 
not familiar with the details of the Linux kernel internals, much less 
those of Windows. As noted earlier, though, I have run tests of several 
OSes using virtual machines.

More information about the mythtv-users mailing list