[mythtv-users] OT: RAID rebuild with the wrong partition

George Mari george_mythusers at mari1938.org
Sun Sep 12 15:22:54 UTC 2010


On 09/11/2010 08:10 PM, freedenizen wrote:
> Before I ask my question here is my layout, using linux software raid
>
> RAID1:
> md1: sda1&  sdb1
> md2: sda2&  sdb2
> md3: sda3&  sdb3
> md4: sda4&  sdb4
>
> RAID6:
> md5: sd[c-h]1
> md6: sd[c-h]2
> md7: sd[c-h]3
>
> So I had some sort of I/O issue going on, things were locked up and I
> rebooted.  When I rebooted the system started to rebuild md3 using
> sda3, and sdc3!  I checked the UUIDs on the partitions and they were
> all correct, but for some reason it tried to redo the mirror using the
> wrong partition.  I have no idea why or how it did that, but now I
> have a bunch of corrupt data in /
> The system is currently up and running now, but obviously some things
> aren't working, it can't find vi for example, and I have no idea how
> much other stuff is messed up.  Currently the arrays are all rebuilt
> using the correct partitions, but I'm scared to reboot.  Any thoughts
> as to how this could even happen?  I checked /etc/mdadm.conf and
> confirmed it has the right UUID data in it.  Any idea how I should
> move forward from here.  / is running xfs.  My thought is to boot off
> a live cd, do an xfs_repair, and go forward from there.

Drive letters can change, if -

1. A drive failed, and is no longer available to the system.  What was 
sdb previously now could be sdc.
2. You boot from a different drive or device.
3. Upon reboot, your BIOS just decides it wants to change them.

If md3 is built with sda3 and sdc3, are md1, md2 and md4 similarly built 
with corresponding partitions on sdc?

Assuming the drives in your RAID1 arrays are different sizes than in 
your RAID6 arrays, you can use fdisk -l to verify that drive letters are 
assigned to the devices you think they should be.

If your md3 is truly built with one incorrect partition (sdc3), I would 
try marking sdc3 as failed, remove it from the array, and re-boot.  Your 
system should come up with the one good member of md3.

Hmmm...on second read of your message, it sounds like you've already 
managed to get the arrays back together again with the right members, 
and now you're just deailing with the filesystem.  Doing an XFS repair 
from a live CD seems logical, although I'm afraid that if the md3 array 
was already re-built and synced with an incorrect member, the damage may 
already be done, and you may have to restore from backups, depending on 
how bad it is.


More information about the mythtv-users mailing list