Bug 16227 - Mount command sees partition 2 and partition 7 as same partition
Summary: Mount command sees partition 2 and partition 7 as same partition
Status: RESOLVED INVALID
Alias: None
Product: File System
Classification: Unclassified
Component: ext3 (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: Eric Sandeen
URL: http://bugs.centos.org/view.php?id=3601
Keywords:
Depends on:
Blocks:
 
Reported: 2010-06-16 16:29 UTC by Eugene Blanchard
Modified: 2012-07-20 13:09 UTC (History)
2 users (show)

See Also:
Kernel Version: 2.6.18-164.6.1.el5
Subsystem:
Regression: No
Bisected commit-id:


Attachments

Description Eugene Blanchard 2010-06-16 16:29:01 UTC
When mounting multiple partitions, partition 7 sees partition 2 instead of partition 7. Partition scheme is as follows:

/dev/hda1 /boot
/dev/hda2 /
/dev/hda3 swap
/dev/hda4 extended partition
/dev/hda5 / (copy of /dev/hda2)
/dev/hda6 / (copy of /dev/hda2)
/dev/hda7 / (copy of /dev/hda2)

After copying /dev/hda2 to each partition using Gparted, I boot to /dev/hda2 and mount /dev/hda5, /dev/hda6 and /dev/hda7 to /mnt/p5, /mnt/p6, /mnt/p7 respectively. /mnt/p5 and /mnt/p6 work as expected.

Here's the problem: /mnt/p7 - any modifications to it's files, modifies /dev/hda2 partition's files and vice versa. 

I have reproduced this across several different machines: HP DL360s, PCs, etc.. using PATA, SATA and SCSI drives. 

If I only mount p7, then partition p7 works as expected. This bug is originally documented in the CentOS bug tracker listed in the URL.
Comment 1 Eugene Blanchard 2010-06-16 16:33:20 UTC
More details available at CentOS bug tracker URL listed above.
Comment 2 Eric Sandeen 2010-06-16 17:58:12 UTC
This is certainly not an ext3 bug, and almost certainly not a kernel bug; my money is on user error, but there's really not enough information to know.

output from:

# fdisk -l /dev/sda

and

# cat /proc/mounts

when you see this problem would be a start.  I'll take a quick look at that output, but I'm hovering over the "NOTABUG" button, at least for the kernel bug tracker.
Comment 3 Eugene Blanchard 2010-06-17 16:59:00 UTC
Using 7 partitions, this bug is reproduceable using SATA, PATA and SCSI drives on multiple servers. I first ran into this in 2009-05-01 and posted to the CentOS bug tracker. Today, I reloaded my server configuration and went to 9 partitions using the exact same documented procedure and all partitions functioned properly without interaction as noted in this bug report. This bug may only pertain to 7 partitions and the interaction between the primary partition 2 and the logical partition 7.

fdisk -l /dev/cciss/c0d0

Disk /dev/cciss/c0d0: 72.8 GB, 72833679360 bytes
255 heads, 63 sectors/track, 8854 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

           Device Boot      Start         End      Blocks   Id  System
/dev/cciss/c0d0p1   *           1          13      104391   83  Linux
/dev/cciss/c0d0p2              14         905     7164990   83  Linux
/dev/cciss/c0d0p3            8758        8854      779152+  82  Linux swap / Solaris
/dev/cciss/c0d0p4             906        8757    63071190    5  Extended
/dev/cciss/c0d0p5             906        1798     7172991   83  Linux
/dev/cciss/c0d0p6            1799        2691     7172991   83  Linux
/dev/cciss/c0d0p7            2692        3584     7172991   83  Linux

cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext3 rw,data=ordered 0 0
/dev /dev tmpfs rw 0 0
/proc /proc proc rw 0 0
/sys /sys sysfs rw 0 0
/proc/bus/usb /proc/bus/usb usbfs rw 0 0
devpts /dev/pts devpts rw 0 0
/dev/cciss/c0d0p1 /boot ext3 rw,data=ordered 0 0
tmpfs /dev/shm tmpfs rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
/etc/auto.misc /misc autofs rw,fd=7,pgrp=2909,timeout=300,minproto=5,maxproto=5,indirect 0 0
-hosts /net autofs rw,fd=13,pgrp=2909,timeout=300,minproto=5,maxproto=5,indirect 0 0
Comment 4 Eric Sandeen 2010-06-17 17:33:57 UTC
I was sorta hoping for /proc/mounts when you saw this problem of one mount affecting the other; you only have /dev/cciss/c0d0p1 mounted there it seems.

My money is on mounting the same device to 2 mountpoints.
Comment 5 Eugene Blanchard 2010-06-17 17:42:29 UTC
After booting to parition p2 root / (p1 is /boot, p3 is swap)

I created the following directories:

/mnt/p5
/mnt/p6
/mnt/p7

Then mounted using the following commands:

mount /dev/cciss/c0d0p5 /mnt/p5
mount /dev/cciss/c0d0p5 /mnt/p6
mount /dev/cciss/c0d0p5 /mnt/p7
Comment 6 Eric Sandeen 2010-06-17 17:44:48 UTC
You have mounted the same device to 3 different mount points; of course a change in one will show up on the other.  It's the same as a bind mount.
Comment 7 Eugene Blanchard 2010-06-17 18:11:48 UTC
That's a typo here, I was just trying to correct it at the same time that you responded. 

mount /dev/cciss/c0d0p5 /mnt/p5
mount /dev/cciss/c0d0p6 /mnt/p6
mount /dev/cciss/c0d0p7 /mnt/p7

If this was a unique situation then I would agree that it could be pebkac (problem exists between keyboard and chair) but it's occurred on 3 different classes of servers: PC, DL320 and DL360. 

I've got a spare server that I can bring up and mount the other partitions. Normally, only one partition is booted to at a time. Post-secondary VoIP lab each student pair gets there own partition/PBX server to work on.
Comment 8 Eugene Blanchard 2010-06-17 18:19:53 UTC
 cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext3 rw,data=ordered 0 0
/dev /dev tmpfs rw 0 0
/proc /proc proc rw 0 0
/sys /sys sysfs rw 0 0
/proc/bus/usb /proc/bus/usb usbfs rw 0 0
devpts /dev/pts devpts rw 0 0
/dev/cciss/c0d0p1 /boot ext3 rw,data=ordered 0 0
tmpfs /dev/shm tmpfs rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
/etc/auto.misc /misc autofs rw,fd=6,pgrp=2472,timeout=300,minproto=5,maxproto=5,indirect 0 0
-hosts /net autofs rw,fd=12,pgrp=2472,timeout=300,minproto=5,maxproto=5,indirect 0 0
/dev/cciss/c0d0p5 /mnt/p5 ext3 rw,data=ordered 0 0
/dev/cciss/c0d0p6 /mnt/p6 ext3 rw,data=ordered 0 0
/dev/cciss/c0d0p7 /mnt/p7 ext3 rw,data=ordered 0 0

Typically, I modify the /etc/hosts file for the server name on the partitions. The strange part is that if I go back and forth between modifying p2 and p7's hosts file then eventually things work out.
Comment 9 Eugene Blanchard 2010-06-17 18:27:51 UTC
Your a little too fast on the resolved draw...
Comment 10 Eric Sandeen 2010-06-17 18:42:57 UTC
(In reply to comment #9)
> Your a little too fast on the resolved draw...

only based on your replies :)

Maybe you can attach a sequence of events:

# fdisk -l /dev/cciss/c0d0
# mount /dev/cciss/c0d0p5 /mnt/p5
# mount /dev/cciss/c0d0p6 /mnt/p6
# mount /dev/cciss/c0d0p7 /mnt/p7
# cat /proc/partitions
# echo "Unique content" > /mnt/p7/unique-filename
# cat /mnt/p2/unique-filename

(or whatever the relevant problematic mountpoints are)
Comment 11 Eugene Blanchard 2010-06-17 18:51:44 UTC
I was thinking the same. It'll take me an hour or so to get the server up to where the problem shows.
Comment 12 Eugene Blanchard 2010-06-17 19:37:26 UTC
 fdisk -l /dev/cciss/c0d0

Disk /dev/cciss/c0d0: 72.8 GB, 72833679360 bytes
255 heads, 32 sectors/track, 17433 cylinders
Units = cylinders of 8160 * 512 = 4177920 bytes

           Device Boot      Start         End      Blocks   Id  System
/dev/cciss/c0d0p1   *           1          26      104391   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/cciss/c0d0p2              27         861     3406800   83  Linux
/dev/cciss/c0d0p3             862        1050      771120   82  Linux swap / Solaris
/dev/cciss/c0d0p4            1051       17433    66842640    5  Extended
/dev/cciss/c0d0p5            1051        1886     3410864   83  Linux
/dev/cciss/c0d0p6            1887        2722     3410864   83  Linux
/dev/cciss/c0d0p7            2723        3558     3410864   83  Linux

root@Alpha-Brown:/mnt $ mount /dev/cciss/c0d0p5 p5
root@Alpha-Brown:/mnt $ mount /dev/cciss/c0d0p6 p6
root@Alpha-Brown:/mnt $ mount /dev/cciss/c0d0p7 p7
root@Alpha-Brown:/mnt $ cat /proc/partitions
major minor  #blocks  name

 104     0   71126640 cciss/c0d0
 104     1     104391 cciss/c0d0p1
 104     2    3406800 cciss/c0d0p2
 104     3     771120 cciss/c0d0p3
 104     4          1 cciss/c0d0p4
 104     5    3410864 cciss/c0d0p5
 104     6    3410864 cciss/c0d0p6
 104     7    3410864 cciss/c0d0p7

Can't reproduce it - hate to say it but must be pebkac. If I run into it again, then I'll know what info you need.
Comment 13 Eugene Blanchard 2011-06-30 19:40:19 UTC
About once a year I run into this bug. I re-image 8 servers with multiple partitions in this manner every summer in preparation for the next school year.

It appears to be a fstab problem. When the boot and root partitions are configured using a LABEL="...." With multiple partitions (c0d0p5 to c0d0p7) on an extended partition (c0d0p4), you then run into problems when you mount the partitions on the root partition (c0d0p2). The last partition mounted (c0d0p7) becomes mounted to c0d0p2 instead of the correct partition. Any changes to c0d0p7 change c0d0p2 and vice versa. 

If you change the fstab from a LABEL=/ and LABEL=/boot to /dev/cciss/c0d0p1 for /boot and /dev/cciss/c0d0p2 for root dir "/" then everything works the way it should. I have verified this bug using kernel 2.6.18-239.9.1. 

It appears that if you reboot the server from c0d0p2 and boot to c0d0p7 then during the boot messages, a reference to the fstab LABEL= pointing to c0d0p2 appears and the server mounts c0d0p2 instead of c0d0p7.

Is there a persistence with the fstab LABELs during a reboot? Maybe a grub issue?

Good fstab:

/dev/cciss/c0d0p2       /                       ext3    defaults        1 1
/dev/cciss/c0d0p1       /boot                   ext3    defaults        1 2
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
/dev/cciss/c0d0p3       swap                    swap    defaults        0 0
Comment 14 Alan 2012-07-20 13:09:00 UTC
LABEL= is a distro tools thing and initrd not kernel
Comment 15 Eugene Blanchard 2012-07-20 13:09:18 UTC
I am on vacation and will be out of the office until August 28. I will be regularly checking my emails and voicemail but mostly I will be sipping sherry on the front porch watching the sun rise during this time period.

Note You need to log in before you can comment on or make changes to this bug.