Bug 197733

Summary: Unable to mount UFS 44bsd partitions
Product: IO/Storage Reporter: Yannick Le Teigner (yannick.leteigner)
Component: Block LayerAssignee: Jens Axboe (axboe)
Status: NEW ---    
Severity: low CC: richard
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: 4.11.6 Subsystem:
Regression: No Bisected commit-id:
Attachments: Patch to recognize FreeBSD partitions using absolute addressing

Description Yannick Le Teigner 2017-11-03 13:32:38 UTC
Starting with kernel 4.11.6, the UFS magic number (1954) is shifted up by 32,256 bytes, making it impossible to mount the partition.
I traced it back to this patch: https://patchwork.kernel.org/patch/9749695/

<-- without the patch:
host # hexdump /dev/nbd0p7 | grep -w '1954'
0002550 0001 0000 0000 0000 0000 0000 1954 0001  << magic number at low offset
0004550 0001 0000 0000 0000 0000 0000 1954 0001  <<
5306550 0001 0000 0000 0000 0000 0000 1954 0001
541ed60 0000 0000 0000 0000 0000 0000 1954 2b49
5447ee0 0000 0000 0000 0000 0000 0000 4ea8 1954
a608550 0001 0000 0000 0000 0000 0000 1954 0001
f90a550 0001 0000 0000 0000 0000 0000 1954 0001
14c0c550 0001 0000 0000 0000 0000 0000 1954 0001
19f0e550 0001 0000 0000 0000 0000 0000 1954 0001
1f210550 0001 0000 0000 0000 0000 0000 1954 0001
1f2c9960 0000 0000 0000 0000 0000 0000 6e0c 1954
1f3037e0 0000 0000 0000 0000 0000 0000 6ff2 1954
24512550 0001 0000 0000 0000 0000 0000 1954 0001
29814550 0001 0000 0000 0000 0000 0000 1954 0001
29873fe0 0000 0000 0000 0000 0000 0000 1954 3292
2eb16550 0001 0000 0000 0000 0000 0000 1954 0001
33e18550 0001 0000 0000 0000 0000 0000 1954 0001
3911a550 0001 0000 0000 0000 0000 0000 1954 0001
3e41c550 0001 0000 0000 0000 0000 0000 1954 0001
3e47fe60 0000 0000 0000 0000 0000 0000 1954 1261

<-- with patch
host # hexdump /dev/nbd0p7 | grep -w '1954'
52fe750 0001 0000 0000 0000 0000 0000 1954 0001 << high offset on the first match
5416f60 0000 0000 0000 0000 0000 0000 1954 2b49
54400e0 0000 0000 0000 0000 0000 0000 4ea8 1954
a600750 0001 0000 0000 0000 0000 0000 1954 0001
f902750 0001 0000 0000 0000 0000 0000 1954 0001
14c04750 0001 0000 0000 0000 0000 0000 1954 0001
19f06750 0001 0000 0000 0000 0000 0000 1954 0001
1f208750 0001 0000 0000 0000 0000 0000 1954 0001
1f2c1b60 0000 0000 0000 0000 0000 0000 6e0c 1954
1f2fb9e0 0000 0000 0000 0000 0000 0000 6ff2 1954
2450a750 0001 0000 0000 0000 0000 0000 1954 0001
2980c750 0001 0000 0000 0000 0000 0000 1954 0001
2986c1e0 0000 0000 0000 0000 0000 0000 1954 3292
2eb0e750 0001 0000 0000 0000 0000 0000 1954 0001
33e10750 0001 0000 0000 0000 0000 0000 1954 0001
39112750 0001 0000 0000 0000 0000 0000 1954 0001
3e414750 0001 0000 0000 0000 0000 0000 1954 0001
3e478060 0000 0000 0000 0000 0000 0000 1954 1261
3ffa5f50 0001 0000 0000 0000 0000 0000 1954 0001  << magic number now at the end
3ffa7f50 0001 0000 0000 0000 0000 0000 1954 0001  <<


0x5306550 - 0x52fe750 = 0x7E00 = 32,256


With the patch, the partition fails to mount:
host # qemu-nbd -c /dev/nbd0 /opt/images/user_images/instances/d40_R1.qcow2
host # partx -l /dev/nbd0
# 1:        63- 41929649 ( 41929587 sectors,  21467 MB)
# 5:        63-  7547376 (  7547314 sectors,   3864 MB)
# 6:   7547377- 21384118 ( 13836742 sectors,   7084 MB)
# 7:  21384119- 23480594 (  2096476 sectors,   1073 MB)
# 8:  23480595- 41929586 ( 18448992 sectors,   9445 MB)
host # mount -t ufs -o ufstype=44bsd /dev/nbd0p7 /mnt/img/
mount: wrong fs type, bad option, bad superblock on /dev/nbd0p7,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

host # dmesg
...
[  507.252786] ufs: ufs_fill_super(): bad magic number


# fdisk -l /dev/nbd0
Disk /dev/nbd0: 20 GiB, 21474836480 bytes, 41943040 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x55000000

Device      Boot Start      End  Sectors Size Id Type
/dev/nbd0p1 *       63 41929649 41929587  20G a5 FreeBSD
Comment 1 Richard Narron 2017-11-03 16:26:12 UTC
Can you give the output of this: uname -a

Can you also give the output of "partx --show /dev/nbd0" after removing the patch?

Before the patch, Linux treated all 3 BSD partition types (FreeBSD, OpenBSD and NetBSD) as having absolute block addressing for the subpartitions.

This may be the way that FreeBSD used to do things but in recent releases (Freebsd 10 and FreeBSD 11) relative block addressing is used.

Another work-around might be to copy the partition somewhere and change the partition id from a5 (FreeBSD) to either a6 (OpenBSD) or a9 (NetBSD).

The fdisk utility using the t command can change it.
Comment 2 Yannick Le Teigner 2017-11-03 17:16:37 UTC
<-- without the patch:
gentoo-vm ~ # uname -a
Linux gentoo-vm 4.11.5 #7 SMP Wed Nov 1 21:36:27 -00 2017 x86_64 Intel Core Processor (Skylake) GenuineIntel GNU/Linux
gentoo-vm ~ # partx --show /dev/nbd0
NR    START      END  SECTORS    SIZE NAME UUID
 1       63 41929649 41929587     20G      55000000-01
 5       63  7547376  7547314    3.6G
 6  7547377 21384118 13836742    6.6G
 7 21384119 23480594  2096476 1023.7M
 8 23480595 41929586 18448992    8.8G

<-- with the patch:
gentoo-vm ~ # uname -a
Linux gentoo-vm 4.11.6-gentoo #8 SMP Wed Oct 25 15:24:33 -00 2017 x86_64 Intel Core Processor (Skylake) GenuineIntel GNU/Linux
gentoo-vm ~ # partx --show /dev/nbd0
NR    START      END  SECTORS    SIZE NAME UUID
 1       63 41929649 41929587     20G      55000000-01
 5       63  7547376  7547314    3.6G
 6  7547377 21384118 13836742    6.6G
 7 21384119 23480594  2096476 1023.7M
 8 23480595 41929586 18448992    8.8G


I've tried to use the workaround, and it seems to work!

gentoo-vm ~ # ls /dev/nbd0*
/dev/nbd0  /dev/nbd0p1  /dev/nbd0p5  /dev/nbd0p6  /dev/nbd0p7  /dev/nbd0p8

gentoo-vm ~ # fdisk /dev/nbd0

Welcome to fdisk (util-linux 2.28.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): p
Disk /dev/nbd0: 20 GiB, 21474836480 bytes, 41943040 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x55000000

Device      Boot Start      End  Sectors Size Id Type
/dev/nbd0p1 *       63 41929649 41929587  20G a5 FreeBSD


Command (m for help): t
Selected partition 1
Partition type (type L to list all types): a6
Changed type of partition 'FreeBSD' to 'OpenBSD'.

Command (m for help): p
Disk /dev/nbd0: 20 GiB, 21474836480 bytes, 41943040 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x55000000

Device      Boot Start      End  Sectors Size Id Type
/dev/nbd0p1 *       63 41929649 41929587  20G a6 OpenBSD

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

gentoo-vm ~ # hexdump /dev/nbd0p7 | grep -w '0001 0000 0000 0000 0000 0000 1954 0001'
0002550 0001 0000 0000 0000 0000 0000 1954 0001
0004550 0001 0000 0000 0000 0000 0000 1954 0001
...
gentoo-vm ~ # mount -t ufs -o ufstype=44bsd /dev/nbd0p7 /mnt/img/
gentoo-vm ~ # df -h | grep nbd
/dev/nbd0p7    1007M  2.0K  926M   1% /mnt/img
Comment 3 Richard Narron 2017-11-03 19:44:38 UTC
I want to try and duplicate the problem, so I will test with old FreeBSD systems.

I will install a machine with FreeBSD 6.0, see if it has the problem and then go from there.
Comment 4 Richard Narron 2017-11-05 17:32:51 UTC
Success.

I duplicated the problem by installing FreeBSD 6.0 on a Linux 4.4.88 x86 system (which apparently now includes the patch).

Linux could not see the FreeBSD subpartitions.

I then used fdisk to change the FreeBSD partition id A5 to A9 (NetBSD), rebooted Linux and then I saw the FreeBSD partitions. 

Here are a couple of ways to fix this issue. 

1) Figure out an easy way to tell the difference between older FreeBSD partitions using absolute addressing versus newer FreeBSD partitions using relative addressing.  

2) Create a Linux kernel parameter to turn off FreeBSD relative subpartition addressing.

I will investigate and try fix 1)

If it is not easy we can go with 2)
Comment 5 Richard Narron 2017-11-06 22:15:53 UTC
Created attachment 260525 [details]
Patch to recognize FreeBSD partitions using absolute addressing

It turns out that there was a easy solution and attached is a patch.

The new patch checks the FreeBSD C partition offset and if it is zero then 
relative addressing is used.
Comment 6 Yannick Le Teigner 2017-11-07 01:08:01 UTC
That was quick! Patch is working fine on my machine with the original FreeBSD test image. :)

Thanks!
Comment 7 Richard Narron 2017-11-08 05:20:41 UTC
After more research, I determined that Linux stopped reading FreeBSD UFS partitions when FreeBSD 9 was released.

My original patch fixed version 9 and later versions but broke FreeBSD 8 and all earlier versions.

This new patch will fix all versions of FreeBSD.
Comment 8 Richard Narron 2018-02-20 03:00:16 UTC
The patch is applied to Linux 4.16 release candidate 1:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0a4b6e2f80aad46fb55a5cf7b1664c0aef030ee0

Look at block/partition/msdos.c

Hopefully the patch will filter back to older systems after 4.16 is formally released.

Thanks to Jens Axboe and Linus Torvalds.