Bug 43511 - Partitions: Amiga RDB partition on 2 TB disk way too big, while OK in AmigaOS 4.1
Partitions: Amiga RDB partition on 2 TB disk way too big, while OK in AmigaOS...
Status: RESOLVED CODE_FIX
Product: IO/Storage
Classification: Unclassified
Component: Block Layer
All Linux
: P1 high
Assigned To: Jens Axboe
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-06-17 06:31 UTC by Martin Steigerwald
Modified: 2012-06-19 19:49 UTC (History)
0 users

See Also:
Kernel Version: 3.4.1
Tree: Mainline
Regression: No


Attachments
mediatoolbox binary profile of RDB, should be just a binary copy (2.00 KB, application/octet-stream)
2012-06-17 06:44 UTC, Martin Steigerwald
Details
screenshot of mediatoolbox showing the RDB informations (158.85 KB, image/png)
2012-06-17 07:22 UTC, Martin Steigerwald
Details
screenshot of mediatoolbox´s view of the partitions with the first one selected (88.58 KB, image/png)
2012-06-17 07:23 UTC, Martin Steigerwald
Details

Description Martin Steigerwald 2012-06-17 06:31:56 UTC
Hi Jens!

I will write an mail to linux-kernel, linux-m68k and possibly linux-ide as well.

With my 2 TB mixed Amiga/Linux backup disk, which I partioned under AmigaOS 4.0/1 with Media Toolbox I get the following in Linux:


Jun 17 07:28:14 merkaba kernel: [30852.968978] sata_sil24 0000:05:00.0: enabling device (0000 -> 0003)
Jun 17 07:28:14 merkaba kernel: [30852.969401] scsi9 : sata_sil24
Jun 17 07:28:14 merkaba kernel: [30852.969533] ata10: SATA max UDMA/100 host m128@0xf1c02000 port 0xf1c00000 irq 19
Jun 17 07:28:16 merkaba kernel: [30855.163712] ata10: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
Jun 17 07:28:16 merkaba kernel: [30855.165014] ata10.00: ATA-8: Hitachi HDS5C3020ALA632, ML6OA580, max UDMA/133
Jun 17 07:28:16 merkaba kernel: [30855.165017] ata10.00: 3907029168 sectors, multi 16: LBA48 NCQ (depth 31/32)
Jun 17 07:28:16 merkaba kernel: [30855.166378] ata10.00: configured for UDMA/100
Jun 17 07:28:16 merkaba kernel: [30855.166477] scsi 9:0:0:0: Direct-Access     ATA      Hitachi HDS5C302 ML6O PQ: 0 ANSI: 5
Jun 17 07:28:16 merkaba kernel: [30855.166653] sd 9:0:0:0: [sdb] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
Jun 17 07:28:16 merkaba kernel: [30855.166699] sd 9:0:0:0: [sdb] Write Protect is off
Jun 17 07:28:16 merkaba kernel: [30855.166702] sd 9:0:0:0: [sdb] Mode Sense: 00 3a 00 00
Jun 17 07:28:16 merkaba kernel: [30855.166726] sd 9:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jun 17 07:28:16 merkaba kernel: [30855.200936]  sdb: RDSK (512) sdb1 (LNX^@)(res 2 spb 1) sdb2 (JXF^D)(res 2 spb 1) sdb3 (DOS^C)(res 2 spb 4)
Jun 17 07:28:16 merkaba kernel: [30855.200943] sdb: p1 size 18446744072560312368 extends beyond EOD, enabling native capacity
Jun 17 07:28:16 merkaba kernel: [30855.201344]  sdb: RDSK (512) sdb1 (LNX^@)(res 2 spb 1) sdb2 (JXF^D)(res 2 spb 1) sdb3 (DOS^C)(res 2 spb 4)
Jun 17 07:28:16 merkaba kernel: [30855.201347] sdb: p1 size 18446744072560312368 extends beyond EOD, truncated
Jun 17 07:28:16 merkaba kernel: [30855.201398] sdb: p2 start 18446744072560314432 is beyond EOD, truncated
Jun 17 07:28:16 merkaba kernel: [30855.201400] sdb: p3 start 18446744073189460080 is beyond EOD, truncated
Jun 17 07:28:16 merkaba kernel: [30855.201570] sd 9:0:0:0: [sdb] Attached SCSI disk


The first partition seems to be way to big:

merkaba:~#1> amiga-fdisk -l /dev/sdc
Disk /dev/sdc: 3 heads, 16 sectors, 81396441 cylinders, RDB: 0
Logical Cylinders from 43 to 81396440, 24576  bytes/Cylinder 

Device     Boot Mount   Begin      End     Size   Pri  BBlks    System
/dev/sdc1       *        43   65536043   1572864024     0      0  Linux native
/dev/sdc2       *    65536044   78643244   314572824     0      0  [unknown]
/dev/sdc3       *    78643245   81396440   66076704     0      0  Amiga FFS Int.

(sdc2 is JXFS, a new filesystem in AmigaOS 4 that is not known to Linux yet)


In cat /proc/partitions I get:

merkaba:~> cat /proc/partitions 
major minor  #blocks  name

 […]
   8       16 1953514584 sdb
   8       17 1953513552 sdb1
merkaba:~> 


Thus the Debian Linux Kernel 3.4.1 I am using here, truncates the first oversized partition and has no space for the other, too, which therefore are inaccessible under Linux.

I didn´t notice this initially, but it happened from the beginning, I have an old amiga-fdisk listing that is exactly the same.

Amiga Mediatoolbox has a different oppinion on the partition layout:

LVM aka sdc1: 1500 GByte, 43 to 65536043 cyl, size 65536001 Zylinder
BAK aka sdc2: 300 GByte, 65536044 to 78643244 cyl, size 13107201 Zylinder
TAUSCH2 aka sdc3: 63,016 GByte, 78643245 to  81396440 cyl, didn´t note the size here, but as far as I remember was okay as well

But it seems to be confused about the whole size of the disk as well:

Logical sizes:

Blocks per cylinder: 48
Cylinders: 81396441
Sectors: -397938128
Blocksize: 512

The sectors seem overflowed.

So it might be a problem with RDB and 2TB disks and nothing to do with Linux. But still on AmigaOS 4.1 I can access the two Amiga partitions after the Linux partition.



I have another 500GB disk for backup up my Sam440ep AmigaOS 4.1 "Amiga", I plan to repartition the 2 TB disk as GPT anyway, but since MediaToolBox in AmigaOS 4.1 has a different meaning about the partioning and this can cause serious data loss, I think its good to look at it.

I had a BTRFS filesystem that had some checksum errors. Maybe thats somehow related to this issue and AmigaOS and/or Linux overwrote something it shouldn´t have touched.

I will report a bug with AmigaOS 4.1 developers as well to get more details.
Comment 1 Martin Steigerwald 2012-06-17 06:32:50 UTC
Okay, detailed kernel version:


merkaba:~> cat /proc/version
Linux version 3.4-trunk-amd64 (Debian 3.4.1-1~experimental.1) (debian-kernel@lists.debian.org) (gcc version 4.6.3 (Debian 4.6.3-1) ) #1 SMP Wed Jun 6 10:34:53 CEST 2012
Comment 2 Martin Steigerwald 2012-06-17 06:44:49 UTC
Created attachment 73771 [details]
mediatoolbox binary profile of RDB, should be just a binary copy

A short intro on the Amiga RDB is at

http://en.wikipedia.org/wiki/Amiga_Rigid_Disk_Block

More detailed in the link there:

http://lclevy.free.fr/adflib/adf_info.html#p6


Seems to be a binary copy of the RDB:

martin@merkaba:~/Computer/Festplatten/Steigerwald/Amiga> hd profile-binary.img | head
00000000  52 44 53 4b 00 00 00 80  ef 69 b5 58 00 00 00 07  |RDSK.....i.X....|
00000010  00 00 02 00 00 00 00 17  ff ff ff ff 00 00 00 01  |................|
00000020  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
*
00000040  04 da 02 d9 00 00 00 10  00 00 00 03 00 00 00 01  |................|
00000050  04 da 02 d9 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000080  00 00 00 00 00 00 08 0f  00 00 00 2b 04 da 02 d8  |...........+....|
00000090  00 00 00 30 00 00 00 00  00 00 00 03 00 00 00 00  |...0............|
Comment 3 Martin Steigerwald 2012-06-17 06:49:50 UTC
Here the relevant information in mountlist format:


LVM aka sdc1:

	Device		= usbdisk.device
	Unit		= 0
	Flags		= 0
	Surfaces	= 3
	SectorsPerBlock	= 1
	BlocksPerTrack	= 16
	Reserved	= 2
	LowCyl		= 43
	HighCyl		= 65536043
	Buffers		= 100
	BufMemType	= 1
	MaxTransfer	= 0x7fffffff
	Mask		= 0x7ffffffc
	DosType		= 0x4c4e5800
	GlobVec		= -1
#


BAK aka sdc2:


	Device		= usbdisk.device
	Unit		= 0
	Flags		= 0
	Surfaces	= 3
	SectorsPerBlock	= 1
	BlocksPerTrack	= 16
	Reserved	= 2
	LowCyl		= 65536044
	HighCyl		= 78643244
	Buffers		= 5000
	BufMemType	= 1
	MaxTransfer	= 0x7fffffff
	Mask		= 0x7ffffffc
	DosType		= 0x4a584604
	GlobVec		= -1
#


TAUSCH2 aka sdc3:

	Device		= usbdisk.device
	Unit		= 0
	Flags		= 0
	Surfaces	= 3
	SectorsPerBlock	= 4
	BlocksPerTrack	= 16
	Reserved	= 2
	LowCyl		= 78643245
	HighCyl		= 81396440
	Buffers		= 2000
	BufMemType	= 1
	MaxTransfer	= 0x7fffffff
	Mask		= 0x7ffffffc
	DosType		= 0x444f5303
	GlobVec		= -1
#


From the Amiga side this looks correct. Even amiga-fdisk prints start and
end cylinders correctly:

merkaba:~> amiga-fdisk -l /dev/sdc
Disk /dev/sdc: 3 heads, 16 sectors, 81396441 cylinders, RDB: 0
Logical Cylinders from 43 to 81396440, 24576  bytes/Cylinder 

Device     Boot Mount   Begin      End     Size   Pri  BBlks    System
/dev/sdc1       *        43   65536043   1572864024     0      0  Linux native
/dev/sdc2       *    65536044   78643244   314572824     0      0  [unknown]
/dev/sdc3       *    78643245   81396440   66076704     0      0  Amiga FFS Int.

Its just the size of the first partitions that seems miscalculated, cause

65536043 - 43 is 65536000, not 1572864024

So are Linux kernel and amiga-fdisk overflowing there?
Comment 4 Martin Steigerwald 2012-06-17 07:22:54 UTC
Created attachment 73781 [details]
screenshot of mediatoolbox showing the RDB informations

All seems good except for the sectors count which seems to be overflowing. Whether thats just a cosmetic problem or related to this issue I do not know yet.

I will report this with AmigaOS developers to find out more.
Comment 5 Martin Steigerwald 2012-06-17 07:23:39 UTC
Created attachment 73791 [details]
screenshot of mediatoolbox´s view of the partitions with the first one selected

MediaToolbox shows correct size of the first partition.
Comment 6 Martin Steigerwald 2012-06-17 08:01:46 UTC
Raising importance to high. Granted not many people might use Amiga RDB disks
with Linux, but this seems to be a *serious* data loss issue. Cause I now found
out that the LVM I created on the disk is of the size of the whole disk, while
at least 300 GB is for the Amiga backup partitions. AmigaOS 4.1 sees the Amiga
backup partition. So the backups from Amiga and Linux may overwrite each other.
This might explain the BTRFS checksum errors as well.

merkaba:~> pvdisplay /dev/sdb1 
  --- Physical volume ---
  PV Name               /dev/sdb1
  VG Name               steigerwald
  PV Size               1,82 TiB / not usable 4,08 MiB
  Allocatable           yes 
  PE Size               4,00 MiB
  Total PE              476931
  Free PE               105731
  Allocated PE          371200
  PV UUID               ZXMECC-JAir-lX8Q-rLhS-W1cS-quwz-b3FXWp

merkaba:~> vgdisplay /dev/sdb1
  Volume group "sdb1" not found
merkaba:~#5> vgdisplay steigerwald
  --- Volume group ---
  VG Name               steigerwald
  System ID             
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  7
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                5
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               1,82 TiB
  PE Size               4,00 MiB
  Total PE              476931
  Alloc PE / Size       371200 / 1,42 TiB
  Free  PE / Size       105731 / 413,01 GiB
  VG UUID               uhjjE1-0yrD-Ch1A-d9qL-P5jY-UDXE-io4bpi


And basically thats explainable by:


merkaba:~> cat /proc/partitions 
major minor  #blocks  name
[…]
   8       16 1953514584 sdb
   8       17 1953513552 sdb1


Actually I see two issues in there:

1) Linux seems to miscalculate the size of the first partitions grossly.

2) It then *truncates* it to the whole disk and risks serious *data loss*
instead of refusing to do *anything* with the disk.

I will report another bug on issue 2. If a partition is out of bounds I think
the safest choice is to leave it untouched completely. Just truncating and
using it is asking for serious trouble.

I will scrub the BTRFS partitions and fsck the other partitions to test whether
the Amiga backup which I aborted after seeing this, to check for errors. Then I
will redo the disk as GPT as I need a reliable backup.
Comment 7 Martin Steigerwald 2012-06-19 19:47:11 UTC
Am Montag, 18. Juni 2012 schrieb jdow:
> On 2012/06/17 14:20, Martin Steigerwald wrote:
> > Am Sonntag, 17. Juni 2012 schrieb jdow:
> >> On 2012/06/17 09:36, Geert Uytterhoeven wrote:
> >>> On Sun, Jun 17, 2012 at 2:58 PM, Martin Steigerwald
> > 
> > <Martin@lichtvoll.de> wrote:
> >>>> Am Sonntag, 17. Juni 2012 schrieb jdow:
> >>>> | JXFS 64 bit file system
> >>>> | 
> >>>> | With AmigaOS 4.x a new file system has been introduced called
> >>>> | JXFS. It is a totally new 64 bit file system that supports
> >>>> | partitions up to 16 TB in size. It is a modern journalling file
> >>>> | system, which means that it reduces data loss if data writes to
> >>>> | the disk are interrupted. It is the fastest and most reliable
> >>>> | file system ever created for AmigaOS.
> >>>> 
> >>>> http://www.amigaos.net/content/1/features
> >>>> 
> >>>> Well I asked AmigaOS 4 developers about this issue as well. Lets
> >>>> see what they say about 2 TB limits.
> >>> 
> >>> 16 TB = 2 TB * 8. Perhaps they increased the block size from 512 to
> >>> 4096?
> >>> 
> >>> block/partitions/amiga.c reads the block size from
> >>> RigidDiskBlock.rdb_BlockBytes,
> >>> but after conversion to 512-byte blocks, all further calculations
> >>> are done on "int", so it will overflow for disks larger than 2
> >>> TiB.
> >>> 
> >>> Note that in your profile-binary.img, the field is 0x200, i.e. 512
> >>> bytes per block,
> >>> so I'll have to get a deeper look into your RDB first...
> > 
> > […]
> > 
> >> Note that the work I did on the Linux RDB code eons ago took full
> >> cognizance of the physical and virtual block sizes.
> > 
> > […]
> > 
> >> I've asked Martin for a digital copy of his RDBs and what he thinks
> >> the partition(s) should look like. I should also be told whether
> >> the disk is supposed to be solely Amiga OSs or not. I gather it's
> >> not.
> > 
> > Its all in the bug report. profile-binary.img should be it.
> > 
> > Its small so I attach it.
> > 
> > Meanwhile I try to get the currently supported maximum disk size out
> > from the OS 4 developers. Maybe JXFS is playing tricks that other
> > filesystems do not play or simply has a different implementation.
> > 
> > Thanks,
> 
> I've sent Jens a proposed fix changing these lines in amiga.c.
> ===8<--- was
>          struct PartitionBlock *pb;
>          int start_sect, nr_sects, blk, part, res = 0;
>          int blksize = 1;        /* Multiplier for disk block size */
> ===8<--- becomes changing line 338 into lines 338-339
>          struct PartitionBlock *pb;
>          sector_t start_sect, nr_sects;
>          int blk, part, res = 0;
>          int blksize = 1;        /* Multiplier for disk block size */
> ===8<---
> 
> (I'm working without proper tools at the moment or I'd make a real
> diff. Sorry.)
> 
> This fix may not be complete. The RDBs contain provisions for
> describing the physical disk block size and how many physical blocks
> are used in a file system's logical blocks. This MAY not work quite
> right large physical block sizes.

Way better:

dmesg:

Jun 19 21:19:09 merkaba kernel: [ 7891.819552] ata8: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
Jun 19 21:19:09 merkaba kernel: [ 7891.821306] ata8.00: ATA-8: Hitachi HDS5C3020ALA632, ML6OA580, max UDMA/133
Jun 19 21:19:09 merkaba kernel: [ 7891.821315] ata8.00: 3907029168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Jun 19 21:19:09 merkaba kernel: [ 7891.823252] ata8.00: configured for UDMA/100
Jun 19 21:19:09 merkaba kernel: [ 7891.823589] scsi 7:0:0:0: Direct-Access     ATA      Hitachi HDS5C302 ML6O PQ: 0 ANSI: 5
Jun 19 21:19:09 merkaba kernel: [ 7891.824126] sd 7:0:0:0: [sdb] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
Jun 19 21:19:09 merkaba kernel: [ 7891.824440] sd 7:0:0:0: [sdb] Write Protect is off
Jun 19 21:19:09 merkaba kernel: [ 7891.824452] sd 7:0:0:0: [sdb] Mode Sense: 00 3a 00 00
Jun 19 21:19:09 merkaba kernel: [ 7891.824531] sd 7:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jun 19 21:19:09 merkaba kernel: [ 7891.843284]  sdb: RDSK (512) sdb1 (LNX^@)(res 2 spb 1) sdb2 (JXF^D)(res 2 spb 1) sdb3 (DOS^C)(res 2 spb 4)
Jun 19 21:19:09 merkaba kernel: [ 7891.844055] sd 7:0:0:0: [sdb] Attached SCSI disk


cat /proc/partitions:
major minor  #blocks  name
[…]
   8       16 1953514584 sdb
   8       17 1572864024 sdb1
   8       18  314572824 sdb2
   8       19   66076704 sdb3

1572864024 + 314572824 + 66076704 = 1953513552 which is below the complete
size of the disk.


Partition start analysis:

merkaba:~> file -sk /dev/sdb1
/dev/sdb1: sticky LVM2 PV (Linux Logical Volume Manager), UUID: ZXMECC-JAir-lX8Q-rLhS-W1cS-quwz-b3FXWp, size: 2000397877248
merkaba:~> file -sk /dev/sdb2
/dev/sdb2: sticky data
merkaba:~> file -sk /dev/sdb3
/dev/sdb3: sticky Amiga Inter FFS disk

merkaba:~> hd /dev/sdb2 | head
00000000  4a 58 46 04 11 1a 0f 0c  00 00 00 00 00 00 00 00  |JXF.............|
00000010  00 00 00 00 00 11 00 00  3e db 3d 54 40 00 00 00  |........>.=T@...|
00000020  00 00 00 00 00 00 00 00  00 00 01 77 00 10 80 00  |...........w....|
00000030  00 00 01 c2 00 10 e0 00  25 80 00 30 00 00 02 00  |........%..0....|
00000040  00 00 00 30 00 00 00 00  00 00 00 00 00 00 00 00  |...0............|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000060  00 00 00 00 00 00 00 00  00 00 08 00 00 00 00 00  |................|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000200  54 52 4f 4b ab ad b0 b1  00 00 00 01 00 00 00 00  |TROK............|

I don´t know the on-disk format for JXFS, but this could be it.


amiga-fdisk looks the same as before:

merkaba:~> amiga-fdisk -l /dev/sdb
Disk /dev/sdb: 3 heads, 16 sectors, 81396441 cylinders, RDB: 0
Logical Cylinders from 43 to 81396440, 24576  bytes/Cylinder 

Device     Boot Mount   Begin      End     Size   Pri  BBlks    System
/dev/sdb1       *        43   65536043   1572864024     0      0  Linux native
/dev/sdb2       *    65536044   78643244   314572824     0      0  [unknown]
/dev/sdb3       *    78643245   81396440   66076704     0      0  Amiga FFS Int.

But obviously its right anyway: It seems to display the size in blocks,
not in cylinders.


Media Toolbox says:

LVM aka sdc1: 1500 GByte, 43 to 65536043 cyl, size 65536001 Zylinder
BAK aka sdc2: 300 GByte, 65536044 to 78643244 cyl, size 13107201 Zylinder
TAUSCH2 aka sdc3: 63,016 GByte, 78643245 to  81396440 cyl, didn´t note the size
here, but as far as I remember was okay as well

with

Blocks per cylinder: 48
Cylinders: 81396441

So thats:

65536001 * 48 / 2 = 1572864024

13107201 * 48 / 2 = 314572824

( 81396440 - 78643245 ) * 48 / 2 = 66076680

(I verified from a windowshot I made that 81396440 - 78643245 = 2753195
is indeed displayed by Media Toolbox as size of the partition)



So this is looking pretty good.

Many thanks, Joanne for your detective work and the fix.

Tested with:

martin@merkaba:~> cat /proc/version
Linux version 3.5.0-rc3-fix-bug-43511+ (martin@merkaba) (gcc version 4.7.0 (Debian 4.7.1-1) ) #1 SMP Tue Jun 19 14:31:56 CEST 2012


Tested-By: Martin Steigerwald <martin@lichtvoll.de>
Reviewed-By: Martin Steigerwald <martin@lichtvoll.de>


Patch from Joanne in diff format:

martin@merkaba:~/Computer/Merkaba/Kernel/linux-2.6> git diff HEAD~ | cat
diff --git a/block/partitions/amiga.c b/block/partitions/amiga.c
index 70cbf44..42d36f9 100644
--- a/block/partitions/amiga.c
+++ b/block/partitions/amiga.c
@@ -29,7 +29,8 @@ int amiga_partition(struct parsed_partitions *state)
        unsigned char *data;
        struct RigidDiskBlock *rdb;
        struct PartitionBlock *pb;
-       int start_sect, nr_sects, blk, part, res = 0;
+       sector_t start_sect, nr_sects;
+       int blk, part, res = 0;
        int blksize = 1;        /* Multiplier for disk block size */
        int slot = 1;
        char b[BDEVNAME_SIZE];



Thanks,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7
Comment 8 Martin Steigerwald 2012-06-19 19:49:14 UTC
Link to the thread with above message from today:

Partitions: Amiga RDB partition on 2 TB disk way too big, while OK in AmigaOS 4.1
https://lkml.org/lkml/2012/6/17/6

Note You need to log in before you can comment on or make changes to this bug.