Bug 216714 - Issue with file system image created with mke2fs parameter -E offset
Summary: Issue with file system image created with mke2fs parameter -E offset
Status: RESOLVED INVALID
Alias: None
Product: File System
Classification: Unclassified
Component: ext4 (show other bugs)
Hardware: Intel Linux
: P1 normal
Assignee: fs_ext4@kernel-bugs.osdl.org
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-11-20 23:41 UTC by tmahmud
Modified: 2022-11-28 15:51 UTC (History)
2 users (show)

See Also:
Kernel Version: Linux 5.4.0-131-generic
Subsystem:
Regression: No
Bisected commit-id:


Attachments

Description tmahmud 2022-11-20 23:41:52 UTC
Overview:
We encountered an issue while using e2fsck to check the file system image created with the command “mke2fs -t ext4 -E offset=10”.
The issue was observed in e2fsprogs 1.46.5 and 1.45.6. 

According to the manpage of mke2fs, the parameter “-E offset”  is used to create the file system at an offset from the beginning of the device or file. This can be useful when creating disk images for virtual machines. No range is specified for this parameter. 

Below are the steps to reproduce:

truncate -s 500M image
mke2fs -t ext4 -E offset=10 image 500M
e2fsck -yf image

mke2fs output log:
mke2fs 1.46.5 (30-Dec-2021)
Discarding device blocks: done                            
Creating filesystem with 512000 1k blocks and 128016 inodes
Filesystem UUID: 6154bed2-c92b-45fe-b6db-97192ae46155
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done


e2fsck output log:
e2fsck 1.46.5 (30-Dec-2021)
ext2fs_open2: Bad magic number in super-block
e2fsck: Superblock invalid, trying backup blocks...
e2fsck: Bad magic number in super-block while trying to open image

The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>
 or
    e2fsck -b 32768 <device>
Comment 1 Lukas Czerner 2022-11-21 08:56:40 UTC
Hi,

the -E offset=10 creates a file system that is offset by 10 bytes from the beginning of the device or a file (as is written in the man page). It is no wonder then that the tools can't recognize the file system on the device or a file because the super block is not where it should be - it is offset by 10 bytes.

If you want to use such a file system you have to present it to the other tools, or even kernel, with the correct offset. It is not going to be automatic.

So for example this works:

mke2fs -q -t ext4 -E offset=10 image 500M
sudo losetup -o 10 --show -f image
/dev/loop0
sudo e2fsck -f /dev/loop0
e2fsck 1.46.6-rc1 (12-Sep-2022)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/loop0: 11/128016 files (0.0% non-contiguous), 26684/512000 blocks

I hope it helps.
Comment 2 tmahmud 2022-11-28 01:00:51 UTC
Hi Lukas,

Thank you for your reply. It makes sense to me now. I tried using the offset parameter with losetup and it works for me. 

I have another question related to this. Is there any way to find out the offset after creating the file system image? For example, after creating the image with mke2fs, I again have to provide the offset to losetup. Is there any tool or utility that tells us the offset value after creating the file system image? It would be very helpful if you could help me regarding this.

Thank you again for your time.
Comment 3 Lukas Czerner 2022-11-28 06:59:04 UTC
Hi,

that's a good question. I don't know of any existing tool that could do the scanning for you. You can either put something together yourself, or you could use blkid with --offset argument and try to guess the offset. I don't think there is any better way other than scan the beginning of the device looking for file system signature stored in superblock.

-Lukas
Comment 4 Theodore Tso 2022-11-28 15:51:17 UTC
I'm curious --- *why* are you wanting to create file systems with an offset to begin with?    The original reason why this feature was added was in a highly specialized case where someone was creating a system image for some embedded system or for a virtual machine.  In this use case, the image contained a partition table, and the offset feature was used to create a filesystem at the appropriate location as specified by the partition table.

A typical use case is as part of an automated build procedure where the system image (say, for an Android mobile device, or some ARM development board, such as a Beaglebone, Arduino, etc., or some Virtual machine), using the mke2fs -d option to pre-populate the file system with the root file system, or some data partition, etc.   Since Best Practices for such automated build systems involve creating a reproducible build, there is nothing precious on the file system that can't be replicated by re-running the the automated build.   So if the offset is wrong (which is to say, inconsistent with the partition table which was laid down using the same automated build system), the developer will just curse to themselves, and can determine the offset by looking at the build image creation script, and then adjust that offset to match with the offset that was set in the partition table.

So I'm a bit perplexed about why you were using a random starting offset for the file system, and why you can't seem to figure out the offset afterwards.  The typical approach is to RTFS (Read The Fine Shellscript) to determine the offset, and then to fix the perhaps not-so-fine shell script.  :-)

Note You need to log in before you can comment on or make changes to this bug.