Bug 95201

Summary:	btrfs snapshots have bogus unreproducible atimes on empty dirs that were originally subvol parents
Product:	File System	Reporter:	Paul Harvey (csirac2+kernel.org)
Component:	btrfs	Assignee:	Josef Bacik (josef)
Status:	NEW ---
Severity:	normal	CC:	csirac2+kernel.org, leho, samjnaa, szg00000
Priority:	P1
Hardware:	All
OS:	Linux
Kernel Version:	4.0.0-rc4 f897522468	Subsystem:
Regression:	No	Bisected commit-id:
Attachments:	shell script reliably demonstrates inconsistent atimes script demonstrating inconsistent atimes and btrfs send/receive missing dirs

Description Paul Harvey 2015-03-22 02:39:26 UTC

Created attachment 171601 [details]
shell script reliably demonstrates inconsistent atimes

I originally reported at [1] using debian kernel 3.16.7-ckt4-3 and btrfs-tools 3.17-1.1. I have since reproduced with latest kernel checked out from torvalds' master branch (see subject) and same version of btrfs-tools.

I have attached a script, also available at [2], which:
- Creates a btrfs filesystem and mounts it
- mkdir empty_dir
- Tests that this plain old empty_dir has a stable, repeatable atime
- btrfs subvolume create subvol
- Tests that this path 'subvol' has a stable, repeatable atime
- Creates a snapshot of the btrfs parent/default filesystem
- Tests that the snapshot's version of 'empty_dir' has a stable, repeatable atime
- Tests that the snapshot's version of 'subvol' has a stable, repeatable atime, which fails.

Expected behaviour:
- Empty directories which happened to be the path of a subvolume in the original filesystem should behave exactly as empty directories created with mkdir

Actual behaviour:
- It seems that reading atime on these problem directories within the snapshot actually returns the time of day.

Impact of this bug:
- I can't "measure" snapshots and report a stable, reproducible sha512sum of their contents.
- I can't create a reproducible PGP signature of same.
- This reduces the usefulness of snapshots as a means for intermediate backups.

Here's an example of a btrfs filesystem mounted under /mnt, and taking a sha512sum measurement which will give a different answer every time you run it:

cd /mnt/.snapshotz/2015-03-10T131520+1100
find . -xdev -print0 | LC_ALL=C sort -z | tar --null -T - --no-recursion \
--preserve-permissions --one-file-system -c --warning=file-ignored \
--to-stdout | sha512sum -b

Instead, I have to make a list of all paths in the snapshot under which subvolumes existed (including other snapshots!), and then save that file somewhere which I can pass to tar as --exclude-from. Here's an example of the final invocation using this work-around:

cd /mnt/.snapshotz/2015-03-10T131520+1100
find . -xdev -print0 | LC_ALL=C sort -z | tar --null -T - --no-recursion \
--preserve-permissions --one-file-system -c --warning=file-ignored \
--to-stdout --exclude-from='./.snapshot_measurements.exclude' | sha512sum -b

The ".snapshot_measurements.exclude" file contains lines like:
.snapshot_measurements.exclude
some/subvol
some/other/subvol
.snapshotz/2015-03-08T131504+1100
.snapshotz/2015-03-09T131512+1100

I initially encountered this issue when trying to make use of read-only snapshots for reproducible builds at work; but I've also hit this in the course of trying to tidy up my backup scripts into something that may be useful to others [3]

[1] http://permalink.gmane.org/gmane.comp.file-systems.btrfs/43452
[2] https://gist.github.com/csirac2/c2b5b2b9d0193b3c08a8
[3] https://github.com/csirac2/snazzer

Comment 1 Paul Harvey 2015-04-23 04:12:42 UTC

I've updated the gist/script at [1] to also demonstrate that there is an inconsistency with btrfs send/receive, i.e. these empty problem dirs with the unstable atimes are not received at all on the destination filesystem with btrfs receive.

This further complicates measuring/comparing snapshot integrity.

[1] https://gist.github.com/csirac2/c2b5b2b9d0193b3c08a8

Comment 2 Paul Harvey 2015-04-23 04:47:13 UTC

Created attachment 174831 [details]
script demonstrating inconsistent atimes and btrfs send/receive missing dirs

This is a refinement so that we now demonstrate that btrfs receive fails to create the problem directories on a target filesystem; and so not only does the source filesystem have bogus atimes on these problem directories, the destination of a send/receive operation doesn't have those directories at all, which complicates snapshot integrity measurements.