Bug 195597 - Subvolume copy fails with "ERROR: empty stream is not considered valid"
Summary: Subvolume copy fails with "ERROR: empty stream is not considered valid"
Status: RESOLVED PATCH_ALREADY_AVAILABLE
Alias: None
Product: File System
Classification: Unclassified
Component: btrfs (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: Josef Bacik
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-04-27 20:07 UTC by Nazar Mokrynskyi
Modified: 2017-05-09 18:13 UTC (History)
4 users (show)

See Also:
Kernel Version: 4.10
Tree: Mainline
Regression: No


Attachments
btrfs-progs: send: fail on first -ENODATA only (1.60 KB, patch)
2017-05-01 13:11 UTC, Christian Brauner
Details | Diff

Description Nazar Mokrynskyi 2017-04-27 20:07:33 UTC
I have 2 partitions and sometimes I synchronize subvolumes between them.

Last time synchronization happened within 15minutes window after 2017-04-04 14:30:06 (my local time) and was successful without any issues.

Now I'm trying to sync subvolumes once again and getting errors:
root@nazar-pc:~# /bin/btrfs send -p "/media/Backup/web/2017-04-04_06:20:52" "/media/Backup/web/2017-04-04_14:30:06" | /bin/btrfs receive "/media/Backup_backup/web"
At subvol /media/Backup/web/2017-04-04_14:30:06
At snapshot 2017-04-04_14:30:06
ERROR: empty stream is not considered valid
root@nazar-pc:~# /bin/btrfs subvolume delete "/media/Backup_backup/web/2017-04-04_14:30:06"
Delete subvolume (no-commit): '/media/Backup_backup/web/2017-04-04_14:30:06'
root@nazar-pc:~# /bin/btrfs send  "/media/Backup/web/2017-04-04_14:30:06" | /bin/btrfs receive "/media/Backup_backup/web"
At subvol /media/Backup/web/2017-04-04_14:30:06
At subvol 2017-04-04_14:30:06
ERROR: empty stream is not considered valid


I've compared two snapshots (one that existed after previous synchronization and new manually copied) with `diff -qr '/media/Backup_backup/web/2017-04-04_09:15:17' '/media/Backup_backup/web/2017-04-04_14:30:06'` and it appears to be sent fine, so looks like false-positive error.

I'm keeping filesystems as is so that I can do more tests as requested.

btrfs-progs at the moment of previous sync was Ubuntu's package with version 4.9.1-1, current version is Ubuntu's package with version 4.9.1-1ubuntu1.
Changelog between these 2 versions is following:
  * Fix btrfs send/receive:
    - 0001-btrfs-progs-fix-btrfs-send-receive-with-e-flag.patch
    - 0002-tests-use-receive-e-to-terminate-on-end-marker.patch
Comment 1 lakshmipathi 2017-04-28 08:27:39 UTC
Comparing above  patch https://patchwork.kernel.org/patch/9643035/ 
with latest btrfs-progs https://github.com/kdave/btrfs-progs/blob/master/cmds-receive.c#L1205 

Seems like above patch not yet accepted into btrfs-progs? I can see below code is missing on btrfs-progs master branch.

> +             if (ret < 0 && ret == -ENODATA) {
> +                     /* Empty stream is invalid */
>                       error("empty stream is not considered valid");
>                       ret = -EINVAL;
>                       goto out;
> +             } else if (ret < 0) {
> +                     goto out;


Can you try with latest btrfs-progs from https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git and see whether you have still this issue?
Comment 2 Nazar Mokrynskyi 2017-04-28 09:19:49 UTC
I've just compiled master branch (commit bd0ab27afbf14370f9f0da1f5f5ecbb0adc654c1) and it works fine:
root@nazar-pc:~# /media/Data/btrfs-progs-master/btrfs send -p "/media/Backup/web/2017-04-04_06:20:52" "/media/Backup/web/2017-04-04_14:30:06" | /media/Data/btrfs-progs-master/btrfs receive "/media/Backup_backup/web"
At subvol /media/Backup/web/2017-04-04_14:30:06
At snapshot 2017-04-04_14:30:06
root@nazar-pc:~# /media/Data/btrfs-progs-master/btrfs --version
btrfs-progs v4.10.2

Apparently it wasn't merged yet. However it is already in Ubuntu's repositories, so if that patch was indeed needed, it should be corrected.

Additional info:
My setup consists of primary SSD where I do regular snapshots, these snapshots are immediately sent to online HDD and then from time to time to the offline HDD. This bug describes sending snapshots from the online HDD to the offline HDD, however, sending snapshots from SSD to the online HDD works fine. There might be some more tricks happening, I'm still keeping affected partition to test possible fix in future.

Here is an exact patch applied to latest version of Ubuntu's package: https://launchpadlibrarian.net/314579804/btrfs-progs_4.9.1-1_4.9.1-1ubuntu1.diff.gz
Comment 3 lakshmipathi 2017-04-28 09:27:23 UTC
(In reply to Nazar Mokrynskyi from comment #2)
> I've just compiled master branch (commit
> bd0ab27afbf14370f9f0da1f5f5ecbb0adc654c1) and it works fine:
> root@nazar-pc:~# /media/Data/btrfs-progs-master/btrfs send -p
> "/media/Backup/web/2017-04-04_06:20:52"
> "/media/Backup/web/2017-04-04_14:30:06" |
> /media/Data/btrfs-progs-master/btrfs receive "/media/Backup_backup/web"
> At subvol /media/Backup/web/2017-04-04_14:30:06
> At snapshot 2017-04-04_14:30:06
> root@nazar-pc:~# /media/Data/btrfs-progs-master/btrfs --version
> btrfs-progs v4.10.2
> 
> Apparently it wasn't merged yet. However it is already in Ubuntu's
> repositories, so if that patch was indeed needed, it should be corrected.


I shared this bug with patch dev, hopefully he will look into it.
Comment 4 Nazar Mokrynskyi 2017-04-28 10:42:34 UTC
I'm sorry for confusion, the same error indeed happens in all cases. Regular backups are happening in cron task so I haven't seen the output, but snapshots look fine.
Comment 5 lakshmipathi 2017-04-28 12:27:27 UTC
thanks for the details.Yes, received your email on mailing list too.
Comment 6 Christian Brauner 2017-04-30 21:01:07 UTC
I've sent a patch to the btrfs mailing list that tries to address this problem.
My tests show that this should work. Please test, if possible.

Patch:
https://mail-archive.com/linux-btrfs@vger.kernel.org/msg63609.html
Comment 7 Nazar Mokrynskyi 2017-04-30 21:40:21 UTC
I've applied latest patch on top of previous patch on top of mentioned upstream 4.10.2 (commit bd0ab27afbf14370f9f0da1f5f5ecbb0adc654c1) and the issue is gone now, thanks!
Comment 8 Christian Brauner 2017-05-01 13:11:09 UTC
Created attachment 256157 [details]
btrfs-progs: send: fail on first -ENODATA only
Comment 9 lakshmipathi 2017-05-01 18:12:32 UTC
I think this bug can be marked as 'fixed'. Nazar, can change the state? (I don't see options for that)
Comment 10 Nazar Mokrynskyi 2017-05-01 18:13:41 UTC
Sure
Comment 11 David Sterba 2017-05-09 18:13:40 UTC
Thank you all, patch merged and scheduled for 4.11 release.

Note You need to log in before you can comment on or make changes to this bug.