Bug 210521 - btrfs replace doesn't honor the -r option on raid5 and raid6
Summary: btrfs replace doesn't honor the -r option on raid5 and raid6
Status: NEW
Alias: None
Product: File System
Classification: Unclassified
Component: btrfs (show other bugs)
Hardware: All Linux
: P1 low
Assignee: BTRFS virtual assignee
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-12-06 19:27 UTC by Andrej Podzimek
Modified: 2021-02-24 18:05 UTC (History)
0 users

See Also:
Kernel Version: 5.9.11
Subsystem:
Regression: No
Bisected commit-id:


Attachments

Description Andrej Podzimek 2020-12-06 19:27:42 UTC
I gradually replaced all disks in an array (-d raid6 -m raid1c3) with SSDs.

For the last replacement I was expecting a lightning fast progress from the -r option, because all other volumes were already SSDs and I thought that -r would avoid reading from the last spinning disk and restore its data from parity instead.

This was not the case and the data transfer rate during the last device replacement was limited by the last spinning disk's capabilities. The spinning disk was, indeed, heavily utilized (based on iostat).

This could become a problem if (e.g.) the spinning disk was faulty and reading at 1 MB/s. In that case one may have to remove the slow/failing disk first and then specify the device to replace using a devid. Yet that comes extra risk (e.g. pulling the wrong drive) and is hard to carry out remotely on a machine with available hot spares.

(As a side note, the btrfs-replace man page only mentions mirrors in its description of -r; it doesn't say whether and how -r works with raid5/6.)
Comment 1 Andrej Podzimek 2021-02-24 18:05:45 UTC
In retrospect, I guess this is/was because the parity is not checksummed, so parity blocks have to be recalculated from the other drives to make sure everything is correct. That's fine.

But in that case I would expect, for an N-drive array, a load of (100% / (N - redundancy)) on drives other than the replaced one. This was quite obviously not the case; the load on the other drives was like >50% on a 8-drive array.

(Admittedly, I can't rule out the possibility that the relativey high load was in fact a symptom of a *low* load combined with power saving features and inefficiency that comes with them.)

Note You need to log in before you can comment on or make changes to this bug.