Bug 89851 - hang when growing array following added device
Summary: hang when growing array following added device
Status: NEW
Alias: None
Product: IO/Storage
Classification: Unclassified
Component: MD (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: io_md
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-12-17 08:56 UTC by Chris Murphy
Modified: 2014-12-18 04:09 UTC (History)
0 users

See Also:
Kernel Version: 3.18.0-2.fc22.i686+debug
Subsystem:
Regression: No
Bisected commit-id:


Attachments
dmesg, sysrq+t (219.41 KB, text/plain)
2014-12-17 08:56 UTC, Chris Murphy
Details
mdadm -D (1.08 KB, text/plain)
2014-12-17 08:58 UTC, Chris Murphy
Details
mdadm -E (3.82 KB, text/plain)
2014-12-17 08:58 UTC, Chris Murphy
Details

Description Chris Murphy 2014-12-17 08:56:44 UTC
Created attachment 160891 [details]
dmesg, sysrq+t

mdadm-3.3.2-1.fc21.i686

REPRODUCE STEPS
Create four devices to use:
1. for i in {1..4}; do fallocate -l 1073741824 "file$i"; done
2. for i in {1..4}; do losetup -f "file$i"; done
Create array:
3. mdadm -C md0 -n 3 -l raid5 /dev/loop[012]
4. cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] 
md127 : active raid5 loop2[3] loop1[1] loop0[0]
      2095104 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]
Add+Grow array:
5. mdadm --add /dev/md/md0 /dev/loop3
mdadm: added /dev/loop3
6. mdadm --grow --raid-devices=4 /dev/md/md0
mdadm: Need to backup 3072K of critical section..

Hang.

dmesg shows:

[ 1043.523780] RAID conf printout:
[ 1043.523857]  --- level:5 rd:4 wd:4
[ 1043.523918]  disk 0, o:1, dev:loop0
[ 1043.523978]  disk 1, o:1, dev:loop1
[ 1043.524087]  disk 2, o:1, dev:loop2
[ 1043.524148]  disk 3, o:1, dev:loop3
[ 1043.534806] md: reshape of RAID array md127
[ 1043.534871] md: minimum _guaranteed_  speed: 1000 KB/sec/disk.
[ 1043.534928] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for reshape.
[ 1043.535063] md: using 128k window, over a total of 1047552k.
[ 1043.739359]  md127: unknown partition table

And after ~10 minutes, no progress on reshape:
# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] 
md127 : active raid5 loop3[4] loop2[3] loop1[1] loop0[0]
      2095104 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]
      [>....................]  reshape =  0.0% (0/1047552) finish=19.5min speed=864K/sec


dmesg still doesn't show any issue. So I've issued a sysrq+t and will attach dmesg.
Comment 1 Chris Murphy 2014-12-17 08:58:31 UTC
Created attachment 160901 [details]
mdadm -D
Comment 2 Chris Murphy 2014-12-17 08:58:47 UTC
Created attachment 160911 [details]
mdadm -E
Comment 3 Chris Murphy 2014-12-18 04:09:56 UTC
Changed to 64-bit hardware, same results, reshape hangs.
kernel-3.17.6-300.fc21.x86_64
mdadm-3.3.2-1.fc21.x86_64

Note You need to log in before you can comment on or make changes to this bug.