Created attachment 298333 [details] xfs-168.full xfstests xfs/168 fails on ppc64le with xfs-linux kernel xfs-5.15-merge-1: FSTYP -- xfs (debug) PLATFORM -- Linux/ppc64le ibm-p9z-06-lp1 5.14.0-rc4+ #1 SMP Mon Aug 16 05:26:46 EDT 2021 MKFS_OPTIONS -- -f -m crc=1,finobt=1,reflink=1,rmapbt=1,bigtime=1,inobtcount=1 -b size=1024 /dev/sda3 MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/sda3 /mnt/xfstests/scratch xfs/168 [failed, exit status 1]- output mismatch (see /var/lib/xfstests/results//xfs/168.out.bad) --- tests/xfs/168.out 2021-08-16 08:00:55.599141174 -0400 +++ /var/lib/xfstests/results//xfs/168.out.bad 2021-08-16 08:03:59.237993851 -0400 @@ -1,2 +1,3 @@ QA output created by 168 -Silence is golden +xfs_repair failed with shrinking 776672 +(see /var/lib/xfstests/results//xfs/168.full for details) ... (Run 'diff -u /var/lib/xfstests/tests/xfs/168.out /var/lib/xfstests/results//xfs/168.out.bad' to see the entire diff) Ran: xfs/168 Failures: xfs/168 Failed 1 of 1 tests Part of .full output: ... ... xfs_growfs: XFS_IOC_FSGROWFSDATA xfsctl failed: No space left on device meta-data=/dev/sda3 isize=512 agcount=2, agsize=517120 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=1 = reflink=1 bigtime=1 inobtcount=1 data = bsize=1024 blocks=776673, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=1024 blocks=6011, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 fsstress: check_cwd failure fsstress: check_cwd failure fsstress: check_cwd failure xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... ALERT: The filesystem has valuable metadata changes in a log which is being ignored because the -n option was used. Expect spurious inconsistencies which may be resolved by first mounting the filesystem to replay the log. - scan filesystem freespace and inode maps... - found root inode chunk Phase 3 - for each AG... - scan (but don't clear) agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 No modify flag set, skipping phase 5 Phase 6 - check inode connectivity... - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify link counts... No modify flag set, skipping filesystem flush and exiting. xfs_repair failed with shrinking 776672 Please check the attachment to get full .full output file
*** Bug 214079 has been marked as a duplicate of this bug. ***
It's not only on ppc64le, I hit it on x86_64 with 1k blocksize xfs: meta-data=/dev/sda4 isize=512 agcount=2, agsize=517120 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=1 = reflink=1 bigtime=1 inobtcount=1 data = bsize=1024 blocks=777569, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=1024 blocks=6011, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem xfs_growfs: /mnt/xfstests/scratch is not a mounted XFS filesystem Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... ALERT: The filesystem has valuable metadata changes in a log which is being ignored because the -n option was used. Expect spurious inconsistencies which may be resolved by first mounting the filesystem to replay the log. - scan filesystem freespace and inode maps... block (1,34732-34741) multiply claimed by cnt space tree, state - 2 block (1,13515-13531) multiply claimed by cnt space tree, state - 2 block (1,24562-24578) multiply claimed by cnt space tree, state - 2 block (1,40696-40712) multiply claimed by cnt space tree, state - 2 block (1,61159-61175) multiply claimed by cnt space tree, state - 2 block (1,103867-103883) multiply claimed by cnt space tree, state - 2 block (0,411364-411366) multiply claimed by cnt space tree, state - 2 block (0,264601-264605) multiply claimed by cnt space tree, state - 2 block (0,323760-323778) multiply claimed by cnt space tree, state - 2 block (0,296448-296452) multiply claimed by cnt space tree, state - 2 block (0,329426-329439) multiply claimed by cnt space tree, state - 2 block (0,387875-387879) multiply claimed by cnt space tree, state - 2 block (0,300749-300755) multiply claimed by cnt space tree, state - 2 block (0,371814-371820) multiply claimed by cnt space tree, state - 2 block (0,409514-409533) multiply claimed by cnt space tree, state - 2 block (0,363353-363363) multiply claimed by cnt space tree, state - 2 block (0,365339-365350) multiply claimed by cnt space tree, state - 2 block (0,406170-406184) multiply claimed by cnt space tree, state - 2 block (0,366667-366681) multiply claimed by cnt space tree, state - 2 block (0,410945-410955) multiply claimed by cnt space tree, state - 2 block (0,364395-364403) multiply claimed by cnt space tree, state - 2 block (0,374209-374211) multiply claimed by cnt space tree, state - 2 block (0,374452-374457) multiply claimed by cnt space tree, state - 2 block (0,362639-362644) multiply claimed by cnt space tree, state - 2 block (0,363071-363091) multiply claimed by cnt space tree, state - 2 block (0,418627-418639) multiply claimed by cnt space tree, state - 2 block (0,418857-418875) multiply claimed by cnt space tree, state - 2 block (0,420484-420499) multiply claimed by cnt space tree, state - 2 block (0,421474-421495) multiply claimed by cnt space tree, state - 2 block (0,424507-424519) multiply claimed by cnt space tree, state - 2 block (0,427415-427438) multiply claimed by cnt space tree, state - 2 rmap rmap btree block claimed (state 2), agno 0, bno 138214, suspect 0 agf_freeblks 22372, counted 20706 in ag 1 record 37 greater than high key of block (0/255742) in rmap tree record 32 greater than high key of block (0/158153) in rmap tree record 33 greater than high key of block (0/158153) in rmap tree record 34 greater than high key of block (0/158153) in rmap tree record 35 greater than high key of block (0/158153) in rmap tree record 36 greater than high key of block (0/158153) in rmap tree record 37 greater than high key of block (0/158153) in rmap tree record 38 greater than high key of block (0/158153) in rmap tree bad rmapbt block count 117, saw 116 agf_freeblks 114622, counted 114729 in ag 0 agf_btreeblks 130, counted 131 in ag 0 inode chunk claims used block, inobt block - agno 0, bno 296416, inopb 2 inode chunk claims used block, finobt block - agno 0, bno 296416, inopb 2 agi unlinked bucket 51 is 577843 in ag 0 (inode=577843) agi unlinked bucket 53 is 577845 in ag 0 (inode=577845) sb_icount 2816, counted 2880 sb_ifree 56, counted 64 sb_fdblocks 137256, counted 135698 - found root inode chunk Phase 3 - for each AG... - scan (but don't clear) agi unlinked lists... found inodes not in the inode allocation tree - process known inodes and perform inode discovery... - agno = 0 data fork in ino 452278 claims free block 362118 data fork in ino 452278 claims free block 362239 data fork in ino 452278 claims free block 362320 data fork in ino 452278 claims free block 362645 data fork in ino 452278 claims free block 363032 data fork in ino 452278 claims free block 300709 data fork in ino 452278 claims free block 362672 data fork in ino 511501 claims free block 338472 data fork in ino 511501 claims free block 363542 data fork in ino 511501 claims free block 363731 data fork in ino 511501 claims free block 363738 data fork in ino 511501 claims free block 365243 data fork in ino 511501 claims free block 363264 data fork in ino 511501 claims free block 363320 data fork in ino 511501 claims free block 371749 data fork in ino 511501 claims free block 362020 data fork in ino 511501 claims free block 416620 data fork in ino 511501 claims free block 416928 data fork in ino 511501 claims free block 417095 data fork in ino 511501 claims free block 418780 data fork in ino 511501 claims free block 418503 data fork in ino 511501 claims free block 418612 data fork in ino 511519 claims free block 406073 data fork in ino 511519 claims free block 406636 data fork in ino 511519 claims free block 421717 data fork in ino 511519 claims free block 421833 data fork in ino 532936 claims free block 338802 data fork in ino 532936 claims free block 362329 data fork in ino 577842 claims free block 410576 data fork in ino 577842 claims free block 410944 data fork in ino 577842 claims free block 370544 data fork in ino 577842 claims free block 397385 data fork in ino 577842 claims free block 397426 data fork in ino 577853 claims free block 410639 data fork in ino 577853 claims free block 427837 data fork in ino 577853 claims free block 382745 data fork in ino 577854 claims free block 424520 data fork in ino 577854 claims free block 419085 data fork in ino 577854 claims free block 360032 data fork in ino 577854 claims free block 360135 data fork in ino 577854 claims free block 424953 data fork in ino 577854 claims free block 426599 data fork in ino 577854 claims free block 427090 data fork in ino 577854 claims free block 427642 data fork in ino 577854 claims free block 427687 data fork in ino 577855 claims free block 349669 data fork in ino 577855 claims free block 425731 data fork in ino 577855 claims free block 425198 data fork in ino 577855 claims free block 366557 data fork in ino 577855 claims free block 425363 data fork in ino 577855 claims free block 427355 data fork in ino 589415 claims free block 362861 data fork in ino 589415 claims free block 362974 data fork in ino 589415 claims free block 421840 data fork in ino 589415 claims free block 421371 - agno = 1 imap claims in-use inode 1108439 is free, would correct imap imap claims in-use inode 1324608 is free, would correct imap imap claims in-use inode 1324660 is free, would correct imap data fork in ino 1343808 claims free block 300003 data fork in ino 1343808 claims free block 300704 data fork in ino 1343808 claims free block 300713 data fork in ino 1343808 claims free block 300731 data fork in ino 1343808 claims free block 327915 data fork in ino 1343808 claims free block 329394 data fork in ino 1343808 claims free block 323741 imap claims in-use inode 1343808 is free, would correct imap imap claims in-use inode 1343818 is free, would correct imap data fork in ino 1408785 claims free block 363797 data fork in ino 1408785 claims free block 365162 imap claims in-use inode 1408785 is free, would correct imap imap claims in-use inode 1408786 is free, would correct imap imap claims in-use inode 1408787 is free, would correct imap data fork in ino 1408788 claims free block 409146 data fork in ino 1408788 claims free block 410237 data fork in ino 1408788 claims free block 410492 data fork in ino 1408788 claims free block 410291 data fork in ino 1408788 claims free block 410339 data fork in ino 1408788 claims free block 423318 data fork in ino 1408788 claims free block 411074 imap claims in-use inode 1408788 is free, would correct imap imap claims in-use inode 1408789 is free, would correct imap imap claims in-use inode 1408790 is free, would correct imap imap claims in-use inode 1408791 is free, would correct imap imap claims in-use inode 1408792 is free, would correct imap imap claims in-use inode 1408793 is free, would correct imap data fork in ino 1408794 claims free block 422637 data fork in ino 1408794 claims free block 423237 data fork in ino 1408794 claims free block 423272 data fork in ino 1408794 claims free block 425964 data fork in ino 1408794 claims free block 426006 imap claims in-use inode 1408794 is free, would correct imap imap claims in-use inode 1408795 is free, would correct imap data fork in ino 1408796 claims free block 418641 data fork in ino 1408796 claims free block 418876 data fork in ino 1408796 claims free block 419004 data fork in ino 1408796 claims free block 428177 data fork in ino 1408796 claims free block 428433 data fork in ino 1408796 claims free block 428551 imap claims in-use inode 1408796 is free, would correct imap data fork in ino 1408797 claims free block 364273 data fork in ino 1408797 claims free block 417392 imap claims in-use inode 1408797 is free, would correct imap data fork in ino 1408798 claims free block 429094 data fork in ino 1408798 claims free block 429211 imap claims in-use inode 1408798 is free, would correct imap data fork in ino 1408799 claims free block 412330 data fork in ino 1408799 claims free block 414298 data fork in ino 1408799 claims free block 419296 imap claims in-use inode 1408799 is free, would correct imap data fork in ino 1472974 claims free block 410739 data fork in ino 1472974 claims free block 411339 imap claims in-use inode 1472974 is free, would correct imap imap claims in-use inode 1472977 is free, would correct imap imap claims in-use inode 1472983 is free, would correct imap data fork in ino 1472987 claims free block 374337 imap claims in-use inode 1472987 is free, would correct imap imap claims in-use inode 1472992 is free, would correct imap data fork in ino 1472999 claims free block 362517 data fork in ino 1472999 claims free block 428975 imap claims in-use inode 1472999 is free, would correct imap data fork in ino 1473006 claims free block 374049 data fork in ino 1473006 claims free block 428899 data fork in ino 1473006 claims free block 428969 imap claims in-use inode 1473006 is free, would correct imap - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... free space (0,177100-177133) only seen by one free space btree unknown block state, ag 0, blocks 225303-225326 free space (0,228796-228822) only seen by one free space btree free space (0,264568-264600) only seen by one free space btree unknown block state, ag 0, blocks 291063-291103 unknown block state, ag 0, blocks 291717-291737 free space (0,296411-296447) only seen by one free space btree unknown block state, ag 0, blocks 304309-304328 unknown block state, ag 0, blocks 304956-304991 unknown block state, ag 0, blocks 306075-306091 unknown block state, ag 0, blocks 307617-307624 unknown block state, ag 0, blocks 308233-308235 unknown block state, ag 0, blocks 308239-308239 unknown block state, ag 0, blocks 308240-308255 unknown block state, ag 0, blocks 310608-310608 unknown block state, ag 0, blocks 314442-314513 unknown block state, ag 0, blocks 319007-319121 unknown block state, ag 0, blocks 319482-319570 unknown block state, ag 0, blocks 322294-322387 unknown block state, ag 0, blocks 326324-326341 unknown block state, ag 0, blocks 326354-326788 unknown block state, ag 0, blocks 332833-332903 unknown block state, ag 0, blocks 334095-334185 free space (0,338993-338997) only seen by one free space btree unknown block state, ag 0, blocks 339255-339501 unknown block state, ag 0, blocks 341326-341410 unknown block state, ag 0, blocks 342037-342103 unknown block state, ag 0, blocks 342153-342261 unknown block state, ag 0, blocks 342312-342344 unknown block state, ag 0, blocks 344923-345038 unknown block state, ag 0, blocks 354074-354190 unknown block state, ag 0, blocks 357233-357340 unknown block state, ag 0, blocks 357994-358067 unknown block state, ag 0, blocks 358262-358520 unknown block state, ag 0, blocks 358526-358626 unknown block state, ag 0, blocks 358847-358919 unknown block state, ag 0, blocks 359008-359019 unknown block state, ag 0, blocks 361142-361253 unknown block state, ag 0, blocks 361388-361444 free space (0,363495-363495) only seen by one free space btree unknown block state, ag 0, blocks 363503-363541 unknown block state, ag 0, blocks 364404-364927 unknown block state, ag 0, blocks 365004-365043 unknown block state, ag 0, blocks 365128-365161 unknown block state, ag 0, blocks 365223-365242 unknown block state, ag 0, blocks 365351-365578 unknown block state, ag 0, blocks 365687-365924 unknown block state, ag 0, blocks 365927-366222 unknown block state, ag 0, blocks 366225-366556 free space (0,366682-366760) only seen by one free space btree unknown block state, ag 0, blocks 366919-367150 unknown block state, ag 0, blocks 367159-367227 unknown block state, ag 0, blocks 377907-377933 unknown block state, ag 0, blocks 382663-382744 unknown block state, ag 0, blocks 382786-382875 unknown block state, ag 0, blocks 384386-384467 unknown block state, ag 0, blocks 390512-390593 unknown block state, ag 0, blocks 391588-391877 unknown block state, ag 0, blocks 391926-392154 unknown block state, ag 0, blocks 395706-395785 unknown block state, ag 0, blocks 399776-399863 free space (0,400713-400737) only seen by one free space btree unknown block state, ag 0, blocks 403540-403657 unknown block state, ag 1, blocks 29919-29919 unknown block state, ag 1, blocks 96138-96166 unknown block state, ag 1, blocks 96613-96620 free space (1,96621-96635) only seen by one free space btree unknown block state, ag 1, blocks 111075-111075 unknown block state, ag 1, blocks 111080-111080 unknown block state, ag 1, blocks 113268-113308 unknown block state, ag 1, blocks 114785-114824 unknown block state, ag 1, blocks 127635-127659 unknown block state, ag 1, blocks 138015-138015 free space (1,203840-203851) only seen by one free space btree unknown block state, ag 1, blocks 206663-206733 unknown block state, ag 1, blocks 216935-216980 unknown block state, ag 1, blocks 217387-217402 free space (1,217403-217418) only seen by one free space btree unknown block state, ag 1, blocks 217908-217908 free space (1,217909-217918) only seen by one free space btree unknown block state, ag 1, blocks 217919-217919 unknown block state, ag 1, blocks 217920-217951 free space (1,228747-228759) only seen by one free space btree unknown block state, ag 1, blocks 229381-229456 unknown block state, ag 1, blocks 230208-230449 unknown block state, ag 1, blocks 230509-230534 unknown block state, ag 1, blocks 231028-231076 free space (1,231077-231087) only seen by one free space btree unknown block state, ag 1, blocks 231088-231135 unknown block state, ag 1, blocks 232343-232390 unknown block state, ag 1, blocks 232715-232806 unknown block state, ag 1, blocks 234485-234574 unknown block state, ag 1, blocks 234700-234776 unknown block state, ag 1, blocks 234897-235021 unknown block state, ag 1, blocks 235024-235137 unknown block state, ag 1, blocks 235223-235279 unknown block state, ag 1, blocks 237701-238253 unknown block state, ag 1, blocks 238322-238541 unknown block state, ag 1, blocks 239248-239279 unknown block state, ag 1, blocks 240441-240519 unknown block state, ag 1, blocks 241027-241142 unknown block state, ag 1, blocks 247224-247232 unknown block state, ag 1, blocks 247373-247381 - check for inodes claiming duplicate blocks... - agno = 1 - agno = 0 entry "d3" in shortform directory 1060672 references non-existent inode 616491 would have junked entry "d3" in directory inode 1060672 entry "c6" in shortform directory 1060672 references non-existent inode 592844 would have junked entry "c6" in directory inode 1060672 entry "cc" in shortform directory 1060672 references non-existent inode 1484440 would have junked entry "cc" in directory inode 1060672 entry "ff" in shortform directory 1060672 references free inode 1324611 would have junked entry "ff" in directory inode 1060672 entry "c14" in shortform directory 1060673 references non-existent inode 1484443 would have junked entry "c14" in directory inode 1060673 entry "f12" in shortform directory 1060673 references non-existent inode 1484418 would have junked entry "f12" in directory inode 1060673 entry "f69" in shortform directory 1108416 references non-existent inode 616510 would have junked entry "f69" in directory inode 1108416 entry "d1d" in shortform directory 1108418 references non-existent inode 616497 would have junked entry "d1d" in directory inode 1108418 entry "f5f" in shortform directory 1108418 references non-existent inode 616493 would have junked entry "f5f" in directory inode 1108418 entry "d54" at block 0 offset 512 in directory inode 1108425 references non-existent inode 616503 would clear inode number in entry at offset 512... entry "d6d" at block 0 offset 640 in directory inode 1108425 references non-existent inode 616499 would clear inode number in entry at offset 640... entry "fa0" at block 0 offset 416 in directory inode 1108443 references non-existent inode 616500 would clear inode number in entry at offset 416... entry "d24" in shortform directory 1108446 references non-existent inode 616505 would have junked entry "d24" in directory inode 1108446 entry "ff1" in shortform directory 1108446 references non-existent inode 616494 would have junked entry "ff1" in directory inode 1108446 entry "cb7" in shortform directory 452260 references non-existent inode 616501 would have junked entry "cb7" in directory inode 452260 entry "f72" in shortform directory 452271 references non-existent inode 616496 would have junked entry "f72" in directory inode 452271 entry "c6d" in shortform directory 454650 references non-existent inode 616482 would have junked entry "c6d" in directory inode 454650 entry "fc3" in shortform directory 504835 references non-existent inode 616486 would have junked entry "fc3" in directory inode 504835 entry "fc1" in shortform directory 504851 references non-existent inode 616484 would have junked entry "fc1" in directory inode 504851 entry "dd9" in shortform directory 504873 references non-existent inode 616481 would have junked entry "dd9" in directory inode 504873 entry "cc6" in shortform directory 510789 references non-existent inode 616488 would have junked entry "cc6" in directory inode 510789 entry "c60" in shortform directory 511491 references non-existent inode 616507 would have junked entry "c60" in directory inode 511491 entry "cc5" in shortform directory 543205 references non-existent inode 616487 would have junked entry "cc5" in directory inode 543205 entry "fcd" in shortform directory 543205 references non-existent inode 616489 would have junked entry "fcd" in directory inode 543205 entry "dce" in shortform directory 1331001 references non-existent inode 616490 would have junked entry "dce" in directory inode 1331001 entry "fc2" in shortform directory 566123 references non-existent inode 616485 would have junked entry "fc2" in directory inode 566123 entry "c88" in shortform directory 566134 references non-existent inode 616480 would have junked entry "c88" in directory inode 566134 entry "fe0" in shortform directory 566134 references non-existent inode 616483 entry "fde" in shortform directory 1343817 references non-existent inode 616504 would have junked entry "fde" in directory inode 1343817 would have junked entry "fe0" in directory inode 566134 entry "c51" in shortform directory 1343820 references non-existent inode 616498 would have junked entry "c51" in directory inode 1343820 entry "ced" in shortform directory 1343820 references non-existent inode 616498 would have junked entry "ced" in directory inode 1343820 entry "d12" in shortform directory 577806 references non-existent inode 1484417 would have junked entry "d12" in directory inode 577806 entry "ca6" in shortform directory 1343824 references non-existent inode 616502 would have junked entry "ca6" in directory inode 1343824 entry "c16" in shortform directory 577841 references non-existent inode 1484437 would have junked entry "c16" in directory inode 577841 entry "c28" in shortform directory 577841 references non-existent inode 592838 would have junked entry "c28" in directory inode 577841 entry "d29" in shortform directory 577841 references non-existent inode 1484431 would have junked entry "d29" in directory inode 577841 entry "ca6" in shortform directory 1392120 references non-existent inode 616492 would have junked entry "ca6" in directory inode 1392120 entry "d2f" in shortform directory 577841 references non-existent inode 1484435 would have junked entry "d2f" in directory inode 577841 entry "f34" in shortform directory 577847 references non-existent inode 592854 would have junked entry "f34" in directory inode 577847 entry "d23" in shortform directory 577848 references non-existent inode 616509 would have junked entry "d23" in directory inode 577848 entry "c3b" in shortform directory 577848 references non-existent inode 592857 would have junked entry "c3b" in directory inode 577848 entry "f21" in shortform directory 577849 references non-existent inode 592832 would have junked entry "f21" in directory inode 577849 entry "f23" in shortform directory 577849 references non-existent inode 592833 would have junked entry "f23" in directory inode 577849 entry "c2e" in shortform directory 577849 references non-existent inode 592856 would have junked entry "c2e" in directory inode 577849 entry "c32" in shortform directory 577849 references non-existent inode 592859 would have junked entry "c32" in directory inode 577849 entry "c16" in shortform directory 1408792 references non-existent inode 1484421 would have junked entry "c16" in directory inode 1408792 entry "c18" in shortform directory 1408792 references non-existent inode 592837 would have junked entry "c18" in directory inode 1408792 entry "c1b" in shortform directory 1408792 references non-existent inode 592843 would have junked entry "c1b" in directory inode 1408792 entry "c31" in shortform directory 1408792 references non-existent inode 1484429 would have junked entry "c31" in directory inode 1408792 entry "d22" in shortform directory 577852 references non-existent inode 1484427 would have junked entry "d22" in directory inode 577852 entry "d39" in shortform directory 577852 references non-existent inode 1484441 would have junked entry "d39" in directory inode 577852 entry "c3c" in shortform directory 577852 references non-existent inode 592847 would have junked entry "c3c" in directory inode 577852 entry "f1b" in shortform directory 1472977 references non-existent inode 1484416 would have junked entry "f1b" in directory inode 1472977 entry "f1f" in shortform directory 1472977 references non-existent inode 1484420 would have junked entry "f1f" in directory inode 1472977 entry "ca9" in shortform directory 589403 references non-existent inode 616511 would have junked entry "ca9" in directory inode 589403 entry "f20" in shortform directory 1472977 references non-existent inode 1484426 would have junked entry "f20" in directory inode 1472977 entry "f24" in shortform directory 1472977 references non-existent inode 1484428 would have junked entry "f24" in directory inode 1472977 entry "d32" in shortform directory 1472977 references non-existent inode 592853 would have junked entry "d32" in directory inode 1472977 entry "c38" in shortform directory 1472977 references non-existent inode 592858 would have junked entry "c38" in directory inode 1472977 entry "ce4" in shortform directory 1472991 references non-existent inode 616492 would have junked entry "ce4" in directory inode 1472991 entry "c1f" in shortform directory 1472992 references non-existent inode 1484419 would have junked entry "c1f" in directory inode 1472992 entry "d28" in shortform directory 1472992 references non-existent inode 592845 would have junked entry "d28" in directory inode 1472992 entry "f29" in shortform directory 1472992 references non-existent inode 1484432 would have junked entry "f29" in directory inode 1472992 entry "l2c" in shortform directory 1472992 references non-existent inode 1484438 would have junked entry "l2c" in directory inode 1472992 entry "f2d" in shortform directory 1472992 references non-existent inode 1484442 would have junked entry "f2d" in directory inode 1472992 entry "f31" in shortform directory 1472992 references non-existent inode 1484444 would have junked entry "f31" in directory inode 1472992 entry "f33" in shortform directory 1472992 references non-existent inode 1484445 would have junked entry "f33" in directory inode 1472992 entry "c8f" in shortform directory 1472996 references non-existent inode 616508 would have junked entry "c8f" in directory inode 1472996 Missing reverse-mapping record for (0/303267) unwritten len 30 owner 423514 off 1634 Missing reverse-mapping record for (0/303297) len 44 owner 379054 off 4150 Missing reverse-mapping record for (0/303341) len 49 owner 379054 off 3415 Missing reverse-mapping record for (0/303390) len 33 owner 1229027 off 1252 Missing reverse-mapping record for (0/303423) len 90 owner 393918 off 6276 Missing reverse-mapping record for (0/303513) unwritten len 34 owner 393918 off 6366 Missing reverse-mapping record for (0/303547) unwritten len 141 owner 1229006 off 3264 Missing reverse-mapping record for (0/303688) len 621 owner 1096605 off 1567 Missing reverse-mapping record for (0/304333) len 200 owner 1096605 off 2212 Missing reverse-mapping record for (0/304533) len 27 owner 393918 off 6921 Missing reverse-mapping record for (0/304560) unwritten len 28 owner 393918 off 6948 Missing reverse-mapping record for (0/304588) len 125 owner 1290366 off 1577 Missing reverse-mapping record for (0/304713) len 16 owner 1096605 off 3326 Missing reverse-mapping record for (0/304729) len 50 owner 543214 off 385 Missing reverse-mapping record for (0/304779) len 63 owner 379018 off 1409 Missing reverse-mapping record for (0/304842) unwritten len 26 owner 454645 off 3502 Missing reverse-mapping record for (0/304868) len 15 owner 1163034 off 713 Missing reverse-mapping record for (0/304883) len 6 owner 379011 off 3013 Missing reverse-mapping record for (0/304889) len 1 owner 379011 bmbt off 0 Missing reverse-mapping record for (0/304890) len 66 owner 566106 off 394 Missing reverse-mapping record for (0/304999) len 276 owner 1163034 off 844 Missing reverse-mapping record for (0/363542) unwritten len 189 owner 511501 off 614 Missing reverse-mapping record for (0/363731) len 7 owner 511501 off 803 Missing reverse-mapping record for (0/363738) unwritten len 59 owner 511501 off 810 Missing reverse-mapping record for (0/363797) len 439 owner 1408785 off 653 Missing reverse-mapping record for (0/364273) len 122 owner 1408797 off 949 Missing reverse-mapping record for (0/376193) unwritten len 80 owner 1108422 off 1644 Missing reverse-mapping record for (0/376273) len 43 owner 1108432 off 4753 Missing reverse-mapping record for (0/376333) unwritten len 714 owner 1108422 off 1784 Missing reverse-mapping record for (0/403920) unwritten len 6 owner 452263 off 5050 Missing reverse-mapping record for (0/408317) len 74 owner 1472972 off 2052 Missing reverse-mapping record for (0/408391) unwritten len 50 owner 1472972 off 2126 Missing reverse-mapping record for (0/408460) len 129 owner 1335513 off 2917 Missing reverse-mapping record for (0/408589) unwritten len 369 owner 1335513 off 1625 Missing reverse-mapping record for (0/408958) len 116 owner 511513 off 4240 Missing reverse-mapping record for (0/409074) unwritten len 72 owner 577820 off 8 Missing reverse-mapping record for (0/409146) len 95 owner 1408788 off 541 Missing reverse-mapping record for (0/414576) len 711 owner 1343834 off 2265 Missing reverse-mapping record for (0/415315) len 41 owner 1343834 off 3004 Missing reverse-mapping record for (0/415356) unwritten len 247 owner 1108442 off 2939 Missing reverse-mapping record for (0/415603) unwritten len 175 owner 1324632 off 1089 Missing reverse-mapping record for (0/415778) len 80 owner 1324632 off 1264 Missing reverse-mapping record for (0/415858) unwritten len 132 owner 1324632 off 1344 Missing reverse-mapping record for (0/415990) unwritten len 630 owner 1343822 off 455 Missing reverse-mapping record for (0/416620) unwritten len 308 owner 511501 off 2905 Missing reverse-mapping record for (0/416928) len 167 owner 511501 off 3213 Missing reverse-mapping record for (0/417095) unwritten len 297 owner 511501 off 3380 Missing reverse-mapping record for (0/417392) len 990 owner 1408797 off 1071 Missing reverse-mapping record for (0/418503) len 109 owner 511501 off 4178 Missing reverse-mapping record for (0/418612) len 15 owner 511501 off 4900 Missing reverse-mapping record for (0/418640) len 1 owner 511501 bmbt off 0 Missing reverse-mapping record for (0/418641) unwritten len 139 owner 1408796 off 241 Missing reverse-mapping record for (0/418780) unwritten len 77 owner 511501 off 3677 Missing reverse-mapping record for (0/418876) unwritten len 84 owner 1408796 off 476 Missing reverse-mapping record for (0/419004) unwritten len 81 owner 1408796 off 604 Missing reverse-mapping record for (0/419085) unwritten len 211 owner 577854 off 736 Missing reverse-mapping record for (0/419296) len 45 owner 1408799 off 1499 Missing reverse-mapping record for (0/421371) len 60 owner 589415 off 4255 Missing reverse-mapping record for (0/421717) len 116 owner 511519 off 7685 Missing reverse-mapping record for (0/421833) unwritten len 7 owner 511519 off 7801 Missing reverse-mapping record for (0/421840) unwritten len 189 owner 589415 off 3554 Missing reverse-mapping record for (0/422637) unwritten len 600 owner 1408794 off 1383 Missing reverse-mapping record for (0/423237) len 35 owner 1408794 off 1983 Missing reverse-mapping record for (0/423272) unwritten len 46 owner 1408794 off 2018 Missing reverse-mapping record for (0/423318) unwritten len 252 owner 1408788 off 3776 Missing reverse-mapping record for (0/424520) unwritten len 433 owner 577854 off 233 Missing reverse-mapping record for (0/424953) unwritten len 245 owner 577854 off 3110 Missing reverse-mapping record for (0/425198) unwritten len 165 owner 577855 off 827 Missing reverse-mapping record for (0/425363) unwritten len 368 owner 577855 off 1102 Missing reverse-mapping record for (0/425731) unwritten len 233 owner 577855 off 594 Missing reverse-mapping record for (0/425964) len 42 owner 1408794 off 2731 Missing reverse-mapping record for (0/426006) unwritten len 43 owner 1408794 off 2773 Missing reverse-mapping record for (0/426599) len 76 owner 577854 off 4097 Missing reverse-mapping record for (0/427090) len 212 owner 577854 off 4588 Missing reverse-mapping record for (0/427355) len 60 owner 577855 off 2360 Missing reverse-mapping record for (0/427642) len 45 owner 577854 off 5140 Missing reverse-mapping record for (0/427687) unwritten len 63 owner 577854 off 5185 Missing reverse-mapping record for (0/427837) len 224 owner 577853 off 174 Missing reverse-mapping record for (0/428177) len 86 owner 1408796 off 930 Missing reverse-mapping record for (0/428433) len 118 owner 1408796 off 3557
dmesg output, when xfs/168 run into fail: [10782.980699] run fstests xfs/168 at 2021-08-16 09:29:52 [10785.856212] XFS (sda4): Mounting V5 Filesystem [10785.960044] XFS (sda4): Ending clean mount [10785.999747] XFS (sda4): EXPERIMENTAL online shrink feature in use. Use at your own risk! [10786.124668] XFS (sda4): Unmounting Filesystem [10788.000206] XFS (sda4): Mounting V5 Filesystem [10788.089115] XFS (sda4): Ending clean mount [-- MARK -- Mon Aug 16 13:30:00 2021] [10792.642459] XFS (sda4): Unmounting Filesystem [10793.229287] XFS (sda4): Mounting V5 Filesystem [10793.289566] XFS (sda4): Ending clean mount [10797.115638] fsstress (111266) used greatest stack depth: 20600 bytes left [10797.859428] XFS (sda4): Unmounting Filesystem [10798.558178] XFS (sda4): Mounting V5 Filesystem [10807.354979] XFS (sda4): Ending clean mount [10810.791370] fsstress (111532) used greatest stack depth: 20312 bytes left [10811.886076] XFS (sda4): xlog_verify_grant_tail: space > BBTOB(tail_blocks) [10812.241353] XFS (sda4): Unmounting Filesystem [10813.075136] XFS (sda4): Mounting V5 Filesystem [10813.176424] XFS (sda4): Ending clean mount [10817.343832] fsstress (111792) used greatest stack depth: 20008 bytes left [10817.807386] XFS (sda4): Unmounting Filesystem [10819.260537] XFS (sda4): Mounting V5 Filesystem [10819.923391] XFS (sda4): Ending clean mount [10824.793331] XFS (sda4): Unmounting Filesystem [10825.701244] XFS (sda4): Mounting V5 Filesystem [10843.028604] XFS (sda4): Ending clean mount [10847.696262] XFS (sda4): Unmounting Filesystem [10849.031471] XFS (sda4): Mounting V5 Filesystem [10849.389481] XFS (sda4): Ending clean mount [10854.117904] XFS (sda4): xlog_verify_grant_tail: space > BBTOB(tail_blocks) [10854.117904] XFS (sda4): xlog_verify_grant_tail: space > BBTOB(tail_blocks) [10854.626542] XFS (sda4): Unmounting Filesystem [10855.791704] XFS (sda4): Mounting V5 Filesystem [10860.128289] XFS (sda4): Ending clean mount [10861.518327] XFS (sda4): Error -28 reserving per-AG metadata reserve pool. [10861.534515] XFS (sda4): Corruption of in-memory data (0x8) detected at xfs_ag_shrink_space+0x739/0xae0 [xfs] (fs/xfs/libxfs/xfs_ag.c:877). Shutting down filesystem [10861.543426] sda4: writeback error on inode 532936, offset 475136, sector 68696454 [10861.551487] XFS (sda4): Please unmount the filesystem and rectify the problem(s) [10862.297180] XFS (sda4): Unmounting Filesystem [10863.893759] XFS (sda5): Unmounting Filesystem [10864.373197] XFS (sda5): Mounting V5 Filesystem [10864.483245] XFS (sda5): Ending clean mount [10883.073207] XFS (sda4): Mounting V5 Filesystem [10883.160863] XFS (sda4): Ending clean mount [10883.190838] XFS (sda4): Unmounting Filesystem [10883.610827] XFS (sda5): Unmounting Filesystem [10884.278500] XFS (sda5): Mounting V5 Filesystem [10884.322675] XFS (sda5): Ending clean mount
FYI: reproduced it on mainline linux 5.14.0-rc6+
Hi, I tried several times without any luck in my own VM, FSTYP -- xfs (debug) PLATFORM -- Linux/x86_64 AliYun 5.13.0-rc4+ #1 SMP Mon Aug 23 20:17:12 CST 2021 MKFS_OPTIONS -- -f -m crc=1,finobt=1,reflink=1,rmapbt=1,bigtime=1,inobtcount=1 -b size=1024 /dev/loop1 MOUNT_OPTIONS -- /dev/loop1 /fs/loop1 xfs/168 197s ... 225s Ran: xfs/168 Passed all 1 tests But with my own understanding, I guess that may be due to the following condition: /* * Roll the transaction before trying to re-init the per-ag * reservation. The new transaction is clean so it will cancel * without any side effects. */ error = xfs_defer_finish(tpp); <--- here we unlock the AGF, new allocation allowed. if (error) return error; error = -ENOSPC; goto resv_init_out; } xfs_ialloc_log_agi(*tpp, agibp, XFS_AGI_LENGTH); xfs_alloc_log_agf(*tpp, agfbp, XFS_AGF_LENGTH); return 0; resv_init_out: err2 = xfs_ag_resv_init(agibp->b_pag, *tpp); <--- so resv init can be failed here. if (!err2) return error; resv_err: xfs_warn(mp, "Error %d reserving per-AG metadata reserve pool.", err2); xfs_force_shutdown(mp, SHUTDOWN_CORRUPT_INCORE); return err2; As I mentioned before, I'm not sure if growfs has similiar race like this.
By adding some debug output: diff --git a/fs/xfs/libxfs/xfs_ag.c b/fs/xfs/libxfs/xfs_ag.c index ee9ec0c50bec..ce5db7195604 100644 --- a/fs/xfs/libxfs/xfs_ag.c +++ b/fs/xfs/libxfs/xfs_ag.c @@ -859,6 +859,7 @@ xfs_ag_shrink_space( * without any side effects. */ error = xfs_defer_finish(tpp); + pr_info("after xfs_defer_finish, error=%d", error); if (error) return error; @@ -870,6 +871,7 @@ xfs_ag_shrink_space( return 0; resv_init_out: err2 = xfs_ag_resv_init(agibp->b_pag, *tpp); + pr_info("after xfs_ag_resv_init [out], err2=%d\n", err2); if (!err2) return error; resv_err: The failed xfs/168 dmesg output as below: [ 820.044203] run fstests xfs/168 at 2021-08-24 11:23:40 [ 822.039473] XFS (dm-2): Mounting V5 Filesystem [ 822.126343] XFS (dm-2): Ending clean mount [ 822.176716] XFS (dm-2): EXPERIMENTAL online shrink feature in use. Use at your own risk! [ 822.232059] XFS (dm-2): Unmounting Filesystem [ 823.999271] XFS (dm-2): Mounting V5 Filesystem [ 824.061384] XFS (dm-2): Ending clean mount [ 829.139729] XFS (dm-2): Unmounting Filesystem [ 829.632030] XFS (dm-2): Mounting V5 Filesystem [ 829.758903] XFS (dm-2): Ending clean mount [ 835.109447] XFS (dm-2): xlog_verify_grant_tail: space > BBTOB(tail_blocks) [ 835.540700] XFS (dm-2): Unmounting Filesystem [ 836.309492] XFS (dm-2): Mounting V5 Filesystem [ 848.784972] XFS (dm-2): Ending clean mount [ 854.679181] XFS (dm-2): Unmounting Filesystem [ 855.658301] XFS (dm-2): Mounting V5 Filesystem [ 855.757461] XFS (dm-2): Ending clean mount [ 862.342901] XFS (dm-2): xlog_verify_grant_tail: space > BBTOB(tail_blocks) [ 862.830414] XFS (dm-2): Unmounting Filesystem [ 864.397318] XFS (dm-2): Mounting V5 Filesystem [ 864.734746] XFS (dm-2): Ending clean mount [ 870.743557] XFS (dm-2): Unmounting Filesystem [ 872.274155] XFS (dm-2): Mounting V5 Filesystem [ 878.607770] XFS (dm-2): Ending clean mount [ 885.327410] XFS (dm-2): Unmounting Filesystem [ 887.102664] XFS (dm-2): Mounting V5 Filesystem [ 887.207968] XFS (dm-2): Ending clean mount [ 888.269530] after xfs_ag_resv_init [out], err2=0 [ 893.921362] XFS (dm-2): xlog_verify_grant_tail: space > BBTOB(tail_blocks) [ 893.921362] XFS (dm-2): xlog_verify_grant_tail: space > BBTOB(tail_blocks) [ 894.343531] XFS (dm-2): Unmounting Filesystem [ 896.329804] XFS (dm-2): Mounting V5 Filesystem [ 896.442648] XFS (dm-2): Ending clean mount [ 897.487766] XFS (dm-2): Reserve blocks depleted! Consider increasing reserve pool size. [ 897.532377] after xfs_ag_resv_init [out], err2=0 [ 897.561337] after xfs_ag_resv_init [out], err2=0 [ 897.593129] after xfs_defer_finish, error=0 [ 897.593136] after xfs_ag_resv_init [out], err2=0 [ 897.627267] after xfs_ag_resv_init [out], err2=0 [ 897.656902] after xfs_defer_finish, error=0 [ 897.656909] after xfs_ag_resv_init [out], err2=0 [ 897.691148] after xfs_ag_resv_init [out], err2=0 [ 897.719510] after xfs_ag_resv_init [out], err2=0 [ 897.747765] after xfs_ag_resv_init [out], err2=0 [ 897.776548] after xfs_ag_resv_init [out], err2=0 [ 897.799510] after xfs_ag_resv_init [out], err2=0 [ 897.822176] after xfs_ag_resv_init [out], err2=0 [ 897.845231] after xfs_ag_resv_init [out], err2=0 [ 902.797518] XFS (dm-2): Unmounting Filesystem [ 905.223385] XFS (dm-2): Mounting V5 Filesystem [ 905.336268] XFS (dm-2): Ending clean mount [ 906.478459] XFS (dm-2): Per-AG reservation for AG 1 failed. Filesystem may run out of space. [ 906.488330] after xfs_defer_finish, error=0 [ 906.488347] after xfs_ag_resv_init [out], err2=0 [ 906.523046] XFS (dm-2): Per-AG reservation for AG 1 failed. Filesystem may run out of space. [ 906.532797] after xfs_defer_finish, error=0 [ 906.532807] after xfs_ag_resv_init [out], err2=0 [ 906.564799] XFS (dm-2): Per-AG reservation for AG 1 failed. Filesystem may run out of space. [ 906.574422] after xfs_ag_resv_init [out], err2=-28 [ 906.579790] XFS (dm-2): Error -28 reserving per-AG metadata reserve pool. [ 906.614881] XFS (dm-2): Corruption of in-memory data (0x8) detected at xfs_ag_shrink_space+0x78f/0xa90 [xfs] (fs/xfs/libxfs/xfs_ag.c:879). Shutting down filesystem [ 906.631506] XFS (dm-2): Please unmount the filesystem and rectify the problem(s) [ 907.104870] XFS (dm-2): Unmounting Filesystem [ 909.747600] XFS (dm-1): Unmounting Filesystem