Bug 205283

Summary: BUG: KASAN: global-out-of-bounds in _copy_to_iter+0x3d4/0x5a8
Product: File System Reporter: Erhard F. (erhard_f)
Component: btrfsAssignee: BTRFS virtual assignee (fs_btrfs)
Status: RESOLVED CODE_FIX    
Severity: normal CC: christophe.leroy, platform_ppc-32
Priority: P1    
Hardware: PPC-32   
OS: Linux   
See Also: https://bugzilla.kernel.org/show_bug.cgi?id=205099
Kernel Version: 5.5-rc1 Subsystem:
Regression: No Bisected commit-id:
Attachments: dmesg (kernel 5.4.0-rc4, PowerMac G4 DP)
5.4.0-rc4 kernel .config (PowerMac G4 DP)
dmesg (kernel 5.5.0-rc2+, PowerMac G4 DP)
5.5.0-rc2+ kernel .config (PowerMac G4 DP)
5.5.0-rc6+ kernel .config (PowerMac G4 DP)
dmesg (kernel 5.5.0-rc6+, PowerMac G4 DP)

Description Erhard F. 2019-10-22 00:05:27 UTC
Created attachment 285605 [details]
dmesg (kernel 5.4.0-rc4, PowerMac G4 DP)

First of all apologies 'cause I am not quite sure under what kernel subsystem tracker I should file this bug. It was triggered running btrfs filesystem tests (misc tests) on a PowerMac G4 DP and seems to touch some memcopy routine:

[...]
[  601.897623] ==================================================================
[  601.905117] BUG: KASAN: global-out-of-bounds in _copy_to_iter+0x3d4/0x5a8
[  601.912512] Write of size 4096 at addr f18b8000 by task modprobe/10589

[  601.927287] CPU: 1 PID: 10589 Comm: modprobe Tainted: G        W         5.4.0-rc4-PowerMacG4+ #20
[  601.934991] Call Trace:
[  601.942534] [eb9cf848] [c0769184] dump_stack+0xb0/0x10c (unreliable)
[  601.950307] [eb9cf878] [c023aea8] print_address_description.isra.5+0x3c/0x420
[  601.958167] [eb9cf908] [c023b470] __kasan_report+0x140/0x188
[  601.966030] [eb9cf948] [c023bea8] check_memory_region+0x28/0x184
[  601.973925] [eb9cf958] [c0239f30] memcpy+0x48/0x74
[  601.981792] [eb9cf978] [c044ab9c] _copy_to_iter+0x3d4/0x5a8
[  601.989705] [eb9cfaa8] [c044af18] copy_page_to_iter+0x90/0x550
[  601.997585] [eb9cfb08] [c01bcc60] generic_file_read_iter+0x5c8/0x7bc
[  602.005374] [eb9cfb78] [c0251e5c] __vfs_read+0x1b0/0x1f4
[  602.013027] [eb9cfca8] [c0251f5c] vfs_read+0xbc/0x124
[  602.020671] [eb9cfcd8] [c0252018] kernel_read+0x54/0x70
[  602.028302] [eb9cfd08] [c025c7d8] kernel_read_file+0x240/0x358
[  602.035930] [eb9cfdb8] [c025c9dc] kernel_read_file_from_fd+0x54/0x74
[  602.043581] [eb9cfdf8] [c010c494] sys_finit_module+0xd8/0x140
[  602.051183] [eb9cff38] [c001a274] ret_from_syscall+0x0/0x34
[  602.058641] --- interrupt: c01 at 0x7062c4
                   LR = 0x88e7c4


[  602.087858] Memory state around the buggy address:
[  602.095160]  f18b7f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[  602.102601]  f18b7f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[  602.109845] >f18b8000: 00 06 fa fa fa fa fa fa 00 00 03 fa fa fa fa fa
[  602.117150]               ^
[  602.124218]  f18b8080: 00 00 04 fa fa fa fa fa 00 03 fa fa fa fa fa fa
[  602.131467]  f18b8100: 00 07 fa fa fa fa fa fa 00 00 03 fa fa fa fa fa
[  602.138638] ==================================================================
Comment 1 Erhard F. 2019-10-22 00:06:22 UTC
Created attachment 285607 [details]
5.4.0-rc4 kernel .config (PowerMac G4 DP)
Comment 2 Erhard F. 2019-12-15 15:49:10 UTC
Re-tried with kernel 5.5-rc1. This is probably connected with bug #205099.

[...]
[   69.181890] ==================================================================
[   69.182220] BUG: KASAN: global-out-of-bounds in _copy_to_iter+0x3c0/0x594
[   69.182472] Write of size 4096 at addr f15ad000 by task modprobe/233

[   69.182738] CPU: 0 PID: 233 Comm: modprobe Tainted: G        W         5.5.0-rc1-PowerMacG4+ #7
[   69.183061] Call Trace:
[   69.183147] [eb7138b8] [c0783b44] dump_stack+0xbc/0x118 (unreliable)
[   69.183387] [eb7138e8] [c024454c] print_address_description.isra.0+0x3c/0x420
[   69.183652] [eb713978] [c0244b0c] __kasan_report+0x138/0x180
[   69.183858] [eb7139b8] [c024551c] check_memory_region+0x24/0x180
[   69.184084] [eb7139c8] [c02435e8] memcpy+0x48/0x74
[   69.184255] [eb7139e8] [c045b5cc] _copy_to_iter+0x3c0/0x594
[   69.184458] [eb713ad8] [c045b984] copy_page_to_iter+0xac/0x564
[   69.184675] [eb713b38] [c01c6d84] generic_file_read_iter+0x5c4/0x7c0
[   69.184914] [eb713ba8] [c025b8dc] __vfs_read+0x1b0/0x1f8
[   69.185106] [eb713cd8] [c025b9e0] vfs_read+0xbc/0x124
[   69.185287] [eb713d08] [c025ba9c] kernel_read+0x54/0x70
[   69.185480] [eb713d38] [c0266514] kernel_read_file+0x23c/0x34c
[   69.185694] [eb713de8] [c0266710] kernel_read_file_from_fd+0x54/0x74
[   69.185929] [eb713e18] [c0111b1c] sys_finit_module+0xd8/0x138
[   69.186139] [eb713f38] [c001a274] ret_from_syscall+0x0/0x34
[   69.186340] --- interrupt: c01 at 0x56af78
                   LR = 0x6f8a14


[   69.186597] Memory state around the buggy address:
[   69.186766]  f15ad500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[   69.187001]  f15ad580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[   69.187235] >f15ad600: 00 00 00 00 00 00 00 00 00 00 00 fa fa fa fa fa
[   69.187466]                                             ^
[   69.187656]  f15ad680: 00 00 00 03 fa fa fa fa 00 00 00 00 00 00 00 00
[   69.187890]  f15ad700: 00 00 00 00 00 00 04 fa fa fa fa fa 00 00 00 00
[   69.188121] ==================================================================
[   69.272710] raid6: altivecx8 gen()  2354 MB/s
[   69.329383] raid6: altivecx4 gen()  3200 MB/s
[   69.386055] raid6: altivecx2 gen()  2178 MB/s
[   69.442705] raid6: altivecx1 gen()  1975 MB/s
[   69.499549] raid6: int32x8  gen()   336 MB/s
[   69.556086] raid6: int32x8  xor()   200 MB/s
[   69.612793] raid6: int32x4  gen()   342 MB/s
[   69.669421] raid6: int32x4  xor()   224 MB/s
[   69.732733] raid6: int32x2  gen()   534 MB/s
[   69.796110] raid6: int32x2  xor()   414 MB/s
[   69.859399] raid6: int32x1  gen()   401 MB/s
[   69.922790] raid6: int32x1  xor()   310 MB/s
[   69.930418] raid6: using algorithm altivecx4 gen() 3200 MB/s
[   69.938166] raid6: using intx1 recovery algorithm
[   70.027661] xor: measuring software checksum speed
[   70.066059]    8regs     :   123.600 MB/sec
[   70.106036]    8regs_prefetch:   122.400 MB/sec
[   70.146029]    32regs    :   126.000 MB/sec
[   70.186045]    32regs_prefetch:   122.400 MB/sec
[   70.226030]    altivec   :   738.000 MB/sec
[   70.233653] xor: using function: altivec (738.000 MB/sec)
[   70.713528] Btrfs loaded, crc32c=crc32c-generic, debug=on
Comment 3 Erhard F. 2019-12-15 15:54:38 UTC
Steps to reproduce:

1.) Configure G4 kernel with KASAN enabled
2.) Disable KASAN in lib/raid6/Makefile with KASAN_SANITIZE := n (to allow the btrfs module to load)
3.) # modprobe -v btrfs
4.) # modprobe -r -v btrfs
5.) # modprobe -v btrfs

The 2nd time btrfs gets loaded I get this hit. But only once. Further attemps to remove/load btrfs don't provoke the KASAN hit.
Comment 4 Christophe Leroy 2019-12-20 06:07:52 UTC
Can you apply https://patchwork.ozlabs.org/patch/1213028/ and select CONFIG_KASAN_VMALLOC
Comment 5 Erhard F. 2019-12-20 20:44:12 UTC
Created attachment 286385 [details]
dmesg (kernel 5.5.0-rc2+, PowerMac G4 DP)
Comment 6 Erhard F. 2019-12-20 20:45:00 UTC
Created attachment 286387 [details]
5.5.0-rc2+ kernel .config (PowerMac G4 DP)
Comment 7 Erhard F. 2019-12-20 20:50:27 UTC
(In reply to Christophe Leroy from comment #4)
> Can you apply https://patchwork.ozlabs.org/patch/1213028/ and select
> CONFIG_KASAN_VMALLOC
Re-tried with 5.5-rc2 your KASAN_VMALLOC patch and this patch (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7de7de7ca0ae0fc70515ee3154af33af75edae2c) to make -rc2 boot again.

Result with KASAN is the same as before. If I enable CONFIG_KASAN_VMALLOC additionally the KASAN hit is gone. modprobing and removing btrfs shows no error then. Interesting!

Double-checked it just to be sure.
Comment 8 Christophe Leroy 2019-12-21 09:13:29 UTC
Thanks for the test.

Looking once more at module_alloc() in arch/powerpc/mm/kasan/kasan_init_32.c, I think we are missing an initialisation.

In module_alloc(), in the call to  __vmalloc_node_range(), could you try and replace  GFP_KERNEL  by  GFP_KERNEL | __GFP_ZERO
Comment 9 Erhard F. 2019-12-21 15:18:14 UTC
module_alloc() in arch/powerpc/mm/kasan/kasan_init_32.c now is:

#if defined(CONFIG_MODULES) && !defined(CONFIG_KASAN_VMALLOC)
void *module_alloc(unsigned long size)
{
        void *base;

        base = __vmalloc_node_range(size, MODULE_ALIGN, VMALLOC_START, VMALLOC_END,
                                    GFP_KERNEL | __GFP_ZERO, PAGE_KERNEL_EXEC, VM_FLUSH_RESET_PERMS,
                                    NUMA_NO_NODE, __builtin_return_address(0));

        if (!base)
                return NULL;

        if (!kasan_init_region(base, size))
                return base;

        vfree(base);

        return NULL;
}
#endif

The change does not seem to influence this bug however. Still a KASAN hit without KASAN_VMALLOC and no hit with KASAN_VMALLOC enabled.

(bug #205099 does not show any change with KASAN_VMALLOC + modification).
Comment 10 Christophe Leroy 2019-12-21 16:43:50 UTC
So it must be something more. By the way, now that KASAN_VMALLOC exists and handle module loading better, I think we should analyse the impact of activating KASAN_VMALLOC at all time.

Regarding #205099, I believe there's a real stack related bug in raid6_pq that needs to be identified. Did you try with INLINE KASAN ?
Comment 11 Christophe Leroy 2020-01-16 09:23:53 UTC
Series at https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=153133 drops the buggy module_alloc() and forces CONFIG_KASAN_VMALLOC when CONFIG_MODULES is selected.
Comment 12 Erhard F. 2020-01-17 20:01:41 UTC
Applied your patch series on top of 5.5-rc6. CONFIG_KASAN_VMALLOC is not non-selectable but forced on by default.

Current situation is that the hit does not show up with KASAN and btrs module unloading/re-loading.

 # grep -i kasan .config
CONFIG_HAVE_ARCH_KASAN=y
CONFIG_HAVE_ARCH_KASAN_VMALLOC=y
CONFIG_CC_HAS_KASAN_GENERIC=y
CONFIG_KASAN=y
CONFIG_KASAN_GENERIC=y
CONFIG_KASAN_OUTLINE=y
# CONFIG_KASAN_INLINE is not set
CONFIG_KASAN_STACK=1
CONFIG_KASAN_VMALLOC=y
# CONFIG_TEST_KASAN is not set
CONFIG_KASAN_SHADOW_OFFSET=0xe0000000

 # modprobe -r -v btrfs
rmmod btrfs
rmmod zlib_inflate
rmmod raid6_pq
rmmod zlib_deflate
rmmod lzo_decompress
rmmod lzo_compress
rmmod zstd_compress
rmmod zstd_decompress
rmmod xor
rmmod blake2b_generic
 # modprobe -v btrfs
insmod /lib/modules/5.5.0-rc6-PowerMacG4+/kernel/lib/zlib_inflate/zlib_inflate.ko 
insmod /lib/modules/5.5.0-rc6-PowerMacG4+/kernel/lib/raid6/raid6_pq.ko 
insmod /lib/modules/5.5.0-rc6-PowerMacG4+/kernel/lib/zlib_deflate/zlib_deflate.ko 
insmod /lib/modules/5.5.0-rc6-PowerMacG4+/kernel/lib/lzo/lzo_decompress.ko 
insmod /lib/modules/5.5.0-rc6-PowerMacG4+/kernel/lib/lzo/lzo_compress.ko 
insmod /lib/modules/5.5.0-rc6-PowerMacG4+/kernel/lib/zstd/zstd_compress.ko 
insmod /lib/modules/5.5.0-rc6-PowerMacG4+/kernel/lib/zstd/zstd_decompress.ko 
insmod /lib/modules/5.5.0-rc6-PowerMacG4+/kernel/crypto/xor.ko 
insmod /lib/modules/5.5.0-rc6-PowerMacG4+/kernel/crypto/blake2b_generic.ko 
insmod /lib/modules/5.5.0-rc6-PowerMacG4+/kernel/fs/btrfs/btrfs.ko
Comment 13 Erhard F. 2020-01-17 20:02:38 UTC
Created attachment 286865 [details]
5.5.0-rc6+ kernel .config (PowerMac G4 DP)
Comment 14 Erhard F. 2020-01-17 20:03:16 UTC
Created attachment 286867 [details]
dmesg (kernel 5.5.0-rc6+, PowerMac G4 DP)
Comment 15 Christophe Leroy 2020-01-18 08:39:01 UTC
(In reply to Erhard F. from comment #12)
> Applied your patch series on top of 5.5-rc6. CONFIG_KASAN_VMALLOC is not
> non-selectable but forced on by default.
> 

I can't understand.

When I'm using 'make menuconfig', as far as 'module support' is selected, CONFIG_KASAN_VMALLOC is selected and cannot be unselected.

How do you manage to select something else than 'on' ?
Comment 16 Erhard F. 2020-01-18 11:30:28 UTC
(In reply to Christophe Leroy from comment #15)
> (In reply to Erhard F. from comment #12)
> > Applied your patch series on top of 5.5-rc6. CONFIG_KASAN_VMALLOC is not
> > non-selectable but forced on by default.
> > 
> I can't understand.
> 
> When I'm using 'make menuconfig', as far as 'module support' is selected,
> CONFIG_KASAN_VMALLOC is selected and cannot be unselected.
> 
> How do you manage to select something else than 'on' ?
Apologies for my fiddly wording. I see now that I wrote it's now 'not non-selectable' which should have been 'not selectable' due to the patch series.

Anyway, of course it's forced on by default as you said. This can be seen in the short grep -i kasan .config I posted above.
Comment 17 Erhard F. 2020-02-26 16:37:53 UTC
Re-tested with 5.6-rc3 + KASAN_VMALLOC + VMAP_STACK + THREAD_SHIFT=14 + INLINE KASAN. Works now, thanks!