Created attachment 147631 [details] kernel trace I notice since kernel 3.16.x that a freeze or a kernel panic can occur randomly if I use a mounted ntfs partition ( USB drive, or internal drive like sata ), I use ntfs-3g, my ntfs drive is mounted like this : type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other,blksize=4096) for example I use virtualbox, my virtual machine is located on a ntfs partition ( internal hard disk sata ), since kernel 3.16 I have freezes of virtualbox and sometimes kernel panic in host, but if I move my virtual machine to an ext4 partition ( my /home ) instead of a ntfs partition then the bug is gone, no problems, and if I downgrade to kernel 3.15.8 then also I don't have the bug, so there is something wrong in kernel 3.16 about ntfs partitions ( or fuse filesystem ), it seems that sometimes kernel 3.16.x can lost the connection with ntfs partitions, which leads to a freeze of the application and sometimes to a kernel paninc, I use archlinux 64 bits and some users have noticed this problem in the official forums of archlinux : https://bbs.archlinux.org/viewtopic.php?id=185841 the kernel versions tested was 3.16, 3.16.1-1 and each time I have the bug, if I test kernel 3.15.8 I don't have the bug, in kernel log I see this : BUG: unable to handle kernel paging request at ffff880143833c92 kernel: Call Trace: kernel: [<ffffffffa0b8fbdb>] ? fuse_copy_do+0xbb/0xd0 [fuse] kernel: [<ffffffffa0b90181>] fuse_copy_page+0xa1/0x5f0 [fuse] kernel: [<ffffffffa0b8fbdb>] ? fuse_copy_do+0xbb/0xd0 [fuse] kernel: [<ffffffffa0b907bf>] fuse_copy_args+0xef/0x140 [fuse] kernel: [<ffffffffa0b9133b>] fuse_dev_do_read.isra.17+0x77b/0x880 [fuse] kernel: [<ffffffff812009cc>] ? fsnotify+0x28c/0x370 kernel: [<ffffffff810a2cd0>] ? wake_up_process+0x50/0x50 kernel: [<ffffffffa0b9171a>] fuse_dev_read+0x7a/0xa0 [fuse] kernel: [<ffffffff811c1137>] do_sync_read+0x67/0xa0 kernel: [<ffffffff811c19f0>] vfs_read+0x90/0x170 kernel: [<ffffffff811c2729>] SyS_read+0x59/0xd0 kernel: [<ffffffff81015bf3>] ? do_device_not_available+0x23/0x30 kernel: [<ffffffff81530769>] system_call_fastpath+0x16/0x1b kernel: Code: e5 53 48 89 fb 75 05 e8 42 fb ff ff 48 8b 43 58 48 2b 43 50 88 43 4e 5b 5d f3 c3 90 90 90 48 89 f8 48 89 d1 48 c1 e9 03 83 e2 07 <f3> 48 a5 89 d1 f3 a4 c3 20 4c 8b 06 4c 8b 4e 08 4c 8b 56 10 4c kernel: RIP [<ffffffff812ae25d>] memcpy+0xd/0x110 kernel: RSP <ffff8800c64c7c30> kernel: CR2: ffff880143833c92 kernel: ---[ end trace be8f9df069af52ee ]---
the use case which triggers the bug : - kernel 3.16.1 ( or kernel 3.16 ) - virtualbox - a virtual machine located on a fuse filesystem partition ( ntfs for example ) - 5 minutes ( or less ) after starting the virtual machine a freeze or a kernel panic will occur in host, especially when heavy I/O ( read/write ) on the virtual machine file are made if I move the virtual machine file to an ext4 partition ( instead of using a ntfs partition ) then there is no bug, no problems, virtual machine files are usually big ( my VM size is 25 Gb ) and probably the changes made in kernel 3.16 about the fuse filesystem code are not suitable for softwares like virtualbox who work on heavy files, I see in LKLM that some FUSE patches were proposed, do you think it's related to my problem ? : https://lkml.org/lkml/2014/8/21/457 Miklos Szeredi and others developpers in this disccusion talk about "synchronous behaviour of fuse_release"
Specifically virtualbox or does running your vm's under kvm do it too ? FUSE should not break under load or with very large files. And if 3.15.x worked and 3.16 doesn't then its a regression and definitely wants fixing.
(In reply to Alan from comment #2) > Specifically virtualbox or does running your vm's under kvm do it too ? > I tried to run my virtualbox vdi file ( located in a ntfs file ) with qemu ( who uses kvm ) and there is no problem, I use these options with qemu : qemu-system-i386 -m 1024 -cpu host -enable-kvm -machine type=pc -smp 2 -soundhw ac97 -vga std -boot order=c -drive file="archlinux.vdi",if=virtio -drive file=linuxswap.vdi,if=virtio it seems that qemu/kvm manages to avoid the bug, maybe virtualbox uses a weird method to access files located on fuse filesystem ? with kernel 3.15.8 I don't have problems with virtualbox when the vdi file is located on a ntfs partition, so it's a regression with kernel 3.16.x
finally I manage to get the bug with qemu/kvm, qemu is frozen, I can move the mouse but I am unable to click something in my virtual machine, and the CPU% is 100%, when I type "top" in a console I can see that the process "mount.ntfs3-g" is at 100% CPU, when the bug occurs I can't access to my NTFS partition, I have to do a ctrl+x in order to terminate qemu, here is the trace : [33720.097127] INFO: task qemu-system-i38:5084 blocked for more than 120 seconds. [33720.097128] Not tainted 3.16.1-1-ARCH #1 [33720.097129] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [33720.097130] qemu-system-i38 D ffffffff81818500 0 5084 2772 0x00000000 [33720.097132] ffff88011025fd40 0000000000200082 ffff880027750a30 0000000000014580 [33720.097134] ffff88011025ffd8 0000000000014580 ffff880027750a30 ffff8800c44faa00 [33720.097136] ffff88012fc945f8 0000000000200082 ffff88011025fc90 ffffffff810a7d08 [33720.097138] Call Trace: [33720.097140] [<ffffffff810a7d08>] ? __enqueue_entity+0x78/0x80 [33720.097142] [<ffffffff81045c4d>] ? native_smp_send_reschedule+0x4d/0x70 [33720.097144] [<ffffffff8109e740>] ? resched_task+0xc0/0xd0 [33720.097146] [<ffffffff810acfa9>] ? check_preempt_wakeup+0x1b9/0x230 [33720.097148] [<ffffffff8152cd39>] schedule+0x29/0x70 [33720.097149] [<ffffffff8152d1a6>] schedule_preempt_disabled+0x16/0x20 [33720.097151] [<ffffffff8152eb25>] __mutex_lock_slowpath+0xe5/0x230 [33720.097153] [<ffffffff8152ec87>] mutex_lock+0x17/0x30 [33720.097156] [<ffffffffa05db493>] fuse_file_write_iter+0xb3/0x2b0 [fuse] [33720.097158] [<ffffffff811d5773>] ? pollwake+0x73/0x90 [33720.097160] [<ffffffff811c180e>] new_sync_write+0x8e/0xd0 [33720.097162] [<ffffffff811c1f87>] vfs_write+0xb7/0x200 [33720.097164] [<ffffffff811c2dca>] SyS_pwrite64+0x9a/0xc0 [33720.097166] [<ffffffff81530be9>] system_call_fastpath+0x16/0x1b so there is a serious problem in kernel 3.16 related to fuse filesystem code
Same problem here, today, immediately after installing the kernel 3.16.1 (from 3.16.0) and before reboot, with VirtualBox 4.3.12 and a VDI file of 33.1 Gb on a NTFS local disk (2 Gb RAM over 8 Gb): Aug 28 11:10:24 machine_name kernel: [12532.148252] BUG: unable to handle kernel paging request at ffff880267dc6a00 Aug 28 11:10:24 machine_name kernel: [12532.148283] IP: [<ffffffff81397d66>] memcpy+0x6/0x110 Aug 28 11:10:24 machine_name kernel: [12532.148303] PGD 1fbe067 PUD 0 Aug 28 11:10:24 machine_name kernel: [12532.148316] Oops: 0000 [#1] SMP Aug 28 11:10:24 machine_name kernel: [12532.148330] Modules linked in: btrfs xor raid6_pq ufs qnx4 hfsplus hfs minix ntfs msdos jfs xfs libcrc32c nvram pci_stub vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) bbswitch(OE) bnep rfcomm dm_crypt binfmt_misc nls_iso8859_1 ath3k uvcvideo btusb bluetooth videobuf2_vmalloc videobuf2_memops videobuf2_core v4l2_common videodev 6lowpan_iphc media snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel asus_nb_wmi asus_wmi sparse_keymap intel_rapl x86_pkg_temp_thermal intel_powerclamp mxm_wmi coretemp kvm_intel arc4 snd_hda_controller ath9k snd_hda_codec kvm ath9k_common ath9k_hw snd_hwdep ath snd_pcm mac80211 dm_multipath nvidia(POE) crct10dif_pclmul crc32_pclmul snd_seq_midi scsi_dh snd_seq_midi_event snd_rawmidi ghash_clmulni_intel snd_seq cfg80211 aesni_intel aes_x86_64 snd_seq_device lrw snd_timer gf128mul mei_me joydev snd mei serio_raw glue_helper soundcore ablk_helper cryptd mac_hid rtsx_pci_ms lpc_ich memstick wmi parport_pc ppdev lp parport dm_mirror dm_region_hash dm_log hid_generic usbhid hid rtsx_pci_sdmmc i915 i2c_algo_bit drm_kms_helper psmouse ahci drm r8169 rtsx_pci libahci mii video Aug 28 11:10:24 machine_name kernel: [12532.148737] CPU: 7 PID: 3977 Comm: mount.ntfs Tainted: P OE 3.16.0-031600-generic #201408031935 Aug 28 11:10:24 machine_name kernel: [12532.148764] Hardware name: ASUSTeK COMPUTER INC. K55VD/K55VD, BIOS K55VD.411 03/11/2013 Aug 28 11:10:24 machine_name kernel: [12532.148786] task: ffff88021c39bcc0 ti: ffff8801d2df4000 task.ti: ffff8801d2df4000 Aug 28 11:10:24 machine_name kernel: [12532.148807] RIP: 0010:[<ffffffff81397d66>] [<ffffffff81397d66>] memcpy+0x6/0x110 Aug 28 11:10:24 machine_name kernel: [12532.148830] RSP: 0018:ffff8801d2df7c80 EFLAGS: 00010202 Aug 28 11:10:24 machine_name kernel: [12532.148845] RAX: ffff8801cc350c60 RBX: ffff8801d2df7e00 RCX: 00000000000003a0 Aug 28 11:10:24 machine_name kernel: [12532.148864] RDX: 00000000000003a0 RSI: ffff880267dc6a00 RDI: ffff8801cc350c60 Aug 28 11:10:24 machine_name kernel: [12532.148884] RBP: ffff8801d2df7cb8 R08: 0000000000000000 R09: ffff8801d2df7c44 Aug 28 11:10:24 machine_name kernel: [12532.148904] R10: 00007f327471d000 R11: 0000000000000001 R12: 00000000000003a0 Aug 28 11:10:24 machine_name kernel: [12532.148924] R13: ffff8801d2df7cf0 R14: ffff8801d2df7ce4 R15: 00000000000003a0 Aug 28 11:10:24 machine_name kernel: [12532.148944] FS: 00007f327471f740(0000) GS:ffff88022afc0000(0000) knlGS:0000000000000000 Aug 28 11:10:24 machine_name kernel: [12532.148966] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 28 11:10:24 machine_name kernel: [12532.148982] CR2: ffff880267dc6a00 CR3: 00000001d2ea5000 CR4: 00000000001427e0 Aug 28 11:10:24 machine_name kernel: [12532.149002] Stack: Aug 28 11:10:24 machine_name kernel: [12532.149009] ffffffff812cf18e 00007f32746fd000 ffff8801d2df7e00 00000000ffffea00 Aug 28 11:10:24 machine_name kernel: [12532.149034] 0000000000000401 ffffea00059f7200 ffff880150b6d800 ffff8801d2df7d28 Aug 28 11:10:24 machine_name kernel: [12532.149058] ffffffff812d0d5b ffff8800ab18db00 ffff8801d2df7e00 ffff880167dc8000 Aug 28 11:10:24 machine_name kernel: [12532.149083] Call Trace: Aug 28 11:10:24 machine_name kernel: [12532.149094] [<ffffffff812cf18e>] ? fuse_copy_do+0xce/0xd0 Aug 28 11:10:24 machine_name kernel: [12532.149111] [<ffffffff812d0d5b>] fuse_copy_page+0x8b/0x220 Aug 28 11:10:24 machine_name kernel: [12532.149127] [<ffffffff812d0fcf>] fuse_copy_args+0xdf/0x150 Aug 28 11:10:24 machine_name kernel: [12532.149144] [<ffffffff812d1c9f>] fuse_dev_do_read.isra.13+0x2bf/0x310 Aug 28 11:10:24 machine_name kernel: [12532.149163] [<ffffffff812d1fa8>] fuse_dev_read+0x68/0x80 Aug 28 11:10:24 machine_name kernel: [12532.149181] [<ffffffff811e07ba>] do_sync_read+0x5a/0x90 Aug 28 11:10:24 machine_name kernel: [12532.149197] [<ffffffff811e1bbb>] vfs_read+0xab/0x180 Aug 28 11:10:24 machine_name kernel: [12532.149212] [<ffffffff811e1e3f>] SyS_read+0x4f/0xb0 Aug 28 11:10:24 machine_name kernel: [12532.149229] [<ffffffff81793fad>] system_call_fastpath+0x1a/0x1f Aug 28 11:10:24 machine_name kernel: [12532.149246] Code: 74 13 48 8b 43 58 48 2b 43 50 88 43 4e 48 83 c4 08 5b 5d c3 90 e8 fb fd ff ff eb e6 90 90 90 90 90 90 90 90 90 48 89 f8 48 89 d1 <f3> a4 c3 03 83 e2 07 f3 48 a5 89 d1 f3 a4 c3 20 4c 8b 06 4c 8b Aug 28 11:10:24 machine_name kernel: [12532.149382] RIP [<ffffffff81397d66>] memcpy+0x6/0x110 Aug 28 11:10:24 machine_name kernel: [12532.149399] RSP <ffff8801d2df7c80> Aug 28 11:10:24 machine_name kernel: [12532.149409] CR2: ffff880267dc6a00 Aug 28 11:10:24 machine_name kernel: [12532.153280] ---[ end trace 17bf6b2347e8612c ]--- Aug 28 11:10:24 machine_name kernel: [12532.153283] note: mount.ntfs[3977] exited with preempt_count 2 and after reboot: Aug 28 11:50:39 machine_name kernel: [ 758.327889] BUG: unable to handle kernel paging request at ffff8802aed6b800 Aug 28 11:50:39 machine_name kernel: [ 758.327921] IP: [<ffffffff81397d66>] memcpy+0x6/0x110 Aug 28 11:50:39 machine_name kernel: [ 758.327941] PGD 1fbe067 PUD 0 Aug 28 11:50:39 machine_name kernel: [ 758.327954] Oops: 0000 [#1] SMP Aug 28 11:50:39 machine_name kernel: [ 758.327969] Modules linked in: nvram pci_stub vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) bbswitch(OE) bnep rfcomm binfmt_misc dm_crypt nls_iso8859_1 uvcvideo ath3k videobuf2_vmalloc videobuf2_memops btusb videobuf2_core v4l2_common videodev bluetooth media 6lowpan_iphc intel_rapl x86_pkg_temp_thermal intel_powerclamp asus_nb_wmi asus_wmi sparse_keymap coretemp mxm_wmi arc4 ath9k snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic ath9k_common ath9k_hw kvm_intel snd_hda_intel snd_hda_controller kvm crct10dif_pclmul ath mac80211 snd_hda_codec cfg80211 snd_hwdep snd_pcm mei_me snd_seq_midi snd_seq_midi_event snd_rawmidi crc32_pclmul ghash_clmulni_intel nvidia(POE) mei snd_seq aesni_intel aes_x86_64 snd_seq_device snd_timer lrw dm_multipath gf128mul scsi_dh glue_helper snd ablk_helper soundcore rtsx_pci_ms memstick lpc_ich cryptd joydev mac_hid wmi serio_raw parport_pc ppdev lp parport dm_mirror dm_region_hash dm_log hid_generic usbhid hid rtsx_pci_sdmmc i915 i2c_algo_bit drm_kms_helper drm psmouse ahci r8169 rtsx_pci libahci mii video Aug 28 11:50:39 machine_name kernel: [ 758.328344] CPU: 1 PID: 4306 Comm: mount.ntfs Tainted: P OE 3.16.1-031601-generic #201408140014 Aug 28 11:50:39 machine_name kernel: [ 758.328373] Hardware name: ASUSTeK COMPUTER INC. K55VD/K55VD, BIOS K55VD.411 03/11/2013 Aug 28 11:50:39 machine_name kernel: [ 758.328395] task: ffff88009782d100 ti: ffff88008a144000 task.ti: ffff88008a144000 Aug 28 11:50:39 machine_name kernel: [ 758.328416] RIP: 0010:[<ffffffff81397d66>] [<ffffffff81397d66>] memcpy+0x6/0x110 Aug 28 11:50:39 machine_name kernel: [ 758.328439] RSP: 0018:ffff88008a147c80 EFLAGS: 00010202 Aug 28 11:50:39 machine_name kernel: [ 758.328455] RAX: ffff8801e3553c60 RBX: ffff88008a147e00 RCX: 00000000000003a0 Aug 28 11:50:39 machine_name kernel: [ 758.328475] RDX: 00000000000003a0 RSI: ffff8802aed6b800 RDI: ffff8801e3553c60 Aug 28 11:50:39 machine_name kernel: [ 758.328495] RBP: ffff88008a147cb8 R08: 0000000000000000 R09: ffff88008a147c44 Aug 28 11:50:39 machine_name kernel: [ 758.328514] R10: 00007f6017642000 R11: 0000000000000001 R12: 00000000000003a0 Aug 28 11:50:39 machine_name kernel: [ 758.328534] R13: ffff88008a147cf0 R14: ffff88008a147ce4 R15: 00000000000003a0 Aug 28 11:50:39 machine_name kernel: [ 758.328555] FS: 00007f6017644740(0000) GS:ffff88022ae40000(0000) knlGS:0000000000000000 Aug 28 11:50:39 machine_name kernel: [ 758.328578] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 28 11:50:39 machine_name kernel: [ 758.328595] CR2: ffff8802aed6b800 CR3: 0000000099c8d000 CR4: 00000000001427e0 Aug 28 11:50:39 machine_name kernel: [ 758.328615] Stack: Aug 28 11:50:39 machine_name kernel: [ 758.328622] ffffffff812cf18e 00007f6017622000 ffff88008a147e00 00000000ffff8800 Aug 28 11:50:39 machine_name kernel: [ 758.328647] 0000000000000401 ffffea0006bb5cc0 ffff88009799ce00 ffff88008a147d28 Aug 28 11:50:39 machine_name kernel: [ 758.328671] ffffffff812d0d5b ffff88009ce2cd00 ffff88008a147e00 ffff8801aed73000 Aug 28 11:50:39 machine_name kernel: [ 758.328696] Call Trace: Aug 28 11:50:39 machine_name kernel: [ 758.328706] [<ffffffff812cf18e>] ? fuse_copy_do+0xce/0xd0 Aug 28 11:50:39 machine_name kernel: [ 758.328723] [<ffffffff812d0d5b>] fuse_copy_page+0x8b/0x220 Aug 28 11:50:39 machine_name kernel: [ 758.328740] [<ffffffff812d0fcf>] fuse_copy_args+0xdf/0x150 Aug 28 11:50:39 machine_name kernel: [ 758.328757] [<ffffffff812d1c9f>] fuse_dev_do_read.isra.13+0x2bf/0x310 Aug 28 11:50:39 machine_name kernel: [ 758.328776] [<ffffffff812d1fa8>] fuse_dev_read+0x68/0x80 Aug 28 11:50:39 machine_name kernel: [ 758.328793] [<ffffffff811e07ba>] do_sync_read+0x5a/0x90 Aug 28 11:50:39 machine_name kernel: [ 758.328809] [<ffffffff811e1bbb>] vfs_read+0xab/0x180 Aug 28 11:50:39 machine_name kernel: [ 758.328824] [<ffffffff811e1e3f>] SyS_read+0x4f/0xb0 Aug 28 11:50:39 machine_name kernel: [ 758.328840] [<ffffffff81793fad>] system_call_fastpath+0x1a/0x1f Aug 28 11:50:39 machine_name kernel: [ 758.328857] Code: 74 13 48 8b 43 58 48 2b 43 50 88 43 4e 48 83 c4 08 5b 5d c3 90 e8 fb fd ff ff eb e6 90 90 90 90 90 90 90 90 90 48 89 f8 48 89 d1 <f3> a4 c3 03 83 e2 07 f3 48 a5 89 d1 f3 a4 c3 20 4c 8b 06 4c 8b Aug 28 11:50:39 machine_name kernel: [ 758.328991] RIP [<ffffffff81397d66>] memcpy+0x6/0x110 Aug 28 11:50:39 machine_name kernel: [ 758.329008] RSP <ffff88008a147c80> Aug 28 11:50:39 machine_name kernel: [ 758.329018] CR2: ffff8802aed6b800 Aug 28 11:50:39 machine_name kernel: [ 758.332731] ---[ end trace 471e6a66f8cc521f ]--- Aug 28 11:50:39 machine_name kernel: [ 758.332733] note: mount.ntfs[4306] exited with preempt_count 2 After the error, trying to open mounted ntfs volume produced a "transport endpoint not connected". So far, no problems with the 3.16.0 without trying the update.
(In reply to RL from comment #5) > > So far, no problems with the 3.16.0 without trying the update. Are you sure ? because I have also the bug with kernel 3.16.0 ( but I use archlinux, not ubuntu ) If I use kernel 3.15.8 I don't have the bug
Just replicated with 3.16.0-031600-generic [Linux Mint 17], when I was applying some Windows updates on the virtual machine (on VirtualBox 4.3.14 r95030) in the middle of the installation: Sep 4 08:02:49 machine_name kernel: [ 1324.669667] BUG: unable to handle kernel paging request at ffff88023f5fba00 Sep 4 08:02:49 machine_name kernel: [ 1324.669697] IP: [<ffffffff81397d66>] memcpy+0x6/0x110 Sep 4 08:02:49 machine_name kernel: [ 1324.669717] PGD 1fbe067 PUD 1fc1067 PMD 0 Sep 4 08:02:49 machine_name kernel: [ 1324.669734] Oops: 0000 [#1] SMP Sep 4 08:02:49 machine_name kernel: [ 1324.669748] Modules linked in: nvram pci_stub vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) bbswitch(OE) bnep rfcomm dm_crypt binfmt_misc nls_iso8859_1 uvcvideo ath3k btusb videobuf2_vmalloc videobuf2_memops bluetooth videobuf2_core v4l2_common videodev media intel_rapl x86_pkg_temp_thermal intel_powerclamp 6lowpan_iphc coretemp kvm_intel kvm crct10dif_pclmul arc4 crc32_pclmul ath9k asus_nb_wmi ghash_clmulni_intel snd_hda_codec_hdmi asus_wmi snd_hda_codec_realtek aesni_intel snd_hda_codec_generic snd_hda_intel ath9k_common sparse_keymap ath9k_hw aes_x86_64 snd_hda_controller lrw snd_hda_codec gf128mul mxm_wmi snd_hwdep snd_pcm ath mac80211 snd_seq_midi snd_seq_midi_event glue_helper snd_rawmidi nvidia(POE) cfg80211 snd_seq snd_seq_device snd_timer ablk_helper mei_me rtsx_pci_ms cryptd mei snd lpc_ich memstick dm_multipath soundcore scsi_dh joydev serio_raw mac_hid wmi parport_pc ppdev lp parport dm_mirror dm_region_hash dm_log hid_generic usbhid hid rtsx_pci_sdmmc i915 i2c_algo_bit psmouse drm_kms_helper ahci libahci r8169 rtsx_pci drm mii video Sep 4 08:02:49 machine_name kernel: [ 1324.670123] CPU: 7 PID: 4214 Comm: mount.ntfs Tainted: P OE 3.16.0-031600-generic #201408031935 Sep 4 08:02:49 machine_name kernel: [ 1324.670149] Hardware name: ASUSTeK COMPUTER INC. K55VD/K55VD, BIOS K55VD.411 03/11/2013 Sep 4 08:02:49 machine_name kernel: [ 1324.670172] task: ffff8800a0651e60 ti: ffff8801dd298000 task.ti: ffff8801dd298000 Sep 4 08:02:49 machine_name kernel: [ 1324.670193] RIP: 0010:[<ffffffff81397d66>] [<ffffffff81397d66>] memcpy+0x6/0x110 Sep 4 08:02:49 machine_name kernel: [ 1324.670215] RSP: 0018:ffff8801dd29bc80 EFLAGS: 00010202 Sep 4 08:02:49 machine_name kernel: [ 1324.670231] RAX: ffff8801bcf05e60 RBX: ffff8801dd29be00 RCX: 00000000000001a0 Sep 4 08:02:49 machine_name kernel: [ 1324.670250] RDX: 00000000000001a0 RSI: ffff88023f5fba00 RDI: ffff8801bcf05e60 Sep 4 08:02:49 machine_name kernel: [ 1324.670270] RBP: ffff8801dd29bcb8 R08: 0000000000000000 R09: ffff8801dd29bc44 Sep 4 08:02:49 machine_name kernel: [ 1324.670296] R10: 00007f8fd3d14000 R11: 0000000000000001 R12: 00000000000001a0 Sep 4 08:02:49 machine_name kernel: [ 1324.670321] R13: ffff8801dd29bcf0 R14: ffff8801dd29bce4 R15: 00000000000001a0 Sep 4 08:02:49 machine_name kernel: [ 1324.670345] FS: 00007f8fd3d16740(0000) GS:ffff88022afc0000(0000) knlGS:0000000000000000 Sep 4 08:02:49 machine_name kernel: [ 1324.670368] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Sep 4 08:02:49 machine_name kernel: [ 1324.670384] CR2: ffff88023f5fba00 CR3: 00000001e8a57000 CR4: 00000000001427e0 Sep 4 08:02:49 machine_name kernel: [ 1324.670404] Stack: Sep 4 08:02:49 machine_name kernel: [ 1324.670411] ffffffff812cf18e 00007f8fd3cf4000 ffff8801dd29be00 00000000ffffea00 Sep 4 08:02:49 machine_name kernel: [ 1324.670436] 0000000000000201 ffffea0004fd7f40 ffff8801d5d05600 ffff8801dd29bd28 Sep 4 08:02:49 machine_name kernel: [ 1324.670460] ffffffff812d0d5b ffff8800a5e35b00 ffff8801dd29be00 ffff88013f5fd000 Sep 4 08:02:49 machine_name kernel: [ 1324.670485] Call Trace: Sep 4 08:02:49 machine_name kernel: [ 1324.670495] [<ffffffff812cf18e>] ? fuse_copy_do+0xce/0xd0 Sep 4 08:02:49 machine_name kernel: [ 1324.670513] [<ffffffff812d0d5b>] fuse_copy_page+0x8b/0x220 Sep 4 08:02:49 machine_name kernel: [ 1324.670529] [<ffffffff812d0fcf>] fuse_copy_args+0xdf/0x150 Sep 4 08:02:49 machine_name kernel: [ 1324.670546] [<ffffffff812d1c9f>] fuse_dev_do_read.isra.13+0x2bf/0x310 Sep 4 08:02:49 machine_name kernel: [ 1324.670565] [<ffffffff812d1fa8>] fuse_dev_read+0x68/0x80 Sep 4 08:02:49 machine_name kernel: [ 1324.670584] [<ffffffff811e07ba>] do_sync_read+0x5a/0x90 Sep 4 08:02:49 machine_name kernel: [ 1324.670600] [<ffffffff811e1bbb>] vfs_read+0xab/0x180 Sep 4 08:02:49 machine_name kernel: [ 1324.670615] [<ffffffff811e1e3f>] SyS_read+0x4f/0xb0 Sep 4 08:02:49 machine_name kernel: [ 1324.670632] [<ffffffff81793fad>] system_call_fastpath+0x1a/0x1f Sep 4 08:02:49 machine_name kernel: [ 1324.670649] Code: 74 13 48 8b 43 58 48 2b 43 50 88 43 4e 48 83 c4 08 5b 5d c3 90 e8 fb fd ff ff eb e6 90 90 90 90 90 90 90 90 90 48 89 f8 48 89 d1 <f3> a4 c3 03 83 e2 07 f3 48 a5 89 d1 f3 a4 c3 20 4c 8b 06 4c 8b Sep 4 08:02:49 machine_name kernel: [ 1324.670792] RIP [<ffffffff81397d66>] memcpy+0x6/0x110 Sep 4 08:02:49 machine_name kernel: [ 1324.670809] RSP <ffff8801dd29bc80> Sep 4 08:02:49 machine_name kernel: [ 1324.670820] CR2: ffff88023f5fba00 Sep 4 08:02:49 machine_name kernel: [ 1324.674521] ---[ end trace 30791b1464fa352e ]--- Sep 4 08:02:49 machine_name kernel: [ 1324.674523] note: mount.ntfs[4214] exited with preempt_count 2 No problem so far when I don't trying updates. But yes, there is also in 3.16.0.
I have the same problem on Debian Jessie since updating the kernel through the standard repositories (not sure which version exactly caused the problem). Current kernel with the problem is "3.16-2-amd64 #1 Debian 3.16.3-2". VirtualBox is the only thing that seems to run into this problem, copying large files from NTFS works fine. I tried Windows 7 and Mac OS X guests and they both run into the problem during their boot process.
I hit the same bug using Debian Jessie with kernel Linux debian 3.16-2-amd64 #1 SMP Debian 3.16.3-2 (2014-09-20) x86_64 GNU/Linux. I am using virtualbox-4.3 (4.3.16-95972~Debian~wheezy) on a Debian host 3.16.3-2 . The virtualbox guest VDI image is on a disk with a NTFS file system. Today Virtualbox was frozen with DMA errors in the guest and I found the following oops in the /var/log/syslog of the Debian 3.16.3-2 host: Oct 11 22:19:09 debian kernel: [21958.576103] BUG: unable to handle kernel paging request at ffff88015257aa00 Oct 11 22:19:09 debian kernel: [21958.576112] IP: [<ffffffff812b39bd>] memcpy+0xd/0x110 Oct 11 22:19:09 debian kernel: [21958.576120] PGD 1af1067 PUD 0 Oct 11 22:19:09 debian kernel: [21958.576124] Oops: 0000 [#1] SMP Oct 11 22:19:09 debian kernel: [21958.576128] Modules linked in: bnep bluetooth 6lowpan_iphc ctr ccm pci_stub vboxpci(O) vboxnetadp(O) vboxnetflt(O) vboxdrv(O) cpufreq_userspace cpufreq_conservative cpufreq_stats cpufreq_powersave nfsd auth_rpcgss oid_registry nfs_acl nfs uvcvideo videobuf2_vmalloc lockd fscache videobuf2_memops sunrpc videobuf2_core v4l2_common videodev media nvidia(PO) joydev acer_wmi mxm_wmi iTCO_wdt iTCO_vendor_support sparse_keymap coretemp kvm_intel psmouse evdev serio_raw pcspkr arc4 iwldvm snd_hda_codec_hdmi kvm mac80211 i2c_i801 snd_hda_codec_realtek snd_hda_codec_generic drm lpc_ich mfd_core snd_hda_intel i2c_core snd_hda_controller iwlwifi cfg80211 rfkill snd_hda_codec shpchp snd_hwdep battery snd_pcm snd_timer snd soundcore wmi ac video button acpi_cpufreq processor loop fuse parport_pc ppdev lp parport autofs4 ext4 crc16 mbcache jbd2 sr_mod cdrom sg sd_mod crc_t10dif crct10dif_generic crct10dif_common hid_generic usbhid hid usb_storage ahci libahci libata scsi_mod sky2 thermal thermal_sys ehci_pci uhci_hcd ehci_hcd usbcore usb_common Oct 11 22:19:09 debian kernel: [21958.576216] CPU: 1 PID: 4323 Comm: mount.ntfs Tainted: P O 3.16-2-amd64 #1 Debian 3.16.3-2 Oct 11 22:19:09 debian kernel: [21958.576219] Hardware name: Acer Aspire 5930 /Eiger , BIOS V1.21 08/22/2008 Oct 11 22:19:09 debian kernel: [21958.576223] task: ffff88006665b4f0 ti: ffff8800667cc000 task.ti: ffff8800667cc000 Oct 11 22:19:09 debian kernel: [21958.576225] RIP: 0010:[<ffffffff812b39bd>] [<ffffffff812b39bd>] memcpy+0xd/0x110 Oct 11 22:19:09 debian kernel: [21958.576230] RSP: 0018:ffff8800667cfc90 EFLAGS: 00010246 Oct 11 22:19:09 debian kernel: [21958.576232] RAX: ffff88006b05ca60 RBX: ffff8800667cfe38 RCX: 00000000000000b4 Oct 11 22:19:09 debian kernel: [21958.576235] RDX: 0000000000000000 RSI: ffff88015257aa00 RDI: ffff88006b05ca60 Oct 11 22:19:09 debian kernel: [21958.576237] RBP: 00000000000005a0 R08: 0000000000000000 R09: ffff8800667cfc6c Oct 11 22:19:09 debian kernel: [21958.576240] R10: ffff880000000328 R11: 0000000000000005 R12: ffff8800667cfcf8 Oct 11 22:19:09 debian kernel: [21958.576242] R13: ffff8800667cfcf4 R14: 00000000000005a0 R15: 0000000000000600 Oct 11 22:19:09 debian kernel: [21958.576245] FS: 00007f1f4cc6b700(0000) GS:ffff8800bf500000(0000) knlGS:0000000000000000 Oct 11 22:19:09 debian kernel: [21958.576248] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Oct 11 22:19:09 debian kernel: [21958.576250] CR2: ffff88015257aa00 CR3: 000000005fe5a000 CR4: 00000000000027e0 Oct 11 22:19:09 debian kernel: [21958.576252] Stack: Oct 11 22:19:09 debian kernel: [21958.576254] ffffffffa02347eb 00000000ffffea00 0000000000000001 ffff88005257c000 Oct 11 22:19:09 debian kernel: [21958.576258] ffffea0001203320 ffff8800667cfe38 ffffffffa0235044 0000000000000001 Oct 11 22:19:09 debian kernel: [21958.576262] ffff88004f07cf00 00007f1f4cac0000 ffffffffa02347eb ffff8800667cfe38 Oct 11 22:19:09 debian kernel: [21958.576266] Call Trace: Oct 11 22:19:09 debian kernel: [21958.576277] [<ffffffffa02347eb>] ? fuse_copy_do+0xbb/0xc0 [fuse] Oct 11 22:19:09 debian kernel: [21958.576285] [<ffffffffa0235044>] ? fuse_copy_page+0x94/0x640 [fuse] Oct 11 22:19:09 debian kernel: [21958.576291] [<ffffffffa02347eb>] ? fuse_copy_do+0xbb/0xc0 [fuse] Oct 11 22:19:09 debian kernel: [21958.576299] [<ffffffffa02356e6>] ? fuse_copy_args+0xf6/0x150 [fuse] Oct 11 22:19:09 debian kernel: [21958.576306] [<ffffffffa023616f>] ? fuse_dev_do_read.isra.14+0x6ef/0x800 [fuse] Oct 11 22:19:09 debian kernel: [21958.576311] [<ffffffff811dfb7c>] ? fsnotify+0x23c/0x300 Oct 11 22:19:09 debian kernel: [21958.576316] [<ffffffff81094a80>] ? wake_up_state+0x10/0x10 Oct 11 22:19:09 debian kernel: [21958.576323] [<ffffffffa023651d>] ? fuse_dev_read+0x6d/0x80 [fuse] Oct 11 22:19:09 debian kernel: [21958.576328] [<ffffffff811a495c>] ? do_sync_read+0x5c/0x90 Oct 11 22:19:09 debian kernel: [21958.576332] [<ffffffff811a5163>] ? vfs_read+0x93/0x170 Oct 11 22:19:09 debian kernel: [21958.576336] [<ffffffff811a5d92>] ? SyS_read+0x42/0xa0 Oct 11 22:19:09 debian kernel: [21958.576341] [<ffffffff8150c26d>] ? system_call_fast_compare_end+0x10/0x15 Oct 11 22:19:09 debian kernel: [21958.576343] Code: 66 0f 1f 44 00 00 e8 03 fc ff ff 48 8b 43 58 48 2b 43 50 88 43 4e eb e6 90 90 90 90 90 90 48 89 f8 48 89 d1 48 c1 e9 03 83 e2 07 <f3> 48 a5 89 d1 f3 a4 c3 20 4c 8b 06 4c 8b 4e 08 4c 8b 56 10 4c Oct 11 22:19:09 debian kernel: [21958.576381] RIP [<ffffffff812b39bd>] memcpy+0xd/0x110 Oct 11 22:19:09 debian kernel: [21958.576385] RSP <ffff8800667cfc90> Oct 11 22:19:09 debian kernel: [21958.576387] CR2: ffff88015257aa00 Oct 11 22:19:09 debian kernel: [21958.576390] ---[ end trace c7ac8b352edfefc6 ]--- Oct 11 22:19:09 debian kernel: [21958.576394] note: mount.ntfs[4323] exited with preempt_count 2 I was barely able to reboot the system using the CTRL-ALT-F1 console. Today I had a previous freeze using VirtualBox, but the system was completely screwed up and I was not able to obtain a kernel trace.
I can confirm the bug, I have the same problem. Running Debian Jessie kernel 3.16.6 (vanilla), virtualbox 4.3.14 (debian package). When virtualizating an x86 OpenSuse machine, after a while it crashes my system. First I lost the keyboard, and after that the system freezes. My syslog: Oct 27 14:29:14 thinkpadBor avahi-daemon[779]: Received response from host 192.168.20.190 with invalid source port 55922 on interface 'eth0.0' Oct 27 14:33:39 thinkpadBor systemd-udevd[182]: Network interface NamePolicy= disabled on kernel commandline, ignoring. Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423441] BUG: unable to handle kernel paging request at ffff8803ccd77a00 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423456] IP: [<ffffffff8124ddb6>] memcpy+0x6/0x110 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423462] PGD 183d067 PUD 0 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423466] Oops: 0002 [#1] SMP Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423469] Modules linked in: binfmt_misc bnep vboxpci(O) vboxnetadp(O) vboxnetflt(O) vboxdrv(O) nfsd auth_rpcgss oid_registry nfs_acl nfs lockd fscache sunrpc ecb uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_core v4l2_common x86_pkg_temp_thermal btusb bluetooth videodev cdc_mbim cdc_ncm media usbnet cdc_wdm joydev cdc_acm mii intel_powerclamp intel_rapl iTCO_wdt coretemp nouveau kvm_intel iTCO_vendor_support kvm snd_hda_codec_realtek snd_hda_codec_generic arc4 mxm_wmi ttm drm_kms_helper drm crc32_pclmul crc32c_intel nvidiafb snd_hda_intel ghash_clmulni_intel vgastate aesni_intel snd_hda_controller aes_x86_64 iwldvm mac80211 iwlwifi i2c_algo_bit snd_hda_codec psmouse evdev glue_helper fb_ddc lrw gf128mul ablk_helper cryptd mei_me serio_raw snd_hwdep cfg80211 pcspkr snd_pcm shpchp snd_timer mei thinkpad_acpi nvram snd video ac i2c_i801 soundcore tpm_tis i2c_core wmi tpm battery lpc_ich mfd_core button rfkill processor fuse autofs4 ext4 crc16 mbcache jbd2 sg sd_mod sr_mod crc_t10dif cdrom crct10dif_generic hid_generic usbhid hid crct10dif_pclmul crct10dif_common ahci libahci libata scsi_mod sdhci_pci sdhci mmc_core ehci_pci ehci_hcd e1000e ptp pps_core xhci_hcd usbcore usb_common thermal thermal_sys Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423595] CPU: 1 PID: 564 Comm: mount.ntfs-3g Tainted: G O 3.16.6 #1 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423599] Hardware name: LENOVO 2349KAG/2349KAG, BIOS G1ETA1WW (2.61 ) 12/03/2013 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423602] task: ffff88030cc5c490 ti: ffff88030a65c000 task.ti: ffff88030a65c000 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423605] RIP: 0010:[<ffffffff8124ddb6>] [<ffffffff8124ddb6>] memcpy+0x6/0x110 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423610] RSP: 0018:ffff88030a65fbb8 EFLAGS: 00010246 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423612] RAX: ffff8803ccd77a00 RBX: 0000000000000600 RCX: 0000000000000600 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423614] RDX: 0000000000000600 RSI: ffff880280df94f0 RDI: ffff8803ccd77a00 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423617] RBP: ffff88030a65fd70 R08: 0000000000000000 R09: ffff88030a65fb94 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423620] R10: 0000000000020000 R11: ffff880000000730 R12: ffff88030a65fc14 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423622] R13: ffff88030a65fc18 R14: 0000000000000600 R15: 0000000000000600 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423625] FS: 00007f6cdc257700(0000) GS:ffff88031dc80000(0000) knlGS:0000000000000000 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.423628] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426375] CR2: ffff8803ccd77a00 CR3: 000000030a6e7000 CR4: 00000000001427e0 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426386] Stack: Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426390] ffffffffa02f877b 00000000ffffea00 ffff88030a65fd01 ffff8802ccd79000 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426409] ffffea0009ccf278 ffff88030a65fd70 ffffffffa02f8f2c 0000008000000000 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426427] ffff880047d47200 ffffffff81048a82 0000000000000000 000006008108af5c Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426445] Call Trace: Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426459] [<ffffffffa02f877b>] ? fuse_copy_do+0x7b/0xc0 [fuse] Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426469] [<ffffffffa02f8f2c>] ? fuse_copy_page+0x9c/0x680 [fuse] Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426475] [<ffffffff81048a82>] ? get_user_pages_fast+0xb2/0x1a0 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426488] [<ffffffffa02f95fe>] ? fuse_copy_args+0xee/0x140 [fuse] Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426499] [<ffffffffa02faba2>] ? fuse_dev_do_write+0x772/0xce0 [fuse] Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426512] [<ffffffffa02f87b3>] ? fuse_copy_do+0xb3/0xc0 [fuse] Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426522] [<ffffffffa02f8e7d>] ? fuse_copy_one+0x3d/0x50 [fuse] Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426534] [<ffffffffa02f955f>] ? fuse_copy_args+0x4f/0x140 [fuse] Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426542] [<ffffffff8114c1d0>] ? do_sync_read+0x90/0x90 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426556] [<ffffffffa02fb469>] ? fuse_dev_write+0x59/0x70 [fuse] Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426564] [<ffffffff8114c2a3>] ? do_sync_readv_writev+0x43/0x70 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426575] [<ffffffff8114dabd>] ? do_readv_writev+0x1bd/0x240 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426585] [<ffffffff8114c33c>] ? new_sync_read+0x6c/0xa0 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426596] [<ffffffff8114dcd1>] ? SyS_writev+0x41/0xd0 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426606] [<ffffffff8147ea6d>] ? system_call_fastpath+0x1a/0x1f Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426615] Code: 48 89 fb 75 05 e8 eb fa ff ff 48 8b 43 58 48 2b 43 50 88 43 4e 5b c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 48 89 f8 48 89 d1 <f3> a4 c3 03 83 e2 07 f3 48 a5 89 d1 f3 a4 c3 20 4c 8b 06 4c 8b Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426740] RIP [<ffffffff8124ddb6>] memcpy+0x6/0x110 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426751] RSP <ffff88030a65fbb8> Oct 27 14:43:59 thinkpadBor kernel: [ 2502.426758] CR2: ffff8803ccd77a00 Oct 27 14:43:59 thinkpadBor kernel: [ 2502.463718] ---[ end trace 1ccb63d055b06936 ]--- Oct 27 14:43:59 thinkpadBor kernel: [ 2502.463721] note: mount.ntfs-3g[564] exited with preempt_count 2 Oct 27 14:43:59 thinkpadBor systemd-udevd[182]: Network interface NamePolicy= disabled on kernel commandline, ignoring.
Adding some info to my comment, my OpenSuse virtual machine is in a ntfs partition.
After one day testing with kernel 3.17.1, I can say it doesn't happen anymore.
Created attachment 155711 [details] test patch Happy to hear it's gone in 3.17, but it would still be good to know what exactly was the cause of the bug and the fix. Could you please try the attached test patch on a 3.16 kernel? After the BUG triggers, it would be also good to have the backtrace for all processes (echo t > /proc/sysrq-trigger).
Created attachment 155751 [details] Full log with backtrace, fuse with patch of Miklos Szeredi @Miklos Szeredi Log with backtraces, kernel 3.16.4 with test patch applied.
Thanks, Looks like it was the following commit in 3.16 that broke it: c9c37e2e6378 "fuse: switch to iov_iter_get_pages()" And the following commits in 3.17 are responsible for fixing it: c7f3888ad7f0 "switch iov_iter_get_pages() to passing maximal number of pages" 2c80929c4c4d "fuse: honour max_read and max_write in direct_io mode"
Sent notification to stable@vger...
Note from Greg KH: I'm not doing any more 3.16-stable releases after the one I'm releasing tomorrow, so these are not going to make it in, sorry.