Created attachment 141461 [details] Additional infos about errors Hello, on a ESXi 4.1 HP ML350G5 host, I continue getting this sort of errors using linux kernel. I tried all sorts of versions from UBUNTU mainline... http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.15.1-utopic/ Please, can someone get me if the right direction? I don't event understand where is locaed the aread that is continuing to generate this kind of errors. After that I have to rebot the system, because the system wil get nearly unresposive and even the reboot process is getting minutes to perfor. I'm attaching all infos I collected... many thanks. Dif tor eh smusma [410224.149680] INFO: rcu_sched detected stalls on CPUs/tasks: {} (detected by 3, t=15020 jiffies, g=996355, c=996354, q=0) [410226.013662] INFO: Stall ended before state dump start [410233.550339] BUG: soft lockup - CPU#1 stuck for 67s! [atop:25014] [410233.594140] Modules linked in: btrfs raid6_pq xor ufs qnx4 hfsplus hfs minix ntfs msdos jfs cpuid tcp_htcp sch_codel sch_fq_codel xt_multiport xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6 ipt_REJECT xt_LOG xt_limit xt_tcpudp xt_addrtype nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp nf_conntrack iptable_filter ip_tables x_tables ppdev vmw_balloon irda coretemp serio_raw crc_ccitt parport_pc vmwgfx ttm drm vmw_vmci mac_hid shpchp i2c_piix4 lp parport xfs libcrc32c psmouse floppy vmw_pvscsi vmxnet3 [410236.060588] CPU: 1 PID: 25014 Comm: atop Not tainted 3.15.1-031501-generic #201406161841 [410236.060596] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 01/15/2013 [410236.060599] task: ffff880028dd1930 ti: ffff880024768000 task.ti: ffff880024768000 [410236.060602] RIP: 0010:[<ffffffff81071fa0>] [<ffffffff81071fa0>] __do_softirq+0x90/0x330 [410236.749024] RSP: 0018:ffff88013fc23f08 EFLAGS: 00000206 [410236.749028] RAX: 0000000000000000 RBX: ffff880028dd1930 RCX: ffff88002476bfd8 [410236.749029] RDX: 0000000000000001 RSI: ffff88013fc23e28 RDI: ffff880028dd1e44 [410236.749031] RBP: ffff88013fc23f68 R08: 000000000000dfbc R09: 0000000000000001 [410236.749032] R10: 0000000000000a7d R11: 0000000000000002 R12: ffff88013fc23e78 [410236.749033] R13: ffffffff817853dd R14: ffff88013fc23f68 R15: 0000000000000000 [410236.749041] FS: 00007f6d6c179740(0000) GS:ffff88013fc20000(0000) knlGS:0000000000000000 [410236.749042] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [410236.749044] CR2: 00007f6b980c8000 CR3: 000000002474c000 CR4: 00000000000006e0 [410236.774877] Stack: [410236.774883] ffff88002476bfd8 00000001061a2c57 0000000a00406100 0000000000000001 [410236.774886] ffff88013fc23f48 ffffffff810a5652 ffff88013fc2e960 0000000000000000 [410236.774889] ffff880138454b90 ffffffff81828c60 ffffffff81829278 0000000000000006 [410236.774891] Call Trace: [410236.806600] <IRQ> [410236.823547] [<ffffffff810a5652>] ? __vtime_account_system+0x32/0x40 [410236.888703] [<ffffffff8107254e>] irq_exit+0x11e/0x140 [410237.203751] [<ffffffff81786a9a>] smp_apic_timer_interrupt+0x4a/0x60 [410237.203769] [<ffffffff817853dd>] apic_timer_interrupt+0x6d/0x80 [410237.203770] <EOI> [410237.203773] [<ffffffff81237d10>] ? proc_alloc_inode+0x20/0xb0 [410237.666280] [<ffffffff8177afa7>] ? _raw_spin_lock+0x37/0x50 [410237.768396] [<ffffffff811ebc0d>] inode_sb_list_add+0x1d/0x70 [410237.768412] [<ffffffff811ee22d>] new_inode+0x2d/0x40 [410237.768416] [<ffffffff811ea6e4>] ? __d_alloc+0x34/0x180 [410237.777696] [<ffffffff8123c455>] proc_pid_make_inode+0x25/0x110 [410237.777704] [<ffffffff8123c634>] proc_pident_instantiate+0x24/0xd0 [410237.777706] [<ffffffff8123c77b>] proc_pident_lookup+0x9b/0xe0 [410237.777708] [<ffffffff8123c81a>] proc_tgid_base_lookup+0x1a/0x20 [410237.955919] [<ffffffff811dc03d>] lookup_real+0x1d/0x60 [410237.955986] [<ffffffff811e1578>] lookup_open+0xc8/0x1d0 [410237.955990] [<ffffffff811e1954>] do_last+0x2d4/0x7b0 [410238.046444] [<ffffffff812ef976>] ? security_file_alloc+0x16/0x20 [410238.046459] [<ffffffff811e1ef4>] path_openat+0xc4/0x4c0 [410238.099704] [<ffffffff81186e44>] ? tlb_flush_mmu_free+0x34/0x60 [410238.099725] [<ffffffff81187844>] ? tlb_flush_mmu+0x24/0x30 [410238.099729] [<ffffffff811e2c63>] do_filp_open+0x43/0xa0 [410238.099733] [<ffffffff811ef9cd>] ? __alloc_fd+0xcd/0x120 [410238.103425] [<ffffffff811d1742>] do_sys_open+0x132/0x220 [410238.272278] [<ffffffff810220dd>] ? syscall_trace_leave+0xdd/0x150 [410238.272296] [<ffffffff811d184e>] SyS_open+0x1e/0x20 [410238.272300] [<ffffffff8178487f>] tracesys+0xe1/0xe6 [410238.272302] Code: 00 00 65 8b 14 25 64 b0 00 00 c7 45 b4 0a 00 00 00 89 55 b8 48 89 4d a0 0f 1f 00 65 c7 04 25 40 19 01 00 00 00 00 00 fb 66 66 90 <66> 66 90 41 bf ff ff ff ff 45 0f bc fe 41 83 c7 01 0f 84 88 00 [410286.360708] INFO: rcu_sched detected stalls on CPUs/tasks: { 0} (detected by 3, t=15020 jiffies, g=996356, c=996355, q=0) [410286.803644] sending NMI to all CPUs: [410286.926271] NMI backtrace for cpu 3 [410286.926302] CPU: 3 PID: 2400 Comm: postgres Not tainted 3.15.1-031501-generic #201406161841 [410286.926304] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 01/15/2013 [410286.926306] task: ffff8800b8ae64c0 ti: ffff880138b04000 task.ti: ffff880138b04000 [410286.926308] RIP: 0010:[<ffffffff8104a310>] [<ffffffff8104a310>] default_send_IPI_mask_sequence_phys+0xa0/0xe0 [410287.139517] RSP: 0018:ffff88013fc63b70 EFLAGS: 00000046 [410287.139521] RAX: 0000000000000400 RBX: 0000000000000002 RCX: 0000000003000000 [410287.139523] RDX: 0000000000000003 RSI: 0000000000000100 RDI: 0000000000000300 [410287.139524] RBP: ffff88013fc63bb0 R08: ffffffff81d1bbc0 R09: 0000000000000000 [410287.139525] R10: 00000000000005a8 R11: 3a73555043206c6c R12: 000000000000b06a [410287.139527] R13: ffffffff81d1bbc0 R14: 0000000000000400 R15: 0000000000000082 [410287.139532] FS: 00007f52d5e11780(0000) GS:ffff88013fc60000(0000) knlGS:0000000000000000 [410287.139534] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [410287.139535] CR2: 00007f52d5e30000 CR3: 0000000138aed000 CR4: 00000000000006e0 [410287.139547] Stack: [410287.139549] ffffffff00000003 0000000000000003 ffff88013fc63bd0 0000000000002710 [410287.139554] 0000000000000001 0000000000000000 ffffffff81d1bbe0 0000000000000087 [410287.139557] ffff88013fc63bc0 ffffffff8104eb97 ffff88013fc63be0 ffffffff8104a4e4 [410287.139559] Call Trace: [410287.139564] <IRQ> [410287.139570] [<ffffffff8104eb97>] physflat_send_IPI_all+0x17/0x20 [410287.139581] [<ffffffff8104a4e4>] arch_trigger_all_cpu_backtrace+0x74/0xb0 [410287.161341] [<ffffffff810d5397>] print_other_cpu_stall+0x1f7/0x230 [410287.161353] [<ffffffff810d5467>] check_cpu_stall.isra.50+0x97/0xb0 [410287.161356] [<ffffffff810d54b8>] __rcu_pending+0x38/0x1d0 [410287.161359] [<ffffffff810d635d>] rcu_check_callbacks+0xdd/0x190 [410287.192078] [<ffffffff8107bd08>] update_process_times+0x48/0x80 [410287.419162] [<ffffffff810e1c83>] tick_sched_handle.isra.14+0x33/0x70 [410287.419177] [<ffffffff810e1dac>] tick_sched_timer+0x4c/0x80 [410287.451605] [<ffffffff81094726>] __run_hrtimer+0x76/0x230 [410287.451626] [<ffffffff810e1d60>] ? tick_nohz_handler+0xa0/0xa0 [410287.451631] [<ffffffff81094eef>] hrtimer_interrupt+0xff/0x260 [410287.451756] [<ffffffff8104875b>] local_apic_timer_interrupt+0x3b/0x60 [410287.451774] [<ffffffff81786a95>] smp_apic_timer_interrupt+0x45/0x60 [410287.451777] [<ffffffff817853dd>] apic_timer_interrupt+0x6d/0x80 [410287.451783] [<ffffffff81071fa0>] ? __do_softirq+0x90/0x330 [410287.451786] [<ffffffff810721b6>] ? __do_softirq+0x2a6/0x330 [410287.451792] [<ffffffff810a5652>] ? __vtime_account_system+0x32/0x40 [410287.451796] [<ffffffff8107254e>] irq_exit+0x11e/0x140 [410287.451798] [<ffffffff81786a9a>] smp_apic_timer_interrupt+0x4a/0x60 [410287.451800] [<ffffffff817853dd>] apic_timer_interrupt+0x6d/0x80 [410287.451802] <EOI> [410287.451820] [<ffffffff8177afa0>] ? _raw_spin_lock+0x30/0x50 [410287.451833] [<ffffffff811ecfdf>] evict+0x6f/0x190 [410287.451837] [<ffffffff811ed8c0>] iput_final+0xe0/0x170 [410287.451839] [<ffffffff811ed98e>] iput+0x3e/0x50 [410287.451844] [<ffffffff811e7c88>] dentry_iput+0xa8/0xf0 [410287.451847] [<ffffffff811e8c96>] __dentry_kill+0xb6/0x160 [410287.451849] [<ffffffff811e92c1>] dput+0xb1/0x100 [410287.451852] [<ffffffff811e0ae8>] SYSC_renameat2+0x428/0x4c0 [410287.643918] [<ffffffff8111cdfc>] ? acct_account_cputime+0x1c/0x20 [410287.643943] [<ffffffff8111cdfc>] ? acct_account_cputime+0x1c/0x20 [410287.643948] [<ffffffff810a5469>] ? account_user_time+0x99/0xb0 [410287.643950] [<ffffffff810a5a9d>] ? vtime_account_user+0x5d/0x70 [410287.643972] [<ffffffff81021ee5>] ? syscall_trace_enter+0x165/0x280 [410287.643974] [<ffffffff810220dd>] ? syscall_trace_leave+0xdd/0x150 [410287.643978] [<ffffffff811e34ee>] SyS_renameat2+0xe/0x10 [410287.643980] [<ffffffff811e352e>] SyS_rename+0x1e/0x20 [410287.643985] [<ffffffff8178487f>] tracesys+0xe1/0xe6 [410287.643987] Code: 0f 1f 00 f3 90 8b 04 25 00 73 5f ff f6 c4 10 75 f2 c1 e1 18 89 0c 25 10 73 5f ff 83 fb 02 44 89 f0 0f 45 c3 89 04 25 00 73 5f ff <eb> 9e 4c 89 ff 57 9d 66 66 90 66 90 48 83 c4 18 5b 41 5c 41 5d [410287.644051] NMI backtrace for cpu 1 [410287.644063] CPU: 1 PID: 25014 Comm: atop Not tainted 3.15.1-031501-generic #201406161841 [410287.644064] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 01/15/2013 [410287.644067] task: ffff880028dd1930 ti: ffff880024768000 task.ti: ffff880024768000 [410287.644069] RIP: 0010:[<ffffffff8177afad>] [<ffffffff8177afad>] _raw_spin_lock+0x3d/0x50 [410287.644080] RSP: 0018:ffff88002476baf8 EFLAGS: 00000202 [410287.644081] RAX: 000000000000053b RBX: ffff8800b8f2e128 RCX: 0000000000005878 [410287.644083] RDX: 000000000000587a RSI: 000000000000587a RDI: ffffffff81c0b400 [410287.644084] RBP: ffff88002476baf8 R08: ffff88013fc37c30 R09: 0000000000000001 [410287.644085] R10: ffffc90000014000 R11: ffffc90000014000 R12: ffff880138454b90 [410287.644087] R13: ffffffff81828c60 R14: ffffffff81829278 R15: 0000000000000006 [410287.644091] FS: 00007f6d6c179740(0000) GS:ffff88013fc20000(0000) knlGS:0000000000000000 [410287.644093] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [410287.644094] CR2: 00007f6b980c8000 CR3: 000000002474c000 CR4: 00000000000006e0 [410287.644103] Stack: [410287.644105] ffff88002476bb18 ffffffff811ebc0d ffff88001b6b46c0 ffff8800b8f2e128 [410287.644108] ffff88002476bb38 ffffffff811ee22d ffffffff811ea6e4 ffff88001b6b46c0 [410287.644110] ffff88002476bb68 ffffffff8123c455 ffff88001b6b46c0 ffffffff81828c60 [410287.644112] Call Trace: [410287.644120] [<ffffffff811ebc0d>] inode_sb_list_add+0x1d/0x70 [410287.644123] [<ffffffff811ee22d>] new_inode+0x2d/0x40 [410287.644126] [<ffffffff811ea6e4>] ? __d_alloc+0x34/0x180 [410287.644130] [<ffffffff8123c455>] proc_pid_make_inode+0x25/0x110 [410287.644132] [<ffffffff8123c634>] proc_pident_instantiate+0x24/0xd0 [410287.644134] [<ffffffff8123c77b>] proc_pident_lookup+0x9b/0xe0 [410287.644137] [<ffffffff8123c81a>] proc_tgid_base_lookup+0x1a/0x20 [410287.644141] [<ffffffff811dc03d>] lookup_real+0x1d/0x60 [410287.644143] [<ffffffff811e1578>] lookup_open+0xc8/0x1d0 [410287.644145] [<ffffffff811e1954>] do_last+0x2d4/0x7b0 [410287.644150] [<ffffffff812ef976>] ? security_file_alloc+0x16/0x20 [410287.644153] [<ffffffff811e1ef4>] path_openat+0xc4/0x4c0 [410287.644157] [<ffffffff81186e44>] ? tlb_flush_mmu_free+0x34/0x60 [410287.644159] [<ffffffff81187844>] ? tlb_flush_mmu+0x24/0x30 [410287.644162] [<ffffffff811e2c63>] do_filp_open+0x43/0xa0 [410287.644164] [<ffffffff811ef9cd>] ? __alloc_fd+0xcd/0x120 [410287.644168] [<ffffffff811d1742>] do_sys_open+0x132/0x220 [410287.644172] [<ffffffff810220dd>] ? syscall_trace_leave+0xdd/0x150 [410287.644174] [<ffffffff811d184e>] SyS_open+0x1e/0x20 [410287.644178] [<ffffffff8178487f>] tracesys+0xe1/0xe6 [410287.644179] Code: 89 c2 c1 ea 10 66 39 c2 75 02 5d c3 83 e2 fe 0f b7 f2 b8 00 80 00 00 eb 0c 0f 1f 44 00 00 f3 90 83 e8 01 74 0a 0f b7 0f 66 39 ca <75> f1 5d c3 66 66 66 90 66 66 90 eb da 66 0f 1f 44 00 00 66 66 [410287.644274] NMI backtrace for cpu 2 [410287.644296] CPU: 2 PID: 2654 Comm: psad Not tainted 3.15.1-031501-generic #201406161841 [410287.644298] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 01/15/2013 [410287.644301] task: ffff8800ba0cb260 ti: ffff8800b4774000 task.ti: ffff8800b4774000 [410287.644304] RIP: 0010:[<ffffffff8177afaa>] [<ffffffff8177afaa>] _raw_spin_lock+0x3a/0x50 [410287.644345] RSP: 0018:ffff8800b47778c8 EFLAGS: 00000202 [410287.644346] RAX: 0000000000006f6d RBX: ffff8800a8b89598 RCX: 0000000000005878 [410287.644348] RDX: 000000000000587c RSI: 000000000000587c RDI: ffffffff81c0b400 [410287.644349] RBP: ffff8800b47778c8 R08: ffff88013fc572c0 R09: 0000000000000000 [410287.644350] R10: ffff880026a375a0 R11: 0000000000000000 R12: ffff8800a8b89598 [410287.644352] R13: 0000000000000000 R14: 0000000000008000 R15: 0000000000000000 [410287.644357] FS: 00007fcc8b7e8740(0000) GS:ffff88013fc40000(0000) knlGS:0000000000000000 [410287.644359] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [410287.644360] CR2: 00007f3682ab7340 CR3: 00000000baed8000 CR4: 00000000000006e0 [410287.644372] Stack: [410287.644374] ffff8800b47778e8 ffffffff811ebc0d ffff8800a8b89400 ffff8800a8b89400 [410287.644380] ffff8800b4777918 ffffffffa006e9da ffff8801021c94d8 ffff880137e1f000 [410287.644382] ffff8801021c94d8 0000000000000000 ffff8800b47779a8 ffffffffa00aca4d [410287.644385] Call Trace: [410287.644409] [<ffffffff811ebc0d>] inode_sb_list_add+0x1d/0x70 [410288.583763] [<ffffffffa006e9da>] xfs_setup_inode+0x3a/0x330 [xfs] [410288.777295] [<ffffffffa00aca4d>] xfs_ialloc+0x4cd/0x6f0 [xfs] [410288.777358] [<ffffffffa00accda>] xfs_dir_ialloc+0x6a/0x280 [xfs] [410288.839088] [<ffffffffa00ab2bc>] ? xfs_ilock+0xdc/0x120 [xfs] [410288.839152] [<ffffffffa00ad440>] xfs_create+0x4d0/0x6c0 [xfs] [410288.839168] [<ffffffffa006de3b>] xfs_generic_create+0xdb/0x250 [xfs] [410288.839183] [<ffffffffa006dfe4>] xfs_vn_mknod+0x14/0x20 [xfs] [410288.839209] [<ffffffffa006e023>] xfs_vn_create+0x13/0x20 [xfs] [410288.839216] [<ffffffff811e143d>] vfs_create+0xed/0x160 [410288.839219] [<ffffffff811e15ee>] lookup_open+0x13e/0x1d0 [410288.839245] [<ffffffff811f2298>] ? __mnt_want_write+0x58/0x70 [410288.839247] [<ffffffff811e1954>] do_last+0x2d4/0x7b0 [410288.839249] [<ffffffff811e1ef4>] path_openat+0xc4/0x4c0 [410288.839270] [<ffffffffa00ab634>] ? xfs_iunlock+0x114/0x120 [xfs] [410288.839284] [<ffffffffa0063fd7>] ? xfs_file_aio_read+0x1f7/0x2f0 [xfs] [410288.839286] [<ffffffff811e2c63>] do_filp_open+0x43/0xa0 [410288.839289] [<ffffffff811ef9cd>] ? __alloc_fd+0xcd/0x120 [410288.839298] [<ffffffff811d1742>] do_sys_open+0x132/0x220 [410288.839310] [<ffffffff810220dd>] ? syscall_trace_leave+0xdd/0x150 [410288.839313] [<ffffffff811d184e>] SyS_open+0x1e/0x20 [410288.839319] [<ffffffff8178487f>] tracesys+0xe1/0xe6 [410288.839321] Code: 0f c1 07 89 c2 c1 ea 10 66 39 c2 75 02 5d c3 83 e2 fe 0f b7 f2 b8 00 80 00 00 eb 0c 0f 1f 44 00 00 f3 90 83 e8 01 74 0a 0f b7 0f <66> 39 ca 75 f1 5d c3 66 66 66 90 66 66 90 eb da 66 0f 1f 44 00 [410288.839403] NMI backtrace for cpu 0 [410288.839431] CPU: 0 PID: 26553 Comm: xfs_db Not tainted 3.15.1-031501-generic #201406161841 [410288.839433] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 01/15/2013 [410288.839435] task: ffff8800ba0ccb90 ti: ffff8800a8008000 task.ti: ffff8800a8008000 [410288.839438] RIP: 0010:[<ffffffff811fad4a>] [<ffffffff811fad4a>] wait_sb_inodes+0x7a/0x120 [410288.863791] RSP: 0018:ffff8800a800bd58 EFLAGS: 00010206 [410288.863794] RAX: ffff8800136bbea0 RBX: ffff8800136bbd98 RCX: ffff88013fc11998 [410288.863796] RDX: 000000000000001c RSI: ffff8800ba0ccb90 RDI: ffff8800136bc220 [410288.863797] RBP: ffff8800a800bd88 R08: ffff8800a8008000 R09: 0000000000000001 [410288.863836] R10: dffa88fd27410620 R11: 0000000000000002 R12: ffff8800360228a0 [410288.863839] R13: ffff8800136bbe20 R14: ffff8800136bc2e8 R15: ffff880011925998 [410288.864238] FS: 00007fa610643780(0000) GS:ffff88013fc00000(0000) knlGS:0000000000000000 [410288.864241] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [410288.864242] CR2: 00007fe95be69000 CR3: 0000000011a5d000 CR4: 00000000000006f0 [410288.864325] Stack: [410288.864327] ffff8800a800bd58 ffff880036022800 ffff8800a800bdd8 ffff8800a800bd98 [410288.864334] ffff88013ae6a080 ffff88013954cc00 ffff8800a800be18 ffffffff811fae93 [410288.864336] 7fffffffffffffff ffff880036022800 ffff8800abab7ca0 0000001000000001 [410288.864339] Call Trace: [410288.864351] [<ffffffff811fae93>] sync_inodes_sb+0xa3/0xd0 [410288.864354] [<ffffffff8120222b>] __sync_filesystem+0x1b/0x60 [410288.864357] [<ffffffff812022eb>] sync_filesystem+0x4b/0x70 [410288.877144] [<ffffffff8120d20d>] fsync_bdev+0x2d/0x60 [410289.052866] [<ffffffff8135837c>] blkdev_ioctl+0x18c/0x810 [410289.052885] [<ffffffff8120b070>] block_ioctl+0x40/0x50 [410289.052908] [<ffffffff811e4cf5>] do_vfs_ioctl+0x75/0x2c0 [410289.052920] [<ffffffff81021f2d>] ? syscall_trace_enter+0x1ad/0x280 [410289.052923] [<ffffffff810220dd>] ? syscall_trace_leave+0xdd/0x150 [410289.052926] [<ffffffff8120bd8e>] ? blkdev_fsync+0x3e/0x50 [410289.052928] [<ffffffff811e4fd1>] SyS_ioctl+0x91/0xb0 [410289.052947] [<ffffffff8178487f>] tracesys+0xe1/0xe6 [410289.052953] Code: 8d 00 00 00 0f 1f 44 00 00 4c 89 ef e8 c0 01 58 00 48 8b 83 08 01 00 00 49 39 c4 48 8d 98 f8 fe ff ff 74 6d 4c 8d ab 88 00 00 00 <4c> 8b 73 30 4c 89 ef e8 1a 02 58 00 f6 83 a0 00 00 00 38 75 c9 [410289.077524] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 2211.030 msecs [410473.734748] BUG: soft lockup - CPU#1 stuck for 67s! [postgres:2477] [410473.734775] BUG: soft lockup - CPU#2 stuck for 67s! [postgres:2400] [410473.734825] Modules linked in: btrfs raid6_pq xor ufs qnx4 hfsplus hfs minix ntfs msdos jfs cpuid tcp_htcp sch_codel sch_fq_codel xt_multiport xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6 ipt_REJECT xt_LOG xt_limit xt_tcpudp xt_addrtype nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp nf_conntrack iptable_filter ip_tables x_tables ppdev vmw_balloon irda coretemp serio_raw crc_ccitt parport_pc vmwgfx ttm drm vmw_vmci mac_hid shpchp i2c_piix4 lp parport xfs libcrc32c psmouse floppy vmw_pvscsi vmxnet3 [410473.734829] CPU: 2 PID: 2400 Comm: postgres Not tainted 3.15.1-031501-generic #201406161841 [410473.734829] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 01/15/2013 [410473.734831] task: ffff8800b8ae64c0 ti: ffff880138b04000 task.ti: ffff880138b04000 [410473.734839] RIP: 0010:[<ffffffff8177afa2>] [<ffffffff8177afa2>] _raw_spin_lock+0x32/0x50 [410473.734840] RSP: 0018:ffff880138b07c98 EFLAGS: 00000216 [410473.734841] RAX: 00000000000010dd RBX: ffffea0002bd71c0 RCX: 00000000000061ec [410473.734842] RDX: 00000000000061f2 RSI: 00000000000061f2 RDI: ffffffff81c0b400 [410473.734842] RBP: ffff880138b07c98 R08: 0000000000000000 R09: 0000000000000000 [410473.734843] R10: ffff8800baa7f000 R11: 0000000000000000 R12: ffff8800360f8380 [410473.734843] R13: 00007f520000003f R14: ffffffffa00af2e6 R15: ffff880138b07cd8 [410473.734846] FS: 00007f52d5e11780(0000) GS:ffff88013fc40000(0000) knlGS:0000000000000000 [410473.734847] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [410473.734847] CR2: 00007f52d5e30000 CR3: 0000000138aed000 CR4: 00000000000006e0 [410473.734855] Stack: [410473.734857] ffff880138b07cc8 ffffffff811ecfdf 0000000000000000 ffff8801057a7d98 [410473.734858] ffff8801057a7e20 ffffffffa00db720 ffff880138b07cf8 ffffffff811ed8c0 [410473.734860] 0000000000000000 ffff8801057a7d98 ffff8801057a7d98 ffff8800baa7f058 [410473.734860] Call Trace: [410473.734866] [<ffffffff811ecfdf>] evict+0x6f/0x190 [410473.742954] [<ffffffff811ed8c0>] iput_final+0xe0/0x170 [410473.742960] [<ffffffff811ed98e>] iput+0x3e/0x50 [410473.742965] [<ffffffff811e7c88>] dentry_iput+0xa8/0xf0 [410473.742968] [<ffffffff811e8c96>] __dentry_kill+0xb6/0x160 [410473.742971] [<ffffffff811e92c1>] dput+0xb1/0x100 [410473.742974] [<ffffffff811e0ae8>] SYSC_renameat2+0x428/0x4c0 [410473.743023] [<ffffffff8111cdfc>] ? acct_account_cputime+0x1c/0x20 [410473.743027] [<ffffffff8111cdfc>] ? acct_account_cputime+0x1c/0x20 [410473.743039] [<ffffffff810a5469>] ? account_user_time+0x99/0xb0 [410473.743041] [<ffffffff810a5a9d>] ? vtime_account_user+0x5d/0x70 [410473.743046] [<ffffffff81021ee5>] ? syscall_trace_enter+0x165/0x280 [410473.743059] [<ffffffff810d4063>] ? rcu_eqs_enter+0x73/0xa0 [410473.743062] [<ffffffff811e34ee>] SyS_renameat2+0xe/0x10 [410473.743064] [<ffffffff811e352e>] SyS_rename+0x1e/0x20 [410473.743069] [<ffffffff8178487f>] tracesys+0xe1/0xe6 [410473.743069] Code: 89 e5 b8 00 00 02 00 [410473.743077] Modules linked in: btrfs raid6_pq xor ufs qnx4 hfsplus hfs minix ntfs msdos jfs cpuid tcp_htcp sch_codel sch_fq_codel xt_multiport xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6 ipt_REJECT xt_LOG xt_limit xt_tcpudp xt_addrtype nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp nf_conntrack iptable_filter ip_tables x_tables ppdev vmw_balloon irda coretemp serio_raw crc_ccitt parport_pc vmwgfx ttm drm vmw_vmci mac_hid shpchp i2c_piix4 lp parport xfs libcrc32c psmouse floppy vmw_pvscsi vmxnet3 [410473.743127] CPU: 1 PID: 2477 Comm: postgres Not tainted 3.15.1-031501-generic #201406161841