Discovered this on one of the kernel.org machines - it's been happening semi-consistently on a pair of boxes. Seems like a double free somewhere and at that point the whole box falls over dead basically. ------------[ cut here ]------------ kernel BUG at mm/slub.c:2835! invalid opcode: 0000 [#1] SMP last sysfs file: /sys/kernel/mm/ksm/run CPU 1 Modules linked in: ocfs2 mptctl mptbase drbd lru_cache nfsd lockd nfs_acl auth_rpcgss sunrpc ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs cpufreq_ondemand powernow_k8 freq_table 8021q garp stp llc ip6t_REJECT nf_conntrack_ipv6 ip6table_filter ip6_tables ipv6 xfs exportfs tg3 hpwdt amd64_edac_mod i2c_amd756 i2c_core edac_core shpchp k8temp amd_rng edac_mce_amd microcode pata_acpi ata_generic cciss pata_amd [last unloaded: scsi_wait_scan] Pid: 1713, comm: snmpd Not tainted 2.6.34.7-56.fc13.x86_64 #1 /ProLiant DL385 G1 RIP: 0010:[<ffffffff811006d6>] [<ffffffff811006d6>] kfree+0x5e/0xcb RSP: 0018:ffff8801f6433df8 EFLAGS: 00010246 RAX: 0040000000000400 RBX: ffff8803ed0eb9b0 RCX: ffff8803e9c92340 RDX: ffffea0000000000 RSI: ffffea0003800000 RDI: ffff880100000002 RBP: ffff8801f6433e18 R08: ffff8803e9c92958 R09: 0000000000000000 R10: 0000000000000011 R11: 0000000000000246 R12: ffff880100000002 R13: ffffffff81125e27 R14: ffffffff8115dbdc R15: ffff8803dd19ea80 FS: 00007ff4a31917a0(0000) GS:ffff880207400000(0000) knlGS:00000000f76fa6d0 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007ff4a31b2000 CR3: 00000001f6455000 CR4: 00000000000006e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process snmpd (pid: 1713, threadinfo ffff8801f6432000, task ffff8801f5688000) Stack: ffff8801f6433e18 ffff8803ed0eb9b0 ffff8803f61ec480 ffff8803ed0eb9b0 <0> ffff8801f6433e48 ffffffff81125e27 ffff8801f6433e38 ffff8803dd19ea80 <0> ffff8803ed0eb9b0 ffff8803e9c92940 ffff8801f6433e78 ffffffff8115dc10 Call Trace: [<ffffffff81125e27>] seq_release_private+0x28/0x44 [<ffffffff8115dc10>] seq_release_net+0x34/0x3d [<ffffffff81155ada>] proc_reg_release+0xd3/0xf0 [<ffffffff8110efbb>] __fput+0x12a/0x1dc [<ffffffff8110f087>] fput+0x1a/0x1c [<ffffffff8110c0f7>] filp_close+0x68/0x72 [<ffffffff8110c19e>] sys_close+0x9d/0xd2 [<ffffffff81009c72>] system_call_fastpath+0x16/0x1b Code: ef ff 13 48 83 c3 08 48 83 3b 00 eb ec 49 83 fc 10 76 7d 4c 89 e7 e8 67 e4 ff ff 48 89 c6 48 8b 00 84 c0 78 14 66 a9 00 c0 75 04 <0f> 0b eb fe 48 89 f7 e8 66 36 fd ff eb 57 48 8b 4d 08 48 8b 7e RIP [<ffffffff811006d6>] kfree+0x5e/0xcb RSP <ffff8801f6433df8> ---[ end trace 1a4b1fd758dd1fdb ]--- general protection fault: 0000 [#2] SMP last sysfs file: /sys/devices/pci0000:00/0000:00:19.3/name CPU 1 Modules linked in: ocfs2 mptctl mptbase drbd lru_cache nfsd lockd nfs_acl auth_rpcgss sunrpc ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs cpufreq_ondemand powernow_k8 freq_table 8021q garp stp llc ip6t_REJECT nf_conntrack_ipv6 ip6table_filter ip6_tables ipv6 xfs exportfs tg3 hpwdt amd64_edac_mod i2c_amd756 i2c_core edac_core shpchp k8temp amd_rng edac_mce_amd microcode pata_acpi ata_generic cciss pata_amd [last unloaded: scsi_wait_scan] Pid: 16274, comm: snmpd Tainted: G D 2.6.34.7-56.fc13.x86_64 #1 /ProLiant DL385 G1 RIP: 0010:[<ffffffff8110136b>] [<ffffffff8110136b>] __kmalloc_track_caller+0xe3/0x14c RSP: 0018:ffff8800380dbcc8 EFLAGS: 00010006 RAX: 0000000000000000 RBX: 0003000000000000 RCX: 000000000000000b RDX: 0000000100000000 RSI: 00000000000006a9 RDI: ffffffff8177d901 RBP: ffff8800380dbd18 R08: ffff880207412570 R09: ffff8800380dbe88 R10: ffff8800380dbf28 R11: 0000000000000000 R12: ffffffff81a28520 R13: 000000000000000b R14: 00000000000000d0 R15: 00000000000000d0 FS: 00007f4afe3857a0(0000) GS:ffff880207400000(0000) knlGS:00000000f6d92b70 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f4afc38cd50 CR3: 00000000f1fe8000 CR4: 00000000000006e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process snmpd (pid: 16274, threadinfo ffff8800380da000, task ffff8801ccdcddc0) Stack: ffff8800380dbcf8 0000000000000003 ffffffff81117373 0000000000000246 <0> ffff8800380dbd18 ffff8803ed0178e0 ffff8803ed017840 ffff8801f0938a50 <0> ffff8801f09046c0 0000000000000000 ffff8800380dbd48 ffffffff810dd233 Call Trace: [<ffffffff81117373>] ? vfs_rename+0xb2/0x3e1 [<ffffffff810dd233>] kstrdup+0x31/0x49 [<ffffffff81117373>] vfs_rename+0xb2/0x3e1 [<ffffffff81116523>] ? __lookup_hash+0x55/0xf1 [<ffffffff811ce973>] ? security_inode_permission+0x21/0x23 [<ffffffff81118cfd>] sys_renameat+0x193/0x20c [<ffffffff810e4e5a>] ? handle_mm_fault+0x452/0x97b [<ffffffff81450195>] ? do_page_fault+0x28e/0x2bb [<ffffffff81118d91>] sys_rename+0x1b/0x1d [<ffffffff81009c72>] system_call_fastpath+0x16/0x1b Code: 90 66 90 48 89 45 c8 fa 66 66 90 66 66 90 65 4c 8b 04 25 90 e8 00 00 49 8b 04 24 49 01 c0 49 8b 18 48 85 db 74 0e 49 63 44 24 18 <48> 8b 04 03 49 89 00 eb 15 48 8b 4d c0 83 ca ff 44 89 fe 4c 89 RIP [<ffffffff8110136b>] __kmalloc_track_caller+0xe3/0x14c RSP <ffff8800380dbcc8> ---[ end trace 1a4b1fd758dd1fdc ]--- block drbd1: write: error=-95 s=1658s block drbd1: Method to ensure write ordering: flush block drbd1: local disk flush failed with status -95 block drbd1: Method to ensure write ordering: drain o2net: accepted connection from node demeter2.kernel.org (num 2) at 172.20.0.20:7777 ocfs2_dlm: Node 2 joins domain FC86A681BA714C7AA126836FFC1D4C8C ocfs2_dlm: Nodes in domain ("FC86A681BA714C7AA126836FFC1D4C8C"): 1 2 general protection fault: 0000 [#3] SMP last sysfs file: /sys/kernel/mm/ksm/run CPU 1 Modules linked in: ocfs2 mptctl mptbase drbd lru_cache nfsd lockd nfs_acl auth_rpcgss sunrpc ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs cpufreq_ondemand powernow_k8 freq_table 8021q garp stp llc ip6t_REJECT nf_conntrack_ipv6 ip6table_filter ip6_tables ipv6 xfs exportfs tg3 hpwdt amd64_edac_mod i2c_amd756 i2c_core edac_core shpchp k8temp amd_rng edac_mce_amd microcode pata_acpi ata_generic cciss pata_amd [last unloaded: scsi_wait_scan] Pid: 29724, comm: lsof Tainted: G D 2.6.34.7-56.fc13.x86_64 #1 /ProLiant DL385 G1 RIP: 0010:[<ffffffff81101615>] [<ffffffff81101615>] __kmalloc+0xeb/0x150 RSP: 0018:ffff8801c9cd5be8 EFLAGS: 00010006 RAX: 0000000000000000 RBX: 0003000000000000 RCX: ffffffff811264ff RDX: 0000000100000000 RSI: 00000000000006a9 RDI: ffffffff8177d901 RBP: ffff8801c9cd5c28 R08: ffff880207412570 R09: ffff8803ea9aed80 R10: ffff8803ce0767c0 R11: 0000000000000000 R12: ffffffff81a28520 R13: 0000000000000010 R14: 00000000000080d0 R15: 00000000000080d0 FS: 00007f364411c7a0(0000) GS:ffff880207400000(0000) knlGS:00000000f5990b70 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000004091c0 CR3: 000000019f488000 CR4: 00000000000006e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process lsof (pid: 29724, threadinfo ffff8801c9cd4000, task ffff8801cd6c8000) Stack: ffffffff811264ff 0000000000000246 ffffc900013ed638 ffff8803ee7dfc00 <0> ffffffff81666660 ffff8803ce0765b0 ffff8801f6bed1f8 ffff8803ee7dfc00 <0> ffff8801c9cd5c58 ffffffff811264ff ffff8801c9cd5c58 ffffffff81e074e0 Call Trace: [<ffffffff811264ff>] ? __seq_open_private+0x25/0x5f [<ffffffff811264ff>] __seq_open_private+0x25/0x5f [<ffffffff8115dcf9>] seq_open_net+0x65/0x8c [<ffffffff814217b6>] unix_seq_open+0x1a/0x1c [<ffffffff81155f64>] proc_reg_open+0xd7/0x163 [<ffffffff8115dbdc>] ? seq_release_net+0x0/0x3d [<ffffffff81155e8d>] ? proc_reg_open+0x0/0x163 [<ffffffff8110c4a1>] __dentry_open+0x173/0x2aa [<ffffffff811ce973>] ? security_inode_permission+0x21/0x23 [<ffffffff8110c6a7>] nameidata_to_filp+0x3f/0x50 [<ffffffff81117e21>] do_last+0x447/0x5b8 [<ffffffff81119868>] do_filp_open+0x217/0x5fe [<ffffffff81214e23>] ? might_fault+0x21/0x23 [<ffffffff811225ca>] ? alloc_fd+0x7b/0x124 [<ffffffff8110c236>] do_sys_open+0x63/0x10f [<ffffffff8110c315>] sys_open+0x20/0x22 [<ffffffff81009c72>] system_call_fastpath+0x16/0x1b Code: 90 66 90 48 89 45 c8 fa 66 66 90 66 66 90 65 4c 8b 04 25 90 e8 00 00 49 8b 04 24 49 01 c0 49 8b 18 48 85 db 74 0e 49 63 44 24 18 <48> 8b 04 03 49 89 00 eb 11 83 ca ff 44 89 fe 4c 89 e7 e8 16 eb RIP [<ffffffff81101615>] __kmalloc+0xeb/0x150 RSP <ffff8801c9cd5be8> ---[ end trace 1a4b1fd758dd1fdd ]---
*** This bug has been marked as a duplicate of bug 20702 ***