hm....surprisingly my firefox is all the time up and running....even to send this bug report. EXT4-fs (sda6): mounted filesystem with ordered data mode Adding 7911972k swap on /dev/sdb5. Priority:-1 extents:1 across:7911972k device: 'vcs4': device_add PM: Adding info for No Bus:vcs4 device: 'vcsa4': device_add PM: Adding info for No Bus:vcsa4 device: 'vcs3': device_add PM: Adding info for No Bus:vcs3 device: 'vcsa3': device_add PM: Adding info for No Bus:vcsa3 device: 'vcs5': device_add PM: Adding info for No Bus:vcs5 device: 'vcsa5': device_add PM: Adding info for No Bus:vcsa5 device: 'vcs2': device_add PM: Adding info for No Bus:vcs2 device: 'vcsa2': device_add PM: Adding info for No Bus:vcsa2 device: 'vcs6': device_add PM: Adding info for No Bus:vcs6 device: 'vcsa6': device_add PM: Adding info for No Bus:vcsa6 device: '0:18': device_add PM: Adding info for No Bus:0:18 firefox: page allocation failure. order:1, mode:0x4020 Pid: 8200, comm: firefox Not tainted 2.6.32-rc5 #1 Call Trace: <IRQ> [<ffffffff81131e74>] __alloc_pages_nodemask+0x640/0x6cc [<ffffffff8115f85a>] alloc_pages_current+0xc2/0xe1 [<ffffffff811672e1>] alloc_slab_page+0x2f/0x83 [<ffffffff811673b8>] new_slab+0x83/0x224 [<ffffffff81165a77>] ? deactivate_slab+0xec/0x170 [<ffffffff81167aa7>] __slab_alloc+0x1f1/0x3df [<ffffffff81d75630>] ? dev_alloc_skb+0x2e/0x58 [<ffffffff8116910f>] __kmalloc_node_track_caller+0xda/0x15f [<ffffffff81d7c696>] ? netif_rx+0xbf/0x149 [<ffffffff81d75630>] ? dev_alloc_skb+0x2e/0x58 [<ffffffff81113b7b>] ? time_hardirqs_off+0x2c/0x42 [<ffffffff81d748e6>] __alloc_skb+0x83/0x192 [<ffffffff810c9e31>] ? trace_hardirqs_off+0x20/0x36 [<ffffffff81d75630>] dev_alloc_skb+0x2e/0x58 [<ffffffff8188e21f>] nv_alloc_rx_optimized+0x5c/0x1b4 [<ffffffff81890c7c>] nv_nic_irq_optimized+0xb5/0x2a2 [<ffffffff810fc0e4>] handle_IRQ_event+0x68/0x158 [<ffffffff810fe51c>] handle_fasteoi_irq+0xa3/0xfa [<ffffffff81113b7b>] ? time_hardirqs_off+0x2c/0x42 [<ffffffff81041714>] handle_irq+0x9d/0xbd [<ffffffff8103da96>] ? exit_idle+0xd/0x67 [<ffffffff81040c6e>] do_IRQ+0x72/0xe8 [<ffffffff8103f3d3>] ret_from_intr+0x0/0xf <EOI> Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 CPU 1: hi: 0, btch: 1 usd: 0 Node 0 DMA32 per-cpu: CPU 0: hi: 186, btch: 31 usd: 40 CPU 1: hi: 186, btch: 31 usd: 72 ================================= irq event stamp: 1208194 hardirqs last enabled at (1208193): [<ffffffff810cb4b1>] debug_check_no_locks_freed+0x12d/0x153 hardirqs last disabled at (1208194): [<ffffffff81f99eb4>] _spin_lock_irqsave+0x2d/0x9f softirqs last enabled at (1207922): [<ffffffff81093924>] __do_softirq+0x1b7/0x1db softirqs last disabled at (1207893): [<ffffffff8103fbbc>] call_softirq+0x1c/0x28 [ INFO: inconsistent lock state ] 2.6.32-rc5 #1 --------------------------------- inconsistent {HARDIRQ-ON-W} -> {IN-HARDIRQ-W} usage. firefox/8200 [HC1[1]:SC0[0]:HE0:SE1] takes: (bdev_lock){?.+...}, at: [<ffffffff811aa1ef>] nr_blockdev_pages+0x2b/0x97 {HARDIRQ-ON-W} state was registered at: [<ffffffff810cc342>] __lock_acquire+0x3a7/0xdff [<ffffffff810cce69>] lock_acquire+0xcf/0x100 [<ffffffff81f99d15>] _spin_lock+0x45/0x8e [<ffffffff811aa1ef>] nr_blockdev_pages+0x2b/0x97 [<ffffffff8112fa00>] si_meminfo+0x4c/0x82 [<ffffffff82b4ac2e>] eventpoll_init+0x23/0x102 [<ffffffff810090b4>] do_one_initcall+0x82/0x1a6 [<ffffffff82b268d3>] kernel_init+0x1b7/0x223 [<ffffffff8103faba>] child_rip+0xa/0x20 irq event stamp: 399036574 hardirqs last enabled at (399036573): [<ffffffff8103ea2c>] sysret_check+0x27/0x62 hardirqs last disabled at (399036574): [<ffffffff8103e877>] save_args+0x67/0x70 softirqs last enabled at (399035768): [<ffffffff81093924>] __do_softirq+0x1b7/0x1db softirqs last disabled at (399035763): [<ffffffff8103fbbc>] call_softirq+0x1c/0x28 other info that might help us debug this: no locks held by firefox/8200. stack backtrace: Pid: 8200, comm: firefox Not tainted 2.6.32-rc5 #1 Call Trace: <IRQ> [<ffffffff810ca732>] valid_state+0x18f/0x1b6 [<ffffffff810cb75d>] ? check_usage_forwards+0x0/0xb1 [<ffffffff810ca87c>] mark_lock+0x123/0x25c [<ffffffff810cc2bb>] __lock_acquire+0x320/0xdff [<ffffffff810c9d70>] ? trace_hardirqs_off_caller+0x32/0xd3 [<ffffffff811aa1ef>] ? nr_blockdev_pages+0x2b/0x97 [<ffffffff8108a7f4>] ? release_console_sem+0x1c9/0x215 [<ffffffff811aa1ef>] ? nr_blockdev_pages+0x2b/0x97 [<ffffffff810cce69>] lock_acquire+0xcf/0x100 [<ffffffff811aa1ef>] ? nr_blockdev_pages+0x2b/0x97 [<ffffffff810c9e31>] ? trace_hardirqs_off+0x20/0x36 [<ffffffff81f99d15>] _spin_lock+0x45/0x8e [<ffffffff811aa1ef>] ? nr_blockdev_pages+0x2b/0x97 [<ffffffff81f96aa9>] ? printk+0x50/0x66 [<ffffffff811aa1ef>] nr_blockdev_pages+0x2b/0x97 [<ffffffff811428c0>] ? next_zone+0x40/0x56 [<ffffffff8112fb6a>] show_free_areas+0x134/0x7ff [<ffffffff810c9d70>] ? trace_hardirqs_off_caller+0x32/0xd3 [<ffffffff810c9e31>] ? trace_hardirqs_off+0x20/0x36 [<ffffffff81f99a97>] ? _spin_unlock_irqrestore+0x50/0x76 [<ffffffff8108a7f4>] ? release_console_sem+0x1c9/0x215 [<ffffffff81f96aa9>] ? printk+0x50/0x66 [<ffffffff814bb35f>] show_mem+0x37/0x1a8 [<ffffffff81f968fe>] ? dump_stack+0x86/0xa3 [<ffffffff81131e79>] __alloc_pages_nodemask+0x645/0x6cc [<ffffffff8115f85a>] alloc_pages_current+0xc2/0xe1 [<ffffffff811672e1>] alloc_slab_page+0x2f/0x83 [<ffffffff811673b8>] new_slab+0x83/0x224 [<ffffffff81165a77>] ? deactivate_slab+0xec/0x170 [<ffffffff81167aa7>] __slab_alloc+0x1f1/0x3df [<ffffffff81d75630>] ? dev_alloc_skb+0x2e/0x58 [<ffffffff8116910f>] __kmalloc_node_track_caller+0xda/0x15f [<ffffffff81d7c696>] ? netif_rx+0xbf/0x149 [<ffffffff81d75630>] ? dev_alloc_skb+0x2e/0x58 [<ffffffff81113b7b>] ? time_hardirqs_off+0x2c/0x42 [<ffffffff81d748e6>] __alloc_skb+0x83/0x192 [<ffffffff810c9e31>] ? trace_hardirqs_off+0x20/0x36 [<ffffffff81d75630>] dev_alloc_skb+0x2e/0x58 [<ffffffff8188e21f>] nv_alloc_rx_optimized+0x5c/0x1b4 [<ffffffff81890c7c>] nv_nic_irq_optimized+0xb5/0x2a2 [<ffffffff810fc0e4>] handle_IRQ_event+0x68/0x158 [<ffffffff810fe51c>] handle_fasteoi_irq+0xa3/0xfa [<ffffffff81113b7b>] ? time_hardirqs_off+0x2c/0x42 [<ffffffff81041714>] handle_irq+0x9d/0xbd [<ffffffff8103da96>] ? exit_idle+0xd/0x67 [<ffffffff81040c6e>] do_IRQ+0x72/0xe8 [<ffffffff8103f3d3>] ret_from_intr+0x0/0xf <EOI> active_anon:190767 inactive_anon:48690 isolated_anon:0 active_file:73804 inactive_file:156012 isolated_file:0 unevictable:0 dirty:591 writeback:0 unstable:0 buffer:135346 free:4687 slab_reclaimable:216458 slab_unreclaimable:34660 mapped:20234 shmem:1390 pagetables:6897 bounce:0 Node 0 DMA free:9588kB min:20kB low:24kB high:28kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:9012kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes lowmem_reserve[]: 0 2946 2946 2946 Node 0 DMA32 free:9160kB min:6932kB low:8664kB high:10396kB active_anon:763068kB inactive_anon:194760kB active_file:295216kB inactive_file:624048kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3017640kB mlocked:0kB dirty:2364kB writeback:0kB mapped:80936kB shmem:5560kB slab_reclaimable:865832kB slab_unreclaimable:138640kB kernel_stack:2352kB pagetables:27588kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:70 all_unreclaimable? no lowmem_reserve[]: 0 0 0 0 Node 0 DMA: 1*4kB 2*8kB 2*16kB 4*32kB 1*64kB 1*128kB 0*256kB 0*512kB 1*1024kB 0*2048kB 2*4096kB = 9588kB Node 0 DMA32: 2004*4kB 37*8kB 1*16kB 0*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 9160kB 231205 total pagecache pages 6 pages in swap cache Swap cache stats: add 64, delete 58, find 0/0 Free swap = 7911716kB Total swap = 7911972kB 778144 pages RAM 34266 pages reserved 237376 pages shared 557474 pages non-shared SLUB: Unable to allocate memory on node -1 (gfp=0x20) cache: kmalloc-4096, object size: 4096, buffer size: 4168, default order: 3, min order: 1 kmalloc-4096 debugging increased min order, use slub_debug=O to disable. node 0: slabs: 95, objs: 665, free: 0 firefox: page allocation failure. order:1, mode:0x4020 Pid: 8200, comm: firefox Not tainted 2.6.32-rc5 #1 Call Trace: <IRQ> [<ffffffff81131e74>] __alloc_pages_nodemask+0x640/0x6cc [<ffffffff81113b7b>] ? time_hardirqs_off+0x2c/0x42 [<ffffffff8115f85a>] alloc_pages_current+0xc2/0xe1 [<ffffffff811672e1>] alloc_slab_page+0x2f/0x83 [<ffffffff811673b8>] new_slab+0x83/0x224 [<ffffffff81165a77>] ? deactivate_slab+0xec/0x170 [<ffffffff81167aa7>] __slab_alloc+0x1f1/0x3df [<ffffffff81d75630>] ? dev_alloc_skb+0x2e/0x58 [<ffffffff8116910f>] __kmalloc_node_track_caller+0xda/0x15f [<ffffffff81d7c696>] ? netif_rx+0xbf/0x149 [<ffffffff81d75630>] ? dev_alloc_skb+0x2e/0x58 [<ffffffff81113b7b>] ? time_hardirqs_off+0x2c/0x42 [<ffffffff81d748e6>] __alloc_skb+0x83/0x192 [<ffffffff810c9e31>] ? trace_hardirqs_off+0x20/0x36 [<ffffffff81d75630>] dev_alloc_skb+0x2e/0x58 [<ffffffff8188e21f>] nv_alloc_rx_optimized+0x5c/0x1b4 [<ffffffff81890c7c>] nv_nic_irq_optimized+0xb5/0x2a2 [<ffffffff810fc0e4>] handle_IRQ_event+0x68/0x158 [<ffffffff810fe51c>] handle_fasteoi_irq+0xa3/0xfa [<ffffffff81113b7b>] ? time_hardirqs_off+0x2c/0x42 [<ffffffff81041714>] handle_irq+0x9d/0xbd [<ffffffff8103da96>] ? exit_idle+0xd/0x67 [<ffffffff81040c6e>] do_IRQ+0x72/0xe8 [<ffffffff8103f3d3>] ret_from_intr+0x0/0xf <EOI> Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 CPU 1: hi: 0, btch: 1 usd: 0 Node 0 DMA32 per-cpu: CPU 0: hi: 186, btch: 31 usd: 90 CPU 1: hi: 186, btch: 31 usd: 176 active_anon:210971 inactive_anon:52728 isolated_anon:32 active_file:78326 inactive_file:136152 isolated_file:0 unevictable:0 dirty:133 writeback:0 unstable:0 buffer:115789 free:4463 slab_reclaimable:208888 slab_unreclaimable:33576 mapped:19624 shmem:1646 pagetables:6797 bounce:0 Node 0 DMA free:9588kB min:20kB low:24kB high:28kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:9012kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes lowmem_reserve[]: 0 2946 2946 2946 Node 0 DMA32 free:8264kB min:6932kB low:8664kB high:10396kB active_anon:843884kB inactive_anon:210912kB active_file:313304kB inactive_file:544608kB unevictable:0kB isolated(anon):128kB isolated(file):0kB present:3017640kB mlocked:0kB dirty:532kB writeback:0kB mapped:78496kB shmem:6584kB slab_reclaimable:835552kB slab_unreclaimable:134304kB kernel_stack:2296kB pagetables:27188kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:64 all_unreclaimable? no lowmem_reserve[]: 0 0 0 0 Node 0 DMA: 1*4kB 2*8kB 2*16kB 4*32kB 1*64kB 1*128kB 0*256kB 0*512kB 1*1024kB 0*2048kB 2*4096kB = 9588kB Node 0 DMA32: 1852*4kB 1*8kB 1*16kB 0*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 8264kB 216138 total pagecache pages 31 pages in swap cache Swap cache stats: add 254, delete 223, find 0/1 Free swap = 7910984kB Total swap = 7911972kB 778144 pages RAM 34266 pages reserved 215694 pages shared 579175 pages non-shared SLUB: Unable to allocate memory on node -1 (gfp=0x20) cache: kmalloc-4096, object size: 4096, buffer size: 4168, default order: 3, min order: 1 kmalloc-4096 debugging increased min order, use slub_debug=O to disable. node 0: slabs: 95, objs: 665, free: 0
Created attachment 23626 [details] .config file used for compilation (from linus git tree).
Created attachment 23627 [details] full dmesg output
page allocation failure is fine (NIC driver, GFP_ATOMIC) lockdep trace is fixed by commit b76146ed1ae7d7acae1d51f9342e31d00c8d5a12 revert "mm: oom analysis: add buffer cache information to show_free_areas()"