Hi, Since upgrading from a Q9450 to 2xE5520 (and upgrading from 2.6.34-rc-something to 2.6.34), I've started seeing these: [605882.372418] swapper: page allocation failure. order:2, mode:0x4020 [605882.378981] Pid: 0, comm: swapper Not tainted 2.6.34 #1 [605882.384617] Call Trace: [605882.387499] <IRQ> [<ffffffff81096d5a>] __alloc_pages_nodemask+0x5b0/0x629 [605882.395068] [<ffffffff81096de5>] __get_free_pages+0x12/0x4f [605882.401103] [<ffffffff810bdeb4>] __kmalloc_track_caller+0x4c/0x156 [605882.407817] [<ffffffff81245986>] ? sock_alloc_send_pskb+0xdd/0x32d [605882.414556] [<ffffffff8124a515>] __alloc_skb+0x66/0x15b [605882.420331] [<ffffffff81245986>] sock_alloc_send_pskb+0xdd/0x32d [605882.426879] [<ffffffff81245be6>] sock_alloc_send_skb+0x10/0x12 [605882.433201] [<ffffffff812d835e>] mld_newpack+0x6d/0x15a [605882.438861] [<ffffffff812d85bd>] add_grhead+0x2e/0x8b [605882.444357] [<ffffffff812da254>] add_grec+0x359/0x38b [605882.449885] [<ffffffff8105aadc>] ? tick_sched_timer+0x0/0x9a [605882.456002] [<ffffffff81056851>] ? ktime_get+0x68/0xc4 [605882.461664] [<ffffffff812da2e7>] mld_send_report+0x61/0xca [605882.467684] [<ffffffff812da367>] mld_gq_timer_expire+0x17/0x29 [605882.474134] [<ffffffff81042a9f>] run_timer_softirq+0x1d4/0x270 [605882.480523] [<ffffffff8105969d>] ? clockevents_program_event+0x73/0x7c [605882.487547] [<ffffffff8105a67d>] ? tick_dev_program_event+0x37/0xf9 [605882.494341] [<ffffffff812da350>] ? mld_gq_timer_expire+0x0/0x29 [605882.500751] [<ffffffff8103be87>] __do_softirq+0xd8/0x195 [605882.506555] [<ffffffff81002e0c>] call_softirq+0x1c/0x28 [605882.512263] [<ffffffff810043e7>] do_softirq+0x33/0x68 [605882.517871] [<ffffffff8103bd0d>] irq_exit+0x36/0x75 [605882.523295] [<ffffffff81003ae5>] do_IRQ+0xaa/0xc1 [605882.528566] [<ffffffff81309353>] ret_from_intr+0x0/0xa [605882.534275] <EOI> [<ffffffff811abd5e>] ? acpi_idle_enter_bm+0x298/0x2c6 [605882.541554] [<ffffffff811abd54>] ? acpi_idle_enter_bm+0x28e/0x2c6 [605882.548107] [<ffffffff81236dfd>] cpuidle_idle_call+0x95/0xf0 [605882.554268] [<ffffffff8100138f>] cpu_idle+0x55/0x8d [605882.559771] [<ffffffff812f7720>] rest_init+0x74/0x76 [605882.565188] [<ffffffff8155bcaf>] start_kernel+0x35d/0x368 [605882.571023] [<ffffffff8155b2a1>] x86_64_start_reservations+0xb1/0xb5 [605882.577840] [<ffffffff8155b386>] x86_64_start_kernel+0xe1/0xe8 [605882.584125] Mem-Info: [605882.586775] DMA per-cpu: [605882.589670] CPU 0: hi: 0, btch: 1 usd: 0 [605882.594895] CPU 1: hi: 0, btch: 1 usd: 0 [605882.600092] CPU 2: hi: 0, btch: 1 usd: 0 [605882.605351] CPU 3: hi: 0, btch: 1 usd: 0 [605882.610547] CPU 4: hi: 0, btch: 1 usd: 0 [605882.615775] CPU 5: hi: 0, btch: 1 usd: 0 [605882.621125] CPU 6: hi: 0, btch: 1 usd: 0 [605882.626285] CPU 7: hi: 0, btch: 1 usd: 0 [605882.631436] CPU 8: hi: 0, btch: 1 usd: 0 [605882.636579] CPU 9: hi: 0, btch: 1 usd: 0 [605882.641758] CPU 10: hi: 0, btch: 1 usd: 0 [605882.652045] CPU 11: hi: 0, btch: 1 usd: 0 [605882.657201] CPU 12: hi: 0, btch: 1 usd: 0 [605882.662348] CPU 13: hi: 0, btch: 1 usd: 0 [605882.667544] CPU 14: hi: 0, btch: 1 usd: 0 [605882.672687] CPU 15: hi: 0, btch: 1 usd: 0 [605882.677832] DMA32 per-cpu: [605882.680942] CPU 0: hi: 186, btch: 31 usd: 33 [605882.686087] CPU 1: hi: 186, btch: 31 usd: 180 [605882.691249] CPU 2: hi: 186, btch: 31 usd: 38 [605882.696395] CPU 3: hi: 186, btch: 31 usd: 166 [605882.701529] CPU 4: hi: 186, btch: 31 usd: 48 [605882.706676] CPU 5: hi: 186, btch: 31 usd: 176 [605882.711827] CPU 6: hi: 186, btch: 31 usd: 2 [605882.716981] CPU 7: hi: 186, btch: 31 usd: 48 [605882.722199] CPU 8: hi: 186, btch: 31 usd: 21 [605882.727346] CPU 9: hi: 186, btch: 31 usd: 170 [605882.732520] CPU 10: hi: 186, btch: 31 usd: 40 [605882.737681] CPU 11: hi: 186, btch: 31 usd: 1 [605882.742909] CPU 12: hi: 186, btch: 31 usd: 181 [605882.748085] CPU 13: hi: 186, btch: 31 usd: 120 [605882.753255] CPU 14: hi: 186, btch: 31 usd: 45 [605882.758418] CPU 15: hi: 186, btch: 31 usd: 53 [605882.763563] Normal per-cpu: [605882.766719] CPU 0: hi: 186, btch: 31 usd: 175 [605882.771860] CPU 1: hi: 186, btch: 31 usd: 171 [605882.777083] CPU 2: hi: 186, btch: 31 usd: 199 [605882.782287] CPU 3: hi: 186, btch: 31 usd: 166 [605882.787461] CPU 4: hi: 186, btch: 31 usd: 184 [605882.792639] CPU 5: hi: 186, btch: 31 usd: 178 [605882.797789] CPU 6: hi: 186, btch: 31 usd: 158 [605882.626285] CPU 7: hi: 0, btch: 1 usd: 0 [605882.631436] CPU 8: hi: 0, btch: 1 usd: 0 [605882.636579] CPU 9: hi: 0, btch: 1 usd: 0 [605882.641758] CPU 10: hi: 0, btch: 1 usd: 0 [605882.652045] CPU 11: hi: 0, btch: 1 usd: 0 [605882.657201] CPU 12: hi: 0, btch: 1 usd: 0 [605882.662348] CPU 13: hi: 0, btch: 1 usd: 0 [605882.667544] CPU 14: hi: 0, btch: 1 usd: 0 [605882.672687] CPU 15: hi: 0, btch: 1 usd: 0 [605882.677832] DMA32 per-cpu: [605882.680942] CPU 0: hi: 186, btch: 31 usd: 33 [605882.686087] CPU 1: hi: 186, btch: 31 usd: 180 [605882.691249] CPU 2: hi: 186, btch: 31 usd: 38 [605882.696395] CPU 3: hi: 186, btch: 31 usd: 166 [605882.701529] CPU 4: hi: 186, btch: 31 usd: 48 [605882.706676] CPU 5: hi: 186, btch: 31 usd: 176 [605882.711827] CPU 6: hi: 186, btch: 31 usd: 2 [605882.716981] CPU 7: hi: 186, btch: 31 usd: 48 [605882.722199] CPU 8: hi: 186, btch: 31 usd: 21 [605882.727346] CPU 9: hi: 186, btch: 31 usd: 170 [605882.732520] CPU 10: hi: 186, btch: 31 usd: 40 [605882.737681] CPU 11: hi: 186, btch: 31 usd: 1 [605882.742909] CPU 12: hi: 186, btch: 31 usd: 181 [605882.748085] CPU 13: hi: 186, btch: 31 usd: 120 [605882.753255] CPU 14: hi: 186, btch: 31 usd: 45 [605882.758418] CPU 15: hi: 186, btch: 31 usd: 53 [605882.763563] Normal per-cpu: [605882.766719] CPU 0: hi: 186, btch: 31 usd: 175 [605882.771860] CPU 1: hi: 186, btch: 31 usd: 171 [605882.777083] CPU 2: hi: 186, btch: 31 usd: 199 [605882.782287] CPU 3: hi: 186, btch: 31 usd: 166 [605882.787461] CPU 4: hi: 186, btch: 31 usd: 184 [605882.792639] CPU 5: hi: 186, btch: 31 usd: 178 [605882.797789] CPU 6: hi: 186, btch: 31 usd: 158 [605882.802953] CPU 7: hi: 186, btch: 31 usd: 183 [605882.808219] CPU 8: hi: 186, btch: 31 usd: 153 [605882.813366] CPU 9: hi: 186, btch: 31 usd: 165 [605882.818587] CPU 10: hi: 186, btch: 31 usd: 179 [605882.823785] CPU 11: hi: 186, btch: 31 usd: 178 [605882.828986] CPU 12: hi: 186, btch: 31 usd: 168 [605882.834135] CPU 13: hi: 186, btch: 31 usd: 162 [605882.839295] CPU 14: hi: 186, btch: 31 usd: 164 [605882.844466] CPU 15: hi: 186, btch: 31 usd: 173 [605882.849615] active_anon:215773 inactive_anon:148687 isolated_anon:0 [605882.849616] active_file:814152 inactive_file:4543587 isolated_file:0 [605882.849617] unevictable:0 dirty:71 writeback:0 unstable:0 [605882.849618] free:90988 slab_reclaimable:321713 slab_unreclaimable:14548 [605882.849619] mapped:14259 shmem:8188 pagetables:5758 bounce:0 [605882.881952] DMA free:15748kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15680kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes [605882.919426] lowmem_reserve[]: 0 2991 24201 24201 [605882.924822] DMA32 free:325328kB min:2460kB low:3072kB high:3688kB active_anon:50288kB inactive_anon:117072kB active_file:168692kB inactive_file:1578832kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:88kB writeback:0kB mapped:2896kB shmem:792kB slab_reclaimable:395096kB slab_unreclaimable:3980kB kernel_stack:320kB pagetables:2012kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [605882.966459] lowmem_reserve[]: 0 0 21210 21210 [605882.971431] Normal free:21760kB min:17440kB low:21800kB high:26160kB active_anon:812804kB inactive_anon:477676kB active_file:3087916kB inactive_file:16596356kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:21719040kB mlocked:0kB dirty:208kB writeback:0kB mapped:54140kB shmem:31960kB slab_reclaimable:891756kB slab_unreclaimable:54212kB kernel_stack:5608kB pagetables:21020kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [605883.014280] lowmem_reserve[]: 0 0 0 0 [605883.018559] DMA: 1*4kB 0*8kB 0*16kB 2*32kB 1*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15748kB [605883.030108] DMA32: 59734*4kB 10689*8kB 17*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 324848kB [605883.042657] Normal: 5152*4kB 0*8kB 0*16kB 2*32kB 1*64kB 0*128kB 0*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 21760kB [605883.054667] 5424252 total pagecache pages [605883.059194] 57991 pages in swap cache [605883.063305] Swap cache stats: add 2686125, delete 2628134, find 13200563/13296693 [605883.071396] Free swap = 17601928kB [605883.075290] Total swap = 18000504kB [605883.159424] 6291440 pages RAM [605883.162810] 119334 pages reserved [605883.166472] 869820 pages shared [605883.169976] 5269316 pages non-shared I wonder if this might be related to my use of huge pages -- at times I've turned the number up to 4096 2M pages (8GB RAM, out of my total 24GB), and the machine has been pretty sluggish just afterwards. I shouldn't have too much memory pressure in general, though. This is what free -m currently looks like: pannekake:~> free -m total used free shared buffers cached Mem: 24109 23981 128 0 2325 19920 -/+ buffers/cache: 1736 22373 Swap: 17578 235 17343 and it's a pretty typical situation for the machine.