Problem Description: My system hangs when it has no more free memory to allocate via malloc(). Steps to reproduce: I have made a program "a.c" : ------------------------------- #include <stdio.h> #include <string.h> #include <stdlib.h> int main(void) { void * ptr; int i=0; while(1) { ptr = malloc(1024); memset(ptr,0,1024); printf("i=%d\n", i++); } } ------------------------------- and a script "b.sh": ------------------------------- a.out & a.out & a.out & a.out & a.out & a.out & ------------------------------- I compile my a.c program # gcc a.c If I execute the script b.sh my linux system hangs. A memdump is displayed continuously in loop on the system console.
Takes about 5 minutes to run into OOM on a dual core opteron with 2GB of memory and a bucket of swap (2.6.24-rc2-ish) x86_64 kernel + userland (F8). Processes seem to be snoozing here: /* Take a nap, wait for some writeback to complete */ if (sc.nr_scanned && priority < DEF_PRIORITY - 2) congestion_wait(WRITE, HZ/10); will run a bit with: http://lkml.org/lkml/2007/9/27/311 reduces the wait for the OOM killer to about 3 minutes, obviously it still kills the wrong processes. We've consumed all memory, all swap, and are scanning like mad, triggering the oom killer left right and center. Not killing the obvious process is pretty sad but aside from that it looks to be doing the right thing.
Closing - we have a no overcommit mode for cases this isn't acceptable.