Bug 19332

Summary:	remove editorializing from malloc man page
Product:	Documentation	Reporter:	landijk-user
Component:	man-pages	Assignee:	Michael Kerrisk (mtk.manpages)
Status:	RESOLVED CODE_FIX
Severity:	normal	CC:	mtk.manpages, pasky, vapier
Priority:	P1
Hardware:	All
OS:	Linux
Kernel Version:		Subsystem:
Regression:	No	Bisected commit-id:
Bug Depends on:	19382
Bug Blocks:
Attachments:	clean up paragraph, point to proc(5)

Description landijk-user 2010-09-30 00:52:21 UTC

The man page for malloc in release 3.23 of man-pages has this paragraph in a section titled "BUGS":

By default, Linux follows an optimistic memory allocation strategy.
This means that when malloc() returns non-NULL there is no guarantee
that the memory really is available. This is a really bad bug. In
case it turns out that the system is out of memory, one or more pro‐
cesses will be killed by the infamous OOM killer. In case Linux is
employed under circumstances where it would be less desirable to sud‐
denly lose some randomly picked processes, and moreover the kernel ver‐
sion is sufficiently recent, one can switch off this overcommitting
behavior using a command like:

# echo 2 > /proc/sys/vm/overcommit_memory

See also the kernel Documentation directory, files vm/overcommit-
accounting and sysctl/vm.txt.

This paragraph has a polemical tone overall. Rather than explain the issues, the paragraph simply advocates a position against overcommit. Note in particular the following issues:

1. "This is a really bad bug." -- Overcommit may be a bug in the sense of nonconformance to a standard, but it is not made clear exactly which standard. Clearly it is not a bug in the sense of unintentional behavior. Whether or not overcommit is "really bad" depends on facts which the paragraph does not present.

2. "...the infamous OOM killer." -- The word "infamous" here serves only as advocacy, and does not help developers understand the issues involved.

3. "...less desirable to suddenly lose some randomly picked processes" -- It is not true that the OOM killer selects processes randomly.

4. "one can switch off this overcommitting behavior using a command..." -- You need to know about more than just overcommit_memory if you want to precisely control overcommit. You also need to know about the overcommit ratio, as well as panic_on_oom. The manual for malloc is not the right place for a proper treatment of tuning kernel memory management.

I do not wish to engage in a flamewar over overcommit. I am only trying to improve the documentation, which better serves developers when it sticks to the facts. I suggest changing the content to the following:

By default, Linux follows an optimistic memory allocation strategy which can overcommit the available memory. The overcommit strategy maximizes the use of available memory, with the tradeoff that when malloc() returns non-NULL there is no guarantee the memory really is available. In case it turns out that the system is out of memory, one or more processes will be killed by the OOM killer. The overcommit behavior is configurable, and processes may be protected from the OOM killer as needed. For more information, see the kernel documentation files vm/overcommit-accounting and sysctl/vm.txt, and the Web site http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html/Deployment_Guide/s2-proc-pid.html.

In the paragraph I claim that "processes may be protected from the OOM killer," and what I am referring to is oom_adj/oom_score. However, I looked in the overcommit-accounting and sysctl/vm pages, and didn't see anything about oom_adj or oom_score.

It appears there is no documentation about oom_adj, based on the below bug filed against Red Hat.

https://bugzilla.redhat.com/show_bug.cgi?id=239313

Red Hat resolved that bug by writing some documentation for their RHEL distribution. Hence I added a link to Red Hat's documentation at the end of the paragraph. I don't know if it is appropriate to link to distribution-specific docs in a man page, but I see no other available documentation, and the information in question seems to be applicable to Linux in general.

Comment 1 Petr Baudis 2010-09-30 08:22:13 UTC

I agree that the paragraph should be rewritten; it always strucks me as really opinionated one as well every time I stumble upon it. Moreover, I'd move it from section BUGS to section NOTES.

I do not think it is wise to remove the information on how to turn it off, though - I see no harm in keeping the extra information. There are already references to more documentation in the manual page; if there is anything extra in the web page referenced, the information should be added to the kernel documentation, I think. I think it is wise to avoid web references in general whenever possible since most URLs actually tend to be quite volatile, and vendor-specific documentation specifically to avoid any bias, informational or otherwise.

Comment 2 landijk-user 2010-09-30 18:36:01 UTC

With regard to removing the instructions on how to disable overcommit, I thought it would be better to simply direct people to the proper documentation, so they can understand all the options.  But I'm not an expert in this area, so I would be glad to put the instructions back in if that's the right thing.

With regard to the link to Red Hat, I think there is no kernel documentation on oom_adj and oom_score.  If the manual page for malloc is going to discuss the OOM killer, it ought to at least point to some documentation about how to tune its behavior.

So I created a new bug for that, and now this bug depends on that one.

Comment 3 Mike Frysinger 2010-09-30 19:01:48 UTC

i'd agree with the linking to a diff man page.  proc(5) already has a section on overcommit_memory.  perhaps that should be extended with the stuff from malloc(3) and then malloc(3) simply makes an explicit note to also read about overcommit_memory in proc(5).

Comment 4 landijk-user 2010-09-30 20:10:11 UTC

Created attachment 32102 [details]
clean up paragraph, point to proc(5)

Indeed the proc man page has plenty of information about overcommit, and seems to to be the right place to treat the subject.  This patch:

* removes inflammatory language
* moves the paragraph to the beginning of the notes section
* points the reader to the overcommit_memory and oom_adj sections of proc(5)

It was tested using "man -l" with man version 2.5.7.

Comment 5 Michael Kerrisk 2010-10-03 15:26:20 UTC

Landijk,

That text got in the page before I became maintainer. The only reason it's still there is that I didn't notice it. I agree the tone is entirely wrong, and I accepted your patch pretty much as you gave it, keeping a reference to one of the kernel source files (Documentation/vm/overcommit-accounting). The fix will be in man-pages-3.28.