Bug 218484

Summary:	virtual memory exhausted in 6.6.16 on i586 after minmax patchset
Product:	Linux	Reporter:	Giuseppe Ghibò (ghibo)
Component:	Kernel	Assignee:	Virtual assignee for kernel bugs (linux-kernel)
Status:	RESOLVED CODE_FIX
Severity:	normal	CC:	regressions
Priority:	P3
Hardware:	i386
OS:	Linux
Kernel Version:		Subsystem:
Regression:	No	Bisected commit-id:

Description Giuseppe Ghibò 2024-02-12 10:36:13 UTC

After upgrading to version 6.6.16, the kernel compilation on a i586 arch (on a 32bit chroot in a 64bit host) fails with a message:

virtual memory exhausted: Cannot allocate memory

this happens even lowering the number of parallel compilation threads. On a x86_64 arch the same problem doesn't occur. It's not clear whether some weird recursion is triggered that exhausts the memory, but it seems that the problem is caused by the patchset 'minmax' added to the 6.6.16 version, in particular it seems caused by these patches:

- minmax-allow-min-max-clamp-if-the-arguments-have-the-same-signedness.patch
- minmax-fix-indentation-of-__cmp_once-and-__clamp_once.patch
- minmax-allow-comparisons-of-int-against-unsigned-char-short.patch
- minmax-relax-check-to-allow-comparison-between-unsigned-arguments-and-signed-constants.patch

Reverting those patches fixes the memory exhaustion problem during compilation.

Comment 1 The Linux kernel's regression tracker (Thorsten Leemhuis) 2024-02-12 13:30:38 UTC

(In reply to Giuseppe Ghibò from comment #0)

> Reverting those patches fixes the memory exhaustion problem during
> compilation.

The important thing to know would be: does the same problem occur with mainline (e.g. 6.8-rc4) as well. Could you test this please?

Comment 2 Giuseppe Ghibò 2024-02-12 15:35:55 UTC

From a quick test the same problem doesn't occur in 6.8-rc4.

Comment 3 The Linux kernel's regression tracker (Thorsten Leemhuis) 2024-02-12 17:22:34 UTC

Forwarded by mail:
https://lore.kernel.org/all/f9f89284-0f48-4971-ad8d-86938a82fafc@leemhuis.info/

Comment 4 The Linux kernel's regression tracker (Thorsten Leemhuis) 2024-02-13 15:02:16 UTC

(In reply to The Linux kernel's regression tracker (Thorsten Leemhuis) from comment #3)
> Forwarded by mail:
> https://lore.kernel.org/all/f9f89284-0f48-4971-ad8d-86938a82fafc@leemhuis.
> info/

Please check that thread: could you confirm that the problem happens when compiling drivers/media/pci/solo6x10/solo6x10-offsets.h? Or is it something else?

Comment 5 The Linux kernel's regression tracker (Thorsten Leemhuis) 2024-02-13 15:18:28 UTC

(In reply to The Linux kernel's regression tracker (Thorsten Leemhuis) from comment #4)
> Please check that thread: could you confirm [...]

Likely not needed anymore, but can't hurt if you have a minute.

Comment 6 Giuseppe Ghibò 2024-02-13 15:48:45 UTC

(In reply to The Linux kernel's regression tracker (Thorsten Leemhuis) from comment #4)
> (In reply to The Linux kernel's regression tracker (Thorsten Leemhuis) from
> comment #3)
> > Forwarded by mail:
> > https://lore.kernel.org/all/f9f89284-0f48-4971-ad8d-86938a82fafc@leemhuis.
> > info/
> 
> Please check that thread: could you confirm that the problem happens when
> compiling drivers/media/pci/solo6x10/solo6x10-offsets.h? Or is it something
> else?

Yes, sort of. I'll try to build with CONFIG_VIDEO_SOLO6X10 disabled and the minmax patchset applied to see whether there are other missed.

Comment 7 Giuseppe Ghibò 2024-02-13 17:13:44 UTC

With CONFIG_VIDEO_SOLO6X10 disabled it compiles smooth without memory exaustion errors.

Now trying with the patch:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/media/pci/solo6x10/solo6x10-offsets.h?id=31e97d7c9ae3de072d7b424b2cf706a03ec10720

Pretty weird that the max() used in solo6x10-offsets.h like max(min(...,...),...)  would cause that big problem; Though the min() or max() macro has always been fragile (because of being macros, at least when defined with the ternary operator, I wonder if we can extract a minimal test case.

Comment 8 Giuseppe Ghibò 2024-02-13 19:33:11 UTC

Confirming with the media-solo6x10-replace-max-a-min-b-c-by-clamp-b-a-c.patch the  above problem doesn't occur anymore.

Comment 9 Giuseppe Ghibò 2024-05-06 11:33:10 UTC

Closing this, as a fix was included in 6.6.17.