Bug 104111

Summary: If CONFIG_NO_HZ_FULL_ALL=y, all user space processes have 0x1 affinity since 4.2.0
Product: Process Management Reporter: Petr Pisar (petr.pisar)
Component: SchedulerAssignee: Ingo Molnar (mingo)
Status: NEW ---    
Severity: normal CC: ansla80, epopevad, eugene.shalygin, fweisbec, kernel, keruspe+bugs-kernel, szg00000
Priority: P1    
Hardware: x86-64   
OS: Linux   
Kernel Version: 4.2.0 Subsystem:
Regression: Yes Bisected commit-id:
Attachments: 4.2.0 kernel log
4.2.0 .config
failing config
Working config

Description Petr Pisar 2015-09-06 18:29:14 UTC
Created attachment 186911 [details]
4.2.0 kernel log

I have a 4-core AMD A10-5700 CPU. I noticed a regression between 4.1.6 and 4.2.0 kernel:

All user space processes stick to first core no matter of CPU utilization. (Kernel threads use more CPUs.) "taskset -p $$" shows "1". If I set affinity to more CPUs, processes do not move. If I exclude currently occupied CPU from the affinity, the process moves to the lowest allowed CPU number.

If I reboot to 4.1.6, "taskset -p $$" shows "f" and processes are distributed evenly across all CPUs.

This is a difference of kernel configurations as created more or less by "make oldconfig":

--- 4.1.6	2015-09-06 19:00:51.240979625 +0200
+++ 4.2.0	2015-09-06 19:00:59.123486618 +0200
-CONFIG_X86_HT=y
-CONFIG_GENERIC_IRQ_LEGACY_ALLOC_HWIRQ=y
+CONFIG_IRQ_DOMAIN_HIERARCHY=y
+CONFIG_GENERIC_MSI_IRQ_DOMAIN=y
-CONFIG_RCU_FANOUT=64
-CONFIG_RCU_FANOUT_LEAF=16
-CONFIG_RCU_KTHREAD_PRIO=0
+CONFIG_ARCH_WANTS_DYNAMIC_TASK_STRUCT=y
+CONFIG_HAVE_COPY_THREAD_TLS=y
+CONFIG_MODULES_TREE_LOOKUP=y
-CONFIG_ARCH_USE_QUEUE_RWLOCK=y
-CONFIG_QUEUE_RWLOCK=y
+CONFIG_ARCH_USE_QUEUED_SPINLOCKS=y
+CONFIG_QUEUED_SPINLOCKS=y
+CONFIG_ARCH_USE_QUEUED_RWLOCKS=y
+CONFIG_QUEUED_RWLOCKS=y
+CONFIG_ARCH_SUPPORTS_DEFERRED_STRUCT_PAGE_INIT=y
+CONFIG_ACPI_REV_OVERRIDE_POSSIBLE=y
+CONFIG_PCI_MSI_IRQ_DOMAIN=y
+CONFIG_MAC80211_STA_HASH_MAX_SIZE=0
+CONFIG_INPUT_LEDS=m
+CONFIG_SND_JACK=y
+CONFIG_SND_PROC_FS=y
-CONFIG_SND_KCTL_JACK=y
+CONFIG_EDAC_ATOMIC_SCRUB=y
+CONFIG_EDAC_SUPPORT=y
+CONFIG_EFI_ESRT=y
-CONFIG_KASAN_SHADOW_OFFSET=0xdffffc0000000000
+CONFIG_SCHED_INFO=y
+CONFIG_CRYPTO_RNG_DEFAULT=m
+CONFIG_CRYPTO_AKCIPHER2=y
+CONFIG_CRYPTO_NULL=m
+CONFIG_CRYPTO_ECHAINIV=m
+CONFIG_CRYPTO_DRBG_MENU=m
+CONFIG_CRYPTO_DRBG_HMAC=y
+CONFIG_CRYPTO_DRBG=m
+CONFIG_CRYPTO_JITTERENTROPY=m
+CONFIG_ARCH_HAS_PMEM_API=y
Comment 1 Petr Pisar 2015-09-06 18:30:24 UTC
Created attachment 186921 [details]
4.2.0 .config
Comment 2 Marc-Antoine Perennou 2015-09-07 08:16:01 UTC
*** Bug 104151 has been marked as a duplicate of this bug. ***
Comment 3 Eugene Shalygin 2015-09-07 10:27:04 UTC
I also have the same problem. Had to make defconfig for 4.2.0 and then tune options manually to get a working kernel.
Comment 4 Eugene Shalygin 2015-09-07 10:28:05 UTC
Created attachment 186971 [details]
failing config

The buggy config created via make oldconfig from 4.1.6
Comment 5 Eugene Shalygin 2015-09-07 10:29:00 UTC
Created attachment 186981 [details]
Working config

The working one, tuned after make defconfig
Comment 6 Marc-Antoine Perennou 2015-09-07 17:49:42 UTC
Ok, After some digging, I found the exact config option causing it.

I'll have a look tomorrow at fixing it.

Here is the .config diff which makes it work:

diff --git a/.config b/.config
index 4391737..87eea22 100644
--- a/.config
+++ b/.config
@@ -106,7 +106,7 @@ CONFIG_NO_HZ_COMMON=y
 # CONFIG_HZ_PERIODIC is not set
 # CONFIG_NO_HZ_IDLE is not set
 CONFIG_NO_HZ_FULL=y
-CONFIG_NO_HZ_FULL_ALL=y
+# CONFIG_NO_HZ_FULL_ALL is not set
 CONFIG_NO_HZ_FULL_SYSIDLE=y
 CONFIG_NO_HZ_FULL_SYSIDLE_SMALL=8
 # CONFIG_NO_HZ is not set
Comment 7 Petr Pisar 2015-09-19 10:58:19 UTC
I can confirm that unsetting the CONFIG_NO_HZ_FULL_ALL makes my scheduler more usable.
Comment 8 Frederic Weisbecker 2015-09-24 12:21:01 UTC
It's expected behaviour since we include all nohz full CPUs to cpu_isolated_map. But we are starting to think that we should change that behaviour...