# cat /proc/cpuinfo | grep model model : 63 model name : Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz ... # cat /sys/devices/system/cpu/cpu*/cpufreq/cpuinfo_cur_freq <unknown><unknown><unknown><unknown><unknown><unknown><unknown><unknown><unknown><unknown><unknown><unknown> # cat /proc/[pid]/stat will also show '0' for utime, stime, cutime, and cstime Certain utilities like 'top' and 'ps' will show '0%' for %CPU for a given process, but not for a given CPU. I've upgraded my kernel a few times in the past several weeks, and I noticed this issue before, although I'm not sure the exact kernel version I started noticing this. This same behavior has been reported for an i7 5930k (also 6-core Haswell-E). Performance degradation that I began noticing around the same time may be related.
What's in /sys/devices/system/cpu/cpu*/cpufreq/scaling_driver ?
And can you try 4.8, please?
4.8 has the same results/symptoms. # cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_driver intel_pstate intel_pstate intel_pstate intel_pstate intel_pstate intel_pstate intel_pstate intel_pstate intel_pstate intel_pstate intel_pstate intel_pstate
Apparently, intel_pstate_get() returns 0 on your system for some reason. Please run "turbostat --debug" (say for around 30s) on it and attach the output.
Created attachment 240901 [details] turbostat --debug output
Looks like the schedule util tick is not getting called back. Since you have issue with all time keeping issues for process stats also, I suspect something to do with tick. Can you run some load and attach same turbostat: # turbostat --debug --msr=0x199 Do you have some special kernel config or running a distro config? Attach .config also. Kernel command line: What is cat /proc/cmdline?
Also attach output of: # perf record -a --event=power:pstate_sample sleep 300
# cat /proc/cmdline initrd=\initramfs-linux.img root=/dev/sda3 rw fbcon=rotate:3 I'm just using the vanilla 4.7 linux kernel from the archlinux [core] repo. For the 4.8 test, I used the vanilla 4.8 kernel from the archlinux [testing] repo. Attaching the requested output after this comment
Created attachment 241141 [details] turbostat --debug --msr=0x199
Created attachment 241151 [details] perf output perf record -a --event=power:pstate_sample sleep 300
(In reply to Mike Lui from comment #10) > Created attachment 241151 [details] > perf output > > perf record -a --event=power:pstate_sample sleep 300 There doesn't seem to be any actual sample data in the file. The header seems O.K., as does all the module stuff.
Same as Doug. ./perf report --header Error: The perf.data file has no samples! # ======== # captured on: Fri Oct 7 14:08:08 2016 # hostname : stravinsky # os release : 4.8.0-1-ARCH # perf version : 4.7.g523d93 # arch : x86_64 # nrcpus online : 12 # nrcpus avail : 12 # cpudesc : Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz # cpuid : GenuineIntel,6,63,2 # total memory : 32844308 kB # cmdline : /usr/bin/perf record -a --event=power:pstate_sample sleep 300 # event : name = power:pstate_sample, , type = 2, size = 112, config = 0x184, { sample_period, sa # HEADER_CPU_TOPOLOGY info available, use -I to display # HEADER_NUMA_TOPOLOGY info available, use -I to display # pmu mappings: cpu = 4, intel_bts = 6, uncore_imc_4 = 17, uncore_sbox_1 = 26, uncore_cbox_5 = 24 # HEADER_CACHE info available, use -I to display # ======== # No samples collected. I guess intel_pstate sched_util not called. -Please also attach your .config In addition do this step: # echo -n 'file intel_pstate.c +p' > /sys/kernel/debug/dynamic_debug/control # cd /sys/device/system/cpu/cpu1 # echo 0 > online # echo 1 > online Take dmesg and send output
Hrmm, yea not sure why the perf.data is empty. I'm not too familiar with using perf for events outside the standard performance profiling. Attaching .config and dmesg after executing the steps from comment #12.
Created attachment 241311 [details] archlinux kernel config
Comment on attachment 241311 [details] archlinux kernel config # zcat /proc/config.gz > .config
Created attachment 241321 [details] dmesg after toggling cpu1
Please attach dmesg from a fresh boot too.
And please boot with dyndbg="file intel_pstate.c +p" in the kernel command line for that.
Created attachment 241581 [details] WARN if the time argument of the governor hook is 0 Also please apply this patch and see if you see the new warning in dmesg output.
I used same kernel config with 4.8 on Arch Linux on CPU model as yours. I couldn't reproduce. So we need to add debug patches. So start with patch in comment #19.
comment #20 leads to believe this is an issue not with my CPU, but instead with my other hardware configuration. I went into my BIOS (UEFI?) on my MSI X99A SLI PLUS and changed 3 settings I must have set a while back: - disabled on-board OC'ing - disabled intel c-states - enabled turbo-boost After this the issues in comment #1 are no longer present. Should this be marked as resolved, or should this still be considered a bug, for such a corner case?
So your current settings are: - disabled on-board OC'ing - disabled intel c-states - enabled turbo-boost What was setting before for these? I still want to reproduce for analysis.
I can reproduce it on my machine running Fedora 24. model name : Intel(R) Core(TM) i7-5930K CPU @ 3.50GHz model : 63 Kernel 4.8.8-200.fc24.x86_64 I also noticed that machine has slowed down significantly. If there's any help I can provide to debug this, please let me know. I use this machine for computing so CPU time is essential for me to benchmark my code.
(In reply to Dominik Gronkiewicz from comment #23) > I can reproduce it on my machine running Fedora 24. > > model name : Intel(R) Core(TM) i7-5930K CPU @ 3.50GHz > model : 63 > > Kernel 4.8.8-200.fc24.x86_64 > > I also noticed that machine has slowed down significantly. > > If there's any help I can provide to debug this, please let me know. I use > this machine for computing so CPU time is essential for me to benchmark my > code. Sorry for posting two comments in a row. Disabling C-states in MSI BIOS indeed brings back full functionality and performance of the CPU. Still, if there's any way I can help in debugging, please let me know.
Please apply patch "WARN if the time argument of the governor hook is 0" attached in this bugzilla. During boot add this to kernel command line: "dyndbg="file intel_pstate.c +p" and send dmesg log.
ping...
Thought I would chime in hear and say that I'm seeing something very similar. model : 63 model name : Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz stepping : 2 The motherboard is a MSI X99A Raider. I did not see an option to disable over clocking, but did disable C-states and made sure turbo boost was enabled. On cold boot the system is fine. The cores fluctuate between 1200 and 3400 Mhz as expected. But on a normal reboot they appear to be locked at 1200 Mhz and never accelerate. Additionally, apps like ps and htop show all processes with a CPU usage of 0%. I can increase the speed by doing a "cpupower frequency-set -g performance", but all processes show 0% CPU usage which is not correct for the apps I'm running.
Created attachment 253741 [details] attachment-3164-0.html OOO till Feb 7th.
After careful examination, I have noticed a similar behavior: the problem seems to occur only after a reboot and never after a cold shutdown.
Mike Lui : - disabled on-board OC'ing - disabled intel c-states - enabled turbo-boost After this the issues in comment #1 are no longer present. @ Mike Lui Can you verify that there is no problem when you use SETUP DEFAULTs? or at least return C-states to their default ENABLED setting)
@ Dominik Gronkiewicz > I also noticed that machine has slowed down significantly. This suggests that timers are broken. Please see if there is anything in the dmesg about this -- such as your TSC being de-activited as a clock source. > Disabling C-states in MSI BIOS > indeed brings back full functionality and performance of the CPU. This is important if this is true, can you verify that re-enabling C-states breaks your system? > the problem seems to occur only after a reboot and never after a cold > shutdown. is that with C-states enabled or disabled?
@ ish@unx.ca > I did not see an option to disable over clocking, > but did disable C-states and made sure turbo boost was enabled. Does the problem go away when C-states are disabled, or is is still possible to see the failure with c-states disabled? > But on a normal reboot they appear to be locked at 1200 Mhz and never > accelerate. Is it true that the failure is NEVER SEEN without a reboot? @ Dominik Gronkiewicz > MSI BIOS Seems all three sightings are on MSI boards, can you identify yours? (see /sys/class/dmi/id/ )
It appears that this has resolved itself for me. I've recently updated to kernel: Linux desktop 4.10.8-200.fc25.x86_64 #1 SMP Fri Mar 31 13:20:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux with Fedora 25. I did a few reboots without power cycle with c-states disabled, which is how I last left my bios configured. Running htop showed CPU percentages after each boot, i7z showed cores hitting max mhz. I then reset my bios to factory default, which enables c-state. And did a few non-power cycle reboots, again seeing expected behaviour with the cores. I did not have to power cycle this time to get full speed. I'll keep monitoring it, but is it possible a kernel update could have fixed it? ===> /sys/class/dmi/id/ ./product_serial =====> Default string ./bios_vendor =====> American Megatrends Inc. ./product_version =====> 5.0 ./chassis_asset_tag =====> Default string ./chassis_serial =====> Default string ./board_vendor =====> MSI ./board_asset_tag =====> To be filled by O.E.M. ./board_version =====> 5.0 ./power/runtime_suspended_time =====> 0 ./power/autosuspend_delay_ms =====> ./power/runtime_active_time =====> 0 ./power/control =====> auto ./power/runtime_status =====> unsupported ./chassis_vendor =====> MSI ./modalias =====> dmi:bvnAmericanMegatrendsInc.:bvrP.50:bd07/19/2016:svnMSI:pnMS-7885:pvr5.0:rvnMSI:rnX99ARAIDER(MS-7885):rvr5.0:cvnMSI:ct3:cvr5.0: ./product_uuid =====> AAAAAAAA-AAAA-AAAA-AAAA-D8CB8AEDA146 ./bios_version =====> P.50 ./sys_vendor =====> MSI ./board_serial =====> To be filled by O.E.M. ./chassis_version =====> 5.0 ./chassis_type =====> 3 ./uevent =====> MODALIAS=dmi:bvnAmericanMegatrendsInc.:bvrP.50:bd07/19/2016:svnMSI:pnMS-7885:pvr5.0:rvnMSI:rnX99ARAIDER(MS-7885):rvr5.0:cvnMSI:ct3:cvr5.0: ./product_name =====> MS-7885 ./bios_date =====> 07/19/2016 ./board_name =====> X99A RAIDER (MS-7885)
(In reply to ish from comment #33) > It appears that this has resolved itself for me. I've recently updated to > kernel: > > Linux desktop 4.10.8-200.fc25.x86_64 #1 SMP Fri Mar 31 13:20:22 UTC 2017 > x86_64 x86_64 x86_64 GNU/Linux > > with Fedora 25. > Good to know. > I did a few reboots without power cycle with c-states disabled, which is how > I last left my bios configured. Running htop showed CPU percentages after > each boot, i7z showed cores hitting max mhz. > > I then reset my bios to factory default, which enables c-state. And did a > few non-power cycle reboots, again seeing expected behaviour with the cores. > I did not have to power cycle this time to get full speed. > > I'll keep monitoring it, but is it possible a kernel update could have fixed > it? Well, maybe, I'm not quite sure, but I do know there is quite a lot of intel-pstate fixes in recent kernels. Anyway, bug closes as it can not be reproduced. Please feel free to reopen it if you can reproduce the problem in the latest upstream kernel again.
(In reply to Len Brown from comment #30) > Mike Lui : > > - disabled on-board OC'ing > - disabled intel c-states > - enabled turbo-boost > > After this the issues in comment #1 are no longer present. > > @ Mike Lui > Can you verify that there is no problem when you use SETUP DEFAULTs? > or at least return C-states to their default ENABLED setting) I got caught up with work recently and just reviewed the comments, since my last comment. The problem appears to be fixed, for me. Unfortunately (or fortunately), the solution was a simple UEFI BIOS update. This points to a problem in the firmware, for my situation, which explains the intermittent nature of the problem.