Bug 217874

Summary:	Slow boot and shutdown/reboot problems with 6.5.0+
Product:	Linux	Reporter:	Marcus Seyfarth (m.seyfarth)
Component:	Kernel	Assignee:	Virtual assignee for kernel bugs (linux-kernel)
Status:	CLOSED UNREPRODUCIBLE
Severity:	normal
Priority:	P3
Hardware:	Intel
OS:	Linux
Kernel Version:	6.5.1	Subsystem:
Regression:	Yes	Bisected commit-id:
Attachments:	journalctl_651.log journalctl_6414.log

Description Marcus Seyfarth 2023-09-05 20:29:20 UTC

Created attachment 305047 [details]
journalctl_651.log

I've observed a slow boot and shutdown/reboot problems that have been verified on the stock Arch 6.5.1, CachyOS 6.5.0-2 and a custom-compiled 6.5.1 Kernel. I originally had seen a RCU-related trace which I thought to be the cause for these problems as well and reported it in https://bugzilla.kernel.org/show_bug.cgi?id=217864. However that specific trace turned out to be caused by an old patch that I carried around with me since 2022.

systemd-analyze with my custom compiled 6.5.1 shows a suspiciously long firmware time:
Startup finished in 2min 42.000s (firmware) + 4.201s (loader) + 6.895s (kernel) + 1.541s (userspace) = 2min 54.640s  
graphical.target reached after 1.539s in userspace.

For comparison, 6.4.14 starts in 16 seconds and doesn't even show a firmware section.

System:
  Host: klx99 Kernel: 6.5.1-3.1-cachyos-lto arch: x86_64 bits: 64
    Desktop: KDE Plasma v: 5.27.7 Distro: CachyOS
Machine:
  Mobo: Lenovo model: X99-TF Gaming v: G368J V1.1, NALEX
    serial: <superuser required> UEFI: American Megatrends v: CX99DE26
    date: 10/10/2020
CPU:
  Info: 18-core model: Intel Xeon E5-2696 v3 bits: 64 type: MT MCP cache:
    L2: 4.5 MiB
Graphics:
  Device-1: AMD Navi 21 [Radeon RX 6950 XT] driver: amdgpu v: kernel
  Display: x11 server: X.Org v: 21.1.99 with: Xwayland v: 23.2.0 driver: X:
    loaded: amdgpu unloaded: modesetting dri: radeonsi gpu: amdgpu
    resolution: 2560x1440~165Hz
  API: OpenGL v: 4.6 Mesa 23.3.0-devel (git-8a32614a18) renderer: AMD
    Radeon RX 6950 XT (navi21 LLVM 16.0.6 DRM 3.54 6.5.1-3.1-cachyos-lto)

Comment 1 Marcus Seyfarth 2023-09-05 20:29:50 UTC

Created attachment 305048 [details]
journalctl_6414.log

Comment 2 Marcus Seyfarth 2023-09-06 06:04:42 UTC

Here is a graphical diff between my Kernel 6.4.14 (left) and 6.5.1 (right): 
https://www.diffchecker.com/dFhL5tn9/

Comment 3 Artem S. Tashkinov 2023-09-06 14:08:46 UTC

Please perform regression testing:

https://docs.kernel.org/admin-guide/bug-bisect.html

Comment 4 Marcus Seyfarth 2023-09-15 11:39:46 UTC

Good news, I cannot reproduce the issue any longer on a fresh install of CachyOS. As I saw upower.service and plasma-powerdevil.service failing to load previously, that was likely the source of the long boot process and failure to shutdown properly.

Maybe there was a problem with the used linux-tools and systemd on that previous installation that was triggered by using the newer 6.5 Kernel?! I cannot say if that is worth investigating further.