Bug 210689 - Random hang at shutdown/poweroff - ASUSTeK Z9PE-D8 motherboard
Summary: Random hang at shutdown/poweroff - ASUSTeK Z9PE-D8 motherboard
Status: CLOSED DOCUMENTED
Alias: None
Product: ACPI
Classification: Unclassified
Component: BIOS (show other bugs)
Hardware: Intel Linux
: P1 normal
Assignee: Zhang Rui
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-12-14 16:57 UTC by Daniele
Modified: 2021-07-01 02:30 UTC (History)
2 users (show)

See Also:
Kernel Version: LTS 5.4.80-2, 5.9.11-3, 5.10.2-2
Subsystem:
Regression: No
Bisected commit-id:


Attachments
Log - hang after shutdown -h now (31.78 KB, text/plain)
2020-12-14 16:57 UTC, Daniele
Details
dmidecode output (41.76 KB, text/plain)
2020-12-21 16:50 UTC, Daniele
Details
acpi dump (4.14 MB, text/plain)
2020-12-21 16:51 UTC, Daniele
Details
kernel panic (246.63 KB, image/jpeg)
2020-12-21 17:09 UTC, Daniele
Details
Original DSDT ASUS Z9PED8-WS (431.86 KB, text/x-csrc)
2020-12-23 07:09 UTC, Daniele
Details
Patched DSDT ASUS Z9PED8-WS (450.05 KB, text/x-csrc)
2020-12-23 07:10 UTC, Daniele
Details
acpidump from Sun Microsystems Ultra 24 (231.42 KB, text/plain)
2021-03-25 03:05 UTC, Julius Henry Marx
Details
dmesg from Sun Microsystems Ultra 24 (62.06 KB, text/plain)
2021-03-25 03:07 UTC, Julius Henry Marx
Details
Config kernel file for Manjaro (227.22 KB, text/plain)
2021-03-27 08:43 UTC, Daniele
Details
Config kernel file for Unraid (112.38 KB, text/plain)
2021-03-27 08:44 UTC, Daniele
Details

Description Daniele 2020-12-14 16:57:32 UTC
Created attachment 294127 [details]
Log - hang after shutdown -h now

Running Manjaro Architect, CLI only, the issue shows with kernel 5.4.80-2 LTS, 5.9.11-3, and with all the stable and LTS kernels issued from a year ago.

I'm using Manjaro Architect always with latest stable updates (20.2) as a base system to run qemu/kvm for a mac os vm with GPU passthrough.

My workstation has 2 gpus, one nvidia quadro 600, dedicated to Manjaro (actually with the nouveau driver), and a GTX titan black passed through to the vm.

Issue is that sometimes (about 30% of the times?), Manjaro hangs at "shutdown -h now" or "poweroff -f".
Restart (shutdown -r now) always worked.
On shutdown hang, fans are still on, usb keyboard and mouse do not respond, power to usb is active (I tried to connect a device and it's charging), monitor is on (black or without quiet in grub conf stuck with "reboot: power down").
The only thing I can do is a forced shutdown by long pressing the power button on my workstation.
The issue seems not related to systemd, I tried the latest Artix linux (runit) and the issue showed both with the installed system and with the bootable usb.

From the log I can't see anything wrong.
The attached log was saved after a forced reboot, with command "journalctl -b -1 -n250"
In the log there are reference to NetworkManager too, but I tried to also uninstall it and have only networkd.

I don't know how to debug this issue that is going to make me crazy and I don't know what to do now..
I'm trying to write and attach here all the info that I think could be useful, if more info are needed I'm here and I'm ready to try all you can suggest.

What I already tried:
1. run the nvidia quadro 600 with nvidia drivers (390xx) (blacklist nouveau)
2. run the nvidia quadro 600 with nouveau drivers
3. added shutdown hook
4. ehci off in bios
5. deleted quiet in grub conf
6. added acpi=off in grub conf
7. added reboot=bios in grub conf
8. added reboot=pci in grub conf
9. added intel_idle.max_cstate=1 in grub conf
10. added acpi_osi=! acpi_osi=‘Windows 2018’ in grub conf
11. added acpi_osi=! acpi_osi=‘Windows 2009’ in grub conf
12. stopped, disabled and masked lvm2-lvmetad and lvm2-monitor
13. added mei_me to the RUNTIME_PM_DRIVER_BLACKLIST configuration of TLP
14. set networks to down before shutdown
15. stopped nfs server before shutdown
16. tried different kernels
17. installed uefi manjaro (it was installed in legacy mode)
18. disabled secure boot
19. Disabled XHCI in the bios (EHCI was already disabled)
20. Disabled ASMEDIA USB 3.0 controller in bios
21. Added Before=basic.target in dbus.service
22. WOL disabled in bios

---------------------------------------------------------------------------

My custom modifications:

nano /etc/modprobe.d/vfio.conf
options vfio-pci ids=10de:100c,10de:0e1a

nano /etc/mkinitcpio.conf
MODULES=(vfio_pci vfio vfio_iommu_type1 vfio_virqfd)

--> this is for gpu/hdmi audio passthrough.

nano /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on iommu=pt quiet udev.log_priority=3 kvm_intel.nested=1"

--> iommu and nested virtualization on.

Through libvirt I'm also passing through to the vm a usb controller (Fresco chipset), the builtin audio of my motherboard (Asus z9pe-d8 ws), the mouse/keyboard dongle and a logitech c920 webcam.

---------------------------------------------------------------------------
System specifications (inxi run on manjaro from mac os vm - ssh connection):

[root@tower tower]# inxi -Fyz
System:
  Kernel: 5.4.80-2-MANJARO x86_64 bits: 64 Console: tty 1 
  Distro: Manjaro Linux 
Machine:
  Type: Server Mobo: ASUSTeK model: Z9PE-D8 WS v: 1.0x serial: <filter> 
  UEFI: American Megatrends v: 5802 date: 06/10/2015 
CPU:
  Info: 2x 8-Core model: 06/2d bits: 64 type: MCP SMP L2 cache: 40.0 MiB 
  Speed: 1444 MHz min/max: 1200/3800 MHz Core speeds (MHz): 1: 1677 2: 1831 
  3: 1204 4: 1204 5: 1215 6: 1204 7: 1204 8: 1204 9: 1879 10: 1720 11: 1612 
  12: 1903 13: 1621 14: 1628 15: 1213 16: 1210 
Graphics:
  Device-1: NVIDIA GF108GL [Quadro 600] driver: nouveau v: kernel 
  Device-2: NVIDIA GK110B [GeForce GTX TITAN Black] driver: vfio-pci v: 0.2 
  Display: server: No display server data found. Headless machine? tty: 80x24 
  Message: Unable to show advanced data. Required tool glxinfo missing. 
Audio:
  Device-1: Intel C600/X79 series High Definition Audio driver: vfio-pci 
  Device-2: NVIDIA GF108 High Definition Audio driver: snd_hda_intel 
  Device-3: NVIDIA GK110 High Definition Audio driver: vfio-pci 
  Sound Server: ALSA v: k5.4.80-2-MANJARO 
Network:
  Device-1: Intel 82574L Gigabit Network driver: e1000e 
  IF: enp6s0 state: up speed: 1000 Mbps duplex: full mac: <filter> 
  Device-2: Intel 82574L Gigabit Network driver: e1000e 
  IF: enp7s0 state: up speed: 100 Mbps duplex: full mac: <filter> 
  IF-ID-1: br0 state: up speed: N/A duplex: N/A mac: <filter> 
  IF-ID-2: br1 state: up speed: N/A duplex: N/A mac: <filter> 
  IF-ID-3: vnet0 state: unknown speed: 10 Mbps duplex: full mac: <filter> 
  IF-ID-4: vnet1 state: unknown speed: 10 Mbps duplex: full mac: <filter> 
Drives:
  Local Storage: total: 8.19 TiB used: 3.41 TiB (41.6%) 
  ID-1: /dev/sda vendor: SanDisk model: SDSSDP256G size: 238.47 GiB 
  ID-2: /dev/sdb vendor: Western Digital model: WD20EZRX-00DC0B0 
  size: 1.82 TiB 
  ID-3: /dev/sdc vendor: Crucial model: CT500MX500SSD1 size: 465.76 GiB 
  ID-4: /dev/sdd vendor: Western Digital model: WD60EZRX-00MVLB1 
  size: 5.46 TiB 
  ID-5: /dev/sde vendor: Hitachi model: HTS542525K9SA00 size: 232.89 GiB 
Partition:
  ID-1: / size: 227.74 GiB used: 3.03 GiB (1.3%) fs: ext4 dev: /dev/sde2 
Swap:
  Alert: No Swap data was found. 
Sensors:
  System Temperatures: cpu: 28.0 C mobo: N/A gpu: nouveau temp: 48.0 C 
  Fan Speeds (RPM): N/A gpu: nouveau fan: 2730 
Info:
  Processes: 295 Uptime: 5h 28m Memory: 62.92 GiB used: 32.78 GiB (52.1%) 
  Init: systemd Shell: Bash inxi: 3.1.08 

---------------------------------------------------------------------------

sda: windows 10 (uefi installation, ntfs+efi, gpt)
sdb: data (gpt, ntfs)
sdc: data (xfs, gpt)
sdd: data (gpt, ntfs)
sde: Manjaro (uefi installtion, ext4+efi, gpt)

---------------------------------------------------------------------------

Modules:

[root@tower tower]# lsmod
Module                  Size  Used by
tun                    57344  4
uvcvideo              114688  0
videobuf2_vmalloc      20480  1 uvcvideo
snd_usb_audio         307200  0
videobuf2_memops       20480  1 videobuf2_vmalloc
videobuf2_v4l2         28672  1 uvcvideo
videobuf2_common       57344  2 videobuf2_v4l2,uvcvideo
snd_usbmidi_lib        40960  1 snd_usb_audio
videodev              249856  3 videobuf2_v4l2,uvcvideo,videobuf2_common
snd_rawmidi            45056  1 snd_usbmidi_lib
snd_seq_device         16384  1 snd_rawmidi
mc                     61440  5 videodev,snd_usb_audio,videobuf2_v4l2,uvcvideo,videobuf2_common
input_leds             16384  0
mousedev               24576  0
joydev                 28672  0
xfs                  1773568  1
fuse                  135168  5
libcrc32c              16384  1 xfs
intel_rapl_msr         20480  0
intel_rapl_common      28672  1 intel_rapl_msr
sb_edac                24576  0
x86_pkg_temp_thermal    20480  0
intel_powerclamp       20480  0
nouveau              2326528  1
coretemp               20480  0
kvm_intel             323584  18
snd_hda_codec_hdmi     73728  4
kvm                   802816  1 kvm_intel
snd_hda_codec_realtek   143360  0
snd_hda_codec_generic    94208  1 snd_hda_codec_realtek
ledtrig_audio          16384  2 snd_hda_codec_generic,snd_hda_codec_realtek
i2c_algo_bit           16384  1 nouveau
crct10dif_pclmul       16384  1
ttm                   118784  1 nouveau
crc32_pclmul           16384  0
snd_hda_intel          53248  0
ghash_clmulni_intel    16384  0
bridge                217088  0
aesni_intel           372736  0
drm_kms_helper        221184  1 nouveau
snd_intel_nhlt         20480  1 snd_hda_intel
snd_hda_codec         159744  4 snd_hda_codec_generic,snd_hda_codec_hdmi,snd_hda_intel,snd_hda_codec_realtek
crypto_simd            16384  1 aesni_intel
syscopyarea            16384  1 drm_kms_helper
stp                    16384  1 bridge
snd_hda_core          102400  5 snd_hda_codec_generic,snd_hda_codec_hdmi,snd_hda_intel,snd_hda_codec,snd_hda_codec_realtek
llc                    16384  2 bridge,stp
cryptd                 24576  2 crypto_simd,ghash_clmulni_intel
iTCO_wdt               16384  0
sysfillrect            16384  1 drm_kms_helper
glue_helper            16384  1 aesni_intel
sysimgblt              16384  1 drm_kms_helper
iTCO_vendor_support    16384  1 iTCO_wdt
snd_hwdep              16384  2 snd_usb_audio,snd_hda_codec
mxm_wmi                16384  1 nouveau
snd_pcm               143360  5 snd_hda_codec_hdmi,snd_hda_intel,snd_usb_audio,snd_hda_codec,snd_hda_core
fb_sys_fops            16384  1 drm_kms_helper
rapl                   16384  0
snd_timer              40960  1 snd_pcm
ib_mthca              172032  0
intel_cstate           16384  0
snd                   110592  12 snd_hda_codec_generic,snd_seq_device,snd_hda_codec_hdmi,snd_hwdep,snd_hda_intel,snd_usb_audio,snd_usbmidi_lib,snd_hda_codec,snd_hda_codec_realtek,snd_timer,snd_pcm,snd_rawmidi
intel_uncore          147456  0
e1000e                294912  0
ib_uverbs             151552  1 ib_mthca
soundcore              16384  1 snd
ib_core               360448  2 ib_mthca,ib_uverbs
pcspkr                 16384  0
mei_me                 45056  0
mei                   118784  1 mei_me
i2c_i801               36864  0
evdev                  28672  2
lpc_ich                28672  0
mac_hid                16384  0
wmi                    36864  2 mxm_wmi,nouveau
nfsd                  487424  13
auth_rpcgss           106496  1 nfsd
nfs_acl                16384  1 nfsd
lockd                 122880  1 nfsd
grace                  16384  2 nfsd,lockd
drm                   520192  4 drm_kms_helper,ttm,nouveau
sunrpc                471040  18 nfsd,auth_rpcgss,lockd,nfs_acl
agpgart                53248  3 ttm,nouveau,drm
ip_tables              36864  0
x_tables               49152  1 ip_tables
ext4                  786432  1
crc32c_generic         16384  0
crc16                  16384  1 ext4
mbcache                16384  1 ext4
jbd2                  135168  1 ext4
hid_generic            16384  0
usbhid                 65536  0
hid                   143360  2 usbhid,hid_generic
sr_mod                 28672  0
cdrom                  77824  1 sr_mod
sd_mod                 61440  8
isci                  167936  0
libsas                 98304  1 isci
ahci                   40960  4
scsi_transport_sas     45056  2 isci,libsas
libahci                40960  1 ahci
libata                286720  3 libahci,ahci,libsas
crc32c_intel           24576  3
xhci_pci               20480  0
firewire_ohci          45056  0
scsi_mod              253952  6 isci,scsi_transport_sas,sd_mod,libsas,libata,sr_mod
xhci_hcd              282624  1 xhci_pci
firewire_core          81920  1 firewire_ohci
ehci_pci               20480  0
crc_itu_t              16384  1 firewire_core
ehci_hcd               98304  1 ehci_pci
vfio_pci               65536  4
irqbypass              16384  10 vfio_pci,kvm
vfio_virqfd            16384  1 vfio_pci
vfio_iommu_type1       36864  1
vfio                   36864  10 vfio_iommu_type1,vfio_pci

---------------------------------------------------------------------------

FSTAB:

UUID=61eacf52-caa6-4a88-bf3d-ea5244eba645       /               ext4            rw,noatime      0 0

UUID=FADE946ADE942145 /media/2TB ntfs defaults,noatime 0 2
UUID=962CBD772CBD52CB /media/6TB ntfs defaults,noatime 0 2
UUID=d92327eb-5bc3-49bd-84b8-69c4ec3f300c /media/VM xfs defaults,noatime 0 2

---------------------------------------------------------------------------

Installed packages:

[root@tower tower]# pacman -Qe
acpi 1.7-3
acpid 2.0.32-2
amd-ucode 20201124.r1786.b362fd4-1
b43-fwcutter 019-3
bash 5.0.018-1
btrfs-progs 5.9-1
bzip2 1.0.8-4
cockpit 233.1-1
cockpit-machines 233.1-1
coreutils 8.32-1
cpupower 5.9-1
crda 4.14-3
cronie 1.5.5-1
cryptsetup 2.3.4-1
device-mapper 2.02.187-3
dhclient 4.4.2-2
dhcpcd 9.3.3-1
diffutils 3.7-3
dmraid 1.0.0.rc16.3-12
dnsmasq 2.82-2
dosfstools 4.1-3
e2fsprogs 1.45.6-3
ecryptfs-utils 111-4
efibootmgr 17-2
exfat-utils 1.3.0-2
f2fs-tools 1.14.0-1
file 5.39-1
filesystem 2020.09-2
findutils 4.7.0-2
gawk 5.1.0-1
gcc-libs 10.2.0-3
gettext 0.21-1
glibc 2.32-5
grep 3.6-1
grub 2.04-12
grub-theme-manjaro 20.2-12
gzip 1.10-3
inetutils 1.9.4-8
intel-ucode 20201118-1
inxi 3.1.08+1-1
iproute2 5.9.0-1
iputils 20200821-1
jfsutils 1.1.15-7
less 563-1
libvirt 1:6.5.0-3
licenses 20200427-1
linux-latest 5.9-1
linux-latest-headers 5.9-1
linux-lts 1:5.4-4
linux-lts-headers 1:5.4-4
logrotate 3.17.0-1
lvm2 2.02.187-3
man-db 2.9.3-1
man-pages 5.09-2
manjaro-firmware 20160419-1
manjaro-release 20.2-1
manjaro-system 20201014-1
manjaro-zsh-config 0.15-1
mdadm 4.1-2
memtest86+ 5.01-4
mhwd 0.6.5-2
mhwd-db 0.6.5-4
mkinitcpio-openswap 0.1.0-3
nano 5.3-1
nfs-utils 2.5.2-2
nilfs-utils 2.2.8-2
ntfs-3g 2017.3.23-5
openssh 8.4p1-2
os-prober 1.77-1
packagekit 1.1.13-1
pacman 5.2.2-3
pciutils 3.7.0-1
perl 5.32.0-3
procps-ng 3.3.16-2
psmisc 23.3-2
qemu 5.1.0-3
reiserfsprogs 3.6.27-3
rsync 3.2.3-1
s-nail 14.9.19-2
sed 4.8-1
shadow 4.8.1-4
spectre-meltdown-checker 0.43-1
sudo 1.9.3.p1-1
sysfsutils 2.1.0-11
systemd 246.6-1
systemd-fsck-silent 239-1
systemd-sysvcompat 246.6-1
tar 1.32-3
texinfo 6.7-3
tlp 1.3.1-2
udisks2 2.9.1-1
usbutils 013-1
util-linux 2.36.1-3
vi 1:070224-4
wget 1.20.3-3
which 2.21-5
wpa_supplicant 2:2.9-7
xfsprogs 5.8.0-1
xz 5.2.5-1

---------------------------------------------------------------------------

Network (managed by systemd-networkd):

[root@tower tower]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br1 state UP group default qlen 1000
    link/ether 10:bf:48:0d:58:7d brd ff:ff:ff:ff:ff:ff
3: enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000
    link/ether 10:bf:48:0d:58:7e brd ff:ff:ff:ff:ff:ff
4: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 8e:bb:f7:af:86:e1 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.202/24 brd 192.168.1.255 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fe80::8cbb:f7ff:feaf:86e1/64 scope link 
       valid_lft forever preferred_lft forever
5: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ba:5b:05:1d:1c:af brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.1/24 brd 192.168.2.255 scope global br1
       valid_lft forever preferred_lft forever
    inet6 fe80::b85b:5ff:fe1d:1caf/64 scope link 
       valid_lft forever preferred_lft forever
6: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UNKNOWN group default qlen 1000
    link/ether fe:2a:14:55:1a:b2 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc2a:14ff:fe55:1ab2/64 scope link 
       valid_lft forever preferred_lft forever
7: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br1 state UNKNOWN group default qlen 1000
    link/ether fe:2a:14:32:2c:ff brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc2a:14ff:fe32:2cff/64 scope link 
       valid_lft forever preferred_lft forever
Comment 1 Daniele 2020-12-15 07:50:04 UTC
I also tried intel_pstate=disable since I have Sandy Bridge, without luck, same issue.

Additional info about the 2 cpus:

[root@tower tower]# cat /proc/cpuinfo
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 1652.543
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6225.32
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 1575.014
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6225.32
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 1538.594
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6225.32
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 1444.410
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6225.32
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 1424.757
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 4
cpu cores	: 8
apicid		: 8
initial apicid	: 8
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6225.32
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 1308.335
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 5
cpu cores	: 8
apicid		: 10
initial apicid	: 10
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6225.32
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 1308.182
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 6
cpu cores	: 8
apicid		: 12
initial apicid	: 12
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6225.32
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 1400.827
cache size	: 20480 KB
physical id	: 0
siblings	: 8
core id		: 7
cpu cores	: 8
apicid		: 14
initial apicid	: 14
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6225.32
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 2503.418
cache size	: 20480 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
initial apicid	: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6239.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 2215.449
cache size	: 20480 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
initial apicid	: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6239.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 2271.570
cache size	: 20480 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
initial apicid	: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6239.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 2582.556
cache size	: 20480 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
initial apicid	: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6239.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 2559.885
cache size	: 20480 KB
physical id	: 1
siblings	: 8
core id		: 4
cpu cores	: 8
apicid		: 40
initial apicid	: 40
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6239.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 3036.701
cache size	: 20480 KB
physical id	: 1
siblings	: 8
core id		: 5
cpu cores	: 8
apicid		: 42
initial apicid	: 42
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6239.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 2964.786
cache size	: 20480 KB
physical id	: 1
siblings	: 8
core id		: 6
cpu cores	: 8
apicid		: 44
initial apicid	: 44
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6239.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: 06/2d
stepping	: 5
microcode	: 0x513
cpu MHz		: 2252.259
cache size	: 20480 KB
physical id	: 1
siblings	: 8
core id		: 7
cpu cores	: 8
apicid		: 46
initial apicid	: 46
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 6239.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:
Comment 2 Daniele 2020-12-15 07:56:28 UTC
Is it normal that "power management" in /proc/cpuinfo is empty?
Comment 3 Daniele 2020-12-15 15:49:35 UTC
I think I made some progress on this, finding the culprit (?)
It seems that the culprit is ehci_pci.

this seems to fix the shutdown/poweroff hang:

From manjaro terminal (only cli):
1- rmmod ehci_pci
2- short press the mechanical power button to issue the shutdown

Since I remove ehci_pci my mouse/keyboard dongle is not there anymore, so I have to short press the button.
So far, no issues.

To resume:
1. Manjaro is started (mouse/keyboard dongle is attached in Manjaro)
2. Mac os vm is started (mouse/keyboard dongle is detached from Manjaro and attached to mac os, the passthrough is defined in libvirt xml)
3. Shutdown Mac os vm from within the vm
4. Mouse/keyboard dongle is detached from the vm and reattached to Manjaro

Same for the logitech webcam c920.

Without rmmod ehci_pci --> sometimes hang on shutdown
With rmmod ehci_pci before shutdown so far no issue

USB controller to which the dongle is attached:
USB controller: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #2 (rev 06) (prog-if 20 [EHCI])

One thing I noticed, one time, after booting Manjaro and without booting the vm, I issued the command rmmod ehci_pci and a message appeared:

#rmmod ehci_pci
[40.951572] hid-generic 0003:045E:0745.0003: can't resubmit intr, 0000:00:1a.0-1.2/input2, status -19

Did it try to reconnect again after the rmmod?

I have 2 of these usb controllers:

00:1a.0 USB controller: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #2 (rev 06) (prog-if 20 [EHCI])
	Subsystem: ASUSTeK Computer Inc. Device 84ef
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 17
	NUMA node: 0
	IOMMU group: 10
	Region 0: Memory at b3107000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [58] Debug port: BAR=1 offset=00a0
	Capabilities: [98] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: ehci-pci
	Kernel modules: ehci_pci
00:1d.0 USB controller: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #1 (rev 06) (prog-if 20 [EHCI])
	Subsystem: ASUSTeK Computer Inc. Device 84ef
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 23
	NUMA node: 0
	IOMMU group: 18
	Region 0: Memory at b3106000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [58] Debug port: BAR=1 offset=00a0
	Capabilities: [98] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: ehci-pci
	Kernel modules: ehci_pci
Comment 4 Daniele 2020-12-17 09:29:46 UTC
Nope...damn...it's not the ehci...again hang...
I'm trying now the cpu scaling drivers, because on a slackware based system (unraid) the issue doesn't show and I noticed that it uses pcc_cpufreq.
Instead my Manjaro automatically loads intel_pstate, and if disabled in grub as a kernel pa rameter it loads acpi_cpufreq.
I blacklisted acpi_cpufre too, but till now I'm not able to force load pcc_cpufreq, so I'm running the os without a driver loaded.
Will report back if this solved the hang.
Comment 5 Daniele 2020-12-19 14:24:11 UTC
no luck..tried intel_pstate, acpi_cpufreq and no driver, in all cases sometimes hang on shutdown.
I tried also kernel 5.10.0 RC5, same issue.

I also tried (in addition):

1. acpi=force reboot=acpi
2. reboot=acpi
3. apm=power-off
4. noapic nolapic (can't boot)
5. video-vesa drivers (+blacklist nouveau)

I'm out of choices now and I don't know how to debug properly.

Windows 10 has no issue and as I said also unraid 6.8.3 (slavkware based, kernel 4.19 --> yes, I tried also this kernel in my manjaro) has no issue.

Appreciate any hints.
Comment 6 Daniele 2020-12-21 16:50:40 UTC
Created attachment 294265 [details]
dmidecode output

dmidecode
Comment 7 Daniele 2020-12-21 16:51:26 UTC
Created attachment 294267 [details]
acpi dump

acpi dump output
Comment 8 Daniele 2020-12-21 17:09:21 UTC
So sad nobody has replied...anyway, I swapped the motherboard with my second workstation, same asus z9ped8 ws, same bios v.5802, different cpus: previous ones were engineering samples QB7R e5-2687w, these are e5-2687w SR0KG: issue not solved.
I swapped the quadro 600 gpu with a GeForce 8400 GS: issue not solved

Tried acpi_osi='!Windows2012' in grub: issue not solved

Tried disabling watchdog and disabling spectre/meltdown security patches: issue not solved

I tried the old 4.9 kernel: issue not solved

It doesn't seem the issue has anything to do with iommu enabled: it happens without any modifications to the grub file and with the live usb.

I tried also ubuntu 20.04 lts server: issue not solved

In all cases the workstation hangs in S5: if I open the case I can see q-code 05.

In only one case during my tests there was a kernel panic visible on monitor:
not syncing - Fatal exception in interrupt (follows image attachment)

I can't say it's related with the current issue or not.
Comment 9 Daniele 2020-12-21 17:09:45 UTC
Created attachment 294269 [details]
kernel panic

kernel panic
Comment 10 Daniele 2020-12-23 07:08:05 UTC
I think I finally found a workaround, about 20 successful shutdowns, it never happened before!

My last attempt was to look at the ACPI tables: luckily I know that they exist, that they can have bugs, that they can be fixed and that they can be injected at boot.
I know that method _PTS is responsible for S states, S5 included.
My first attempt was to dump my DSDT and fix only all the errors (mostly some wrong lengths, invalid objects and other minor things): compiled and injected, it didn't work, always randomly hangs on shutdown.

Then, I modified the code of the _PTS method.
Original code:
    Method (_PTS, 1, NotSerialized)  // _PTS: Prepare To Sleep
    {
        If (((Arg0 == 0x04) && (OSFL () == 0x02)))
        {
            Sleep (0x0BB8)
        }

        PTS (Arg0)
        DBG8 = Arg0
        WAKP [Zero] = Zero
        WAKP [One] = Zero
        WSSB = ASSB /* \ASSB */
        WOTB = AOTB /* \AOTB */
        WAXB = AAXB /* \AAXB */
        ASSB = Arg0
        AOTB = OSFL ()
        AAXB = Zero
        \_SB.SLPS = One
    }


Modified code:
    Method (_PTS, 1, NotSerialized)
   {
       If (LEqual (Arg0, 0x05)) {}
       Else
       {
           Store (Arg0, DBG8)
           If (LAnd (LEqual (Arg0, 0x04), LEqual (OSFL (), 0x02)))
           {
               Sleep (0x0BB8)
           }

           PTS (Arg0)
           Store (Zero, Index (WAKP, Zero))
           Store (Zero, Index (WAKP, One))
           Store (ASSB, WSSB)
           Store (AOTB, WOTB)
           Store (AAXB, WAXB)
           Store (Arg0, ASSB)
           Store (OSFL (), AOTB)
           Store (Zero, AAXB)
           Store (One, \_SB.SLPS)
       }
   }


I really know near nothing about code in the DSDT, the "fixed" code is found on internet, to fix sleep/restart on hackintosh machines for a different mainboard.
I noticed that most of the variables were identical to mine and I noticed a new if cycle containing Arg0=0x05, which complains the S5 state.
Compiled and injected, no more issues!

I really don't know why this could randomly cause hangs at shutdown, but it would be good if the kernel could be modified to work without injecting a custom DSDT: I know it's a firmware bug, and I don't know if it can be fixed kernel side, maybe some guru can give more light on this.
I noticed a lot of new discussions around about hangs of shutdown, without solutions, maybe DSDT is the workaround to go in these situations.

Following attachments of the dsl files of the original DSDT and the patched one, for ASUS Z9PED8-WS, bios v. 5802
Comment 11 Daniele 2020-12-23 07:09:07 UTC
Created attachment 294311 [details]
Original DSDT ASUS Z9PED8-WS

Dumped original DSDT from ASUS Z9PED8-WS, bios v. 5802
Comment 12 Daniele 2020-12-23 07:10:09 UTC
Created attachment 294313 [details]
Patched DSDT ASUS Z9PED8-WS

Patched DSDT for ASUS Z9PED8-WS --> Fixes random shutdown hangs
Comment 13 Zhang Rui 2021-03-21 15:57:52 UTC
so your workaround is to ignore _PTS for S5, right?
I'm not sure if there are some side effect with this solution, but at least you didn't observe any, right?
why not raise the issue to the mother board vendor, as this is a firmware problem.
BTW, do you have any chance to run Windows with this hardware?
Comment 14 Daniele 2021-03-21 16:04:04 UTC
(In reply to Zhang Rui from comment #13)
> so your workaround is to ignore _PTS for S5, right?
> I'm not sure if there are some side effect with this solution, but at least
> you didn't observe any, right?
> why not raise the issue to the mother board vendor, as this is a firmware
> problem.
> BTW, do you have any chance to run Windows with this hardware?

Hi Zhang, yes, ignore all for S5, so far so good, from late december 2020 I never had any improper shutdown, the workstation always shutdown successful.
About contacting Asus, I'd want, I know it's a firmware issue and I received the same suggestion from another bugtracker, however the mainboard is end of life, no more updates for the firmware.

Yes, I'm currently dual booting linux manjaro and windows, no issues in both systems.

In the end I also patched myself the bios with the patched dsdt and flashed to the motherboard successfully.
Comment 15 Daniele 2021-03-21 16:39:39 UTC
Anyway, I filled Asus form and sent it to them...
Comment 16 Zhang Rui 2021-03-22 12:59:45 UTC
(In reply to Daniele from comment #14)
> (In reply to Zhang Rui from comment #13)
> > so your workaround is to ignore _PTS for S5, right?
> > I'm not sure if there are some side effect with this solution, but at least
> > you didn't observe any, right?
> > why not raise the issue to the mother board vendor, as this is a firmware
> > problem.
> > BTW, do you have any chance to run Windows with this hardware?
> 
> Hi Zhang, yes, ignore all for S5, so far so good, from late december 2020 I
> never had any improper shutdown, the workstation always shutdown successful.
> About contacting Asus, I'd want, I know it's a firmware issue and I received
> the same suggestion from another bugtracker, however the mainboard is end of
> life, no more updates for the firmware.
> 
> Yes, I'm currently dual booting linux manjaro and windows, no issues in both
> systems.

But windows is not running with your patched dsdt, right?

> 
> In the end I also patched myself the bios with the patched dsdt and flashed
> to the motherboard successfully.

What do you mean? You patched this change to the BIOS binary and flashed it?
Comment 17 Daniele 2021-03-22 13:23:50 UTC
(In reply to Zhang Rui from comment #16)
> But windows is not running with your patched dsdt, right?
Windows IS running with no issue with the patched DSDT. Windows was working good also with the not patched DSDT.

> What do you mean? You patched this change to the BIOS binary and flashed it?
Yes, instead of injecting the DSDT at boot I patched the Bios binary and flashed the chip with the modified DSDT table.
Comment 18 Daniele 2021-03-22 16:15:09 UTC
Asus reply, as expected:
In merito a quanto segnalato, purtroppo la scheda madre Z9PE-D8 WS non è più supportata da Asus e non sarà pubblicata una nuova versione Bios che potrebbe risolvere il problema dell'ACPI riscontrato. Inoltreremo la sua segnalazione al reparto competente. 

Translation (briefly):
Z9PE-D8 WS EOL, no bios update, issue forwarded to relevant department. <-- ?????


--------
issue forwarded to relevant department. <-- ????? (hand in face...)
Comment 19 Julius Henry Marx 2021-03-23 18:53:11 UTC
Hello:

Running Devuan Beowulf 3.1.0 on a a Sun Microsystems Ultra 24 with an Intel Q9550 + 8Gb RAM.

The issue has been present from when I first set up Linux in this box, many kernels ago.

Originally reported it here in 2019:
https://bugzilla.kernel.org/show_bug.cgi?id=201965#c3

Like with the OP's hardware, there's no possible update, upgrade, support, etc. from Sun (which is dead) or Oracle for that matter.

My problem is *exactly* like the one the OP originally reported.
When it occurrs, the screen output is exactly the same.

The OP was able to solve the problem by finding precisely the *right* lines to change in the `_PTS` area of the `DSDT table he extracted from his motherboard's BIOS.

He then managed to actually patch the bios.rom without bricking his box.

I had already tried modding my BIOS DSDT table with some MS based utilities and actually managed to reduce the 'Warning' count on recompilaton to '0', but the problem subsists even after adding the OPs _PTS modifications to my DSDT table.

The modification I added under the OP's looks like this:

[code]
    Method (_PTS, 1, NotSerialized)
    {
        If (LEqual (Arg0, 0x05)) {}
        Else
        {
            Store (Arg0, DBG8)
            PTS (Arg0)
            \_SB.PCI0.SBRG.SIOR.SIOL (Arg0)
            Store (Zero, Index (WAKP, Zero))
            Store (Zero, Index (WAKP, One))
            If (LAnd (LEqual (Arg0, 0x04), LEqual (OSFL (), 0x02)))
            {
                Sleep (0x0BB8)
            }

            Store (ASSB, WSSB)
            Store (AOTB, WOTB)
            Store (AAXB, WAXB)
            Store (Arg0, ASSB)
            Store (OSFL (), AOTB)
            Store (OSYS (), OSTP)
            Store (Zero, AAXB)
        }
    }
[/code]

Unfortunately it does not work and I have not found any "fixed" code on the web save the snippets I got from using the DSDT modding utilities mentioned earlier, probably because no one tried to run OSx on a Sun Ultra 24.

I'd appreciate it if I could get a pointer or two to see if I can solve this: envy of the OP is killing me.  8^D !

Thanks in advance,

JHM
Comment 20 Zhang Rui 2021-03-25 01:36:35 UTC
(In reply to Julius Henry Marx from comment #19)
> Hello:
> 
> Running Devuan Beowulf 3.1.0 on a a Sun Microsystems Ultra 24 with an Intel
> Q9550 + 8Gb RAM.
> 
> The issue has been present from when I first set up Linux in this box, many
> kernels ago.
> 
> Originally reported it here in 2019:
> https://bugzilla.kernel.org/show_bug.cgi?id=201965#c3
> 
> Like with the OP's hardware, there's no possible update, upgrade, support,
> etc. from Sun (which is dead) or Oracle for that matter.

"OP" means?

> 
> My problem is *exactly* like the one the OP originally reported.

If you're referring to https://bugzilla.kernel.org/show_bug.cgi?id=201965
then it might be a different issue.
Can you please open a new bug report, and give detailed descriptions of the problem you encountered?
Together with acpidump and dmesg output after boot.
Comment 21 Zhang Rui 2021-03-25 01:38:59 UTC
Daniele, may I know the model of your hardware?
I will check if I can reach their BIOS team to understand why _PTS breaks S5.
Comment 22 Julius Henry Marx 2021-03-25 03:05:40 UTC
Created attachment 296043 [details]
acpidump from Sun Microsystems Ultra 24

Belongs to comment #20

Mobo: Sun Microsystems
model: Ultra 24 v: 50 
BIOS: American Megatrends v: 1.56 date: 01/21/2011
CPU: Quad Core model: Intel Core2 Quad Q9550
Comment 23 Julius Henry Marx 2021-03-25 03:07:51 UTC
Created attachment 296045 [details]
dmesg from Sun Microsystems Ultra 24

Belongs to comment #20

Mobo: Sun Microsystems
model: Ultra 24 v: 50 
BIOS: American Megatrends v: 1.56 date: 01/21/2011
CPU: Quad Core model: Intel Core2 Quad Q9550

Running Linux devuan 4.19.0-14-amd64 #1
SMP Debian 4.19.171-2 (2021-01-30) x86_64 GNU/Linux
Comment 24 Julius Henry Marx 2021-03-25 03:10:32 UTC
Hello:

"OP" means "Original Poster" or "Original Post".
Sorry, I thought it was understood.
My bad.

> Can you please open a new bug report ...
No, let's please stay with this one.

I have compared notes with the author of this post (Daniele) and we have come to the conclusion that we have *same* issue, albeit on different machines.

The basic difference is that he was able to fix it with a piece of DSDT code he found on the web.

Unfortunately, it did not work in my machine.

For testing purposes, this morning I blacklisted acpi_cpufreq and will report back in 30 days or sooner if blacklisting the module does not have any effect and I get another bad shutdown episode.

I'm attaching acpidump and dmesg of the machine with a modified DSDT table and blacklisted acpi_cpufreq module.

Please ask if you need anything else.

Thanks in advance.

JHM
Comment 25 Zhang Rui 2021-03-25 04:13:48 UTC
(In reply to Julius Henry Marx from comment #24)
> Hello:
> 
> "OP" means "Original Poster" or "Original Post".
> Sorry, I thought it was understood.
> My bad.
> 
> > Can you please open a new bug report ...
> No, let's please stay with this one.
> 
https://bugzilla.kernel.org/show_bug.cgi?id=201965
is a different one as it is on AMD CPUs, which means the BIOS code are totally different.

So this thread,
This is from Deniele' post
Machine:
  Type: Server Mobo: ASUSTeK model: Z9PE-D8 WS v: 1.0x serial: <filter> 
  UEFI: American Megatrends v: 5802 date: 06/10/2015 
CPU:
  Info: 2x 8-Core model: 06/2d bits: 64 type: MCP SMP L2 cache: 40.0 MiB 

and your platform is 

Mobo: Sun Microsystems
model: Ultra 24 v: 50 
BIOS: American Megatrends v: 1.56 date: 01/21/2011
CPU: Quad Core model: Intel Core2 Quad Q9550

They are two different hardwares and the BIOS code may also be very different.

So they're very likely to be different problems also the symptoms are similar.

For now, I want to focus on one issue in one bug report, so that I won't be confused by different debug information from different reporter.

If I found they are the same root cause, then I will mark these two bug reports as duplicates.

Does this sound good for you?
Comment 26 Julius Henry Marx 2021-03-25 05:16:24 UTC
Hello:

> ... two different hardwares ...
> ... BIOS code may also be very different.
I understand.

But the BIOS codes share the same disease:
Lack of ACPI compliance by the OEMs.
Maybe too much 'cut and paste' when writing the code?
Totally unacceptable for the likes of Sun Microsystems. 

> ... very likely to be different problems
> ...symptoms are similar.
Actually, the symptoms are identical.

> ... want to focus on one issue in one bug report ...
Makes sense.

> If I found they are the same root cause ...
> ... will mark these two bug reports as duplicates.
> Does this sound good for you?
Sure.
No problem.

After all, *you* are the one who understands all this. 8^D

I post back in ~30 days or sooner if blacklisting the `acpi_cpufreq` module does not have any effect.

That may give you some additional data to consider.

Thanks for your input.

Best,

JHM
Comment 27 Julius Henry Marx 2021-03-25 15:31:25 UTC
Hello:

> ... back in ~30 days or sooner ...
> ... may give you some additional data ...
>

Well, that *was* soon. 8^7

I just had another shutdown episode.

We now know for certain that blacklisting the `acpi_cpufreq` module does not prevent the bad shutdown I am experiencing.

> ... please open a new bug report, and give detailed descriptions of the
> problem you encountered?

> ... with acpidump and dmesg output after boot. 

Will do ASAP.

Best,

JHM
Comment 28 Julius Henry Marx 2021-03-25 18:05:25 UTC
Hello:

> ... please open a new bug report, and give detailed descriptions of the
> problem you encountered?

> ... with acpidump and dmesg output after boot. 

Done.
https://bugzilla.kernel.org/show_bug.cgi?id=212443

Best,

JHM
Comment 29 Daniele 2021-03-27 08:42:18 UTC
Hi,
as written in a previous message, with the NOT patched DSDT I never had the shutdown issue with Unraid (a paid os, based on slackware system); this is the only linux operating system that didn't show the shutdown issue.
I don't know if it can be of any help, but I'm attaching the kernel config files for both systems (Unraid and Manjaro), maybe it can be of some help.

P.S. Unraid (6.8.3) is based on kernel v. 4.19.107: as written I tried the same version in Manjaro without solving the shutdown issue.
Comment 30 Daniele 2021-03-27 08:43:24 UTC
Created attachment 296085 [details]
Config kernel file for Manjaro

Config kernel file for Manjaro, kernel 5.10.26-1
Comment 31 Daniele 2021-03-27 08:44:27 UTC
Created attachment 296087 [details]
Config kernel file for Unraid

Config kernel file for Unraid, kernel 4.19.107
Comment 32 Daniele 2021-04-14 07:39:56 UTC
(In reply to Zhang Rui from comment #13)
> I'm not sure if there are some side effect with this solution, but at least
> you didn't observe any, right?

I just found (I think) one minor drawback (at least for me).
In bios there's a setting to set what to do in case of a power failure: it was set to "restore to last state".
Since the modified dsdt does nothing on shutdown, I think the bios doesn't save the S5 somewhere, so when you cut off the power (and the server is shutdown) and you connect again the power the server starts automatically.
Not a problem for me, because I don't want the system to boot automatically after a power failure, I just changed in bios from "restore to last state" to "power off".
Comment 33 Zhang Rui 2021-06-02 07:43:02 UTC
(In reply to Daniele from comment #32)
> (In reply to Zhang Rui from comment #13)
> > I'm not sure if there are some side effect with this solution, but at least
> > you didn't observe any, right?
> 
> I just found (I think) one minor drawback (at least for me).
> In bios there's a setting to set what to do in case of a power failure: it
> was set to "restore to last state".
> Since the modified dsdt does nothing on shutdown, I think the bios doesn't
> save the S5 somewhere, so when you cut off the power (and the server is
> shutdown) and you connect again the power the server starts automatically.
> Not a problem for me, because I don't want the system to boot automatically
> after a power failure, I just changed in bios from "restore to last state"
> to "power off".

Yeah, this is not something unexpected when we skip the _PTS method during shutdown.

For this issue, maybe you need to live with this DSDT workaround.
And I'd prefer we close this bug as I don't think there is anything we can do in kernel, for this firmware issue. What do you think?

Note You need to log in before you can comment on or make changes to this bug.