On launch of VM these messages are generated. Filing this report since hinted at a reason for improvement such as for patch 7670771. First ever report, sorry if violating any good habits. pvx kernel: [ 77.102686] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x3a pvx kernel: [ 77.102712] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0xd90 pvx kernel: [ 77.102736] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x570 pvx kernel: [ 77.102755] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x571 pvx kernel: [ 77.102773] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x572 pvx kernel: [ 77.102792] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x560 pvx kernel: [ 77.102811] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x561 pvx kernel: [ 77.102829] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x580 pvx kernel: [ 77.102847] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x581 pvx kernel: [ 77.102866] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x582 pvx kernel: [ 77.102686] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x3a pvx kernel: [ 77.102712] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0xd90 pvx kernel: [ 77.102736] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x570 pvx kernel: [ 77.102755] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x571 pvx kernel: [ 77.102773] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x572 pvx kernel: [ 77.102792] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x560 pvx kernel: [ 77.102811] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x561 pvx kernel: [ 77.102829] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x580 pvx kernel: [ 77.102847] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x581 pvx kernel: [ 77.102866] kvm [3464]: vcpu1, guest rIP: 0xffffffff8666b9e4 ignored rdmsr: 0x582
iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=DD:00.0 address=0x7fb99eeb0] iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=DD:00.0 address=0x7fb99ef10] iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=DD:00.0 address=0x7fb99ef50] iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=DD:00.0 address=0x7fb99ef70] vfio-pci 0000:28:00.0: timed out waiting for pending transaction; performing function level reset anyway device=DD < anonimised
More of the same [ 507.188509] vfio-pci 0000:17:00.0: vfio_ecap_init: hiding ecap 0x19@0x270 [ 507.188524] vfio-pci 0000:17:00.0: vfio_ecap_init: hiding ecap 0x1b@0x2d0 [ 513.661635] kvm [19941]: vcpu0, guest rIP: 0xfffff8000afda0aa ignored rdmsr: 0xc0011023 [ 513.661644] kvm [19941]: vcpu0, guest rIP: 0xfffff8000afda0bf ignored wrmsr: 0xc0011023 data 0x100 [ 513.703575] kvm [19941]: vcpu1, guest rIP: 0xfffff8000afda0aa ignored rdmsr: 0xc0011023 [ 513.703588] kvm [19941]: vcpu1, guest rIP: 0xfffff8000afda0bf ignored wrmsr: 0xc0011023 data 0x100 [ 513.704769] kvm [19941]: vcpu2, guest rIP: 0xfffff8000afda0aa ignored rdmsr: 0xc0011023 [ 513.704782] kvm [19941]: vcpu2, guest rIP: 0xfffff8000afda0bf ignored wrmsr: 0xc0011023 data 0x100 [ 513.705948] kvm [19941]: vcpu3, guest rIP: 0xfffff8000afda0aa ignored rdmsr: 0xc0011023 [ 513.705961] kvm [19941]: vcpu3, guest rIP: 0xfffff8000afda0bf ignored wrmsr: 0xc0011023 data 0x100
The "ignored {rd,wr}msr" output is expected when kvm.ignore_msrs=1, which is not the default. Is that enabled for any particular reason? Or maybe a better question is, what is it that you're trying to do and what, if any, errors are occurring? E.g. in the first trace, it looks like you're advertising an Intel vCPU to the guest on top of AMD hardware. That can be made to work, but it's not recommended. Ignoring MSRs in that scenario is almost guaranteed to cause weirdness.
These message are observed when starting a Proxmox VE 6.2 VM guest with GPU passthrough. Current configuration is at the point there are not other error messages visible related to VM activity. Available to respond to and provide for actions to improve documenting these messages etc. The vcpu0, guest rIP messages persist while a vm is running. Stopping and staring a VM results in no screen. messages however now show (repeat kvm ... msr messages, repeeat block of kvm, vcpu... rIP messages) followed by ecap messages. [ 2762.744734] kvm_get_msr_common: 275 callbacks suppressed [ 2762.744738] kvm [19941]: vcpu2, guest rIP: 0xfffff8001123174b ignored rdmsr: 0xc0010064 [ 2762.744775] kvm [19941]: vcpu2, guest rIP: 0xfffff8001123174b ignored rdmsr: 0xc0010065 [ 2762.744853] kvm [19941]: vcpu2, guest rIP: 0xfffff8001123174b ignored rdmsr: 0xc0010066 [ 2762.744885] kvm [19941]: vcpu2, guest rIP: 0xfffff8001123174b ignored rdmsr: 0xc0010067 [ 2762.744917] kvm [19941]: vcpu2, guest rIP: 0xfffff8001123174b ignored rdmsr: 0xc0010068 [ 2762.744948] kvm [19941]: vcpu2, guest rIP: 0xfffff8001123174b ignored rdmsr: 0xc0010069 [ 2762.744983] kvm [19941]: vcpu2, guest rIP: 0xfffff8001123174b ignored rdmsr: 0xc001006a [ 2762.745014] kvm [19941]: vcpu2, guest rIP: 0xfffff8001123174b ignored rdmsr: 0xc001006b [ 2762.745044] kvm [19941]: vcpu2, guest rIP: 0xfffff8001123174b ignored rdmsr: 0xc0010293 [ 2762.750132] kvm [19941]: vcpu2, guest rIP: 0xfffff8001123174b ignored rdmsr: 0xe8 [ 2844.034627] vfio-pci 0000:17:00.0: vfio_ecap_init: hiding ecap 0x19@0x270 [ 2844.034641] vfio-pci 0000:17:00.0: vfio_ecap_init: hiding ecap 0x1b@0x2d0 [ 2845.308346] vfio-pci 0000:17:00.1: vfio_bar_restore: reset recovery - restoring BARs [ 2845.324311] vfio-pci 0000:17:00.0: vfio_bar_restore: reset recovery - restoring BARs [ 2845.566461] vfio-pci 0000:17:00.0: vfio_bar_restore: reset recovery - restoring BARs [ 2845.569777] vfio-pci 0000:17:00.1: vfio_bar_restore: reset recovery - restoring BARs [ 2845.599693] vfio-pci 0000:17:00.0: vfio_bar_restore: reset recovery - restoring BARs [ 2845.602566] vfio-pci 0000:17:00.1: vfio_bar_restore: reset recovery - restoring BARs [ 2845.625243] vfio-pci 0000:17:00.0: vfio_bar_restore: reset recovery - restoring BARs [ 2845.627436] vfio-pci 0000:17:00.1: vfio_bar_restore: reset recovery - restoring BARs [ 2845.659282] vfio-pci 0000:17:00.0: vfio_bar_restore: reset recovery - restoring BARs [ 2845.665352] vfio-pci 0000:17:00.0: vfio_bar_restore: reset recovery - restoring BARs [ 2845.673673] vfio-pci 0000:17:00.0: vfio_bar_restore: reset recovery - restoring BARs [ 2845.673924] vfio-pci 0000:17:00.0: vfio_bar_restore: reset recovery - restoring BARs [ 2845.673948] vfio-pci 0000:17:00.0: vfio_bar_restore: reset recovery - restoring BARs [ 2845.674104] vfio-pci 0000:17:00.0: vfio_bar_restore: reset recovery - restoring BARs [ 2845.674870] vfio-pci 0000:17:00.1: vfio_bar_restore: reset recovery - restoring BARs [ 2845.675040] vfio-pci 0000:17:00.1: vfio_bar_restore: reset recovery - restoring BARs [ 2845.675063] vfio-pci 0000:17:00.1: vfio_bar_restore: reset recovery - restoring BARs [ 2845.675210] vfio-pci 0000:17:00.1: vfio_bar_restore: reset recovery - restoring BARs [ 2845.708785] vfio-pci 0000:17:00.0: vfio_bar_restore: reset recovery - restoring BARs [ 2845.710674] vfio-pci 0000:17:00.1: vfio_bar_restore: reset recovery - restoring BARs [ 2845.727404] vfio-pci 0000:17:00.0: vfio_bar_restore: reset recovery - restoring BARs [ 2845.727598] vfio-pci 0000:17:00.0: vfio_bar_restore: reset recovery - restoring BARs [ 2845.728296] vfio-pci 0000:17:00.1: vfio_bar_restore: reset recovery - restoring BARs [ 2845.728484] vfio-pci 0000:17:00.1: vfio_bar_restore: reset recovery - restoring BARs [ 2845.764636] vfio-pci 0000:17:00.0: vfio_bar_restore: reset recovery - restoring BARs [ 2845.765256] vfio-pci 0000:17:00.1: vfio_bar_restore: reset recovery - restoring BARs [ 2845.933638] vfio-pci 0000:17:00.0: vfio_bar_restore: reset recovery - restoring BARs now the VM no longer displays the desktop.
(In reply to Sean Christopherson from comment #3) > The "ignored {rd,wr}msr" output is expected when kvm.ignore_msrs=1, which is > not the default. Is that enabled for any particular reason? Or maybe a > better question is, what is it that you're trying to do and what, if any, > errors are occurring? > > E.g. in the first trace, it looks like you're advertising an Intel vCPU to > the guest on top of AMD hardware. That can be made to work, but it's not > recommended. Ignoring MSRs in that scenario is almost guaranteed to cause > weirdness. Thanks Sean. I am cleaning up this config. Will check. This is indeed a Ryzen CPU with Qemu/Proxmox on top of it. For GPU Passthrough to a VM the kvm.ignore_msrs=1 is often recommended. I brief test shows the VM is not working properly without this parameter, even when booting the VM with CPU:host. br, Joris
These i get with kvm ignore.msrs=1 enabled and CPU=host [ 46.512153] kvm [7509]: vcpu0, guest rIP: 0xfffff8039588c0aa ignored rdmsr: 0xc0011023 [ 46.512161] kvm [7509]: vcpu0, guest rIP: 0xfffff8039588c0bf ignored wrmsr: 0xc0011023 data 0x100 [ 46.555401] kvm [7509]: vcpu1, guest rIP: 0xfffff8039588c0aa ignored rdmsr: 0xc0011023 [ 46.555406] kvm [7509]: vcpu1, guest rIP: 0xfffff8039588c0bf ignored wrmsr: 0xc0011023 data 0x100 [ 46.556809] kvm [7509]: vcpu2, guest rIP: 0xfffff8039588c0aa ignored rdmsr: 0xc0011023 [ 46.556823] kvm [7509]: vcpu2, guest rIP: 0xfffff8039588c0bf ignored wrmsr: 0xc0011023 data 0x100 [ 46.558065] kvm [7509]: vcpu3, guest rIP: 0xfffff8039588c0aa ignored rdmsr: 0xc0011023 [ 46.558078] kvm [7509]: vcpu3, guest rIP: 0xfffff8039588c0bf ignored wrmsr: 0xc0011023 data 0x100
With the kvm ignor.msrs parameter removed i do indeed see different results. Now i am back at these messages. Sorry for the ignorance, learning as i read the feedback. [10427.220982] vfio-pci 0000:17:00.0: timed out waiting for pending transaction; performing function level reset anyway [10428.469006] vfio-pci 0000:17:00.0: not ready 1023ms after FLR; waiting [10429.524940] vfio-pci 0000:17:00.0: not ready 2047ms after FLR; waiting [10431.668982] vfio-pci 0000:17:00.0: not ready 4095ms after FLR; waiting [10436.020672] vfio-pci 0000:17:00.0: not ready 8191ms after FLR; waiting [10444.469003] vfio-pci 0000:17:00.0: not ready 16383ms after FLR; waiting [10462.133026] vfio-pci 0000:17:00.0: not ready 32767ms after FLR; waiting [10496.948661] vfio-pci 0000:17:00.0: not ready 65535ms after FLR; giving up
followed by [10956.384880] vfio-pci 0000:17:00.0: Refused to change power state, currently in D3 [10957.109026] vfio-pci 0000:17:00.0: timed out waiting for pending transaction; performing function level reset anyway [10958.356694] vfio-pci 0000:17:00.0: not ready 1023ms after FLR; waiting [10959.412727] vfio-pci 0000:17:00.0: not ready 2047ms after FLR; waiting [10961.588924] vfio-pci 0000:17:00.0: not ready 4095ms after FLR; waiting [10965.940829] vfio-pci 0000:17:00.0: not ready 8191ms after FLR; waiting
eventually, boot time parameters resolved all of these issues, sharing in case they matter. iommu=pt amd_iommu=forced_isolation video=vesafb:off,efifb:off rcu_nocbs=0-15 amdgpu.runpm=0 pci=nobar,noats,big_root_window pcie_acs_override=downstream,multifunction pcie_pme=nomsi pcie_aspm=force pcie_ports=native