Bug 206877

Summary: Nested virt on AMD (and probably older Intel) doesn't work with ignore_msrs=Y on L0 (fails with UMWAIT error)
Product: Virtualization Reporter: Stefan Reiter (s.reiter)
Component: kvmAssignee: virtualization_kvm
Status: NEW ---    
Severity: normal CC: maximlevitsky
Priority: P1    
Hardware: i386   
OS: Linux   
Kernel Version: 5.6.0-rc6 Subsystem:
Regression: No Bisected commit-id:

Description Stefan Reiter 2020-03-18 10:22:28 UTC
I'm not sure if this is a supported configuration, but since the commit mentioned below nested virtualization on AMD when the host has 'ignore_msrs=Y' is broken. QEMU fails with:

kvm: error: failed to set MSR 0xe1 to 0x0
kvm: /qemu/target/i386/kvm.c:2947: kvm_put_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.

If this is supposed to work, it's a regression from 6e3ba4abcea5 ("KVM: vmx: Emulate MSR IA32_UMWAIT_CONTROL"), I can confirm that reverting this commit for the guest kernel makes everything work again. Ignoring UMWAIT in QEMU (kvm_get_supported_msrs) does the trick too.

I *think* this happens since MSR_IA32_UMWAIT_CONTROL (in msrs_to_save_all) is added to the guest CPUID with the only condition being that 'rdmsr_safe' in 'kvm_init_msr_list' succeeds - which it does, since the host ignores it. However, since the CPU doesn't actually support UMWAIT (in my case since it's an AMD chip, but I suppose the same happens on older Intel ones) the MSR set for the L2 guest fails.
Comment 1 Maxim Levitsky 2020-07-02 13:39:45 UTC
 I fixed this upstream, the patch should be in 5.8 kernel I think