Hello, when I try to run a freedos nested in linux kvm guest on kernel 5.2-rc6, I get this message in L1 guest: debian login: [ 13.291265] FS-Cache: Loaded [ 13.295627] 9p: Installing v9fs 9p2000 file system support [ 13.296946] FS-Cache: Netfs '9p' registered for caching [ 19.200271] ------------[ cut here ]------------ [ 19.201265] kernel BUG at arch/x86/kvm/x86.c:358! [ 19.202069] invalid opcode: 0000 [#1] SMP NOPTI [ 19.202850] CPU: 0 PID: 270 Comm: qemu-system-i38 Not tainted 5.2.0-rc6-bughunt+ #1 [ 19.203568] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 19.203568] EIP: kvm_spurious_fault+0x8/0x10 [kvm] [ 19.203568] Code: 08 ff 75 18 ff 75 14 ff 75 10 ff 75 0c 57 56 e8 5e 09 3d fa 83 c4 30 8d 65 f8 5e 5f 5d c3 8d 74 26 00 0f 1f 44 00 00 55 89 e5 <0f> 0b 8d b6 00 00 00 00 0f 1f 44 00 00 55 89 e5 57 56 53 83 ec 04 [ 19.203568] EAX: 0004c000 EBX: 00000000 ECX: 00000000 EDX: 00000663 [ 19.203568] ESI: 00000000 EDI: 00000000 EBP: c0037da4 ESP: c0037da4 [ 19.203568] DS: 007b ES: 007b FS: 0000 GS: 00e0 SS: 0068 EFLAGS: 00210246 [ 19.203568] CR0: 80050033 CR2: 00cd813b CR3: 04e7f820 CR4: 000006f0 [ 19.203568] Call Trace: [ 19.203568] svm_vcpu_run+0x1a5/0x650 [kvm_amd] [ 19.203568] ? kvm_ioapic_scan_entry+0x62/0xe0 [kvm] [ 19.203568] ? kvm_arch_vcpu_ioctl_run+0x596/0x1a80 [kvm] [ 19.203568] ? _cond_resched+0x17/0x30 [ 19.203568] ? kvm_vcpu_ioctl+0x214/0x590 [kvm] [ 19.203568] ? kvm_vcpu_ioctl+0x214/0x590 [kvm] [ 19.203568] ? __bpf_trace_kvm_async_pf_nopresent_ready+0x20/0x20 [kvm] [ 19.203568] ? do_vfs_ioctl+0x9a/0x6c0 [ 19.203568] ? tomoyo_path_chmod+0x20/0x20 [ 19.203568] ? tomoyo_file_ioctl+0x19/0x20 [ 19.203568] ? security_file_ioctl+0x30/0x50 [ 19.203568] ? ksys_ioctl+0x56/0x80 [ 19.203568] ? sys_ioctl+0x16/0x20 [ 19.203568] ? do_fast_syscall_32+0x87/0x1df [ 19.203568] ? entry_SYSENTER_32+0x6b/0xbe [ 19.203568] Modules linked in: 9p fscache ppdev edac_mce_amd kvm_amd 9pnet_virtio kvm bochs_drm 9pnet irqbypass ttm snd_pcm snd_timer snd drm_kms_helper soundcore joydev pcspkr evdev serio_raw drm sg parport_pc parport button ip_tables x_tables autofs4 ext4 crc32c_generic crc16 mbcache jbd2 sr_mod cdrom sd_mod ata_generic ata_piix libata psmouse virtio_pci virtio_ring virtio e1000 scsi_mod i2c_piix4 floppy [ 19.203568] ---[ end trace c0c327b925400cd6 ]--- [ 19.203568] EIP: kvm_spurious_fault+0x8/0x10 [kvm] [ 19.203568] Code: 08 ff 75 18 ff 75 14 ff 75 10 ff 75 0c 57 56 e8 5e 09 3d fa 83 c4 30 8d 65 f8 5e 5f 5d c3 8d 74 26 00 0f 1f 44 00 00 55 89 e5 <0f> 0b 8d b6 00 00 00 00 0f 1f 44 00 00 55 89 e5 57 56 53 83 ec 04 [ 19.203568] EAX: 0004c000 EBX: 00000000 ECX: 00000000 EDX: 00000663 [ 19.203568] ESI: 00000000 EDI: 00000000 EBP: c0037da4 ESP: c3a71dfc [ 19.203568] DS: 007b ES: 007b FS: 0000 GS: 00e0 SS: 0068 EFLAGS: 00210246 [ 19.203568] CR0: 80050033 CR2: 00cd813b CR3: 04e7f820 CR4: 000006f0 svm_vcpu_run+0x1a5 is the vmsave instruction. The bug seems to be only dependent on the L0 host, where version 5.1 works, but 5.2 fails. Linux version of the L1 guest doesn't seem to matter. (I'm running a pae system but didn't actually test if it has anything to do with pae) Any ideas about what could have caused it?
So, I have bisected this and found the culprit to be: commit 8c5fbf1a723107814c20c3f4d6343ab9d694a705 (refs/bisect/bad) Author: KarimAllah Ahmed <karahmed@amazon.de> Date: Thu Jan 31 21:24:40 2019 +0100 KVM/nSVM: Use the new mapping API for mapping guest memory Use the new mapping API for mapping guest memory to avoid depending on "struct page". Signed-off-by: KarimAllah Ahmed <karahmed@amazon.de> Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Created attachment 283501 [details] Patch that allows me to run nested kvm instances again After much looking on the offending commit, which doesn't seem to change any functionality, but just changes the API, I found out the gfns used inside __kvm_map_gfn were in fact far off. Further looking into it I found that these gfns were the result of gfn_to_gpa in some (but not all) places that used the new api. However, what was clearly meant was gpa_to_gfn. So I prepared this patch and tested that it allows to run nested linux instances in kvm successfully on my AMD system.
Already fixed by 8f38302c0be2d