Bug 42779

Created attachment 72393 [details]
Configuration of the guest kernel

Created attachment 72394 [details]
Result of 'registers info' and 'x/30i $eip' after fault

Couldn't attach the trace I recorded of the fault occurring since it's 3 MB compressed with xz, bigger still with other formats. I can email it if it will be useful.

Same problem occurs with qemu-kvm 1.0 from https://launchpad.net/~bderzhavets/+archive/lib-usbredir39:

$ sudo kvm -M pc-1.0 -enable-kvm -m 1024 -drive file=/var/lib/libvirt/images/eve.img,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=21,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:b5:f4:00,bus=pci.0,addr=0x3 -chardev vc,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0 -vga cirrus
kvm: -netdev tap,fd=21,id=hostnet0: TUNGETIFF ioctl() failed: Bad file descriptor
TUNSETOFFLOAD ioctl() failed: Bad file descriptor
kvm: -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:b5:f4:00,bus=pci.0,addr=0x3: pci_add_option_rom: failed to find romfile "pxe-e1000.rom"
KVM internal error. Suberror: 1
emulation failure
EAX=f6810000 EBX=0000003e ECX=0000003e EDX=c00b8000
ESI=c00b8000 EDI=c15b0000 EBP=c15b1f74 ESP=c15b1f58
EIP=c1228905 EFL=00010206 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =007b 00000000 ffffffff 00c0f300 DPL=3 DS   [-WA]
CS =0060 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA]
SS =0068 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
DS =007b 00000000 ffffffff 00c0f300 DPL=3 DS   [-WA]
FS =0000 00000000 ffffffff 00000000
GS =0000 00000000 ffffffff 00000000
LDT=0000 00000000 ffffffff 00000000
TR =0080 c15b6300 0000206b 00008b00 DPL=0 TSS32-busy
GDT=     c15b3000 000000ff
IDT=     c15b2000 000007ff
CR0=80050033 CR2=ffee4000 CR3=01663000 CR4=00000690
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=8e 2b 01 00 00 8b 4d f0 89 f2 8b 45 ec 0f 0d 82 40 01 00 00 <0f> 6f 02 0f 6f 4a 08 0f 6f 52 10 0f 6f 5a 18 0f 7f 00 0f 7f 48 08 0f 7f 50 10 0f 7f 58 18

(In reply to comment #3)
> Couldn't attach the trace I recorded of the fault occurring since it's 3 MB
> compressed with xz, bigger still with other formats. I can email it if it
> will
> be useful.

Can you do "tail -10000" on it and attach it here?

Created attachment 72395 [details]
Last 10k lines of a trace showing the fault

Per Gleb's request

Have you installed trace-cmd before capturing the trace? It failed to parse kvm events. qemu haven't paused the guest after emulation error (looks like a bug), so 'x/30i $eip' output is not useful either. Can you do 'x/30i 0xXXX' where XXX is the address in EIP from register dump you see after instruction emulation failure message (c1228905 in your output from comment #4)?

Not sure what you mean by installing trace-cmd before capturing the trace--I did do that, otherwise I wouldn't have had a trace-cmd to run. The package version is trace-cmd 1.0.3-0ubuntu1 if that helps. I tried running it again against qemu 1.0 (the last one was for qemu 0.14), still contained a bunch of [FAILED TO PARSE] messages.

Attaching the output you requested separately.

Created attachment 72397 [details]
Register state and code disassembly at failure point with qemu 1.0

(In reply to comment #8)
> Not sure what you mean by installing trace-cmd before capturing the trace--I
> did do that, otherwise I wouldn't have had a trace-cmd to run. The package
I meant that if you compile it by yourself you need to run "make install" instead of running it from where it was compiled. Otherwise it does not find plugins. If you install it from package manager this is not relevant of course.

> version is trace-cmd 1.0.3-0ubuntu1 if that helps. I tried running it again
> against qemu 1.0 (the last one was for qemu 0.14), still contained a bunch of
> [FAILED TO PARSE] messages.
> 
Something wrong with ubuntu trace-cmd then. Either it does not have kvm plugin or it is too old.

(In reply to comment #4)
> Code=8e 2b 01 00 00 8b 4d f0 89 f2 8b 45 ec 0f 0d 82 40 01 00 00 <0f> 6f 02
> 0f
> 6f 4a 08 0f 6f 52 10 0f 6f 5a 18 0f 7f 00 0f 7f 48 08 0f 7f 50 10 0f 7f 58 18

This is "movq (%rdx),%mm0". Possibly still unsupported by the emulator.

Can you try disabling CONFIG_FB in your guest kernel?

Indeed the emulator doesn't support mmx yet.  But why is the kernel using mmx instead of sse?

Because the selected CPU type doesn't support SSE? Didn't check the config yet, but I bet that's the reason.

I set the kernel up for an AMD Geode LX800 (CONFIG_MGEODE_LX), which does supposedly support SSE. No idea why it would have built using MMX if that is now deprecated in favor of SSE.

I can try disabling the framebuffer (CONFIG_FB/CONFIG_FB_GEODE/CONFIG_FB_GEODE_LX) for testing purposes; unfortunately this wouldn't work for the real thing, since we need it for the Qt-based user interface.

Background: I'm using KVM as a test platform for code destined for a panel PC using a Geode LX800.

(In reply to comment #15)
> I set the kernel up for an AMD Geode LX800 (CONFIG_MGEODE_LX), which does
> supposedly support SSE. No idea why it would have built using MMX if that is
> now deprecated in favor of SSE.

At least not according to the Linux configuration: MGEODE_LX only sets X86_USE_3DNOW, thus MMX.

As a workaround you can use -cpu blah,-3dnow.  But we'll have to implement mmx movq.

Doesn't seem to help. I used -cpu core2duo,-3dnow (since core2duo is the name that libvirt, my usual launcher for qemu, has been using), failed on the same movq. Also tried -cpu core2duo,-3dnow,-mmx, same results. I'll see if I can build a kernel with the Geode framebuffer driver but without setting the CPU to Geode (thus eliminating the use of MMX).

Please try

  git://git.kernel.org/pub/scm/virt/kvm/kvm.git emulator-movq

this emulates the two movq instructions used.  There may be other unemulated instructions though.

Sorry for the delay, I've been really busy lately. Is this kernel reasonably safe to run on my main machine? I'm not worried about it crashing, but if it takes a filesystem with it...

No guarantees of course, but this is just 3.3-rc5 with a few patches on top.  It should be fine for a short test.

Now it dies in a different place, still with a movq instruction (movq   (%edx),%mm0). I think the difference is because of a configuration change or package update since the last test, because the fault happens at the same address and same instruction under both the patched 3.3rc5 and 3.0.20. Unfortunately I've no idea what the configuration change might have been, and I didn't think to save a VM disk image from the previous tests.

I also got a more recent trace-cmd built and installed, so there is usable trace output now. Still huge, so I'll post the last 10k lines as before.

Created attachment 72788 [details]
Register state and code disassembly at failure point, kernel 3.3rc5 with patches

Created attachment 72789 [details]
Last 10k lines of a trace showing the fault, kernel 3.3rc5 with patches

Strange, looks like the patches did not take effect at all.  The opcode is 0f 6f 02, which should have been decoded as movq.

A patch referencing this bug report has been merged in Linux v3.5-rc1:

commit e59717550e5cf0e7159c5b7af1d1ead35fef49dd
Author: Avi Kivity <avi@redhat.com>
Date:   Mon Apr 9 18:40:03 2012 +0300

    KVM: x86 emulator: implement MMX MOVQ (opcodes 0f 6f, 0f 7f)