Hello, I am having migration problems and would appreciate very much help in troubleshooting this further. I have installed two wheezy hosts (qemu-1.1.2) using glusterfs as backend storage and are running about 12 debian wheezy guests. Node A has newer hardware (AMD FX(tm)-8320 Eight-Core Processor) and node B is older (AMD Phenom(tm) II X6 1055T Processor). I am using libvirt and when migrating hosts from Node A to node B the always freeze indicating high cpu utilization in virt-manager for example. Migrations from Node B to Node A always works. I can start all gests on both hosts without problems. To rule out glusterfs as the problem I have also used nfs as backend storage with the same results. I have installed a freebsd-9.1 guest to rule out the guests linux kernel as the likely problem. The guests also freeze if they are in grub or the freebsd bootloader while being migrated. I have tried using this kernel parameter processor.max_cstate=1 and using changing clocksource to acpi_pm (defaults to kvm-clock). Neither of these had any effect on my problem. I have tried disabling the balloon driver and that didn't help either. Both hosts and all guests use ntp so the time is synced. At one point I reinstalled both machines as ubuntu 13.04 to get newer versions of qemu (1.4) and libvirt but the exact same problem was there also, as I prefer debian I went back to wheezy. -no-kvm-irqchip and -no-kvm-pit doesn't help either. Using -no-kvm the migration works both ways. My gut feeling is that it has something to do with the different cpu's, but I read that it was possible to migrate between intel and amd I guess this shouldn't be a problem. Anyone have any ide what could be the problem here? Best Regards Magnus Diff between cpuinfo of linux guest when booted on each host so no cpu flags differ in the guest. I have tried with Opteron_G2, qemu64 and kvm64 cpus all causes the guest to freeze. root@lazy:~# diff -Nru iron1-cpuinfo.txt iron2-cpuinfo.txt --- iron1-cpuinfo.txt 2013-09-19 20:59:24.407659093 +0200 +++ iron2-cpuinfo.txt 2013-09-19 21:01:03.326022357 +0200 @@ -5,14 +5,14 @@ model name : QEMU Virtual CPU version 1.1.2 stepping : 3 microcode : 0x1000065 -cpu MHz : 3511.722 +cpu MHz : 2812.460 cache size : 512 KB fpu : yes fpu_exception : yes cpuid level : 4 wp : yes flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm up nopl pni cx16 popcnt hypervisor lahf_lm svm abm sse4a -bogomips : 7023.44 +bogomips : 5624.92 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 Node A cpuinfo processor : 7 vendor_id : AuthenticAMD cpu family : 21 model : 2 model name : AMD FX(tm)-8320 Eight-Core Processor stepping : 0 microcode : 0x600081f cpu MHz : 1400.000 cache size : 2048 KB physical id : 0 siblings : 8 core id : 7 cpu cores : 4 apicid : 23 initial apicid : 7 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1 bogomips : 7023.38 TLB size : 1536 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro Node B cpuinfo processor : 5 vendor_id : AuthenticAMD cpu family : 16 model : 10 model name : AMD Phenom(tm) II X6 1055T Processor stepping : 0 microcode : 0x10000dc cpu MHz : 800.000 cache size : 512 KB physical id : 0 siblings : 6 core id : 5 cpu cores : 6 apicid : 5 initial apicid : 5 fpu : yes fpu_exception : yes cpuid level : 6 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_save pausefilter bogomips : 5624.50 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate cpb Libvirt starts the guests like this: LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-1.1 -enable-kvm -m 512 -smp 2,sockets=2,cores=1,threads=1 -name jefster -uuid f94a85d4-629e-bdba-b585-282638526800 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/jefster.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/vmimages/gv0/jefster.img,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=29 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:61:9b:f3,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 127.0.0.1:5 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 Hosts kernel: Linux iron1 3.10-0.bpo.2-amd64 #1 SMP Debian 3.10.5-1~bpo70+1 (2013-08-11) x86_64 GNU/Linux
Hello, Since the kernel I used was a debian backport I compiled my own kernel from kernel.org 3.11.1 and it has the same problem. Regards Magnus
Hello, For testing I took an old machine and installed it like the other two except no need for local storage so no glusterfs. gluster volumes are mounted from the other servers. Migration Node A -> (B OR C) Guest freeze Node (B OR C) -> Node A works Node B <-> C Works Any debugging or troubleshooting steps I can do I am happy for any pointers in the right direction. Regards Magnus
Can anyone help with troubleshooting this further? Regards Magnus
So new to old always hangs? What "info status" shows when guest freeze? Can you try latest qemu?
Hello, It says this, I haven't found any easy way to upgrade to later versions of qemu in debian but I will try compiling it from source. I did try 1.4.0 and 1.5.0 in ubuntu before reverting back to my prefered distro debian. virsh # qemu-monitor-command --hmp lazy "info status" VM status: running Thanks Magnus
Sorry missed to answer your question: So new to old always hangs? Yes it always hangs.
Can you copy here the cpuinfo of the two hosts?
Node A processor : 0 vendor_id : AuthenticAMD cpu family : 21 model : 2 model name : AMD FX(tm)-8320 Eight-Core Processor stepping : 0 microcode : 0x600081f cpu MHz : 1400.000 cache size : 2048 KB physical id : 0 siblings : 8 core id : 0 cpu cores : 4 apicid : 16 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1 bogomips : 7023.50 TLB size : 1536 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro processor : 1 vendor_id : AuthenticAMD cpu family : 21 model : 2 model name : AMD FX(tm)-8320 Eight-Core Processor stepping : 0 microcode : 0x600081f cpu MHz : 1400.000 cache size : 2048 KB physical id : 0 siblings : 8 core id : 1 cpu cores : 4 apicid : 17 initial apicid : 1 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1 bogomips : 7023.50 TLB size : 1536 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro processor : 2 vendor_id : AuthenticAMD cpu family : 21 model : 2 model name : AMD FX(tm)-8320 Eight-Core Processor stepping : 0 microcode : 0x600081f cpu MHz : 1400.000 cache size : 2048 KB physical id : 0 siblings : 8 core id : 2 cpu cores : 4 apicid : 18 initial apicid : 2 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1 bogomips : 7023.50 TLB size : 1536 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro processor : 3 vendor_id : AuthenticAMD cpu family : 21 model : 2 model name : AMD FX(tm)-8320 Eight-Core Processor stepping : 0 microcode : 0x600081f cpu MHz : 1400.000 cache size : 2048 KB physical id : 0 siblings : 8 core id : 3 cpu cores : 4 apicid : 19 initial apicid : 3 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1 bogomips : 7023.50 TLB size : 1536 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro processor : 4 vendor_id : AuthenticAMD cpu family : 21 model : 2 model name : AMD FX(tm)-8320 Eight-Core Processor stepping : 0 microcode : 0x600081f cpu MHz : 1400.000 cache size : 2048 KB physical id : 0 siblings : 8 core id : 4 cpu cores : 4 apicid : 20 initial apicid : 4 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1 bogomips : 7023.50 TLB size : 1536 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro processor : 5 vendor_id : AuthenticAMD cpu family : 21 model : 2 model name : AMD FX(tm)-8320 Eight-Core Processor stepping : 0 microcode : 0x600081f cpu MHz : 1400.000 cache size : 2048 KB physical id : 0 siblings : 8 core id : 5 cpu cores : 4 apicid : 21 initial apicid : 5 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1 bogomips : 7023.50 TLB size : 1536 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro processor : 6 vendor_id : AuthenticAMD cpu family : 21 model : 2 model name : AMD FX(tm)-8320 Eight-Core Processor stepping : 0 microcode : 0x600081f cpu MHz : 1400.000 cache size : 2048 KB physical id : 0 siblings : 8 core id : 6 cpu cores : 4 apicid : 22 initial apicid : 6 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1 bogomips : 7023.50 TLB size : 1536 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro processor : 7 vendor_id : AuthenticAMD cpu family : 21 model : 2 model name : AMD FX(tm)-8320 Eight-Core Processor stepping : 0 microcode : 0x600081f cpu MHz : 1400.000 cache size : 2048 KB physical id : 0 siblings : 8 core id : 7 cpu cores : 4 apicid : 23 initial apicid : 7 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1 bogomips : 7023.50 TLB size : 1536 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro
Node B processor : 0 vendor_id : AuthenticAMD cpu family : 16 model : 10 model name : AMD Phenom(tm) II X6 1055T Processor stepping : 0 microcode : 0x10000dc cpu MHz : 800.000 cache size : 512 KB physical id : 0 siblings : 6 core id : 0 cpu cores : 6 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 6 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_save pausefilter bogomips : 5625.25 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate cpb processor : 1 vendor_id : AuthenticAMD cpu family : 16 model : 10 model name : AMD Phenom(tm) II X6 1055T Processor stepping : 0 microcode : 0x10000dc cpu MHz : 800.000 cache size : 512 KB physical id : 0 siblings : 6 core id : 1 cpu cores : 6 apicid : 1 initial apicid : 1 fpu : yes fpu_exception : yes cpuid level : 6 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_save pausefilter bogomips : 5625.25 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate cpb processor : 2 vendor_id : AuthenticAMD cpu family : 16 model : 10 model name : AMD Phenom(tm) II X6 1055T Processor stepping : 0 microcode : 0x10000dc cpu MHz : 800.000 cache size : 512 KB physical id : 0 siblings : 6 core id : 2 cpu cores : 6 apicid : 2 initial apicid : 2 fpu : yes fpu_exception : yes cpuid level : 6 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_save pausefilter bogomips : 5625.25 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate cpb processor : 3 vendor_id : AuthenticAMD cpu family : 16 model : 10 model name : AMD Phenom(tm) II X6 1055T Processor stepping : 0 microcode : 0x10000dc cpu MHz : 800.000 cache size : 512 KB physical id : 0 siblings : 6 core id : 3 cpu cores : 6 apicid : 3 initial apicid : 3 fpu : yes fpu_exception : yes cpuid level : 6 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_save pausefilter bogomips : 5625.25 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate cpb processor : 4 vendor_id : AuthenticAMD cpu family : 16 model : 10 model name : AMD Phenom(tm) II X6 1055T Processor stepping : 0 microcode : 0x10000dc cpu MHz : 800.000 cache size : 512 KB physical id : 0 siblings : 6 core id : 4 cpu cores : 6 apicid : 4 initial apicid : 4 fpu : yes fpu_exception : yes cpuid level : 6 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_save pausefilter bogomips : 5625.25 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate cpb processor : 5 vendor_id : AuthenticAMD cpu family : 16 model : 10 model name : AMD Phenom(tm) II X6 1055T Processor stepping : 0 microcode : 0x10000dc cpu MHz : 800.000 cache size : 512 KB physical id : 0 siblings : 6 core id : 5 cpu cores : 6 apicid : 5 initial apicid : 5 fpu : yes fpu_exception : yes cpuid level : 6 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_save pausefilter bogomips : 5625.25 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate cpb
Node C processor : 0 vendor_id : AuthenticAMD cpu family : 16 model : 2 model name : AMD Phenom(tm) 8550 Triple-Core Processor stepping : 3 microcode : 0x1000065 cpu MHz : 1100.000 cache size : 512 KB physical id : 0 siblings : 3 core id : 0 cpu cores : 3 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs hw_pstate npt lbrv svm_lock bogomips : 4420.10 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate processor : 1 vendor_id : AuthenticAMD cpu family : 16 model : 2 model name : AMD Phenom(tm) 8550 Triple-Core Processor stepping : 3 microcode : 0x1000065 cpu MHz : 1100.000 cache size : 512 KB physical id : 0 siblings : 3 core id : 1 cpu cores : 3 apicid : 1 initial apicid : 1 fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs hw_pstate npt lbrv svm_lock bogomips : 4420.10 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate processor : 2 vendor_id : AuthenticAMD cpu family : 16 model : 2 model name : AMD Phenom(tm) 8550 Triple-Core Processor stepping : 3 microcode : 0x1000065 cpu MHz : 1100.000 cache size : 512 KB physical id : 0 siblings : 3 core id : 2 cpu cores : 3 apicid : 2 initial apicid : 2 fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs hw_pstate npt lbrv svm_lock bogomips : 4420.10 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate
This is XSAVE -> non-XSAVE migration. Alexander Graf reported to me on IRC that the XSAVE patches we recently committed to kvm/next fixed this for him. xstate_bv is saved as 7 and this check fails on the destination: if (xstate_bv & ~XSTATE_FPSSE) return -EINVAL; I think it is because your CPU has xsave but not xsaveopt. However, those patches introduced regressions that have to be fixed. I'll look at those soon, but your bug should get fixed in 3.13.
Perfect. Thanks for the response. I will try 3.13 as soon as it is released. If you need any beta testing I am available.
(In reply to Magnus Boden from comment #12) > Perfect. Thanks for the response. > > I will try 3.13 as soon as it is released. > > If you need any beta testing I am available. Can you try current kvm.git next branch? The bugs that is there may not affect your case.
I will try that. I am going away tonight but I will try it tomorrow and report back tomorrow evening. Thanks for the support Magnus
I have no upgraded all three hosts with the kernel from git://git.kernel.org/pub/scm/virt/kvm/kvm.git Using the branch next, last commit on that tree was: commit d570142674890fe10b3d7d86aa105e3dfce1ddfa Merge: f2e1066 a7265fb Author: Gleb Natapov <gleb@redhat.com> Date: Wed Oct 16 15:30:32 2013 +0300 Merge tag 'kvm-arm-for-3.13-1' of git://git.linaro.org/people/cdall/linux-kvm-arm into next Updates for KVM/ARM including cpu=host and Cortex-A7 support Now the symptoms are the other way around. Migrations B or C to A guests freeze (both freebsd and linux tested). Migrations A to B or C works. Regards Magnus
Yes, that's expected. You also need the patch I send on Thursday, that you were CCed on (KVM: x86: fix KVM_SET_XCRS for CPUs that do not support XSAVE).
Sorry I missed the patch. I have patched my kernel now and can report that migrations work in every direction between my three hosts. Thanks very much for the support Magnus