Bug 55421 - igb VF can't work in KVM guest
Summary: igb VF can't work in KVM guest
Alias: None
Product: Virtualization
Classification: Unclassified
Component: kvm (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: virtualization_kvm
Depends on:
Reported: 2013-03-18 15:05 UTC by Jay Ren
Modified: 2013-04-22 14:14 UTC (History)
2 users (show)

See Also:
Kernel Version: 3.9.0-rc1
Regression: No
Bisected commit-id:

host dmesg (83.16 KB, text/plain)
2013-03-18 15:11 UTC, Jay Ren
guest dmesg (18.85 KB, text/plain)
2013-03-18 15:12 UTC, Jay Ren
lspci info of the igbvf in guest (1.68 KB, text/plain)
2013-03-18 15:12 UTC, Jay Ren

Description Jay Ren 2013-03-18 15:05:57 UTC
Host OS (ia32/ia32e/IA64):ia32e
Guest OS (ia32/ia32e/IA64):ia32e
Guest OS Type (Linux/Windows):Linux
kvm.git next branch Commit:5da596078f915a62e39a20e582308eab91b88c9a
qemu-kvm uq/master branch Commit:3e41a753551a906dd9ed66fb0fc34167a6af3ba0
Host Kernel Version:3.9.0-rc1
Hardware:SNB-EP, WSM-EP

Bug detailed description:
After assigning a igb VF to a guest or hot-plugging a VF to a guest, the VF cannot work in the guest.
1. The VF of Intel 82599 NIC (ixgbe) can work in guest
2. The PF of Intel 82576 NIC or Intel I350 NIC (using igb driver) can work in guest

This should be a kvm bug.
kvm      + qemu-kvm   =  result
5da59607 + 3e41a753   = bad
3ab66e8a + 3e41a753   = good

Reproduce steps:
1.start up a KVM host
2.assign a VF to a guest in qemu commandline: 
qemu-system-x86_64 --enable-kvm -m 1024 -smp 2 -device pci-assign,host=05:10.0 -net none -hda rhel6u3.img

Current result:
the VF of cannot work in guest 

Expected result:
the VF can work in the guest
Comment 1 Jay Ren 2013-03-18 15:11:16 UTC
Created attachment 95741 [details]
host dmesg
Comment 2 Jay Ren 2013-03-18 15:12:02 UTC
Created attachment 95751 [details]
guest dmesg

in the dmesg in guest:
igbvf 0000:00:03.0: irq 26 for MSI/MSI-X
igbvf 0000:00:03.0: Invalid MAC Address: 00:00:00:00:00:00
igbvf: probe of 0000:00:03.0 failed with error -5
Comment 3 Jay Ren 2013-03-18 15:12:33 UTC
Created attachment 95761 [details]
lspci info of the igbvf in guest
Comment 4 Alex Williamson 2013-03-21 21:29:09 UTC
Further bisected to:

commit 5ac6f91d39e0884813dc010e14552143cd1d0d8b
Author: Mitch A Williams <mitch.a.williams@intel.com>
Date:   Fri Jan 18 08:57:20 2013 +0000

    igb: Don't give VFs random MAC addresses
    If the user has not assigned a MAC address to a VM, then don't give it a
    random one. Instead, just give it zeros and let it figure out what to do
    with them.
    Signed-off-by: Mitch Williams <mitch.a.williams@intel.com>
    CC: Andy Gospodarek <andy@greyhouse.net>
    CC: Stefan Assmann <sassmann@kpanic.de>
    Tested-by: Aaron Brown <aaron.f.brown@intel.com>
    Tested-by: Stefan Assmann <sassmann@redhat.com>
    Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>

So, for whatever reason we no longer assign a random MAC address when using the device in a VM (but we do still use one if attached to igbvf in the host).  I expect we'll eventually see this on all the Intel SR-IOV NICs.  The solution is to use the ip command to assign the VF a valid MAC address prior to using it with KVM.  I'll let those who made the change defend it further if they wish.
Comment 5 Jay Ren 2013-04-22 14:14:26 UTC
As this is the latest igb drive change, I'll close this bug.

There are two methods to work around or fix this issue.
1. update igbvf driver or kernel of the guest
2. use ip command to set VF's MAC before assigning to a guest
(e.g. ip link set eth0 vf 0 mac 00:1E:67:65:93:01 )

Note You need to log in before you can comment on or make changes to this bug.