Bug 43219

Summary: Fail to hot-remove multiple function devices in Linux VM
Product: Drivers Reporter: Amos Kong (kongjianjun)
Component: PCIAssignee: Amos Kong (kongjianjun)
Status: RESOLVED CODE_FIX    
Severity: normal CC: alan, florian, jbarnes
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: 3.3.0 Subsystem:
Regression: No Bisected commit-id:
Attachments: Dmesg -- acpiphp register
seabios ACPI DSDT (acpi-dsdt.dsl.i ssdt-pcihp.dsl.i)

Description Amos Kong 2012-05-09 15:43:04 UTC
Created attachment 73228 [details]
Dmesg -- acpiphp register

Boot up a Linux VM with 8 pci block devices which
are the 8 functions in one pci slot.
| # qemu-kvm ...
| -drive file=images/u0,if=none,id=drv0,format=qcow2,cache=none \
| -device virtio-blk-pci,drive=drv0,id=v0,multifunction=on,addr=0x03.0 \
| ....
| -drive file=images/u7,if=none,id=drv7,format=qcow2,cache=none \
| -device virtio-blk-pci,drive=drv7,id=v7,multifunction=on,addr=0x03.7 \

Check devices in guest.
| vm)# ls /dev/vd*
|    vda vdb vdc vde vdf vdg vdh
| vm)# lspci |grep block
| 00:03.0 SCSI storage controller: Red Hat, Inc Virtio block device
|    ...
| 00:03.7 SCSI storage controller: Red Hat, Inc Virtio block device
|

Func1~7 still exist in guest after hot-removing the whole slot
through qemu monitor.  
| vm)# lspci |grep block    (00:03.0 disappeared)
| 00:03.1 SCSI storage controller: Red Hat, Inc Virtio block device (rev ff)
|    ...
| 00:03.7 SCSI storage controller: Red Hat, Inc Virtio block device (rev ff)
| vm)# ls /dev/vd*          (vda disappeared)
|    vdb vdc vde vdf vdg vdh
| vm)# mkfs /dev/vdb
|    INFO: task mkfs.ext2:1784 blocked for more than 120 seconds.


expected result:

all functions are removed in guest
Comment 1 Amos Kong 2012-05-09 15:44:04 UTC
Created attachment 73229 [details]
seabios ACPI DSDT (acpi-dsdt.dsl.i ssdt-pcihp.dsl.i)
Comment 2 Amos Kong 2012-05-09 15:45:37 UTC
Posted patch to linux-pci maillist

[PATCH v2] pci: clean all funcs when hot-removing multifunc device
http://marc.info/?l=linux-pci&m=133657729118731&w=2
Comment 3 Amos Kong 2012-05-10 15:29:02 UTC
>>> Hotplug CallTrace:

    int acpiphp_enable_slot(struct acpiphp_slot *slot)
        \_enable_device(slot);
           \_pci_bus_add_devices(bus); // all un-added new devs will be added
                list_for_each_entry(dev, &bus->devices, bus_list) {
                    if (dev->is_added)
                            continue;
                    pci_bus_add_device(dev);
                    device_add(&dev->dev);
                    dev->is_added = 1;
    
    'dev->is_added' is used to track if pci dev is added to bus, all funcs in
    same slot would be added to bus in enable_device(slot). So we need to clean
    all funcs of same slot in disable_device(slot).
Comment 4 Florian Mickler 2012-08-04 19:06:44 UTC
A patch referencing this bug report has been merged in Linux v3.6-rc1:

commit ce29ca3ea40744f24c2b5d88431e8ac566d257cc
Author: Amos Kong <kongjianjun@gmail.com>
Date:   Wed May 23 10:20:35 2012 -0600

    PCI: acpiphp: remove all functions in slot, even without ACPI _EJx