Bug 199035

Summary: BCM2046B1 and hid2hci generates highcpu usage due to udev since 4.14 kernel
Product: Drivers Reporter: Tournier Mathieu (mathieutournier)
Component: BluetoothAssignee: linux-bluetooth (linux-bluetooth)
Status: RESOLVED OBSOLETE    
Severity: high CC: boro, brunofcosouza, f.dittmer, gustavo.willer, jgwphd, kai.heng.feng, lucent, rickfharris, sobik.szymon, stephan, tlinux, ysg
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: 4.14 4.15 Subsystem:
Regression: Yes Bisected commit-id:
Attachments: dmesg
revert-bind_unbind-uevents.patch

Description Tournier Mathieu 2018-03-06 21:09:45 UTC
Created attachment 274593 [details]
dmesg

Hi,
I'm currently running ubuntu 18.04 with a 4.15 kernel and i can observe very high cpu usage to the  systemd-udevd deamon.

removing the rule :
ATTR{bInterfaceClass}=="03", ATTR{bInterfaceSubClass}=="01", ATTR{bInterfaceProtocol}=="02", \
  ATTRS{bDeviceClass}=="00", ATTRS{idVendor}=="413c", ATTRS{bmAttributes}=="e0", \
  RUN+="hid2hci --method=dell --devpath=%p", ENV{HID2HCI_SWITCH}="1"

Solved the high CPU usage eventhough my bluetooth card is not available anymore (as not in hci mode)

It seems that the command hid2hci creates a bind/unbind loop in udev that is looping trying to set the device in hci mode. (that what udevadm monitor seems to show, looping from bind to unbind for the device)

I suspect a0085f2510e8976614ad8f766b209448b385492f introduced a regression (i have not tried to revert it yet).

Please not that there also seem to be a bug in hid2hci.c from bluez l148 :
	if (err == 0) {
		err = -1;
		errno = EALREADY;
	}
Correcting this and recompile bluez desn't solve the issue as cpu usage remains very high.

Using a 4.13 kernel result in a normal CPU usage, this seems a regression in 4.14.

Other people seems to have the same issue, here is a bug report related to this :
https://dev.solus-project.com/T5224

Thanks a lot for your support,

Mathieu Tournier
Comment 1 Pauli Borodulin 2018-05-04 05:57:10 UTC
I have the very same problem using the same distro (Ubuntu 18.04, 64-bit) on Dell Latitude E5400 laptop. Disabling BT from BIOS or removing the aforementioned rule solves the problem but leaves BT unusable.
Comment 2 Eric Shattow 2018-05-09 23:02:06 UTC
Bus 001 Device 009: ID 0a5c:4500 Broadcom Corp. BCM2046B1 USB 2.0 Hub (part of BCM2046 Bluetooth)
Bus 001 Device 004: ID 0a5c:0000 Broadcom Corp. 

Linux zontar 4.16.0-1-amd64 #1 SMP Debian 4.16.5-1 (2018-04-29) x86_64 GNU/Linux

May 09 15:59:00 hostname upowerd[14610]: unhandled action 'bind' on /sys/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.6/1-1.6.2/1-1.6.2:1.0
May 09 15:59:00 hostname upowerd[14610]: unhandled action 'unbind' on /sys/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.6/1-1.6.2/1-1.6.2:1.0
May 09 15:59:00 hostname upowerd[14610]: unhandled action 'bind' on /sys/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.6/1-1.6.2/1-1.6.2:1.0
May 09 15:59:00 hostname upowerd[14610]: unhandled action 'unbind' on /sys/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.6/1-1.6.2/1-1.6.2:1.0
...repeating...

15632 root      20   0  233136 186656   2664 R  94.1%   4.7%  62:14.67 systemd-udevd                                                                                                             
16222 root      20   0   88252   2432   1888 R  35.3%   0.1%  25:21.87 systemd-udevd                                                                                                             

Looks like same problem affects this Dell Precision M6500 laptop.
Comment 3 Florian Dittmer 2018-05-20 07:57:41 UTC
I observed the high cpu usage after upgrading from udev-233 to udev-236/udev-238 on Gentoo Linux. Downgrading back to udev-233 lets me use newer kernels (currently running 4.15.18) without any problems.
Using DELL Latitude E6400 from 2009, with same Broadcom Bluetooth module as mentioned above.
Comment 4 Rick Harris 2018-06-13 16:50:40 UTC
As per https://patchwork.kernel.org/patch/10384111/ editing 97-hid2hci.rules as follows works around the new uevents added to the kernel in https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1455cf8dbfd06aa7651dcfccbadb7a093944ca65

-ACTION=="remove", GOTO="hid2hci_end"
+ACTION!="add", GOTO="hid2hci_end"

Bluetooth now works with kernels above and below 4.14.
Comment 5 Rick Harris 2018-06-14 00:32:13 UTC
Scratch that last comment, on further testing found I was booted into incorrect kernel :/

Editing of 97-hid2hci.rules is not a solution.

Only surefire way of getting bluetooth back and working with kernels >=4.14, was to revert the bogus kernel commit that added the bind/unbind uevents.
Comment 6 Rick Harris 2018-06-14 00:33:52 UTC
Created attachment 276537 [details]
revert-bind_unbind-uevents.patch
Comment 7 Szymon 2018-08-23 15:22:32 UTC
I use ubuntu 18.04 with 4.15 kernel
I have hit the same problem, but with synaptics touchpad.

I have bisected 4.13-4.14 and the problem was in commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (as in the attached patch)

I am unable to use USB bus because udev is bogged down with these bind/unbind events

```
UDEV  [496.168312] bind     /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0 (usb)
KERNEL[496.175932] bind     /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0 (usb)
KERNEL[496.176947] unbind   /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0 (usb)
UDEV  [496.177602] unbind   /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0 (usb)
KERNEL[496.185259] bind     /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0 (usb)
KERNEL[496.185479] unbind   /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0 (usb)
UDEV  [496.186342] bind     /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0 (usb)
KERNEL[496.196813] bind     /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0 (usb)
KERNEL[496.197103] unbind   /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0 (usb)
UDEV  [496.197501] unbind   /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0 (usb)
KERNEL[496.207283] bind     /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0 (usb)
```

```
> cat /sys/bus/usb/devices/2-1.4/2-1.4.2/id*
8162
413c
```

```
> lsusb -d 413c:8162
Bus 002 Device 006: ID 413c:8162 Dell Computer Corp. Integrated Touchpad [Synaptics]
```
Comment 8 Y S Gupta 2018-09-13 05:50:19 UTC
I also observed in my Dell laptop similar to Szymon that it relates to Touchpad.

My workarounds-
Soon after booting, stopping and starting systed-udev eliminates all bind and unbind problems and response drastically improves. I used the following commands in sequence-

sudo systemctl stop systemd-udevd systemd-udevd-kernel.socket systemd-udevd-control.socket

sudo systemctl start systemd-udevd systemd-udevd-kernel.socket systemd-udevd-control.socket

My understanding-
Before all hardware is discovered properly, bind/unbind start executing when no procedures are available and does not get reinitialized. After stopping and starting, it gets all the procedures in place. Probably, it is booting sequence problem.
Comment 9 brunosouza 2018-10-18 17:53:33 UTC
(In reply to Y S Gupta from comment #8)


Same here. I used the commands from Szymon to find out what was going on and got the Synaptics Touchpad, too.

The workaround by Y S Gupta works like a charm. Put it in a startup script and everything is ok. THANK YOU!
Comment 10 Tournier Mathieu 2018-12-22 17:54:47 UTC
Command from Y S Gupta fixed to high cpu load for me.
As this is just a workaround. I suppose this bug should remain open.
Thanks for this, i can now reuse my bluetooth card.
Comment 11 John 2019-01-08 18:42:34 UTC
When I boot up every day without exception, my machine starts up with one of the CPU cores running at 100%. I see lots of posts on other forums (Unbuntu  etc) going back over a year or more blaming touchpads or nvidia or WiFi. Some even say they can't use their thumb drive if it isn't plugged in when they boot. The problem also mimics a defective thumb drive where you plug it in and Ubuntu doesn't see it (because systemd-udevd doesn't have the cycles to process the newly plugged in USB device). Losing one core makes my machine slower but not too noticeably so. I do see much longer boot times and sometime it will hang entirely during boot. I assume a single core or dual core machine will be drastically slowed down or even unusable. When I search I find other non-ubuntu os's complaining about similar problems.

I have 18.10 running on my Dell studio XPS with an AMD® Phenom(tm) ii x4 945 processor × 4 and AMD® Juniper graphics. It's a quad-core 64 bit machine. I have wireless mouse and keyboard for Logitech. I have a pretty vanilla set-up. I DO NOT have a touchpad or nvidia or WiFi!

I can verify that the problem can be managed by stopping and starting systemd-udevd. I used the following commands, suggested in this bug report, in sequence in the terminal which corrects the problem until I boot again. 

sudo systemctl stop systemd-udevd systemd-udevd-kernel.socket systemd-udevd-control.socket

sudo systemctl start systemd-udevd systemd-udevd-kernel.socket systemd-udevd-control.socket

Also the problem will "sometimes" re-appear by plugging in a thumb drive! 

This is a serious kernel problem and can manifest its presence in a number of ways depending on your hardware configuration. 

This is a very very very annoying problem will someone PLEASE fix it soon! ...did I mention that this is a serious problem impacting many people!
Comment 12 John 2019-01-08 19:38:50 UTC
BTW, add to my previous comment an additional symptom:  sometimes my system will "appear to hang" entirely during boot (but it will power down normally by briefly touching the power off button when it "looks like" it is stuck).
Comment 13 gustavo.willer 2019-04-13 14:02:46 UTC
I have the same problem, that John related.

I have dell n5010, with SSD. Run F30 and suffer to connect bluedio t4. 

If I Uninstall the package bluez-hid2hci, the problem of boot cpu with systemd-udevd dissappear. But at same time, I lost the connection of bluetooth device.
Comment 14 John 2019-04-13 17:23:25 UTC
An update: The problem I reported in January went away on its own after a few weeks of having the problem every single day when I booted each morning. I was installing Ubuntu software updates when they became available during that time. There is something going on during boot that causes this problem. 

"my speculation" is that I suspect a timing issue with the way things are initialized during boot up that leads to the problem. Some strange timing issue is occurring which causes systemd-udevd to fall into an infinite loop (continuously looking for something or to be notified about an action completion) and drive one of the processor cores to 100%. When a new kernel was installed or something was done to change initialization activity during boot the problem went away because the timing problem that created it was changed. I believe that some kind of race condition may be at the root of the problem.

From my earlier search on the problem symptoms, it seems that when systemd-udevd isn't working correctly you can get numerous confusing symptoms that mysteriously go away when the something changes timing conditions during boot. Since the problem is in the kernel then it effects other systems besides Ubuntu.
Comment 15 gustavo.willer 2019-04-13 17:34:37 UTC
I fix the problem with this approach: 

https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1759836/comments/70

from "Florian Dittmer (fd81)"


"
Following the instructions mentioned by one user in the comments helped me to solve the cpu load issue with udev-239 and kernel 4.18.17, while Bluetooth still works.

Run the following command:

/lib/systemd/systemd-udevd -D

This should print garbage in endless loop containing ".../97-hid2hci.rules:"

If so, edit /lib/udev/rules.d/97-hid2hci.rules

and add

ACTION=="add",

in front of line mentioned by above command.

In my case, I had to change the following lines in 97-hid2hci.rules from:

ATTR{bInterfaceClass}=="03", ATTR{bInterfaceSubClass}=="01", ATTR{bInterfaceProtocol}=="02", \
  ATTRS{bDeviceClass}=="00", ATTRS{idVendor}=="413c", ATTRS{bmAttributes}=="e0", \
  RUN+="hid2hci --method=dell --devpath=%p", ENV{HID2HCI_SWITCH}="1"

to:

ACTION=="add", ATTR{bInterfaceClass}=="03", ATTR{bInterfaceSubClass}=="01", ATTR{bInterfaceProtocol}=="02", \
  ATTRS{bDeviceClass}=="00", ATTRS{idVendor}=="413c", ATTRS{bmAttributes}=="e0", \
  RUN+="hid2hci --method=dell --devpath=%p", ENV{HID2HCI_SWITCH}="1"

And this fixed the issue (after reboot).
"

from
Comment 16 John 2019-04-13 18:30:30 UTC
These nerd-level suggestions all work but the root cause (the real problem) is not solved and will pop up again. How is the non-nerd supposed to do this? ...would you want your neighbor editing your system files????  Someone needs to fix this permanently!
Comment 17 gustavo.willer 2019-04-13 23:27:55 UTC
I agree with you, its not the best solution. This bug impact many users! At next release of this softwares this bug has to be fixed.

 
But I lost many days to fix this problem, and I'm sharing this 'hard approach' at forums I had visited before. So other people do not waste so much time!
Comment 18 John 2019-04-14 14:22:24 UTC
This problem has been around for a long time and a lot of people have struggled with it. You can find examples such as the following on quite a few help sites, for example

https://askubuntu.com/questions/1028883/ubuntu-18-04-systemd-udevd-uses-high-cpu-conflict-with-wifi
Comment 19 Tournier Mathieu 2022-04-13 15:14:44 UTC
Finnally solved !
5.15 kernel and ubuntu 22.04 upgrade solved totally the issue for me.
I can enable my integrated Dell BT card again :)