Bug 197147 - iwlwifi: 8260: BT coex issues - WIFILNX-1690
Summary: iwlwifi: 8260: BT coex issues - WIFILNX-1690
Status: CLOSED CODE_FIX
Alias: None
Product: Drivers
Classification: Unclassified
Component: Bluetooth (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: DO NOT USE - assign "network-wireless-intel" component instead
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-10-06 09:20 UTC by John Tanner
Modified: 2017-12-06 20:05 UTC (History)
13 users (show)

See Also:
Kernel Version: 4.13.0 through to 4.14rc3
Subsystem:
Regression: Yes
Bisected commit-id:


Attachments
Firmware Core Dump (4.69 MB, application/octet-stream)
2017-10-21 16:23 UTC, John Tanner
Details
dmesg for first two cases (98.54 KB, text/plain)
2017-10-22 12:54 UTC, John Tanner
Details
dmesg for third case (59.92 KB, text/plain)
2017-10-22 12:55 UTC, John Tanner
Details
8260 firmware with debug enabled (2.20 MB, application/octet-stream)
2017-10-22 19:53 UTC, Emmanuel Grumbach
Details
231017-1 Dump (1.20 MB, application/gzip)
2017-10-23 21:45 UTC, John Tanner
Details
231017-2 Dump (1.25 MB, application/gzip)
2017-10-23 21:46 UTC, John Tanner
Details
dmesg for above dumps (73.78 KB, text/plain)
2017-10-23 21:50 UTC, John Tanner
Details
Fix candidate (2.21 MB, application/octet-stream)
2017-11-14 12:09 UTC, Emmanuel Grumbach
Details
Fix candidate 8265 (2.20 MB, application/octet-stream)
2017-11-14 14:03 UTC, Emmanuel Grumbach
Details
Fix candidate with debug logs (2.21 MB, application/octet-stream)
2017-11-15 11:11 UTC, Emmanuel Grumbach
Details
Fix candidate 8265 with debug logs (2.20 MB, application/octet-stream)
2017-11-20 20:14 UTC, Emmanuel Grumbach
Details
dmesg with attachment 260675 loaded (95.97 KB, text/plain)
2017-11-23 15:21 UTC, Robin Lee
Details
Core dumps with latest firmware (937.60 KB, application/x-xz)
2017-11-23 16:33 UTC, John Tanner
Details
firmware dump during reboot with attachment 260675 loaded (340.68 KB, application/x-xz)
2017-11-26 08:15 UTC, Robin Lee
Details
firmware dump during pm suspend/resume with attachment 260675 loaded (747.94 KB, application/x-xz)
2017-11-26 08:17 UTC, Robin Lee
Details
New BT firmware (783.05 KB, application/gzip)
2017-11-27 10:58 UTC, Emmanuel Grumbach
Details
btmon and dmesg logs (16.27 KB, application/x-xz)
2017-11-27 22:13 UTC, John Tanner
Details
dmesg in the F case (102.80 KB, text/plain)
2017-11-28 16:37 UTC, Tim Ruffing
Details

Description John Tanner 2017-10-06 09:20:40 UTC
I'm currently running kernel 4.13.0-12 (and 4.14 RC3) on Ubuntu 17.10 (daily) and have noticed that with my Intel Corporation Wireless 8260 (rev 3a) there appears to be a problem with buetooth and wifi coexistence.

Unless I disable bluetooth and/or wifi I am unable to connect to the opposite to which I have disabled.

Adding:

options iwlwifi bt_coex_active=0 swcrypto=1 11n_disable=8

To:

/etc/modprobe.d/iwlwifi.conf

Solves the problem. This is a reversion to less functionality as on earlier kernels (default kernel on 17.04) this did not occur, but in later kernels (including 4.14 RC3) 

This would appear to be a problem that has previously been identified with earlier versions of the Intel hardware (see https://wiki.debian.org/iwlwifi#Troubleshooting) but not with the 8260 line.

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1721271
Comment 1 Emmanuel Grumbach 2017-10-08 19:20:33 UTC
FWIW: swcrypto=1 11n_disable=8 shouldn't be changing anything. You can remove those and use the defaults.

Need to debug the firmware to check what happens.
Comment 2 Emmanuel Grumbach 2017-10-08 19:23:05 UTC
Please look at https://bugzilla.kernel.org/show_bug.cgi?id=197039#c11 and do the steps described there.

In your case, since the firmware does not crash, you will need to create a dump manually with the debugfs hook.
Comment 3 Emmanuel Grumbach 2017-10-16 18:56:30 UTC
ping?
Comment 4 John Tanner 2017-10-17 23:16:47 UTC
I'm afraid I'm away from my computer; I'll get around to this soon. Thanks.
Comment 5 John Tanner 2017-10-21 16:23:48 UTC
Created attachment 260311 [details]
Firmware Core Dump

Firmware Core Dump as requested.

Generated using Kernel 4.13.0-16-generic on Ubuntu 17.10.
Comment 6 Emmanuel Grumbach 2017-10-22 10:12:17 UTC
Thanks for the data.

Can you please confirm that this was run with the bt_coex_active parameter was set back to default?
Comment 7 John Tanner 2017-10-22 10:22:16 UTC
Not a problem.

Yes, everything set back to default with a vanilla installation of Ubuntu 17.10.

JT
Comment 8 Emmanuel Grumbach 2017-10-22 10:26:17 UTC
So, to sum up.

Are you positive that you can't associate with WiFi when BT connected?
If so, it'll make our lives easier in the debugging process.

Can you please share a dmesg output of that flow?
Based on that, I'll be able to prepare another debug firmware with proper triggers so that our firmware team will see the right data.

They already looked at the dump you provided, and unfortunately, the data inside the dump doesn't contain the right events because of timing problems.
We can fix that by using the right triggers, but for that, I first need to see exactly what scenario is failing.

Thanks.
Comment 9 John Tanner 2017-10-22 10:29:10 UTC
Leave it with me today and I'll produce a full timeline of events that lead to failure, along with the dmesg's.

JT
Comment 10 Emmanuel Grumbach 2017-10-22 10:30:59 UTC
Thing is that based on the dmesg, I need to prepare a special firmware for your specific issue.

What I can do is to prepare a firmware that will produce a dump anytime we have WiFi association issues.

That will work?
Comment 11 John Tanner 2017-10-22 12:54:14 UTC
Taking each case in turn:

initial conditions: wifi off/bluetooth on

Bluetooh pairs with mouse until wifi is switched back on. The bluetooth ceases to work and wifi connects. Nothing will bring back bluetooth until wifi is turned off again.

initial conditions: wifi on/bluetooth off

wifi connects and stays connected even if bluetooth is turned on; bluetooth does not connect.

initial conditions: wifi on/bluetooth on

wifi conencts but bluetooth doesn't.

I've included two dmesg logs:

- dmesg1 covers the first two sets of initial conditions
- dmesg2 covers the third set of initial conditions and was generated immediately after a reboot.

It would appear to me that wifi is the cause of the issue with bluetooth, rather than the other way around.

John
Comment 12 John Tanner 2017-10-22 12:54:50 UTC
Created attachment 260315 [details]
dmesg for first two cases
Comment 13 John Tanner 2017-10-22 12:55:14 UTC
Created attachment 260317 [details]
dmesg for third case
Comment 14 Emmanuel Grumbach 2017-10-22 12:57:21 UTC
Maybe the problem is with BT, we can't know right now. But for sure, WiFi's connection succeeds which means that I can't trigger the debug collection on WiFi's failure to associate...

I'll check what I can do.
Comment 15 Emmanuel Grumbach 2017-10-22 19:53:28 UTC
Created attachment 260321 [details]
8260 firmware with debug enabled

Hi,

please retry with this firmware. It has more BT COEX specific probes and will automatically trigger a dump when you connect with WiFi.

So the flow we can try is:

wifi disconnected / BT connected
Wifi successfully connects and BT disconnects.

around 1500 after the start of the WiFi connection, the driver will produce a dump. Please send that dump.

If you could run this flow 2 or 3 times, it'd be awesome.

Thank you!
Comment 16 John Tanner 2017-10-23 21:45:53 UTC
Created attachment 260353 [details]
231017-1 Dump
Comment 17 John Tanner 2017-10-23 21:46:36 UTC
Created attachment 260355 [details]
231017-2 Dump
Comment 18 John Tanner 2017-10-23 21:48:29 UTC
Hiya,

I created two dumps as per the use case you described. I wasn't sure where (if) the dumps were created so I followed the instructions at https://wireless.wiki.kernel.org/en/users/drivers/iwlwifi/debugging#firmware_debugging to create the above.

Let me know if the dump goes elsewhere and I'll upload those.

JT :)
Comment 19 John Tanner 2017-10-23 21:50:23 UTC
Created attachment 260357 [details]
dmesg for above dumps
Comment 20 Emmanuel Grumbach 2017-10-23 21:58:22 UTC
Ok - I looked at the first dump briefly and it looks fine, but I'd need to have the experts to look at it was well.
I'll let you know.
Comment 21 John Tanner 2017-10-24 20:34:41 UTC
Super, thanks for all of your help :)
Comment 22 Emmanuel Grumbach 2017-10-25 13:50:23 UTC
So the coex guys say the bug is on the BT side. Not sure how to proceed from. You may want to revert to an earlier BT firmware.
Comment 23 John Tanner 2017-10-25 14:46:20 UTC
OK, should I file a new bug for bluetooth or will they be notified?

JT
Comment 24 Emmanuel Grumbach 2017-10-25 14:52:48 UTC
I'll add them here and transfer the bug.

We are reaching out to them internally. From point on, I can't do much to help further.
Comment 25 John Tanner 2017-10-25 14:53:56 UTC
Emmanuel, you have been a brilliant help! Thanks for taking the time to assist; I'll keep an eye on things :)

John
Comment 26 Emmanuel Grumbach 2017-11-14 11:17:46 UTC
Hi,

the WiFi Coex firmware team thinks it has a fix candidate. I'll post it here in a bit.
Comment 27 Emmanuel Grumbach 2017-11-14 12:09:12 UTC
Created attachment 260647 [details]
Fix candidate

Hi,

This firmware includes a fix candidate. Please try. Thanks.
Comment 28 Martin 2017-11-14 12:53:11 UTC
I can confirm the described regressive behaviour in both 4.13.x and 4.14 with intel 8265 adapter. The proposed firmware fix candidate doesn't fix the problem (for me).
Only work-around still is bt_coex_active=N.
Comment 29 Emmanuel Grumbach 2017-11-14 12:55:50 UTC
You couldn't be testing the fix candidate since it is only for 8260 and not for 8265. Trying to get a fix candidate for 8265 as well.
Comment 30 Emmanuel Grumbach 2017-11-14 14:03:31 UTC
Created attachment 260653 [details]
Fix candidate 8265

Hi,

here is the fix candidate for 8265.
Comment 31 Martin 2017-11-14 15:11:37 UTC
My apologies for the noise I created for different hardware, I was misled by the 8260 family-like reference but apparently it's different. The new fix, for 8265 does seem to make the difference.
Earlier regression (4.13.x) involved waking up from stand-by, so I'm careful in declaring this fixed at the moment. I do have BT and Wifi on now, without the bt_coex_active=N option.
Comment 32 Emmanuel Grumbach 2017-11-14 15:29:11 UTC
Thanks Martin. Keep me posted.
Comment 33 John Tanner 2017-11-14 15:43:26 UTC
Sadly not seeing any changes with the latest 4.14 kernel and this firmware. Bluetooth mouse connects for a second or so, then disconnects. 

JT
Comment 34 John Tanner 2017-11-14 15:51:26 UTC
Not sure it will help, but just as the mouse fails I'm getting:

[  624.595743] Bluetooth: Failed to disable LE scan: status 0x0c

In DMESG

JT
Comment 35 Emmanuel Grumbach 2017-11-14 15:53:16 UTC
Are you sure you use this firmware ?

If you have 4.14 you may be loading 34.ucode by mistake.
Comment 36 John Tanner 2017-11-14 16:04:48 UTC
Maximum firmware I've got is 31; tested on both 4.13 and 4.14...no difference sadly.
Comment 37 Emmanuel Grumbach 2017-11-14 16:09:03 UTC
Ok, let's open a new bug for this. I'll inform the firmware team. I'll need you to collect data. Does BT work when wifi is disconnected?
Comment 38 John Tanner 2017-11-14 16:14:43 UTC
Yes, BT is fine without Wifi enabled; it immediately cuts off when wifi is enabled.

Happy to help out in any way I can.

JT
Comment 39 Emmanuel Grumbach 2017-11-14 16:17:24 UTC
Please try to disconnect from the network and leave wifi enabled. Does BT work then?
Comment 40 Jared Ravetch 2017-11-14 22:41:36 UTC
I have the same issue on Fedora 27 with kernel 4.13.11 with Intel 8265.  I just tried the fix candidate posted by Emmanuel and my bluetooth headphones now stay connected.  Awesome work!  I'm happy to keep testing and look forward to this patch getting upstreamed.
Comment 41 Jared Ravetch 2017-11-14 23:03:06 UTC
I may have spoken too soon.  Kernel 4.13.12 was just released.  After rebooting, bluetooth devices again were not connecting.  After a second reboot, my headphones required several attempts to connect and they finally did, but I do see in dmesg:

[   74.126900] Bluetooth: Failed to disable LE scan: status 0x0c

Then bluetooth crashed with a stack trace in dmesg.  Should I attach that stack trace to this ticket?
Comment 42 Emmanuel Grumbach 2017-11-15 03:57:12 UTC
Please disable wifi completely and check what happens.
Comment 43 Martin 2017-11-15 07:29:10 UTC
I just opened my laptop from standby and am left with non-functional BT, unable to (re)connect to my BT mouse. Disabling wifi (uncheck KDE nm widget checkbox?) does not help. Cycling BT (KDE BT widget checkbox) while wifi is off enables BT again. Mouse is now connected and seems stable (have waited couple of minutes before posting this) while having re-enabled wifi.
Comment 44 Emmanuel Grumbach 2017-11-15 07:33:20 UTC
Thanks Martin.

This looks like a BT standalone problem since it didn't work even with WiFi disabled.

Looks like there are BT standalone issues and Coex issues :(
Comment 45 Martin 2017-11-15 07:51:38 UTC
I don't agree, because my previous stand-by problems were also solved by disabling bt_coex. That was my work-around before I commented in this bug and before I compiled 4.14. The only difference between 4.13 and 4.14 is that BT doesn't even start to work on a fresh boot without bt_coex disabled (on 4.14), while on 4.13 it takes a stand-by cycle before things run amok, just like now on 4.14 with firmware fix candidate.

So, long story short: I'm back at 4.13 symptoms running 4.14 using fix candidate firmware.
Comment 46 Emmanuel Grumbach 2017-11-15 08:22:27 UTC
When you use 4.14 you may be using -34.ucode that does NOT have the fix candidate.
Can you please remove -34.ucode to make sure that you use the fix candidate regardless of the kernel version?
Note that the kernel version should not have any impact on this matter (besides the fact that 4.14 will try to load -34.ucode and 4.13 won't).

Thanks.
Comment 47 Martin 2017-11-15 08:47:14 UTC
There's no -34.ucode in my firmware directory:

/lib/firmware# ls -1 iwlwifi-8265*
iwlwifi-8265-21.ucode
iwlwifi-8265-22.ucode
iwlwifi-8265-27.ucode
iwlwifi-8265-31.ucode
iwlwifi-8265-31.ucode.bak

$ dmesg  | grep iwlwifi
[    2.996126] iwlwifi 0000:3b:00.0: Direct firmware load for iwlwifi-8265-34.ucode failed with error -2
[    2.996454] iwlwifi 0000:3b:00.0: Direct firmware load for iwlwifi-8265-33.ucode failed with error -2
[    2.996460] iwlwifi 0000:3b:00.0: Direct firmware load for iwlwifi-8265-32.ucode failed with error -2
[    2.999870] iwlwifi 0000:3b:00.0: loaded firmware version 31.622545.1 op_mode iwlmvm

And please remind that my reports are of anecdotal quality, I have not done any extensive observation/testing of the bug, apart from what I noticed in daily use.
Comment 48 John Tanner 2017-11-15 10:36:28 UTC
(In reply to Emmanuel Grumbach from comment #39)
> Please try to disconnect from the network and leave wifi enabled. Does BT
> work then?

So, if you start my computer without an active wifi connection (but wifi on) then bluetooth works fine. As soon as wifi connects to an available network, bluetooth ceases to work and doesn't come back even if wifi is either completely disabled or the active network is "forgotten".

Still seeing the error:

[ 4201.900708] Bluetooth: Failed to disable LE scan: status 0x0c

when bluetooth falls over.

JT
Comment 49 Emmanuel Grumbach 2017-11-15 11:11:24 UTC
Created attachment 260675 [details]
Fix candidate with debug logs

Here is the same fix candidate with debug configured.
Please use that one.
It'll trigger a dump one second after you connect and hopefully this will bring data that the firmware team will be able to analyze.
Comment 50 Jared Ravetch 2017-11-20 19:43:23 UTC
Any update?  Also us there a fix candidate for 8265 with debug configured?
Comment 51 Emmanuel Grumbach 2017-11-20 20:14:34 UTC
Created attachment 260747 [details]
Fix candidate 8265 with debug logs

Here you go
Comment 52 John Tanner 2017-11-22 23:02:15 UTC
Hey Emmanuel,

Firmware installed...is there a particular failure mode you want me to try?

JT
Comment 53 Robin Lee 2017-11-23 01:08:59 UTC
(In reply to Emmanuel Grumbach from comment #51)
> Created attachment 260747 [details]
> Fix candidate 8265 with debug logs
> 
> Here you go

Which version of Linux kernel can company with this firmware?
Comment 54 Emmanuel Grumbach 2017-11-23 06:58:01 UTC
(In reply to John Tanner from comment #52)
> 
> Firmware installed...is there a particular failure mode you want me to try?
> 

Anything that failed before :)
You should get a dump after WiFi association which should include relevant data for the firmware team.

(In reply to Robin Lee from comment #53)
> Which version of Linux kernel can company with this firmware?

This is supported from kernel 4.13 and up.
Comment 55 Robin Lee 2017-11-23 15:21:39 UTC
Created attachment 260805 [details]
dmesg with attachment 260675 [details] loaded
Comment 56 Emmanuel Grumbach 2017-11-23 15:47:58 UTC
And when you connected WiFi, BT was working? Stopped working? Not connected?
Comment 57 Robin Lee 2017-11-23 15:52:05 UTC
(In reply to Emmanuel Grumbach from comment #56)
> And when you connected WiFi, BT was working? Stopped working? Not connected?

BT device cannot connect. My BT keyboard can be scanned out but fail to pair, and cannot type any key even during pairing.
Comment 58 John Tanner 2017-11-23 16:33:31 UTC
Created attachment 260807 [details]
Core dumps with latest firmware

Two core dumps with the latest firmware.
Comment 59 Emmanuel Grumbach 2017-11-23 16:41:24 UTC
(In reply to Robin Lee from comment #57)
> 
> BT device cannot connect. My BT keyboard can be scanned out but fail to
> pair, and cannot type any key even during pairing.


Please do what is written in https://bugzilla.kernel.org/show_bug.cgi?id=197039#c11 without forgetting to consult our privacy notice.
Comment 60 Robin Lee 2017-11-26 08:15:34 UTC
Created attachment 260877 [details]
firmware dump during reboot with attachment 260675 [details] loaded
Comment 61 Robin Lee 2017-11-26 08:17:19 UTC
Created attachment 260879 [details]
firmware dump during pm suspend/resume with attachment 260675 [details] loaded
Comment 62 Jack Daniels 2017-11-26 13:31:10 UTC
Hey guys,

I have same problem. I'm on Linux Mint 18.2 Sonya on the latest Lenovo Carbon X1. It started after I upgraded to some build of 4.12 kernel.

My bluetooth mouse disconnect the moment I connect to 2.4G Hz wifi network. It works if connect to  5GHz however. I don't have anything suspicious in logs when it happens. Below is output of different commands.

Let me know if I can help with collecting debugging info.

> uname -a
Linux expert-ThinkPad 4.14.2-041402-generic #201711240330 SMP Fri Nov 24 08:32:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

> sudo modinfo iwlwifi | grep -v 'alias'

filename:       /lib/modules/4.14.2-041402-generic/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko
license:        GPL
author:         Copyright(c) 2003- 2015 Intel Corporation <linuxwifi@intel.com>
description:    Intel(R) Wireless WiFi driver for Linux
firmware:       iwlwifi-100-5.ucode
firmware:       iwlwifi-1000-5.ucode
firmware:       iwlwifi-135-6.ucode
firmware:       iwlwifi-105-6.ucode
firmware:       iwlwifi-2030-6.ucode
firmware:       iwlwifi-2000-6.ucode
firmware:       iwlwifi-5150-2.ucode
firmware:       iwlwifi-5000-5.ucode
firmware:       iwlwifi-6000g2b-6.ucode
firmware:       iwlwifi-6000g2a-6.ucode
firmware:       iwlwifi-6050-5.ucode
firmware:       iwlwifi-6000-6.ucode
firmware:       iwlwifi-7265D-29.ucode
firmware:       iwlwifi-7265-17.ucode
firmware:       iwlwifi-3168-29.ucode
firmware:       iwlwifi-3160-17.ucode
firmware:       iwlwifi-7260-17.ucode
firmware:       iwlwifi-8265-34.ucode
firmware:       iwlwifi-8000C-34.ucode
firmware:       iwlwifi-9260-th-b0-jf-b0--34.ucode
firmware:       iwlwifi-9260-th-a0-jf-a0--34.ucode
firmware:       iwlwifi-9000-pu-a0-jf-b0--34.ucode
firmware:       iwlwifi-9000-pu-a0-jf-a0--34.ucode
firmware:       iwlwifi-QuQnj-a0-hr-a0--34.ucode
firmware:       iwlwifi-QuQnj-a0-jf-b0--34.ucode
firmware:       iwlwifi-QuQnj-f0-hr-a0--34.ucode
firmware:       iwlwifi-Qu-a0-jf-b0--34.ucode
firmware:       iwlwifi-Qu-a0-hr-a0--34.ucode
srcversion:     CD38ECD26D0F8834E65D1FC
depends:        cfg80211
intree:         Y
name:           iwlwifi
vermagic:       4.14.2-041402-generic SMP mod_unload 
parm:           swcrypto:using crypto in software (default 0 [hardware]) (int)
parm:           11n_disable:disable 11n functionality, bitmap: 1: full, 2: disable agg TX, 4: disable agg RX, 8 enable agg TX (uint)
parm:           amsdu_size:amsdu size 0: 12K for multi Rx queue devices, 4K for other devices 1:4K 2:8K 3:12K (default 0) (int)
parm:           fw_restart:restart firmware in case of error (default true) (bool)
parm:           antenna_coupling:specify antenna coupling in dB (default: 0 dB) (int)
parm:           nvm_file:NVM file name (charp)
parm:           d0i3_disable:disable d0i3 functionality (default: Y) (bool)
parm:           lar_disable:disable LAR functionality (default: N) (bool)
parm:           uapsd_disable:disable U-APSD functionality bitmap 1: BSS 2: P2P Client (default: 3) (uint)
parm:           bt_coex_active:enable wifi/bt co-exist (default: enable) (bool)
parm:           led_mode:0=system default, 1=On(RF On)/Off(RF Off), 2=blinking, 3=Off (default: 0) (int)
parm:           power_save:enable WiFi power management (default: disable) (bool)
parm:           power_level:default power save level (range from 1 - 5, default: 1) (int)
parm:           fw_monitor:firmware monitor - to debug FW (default: false - needs lots of memory) (bool)
parm:           d0i3_timeout:Timeout to D0i3 entry when idle (ms) (uint)
parm:           disable_11ac:Disable VHT capabilities (default: false) (bool)

> sudo modinfo btintel
filename:       /lib/modules/4.14.2-041402-generic/kernel/drivers/bluetooth/btintel.ko
firmware:       intel/ibt-12-16.ddc
firmware:       intel/ibt-12-16.sfi
firmware:       intel/ibt-11-5.ddc
firmware:       intel/ibt-11-5.sfi
license:        GPL
version:        0.1
description:    Bluetooth support for Intel devices ver 0.1
author:         Marcel Holtmann <marcel@holtmann.org>
srcversion:     0D6D86648AF46F411426FFA
depends:        bluetooth
intree:         Y
name:           btintel
vermagic:       4.14.2-041402-generic SMP mod_unload

> dmesg | grep iwlwifi
[   10.675841] iwlwifi 0000:04:00.0: enabling device (0000 -> 0002)
[   10.687471] iwlwifi 0000:04:00.0: Direct firmware load for iwlwifi-8265-34.ucode failed with error -2
[   10.687837] iwlwifi 0000:04:00.0: Direct firmware load for iwlwifi-8265-33.ucode failed with error -2
[   10.687925] iwlwifi 0000:04:00.0: Direct firmware load for iwlwifi-8265-32.ucode failed with error -2
[   10.690806] iwlwifi 0000:04:00.0: loaded firmware version 31.532993.0 op_mode iwlmvm
[   10.726099] iwlwifi 0000:04:00.0: Detected Intel(R) Dual Band Wireless AC 8265, REV=0x230
[   10.789284] iwlwifi 0000:04:00.0: base HW address: a0:af:bd:e7:10:e4
[   11.620460] iwlwifi 0000:04:00.0 wlp4s0: renamed from wlan0

> dmesg | grep -i blue
[   10.590724] Bluetooth: Core ver 2.22
[   10.590737] Bluetooth: HCI device and connection manager initialized
[   10.590739] Bluetooth: HCI socket layer initialized
[   10.590741] Bluetooth: L2CAP socket layer initialized
[   10.590745] Bluetooth: SCO socket layer initialized
[   10.635779] Bluetooth: HCI UART driver ver 2.3
[   10.635780] Bluetooth: HCI UART protocol H4 registered
[   10.635781] Bluetooth: HCI UART protocol BCSP registered
[   10.635800] Bluetooth: HCI UART protocol LL registered
[   10.635801] Bluetooth: HCI UART protocol ATH3K registered
[   10.635801] Bluetooth: HCI UART protocol Three-wire (H5) registered
[   10.635822] Bluetooth: HCI UART protocol Intel registered
[   10.635841] Bluetooth: HCI UART protocol Broadcom registered
[   10.635842] Bluetooth: HCI UART protocol QCA registered
[   10.635842] Bluetooth: HCI UART protocol AG6XX registered
[   10.635843] Bluetooth: HCI UART protocol Marvell registered
[   10.732977] thinkpad_acpi: rfkill switch tpacpi_bluetooth_sw: radio is unblocked
[   10.790777] Bluetooth: hci0: Bootloader revision 0.0 build 26 week 38 2015
[   10.791781] Bluetooth: hci0: Device revision is 16
[   10.791782] Bluetooth: hci0: Secure boot is enabled
[   10.791782] Bluetooth: hci0: OTP lock is enabled
[   10.791783] Bluetooth: hci0: API lock is enabled
[   10.791783] Bluetooth: hci0: Debug lock is disabled
[   10.791784] Bluetooth: hci0: Minimum firmware build 1 week 10 2014
[   10.792666] Bluetooth: hci0: Found device firmware: intel/ibt-12-16.sfi
[   12.303785] Bluetooth: hci0: Waiting for firmware download to complete
[   12.304758] Bluetooth: hci0: Firmware loaded in 1480595 usecs
[   12.304785] Bluetooth: hci0: Waiting for device to boot
[   12.316766] Bluetooth: hci0: Device booted in 11706 usecs
[   12.316935] Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-12-16.ddc
[   12.318792] Bluetooth: hci0: Applying Intel DDC parameters completed
[   12.378436] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[   12.378437] Bluetooth: BNEP filters: protocol multicast
[   12.378440] Bluetooth: BNEP socket layer initialized
[   13.572479] input: BluetoothMouse3600 as /devices/virtual/misc/uhid/0005:045E:0916.0003/input/input17
[   13.573134] hid-generic 0005:045E:0916.0003: input,hidraw2: BLUETOOTH HID v1.00 Mouse [BluetoothMouse3600] on A0:AF:BD:E7:10:E8
[   27.925920] Bluetooth: RFCOMM TTY layer initialized
[   27.925925] Bluetooth: RFCOMM socket layer initialized
[   27.925930] Bluetooth: RFCOMM ver 1.11
[14691.361452] Bluetooth: hci0: Bootloader revision 0.0 build 26 week 38 2015
[14691.362454] Bluetooth: hci0: Device revision is 16
[14691.362460] Bluetooth: hci0: Secure boot is enabled
[14691.362461] Bluetooth: hci0: OTP lock is enabled
[14691.362463] Bluetooth: hci0: API lock is enabled
[14691.362464] Bluetooth: hci0: Debug lock is disabled
[14691.362466] Bluetooth: hci0: Minimum firmware build 1 week 10 2014
[14691.362471] Bluetooth: hci0: Found device firmware: intel/ibt-12-16.sfi
[14692.860061] Bluetooth: hci0: Waiting for firmware download to complete
[14692.860479] Bluetooth: hci0: Firmware loaded in 1464403 usecs
[14692.860536] Bluetooth: hci0: Waiting for device to boot
[14692.872523] Bluetooth: hci0: Device booted in 11756 usecs
[14692.872527] Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-12-16.ddc
[14692.874528] Bluetooth: hci0: Applying Intel DDC parameters completed
Comment 63 Emmanuel Grumbach 2017-11-27 10:58:36 UTC
Created attachment 260891 [details]
New BT firmware

Here is a pre-release of the new firmware for BT.
Please copy the files to /lib/firmware/intel/

You'll need a hard power off / on cycle.

After boot,  run btmon in 1 terminal and in the other terminal give the command hcitool cmd 3f 05 in root permission.
After giving the hcitool cmd 3f 05 the btmon log will show the version information.

Please provide the btmon log.

Let us know if things feel better.

Note: this includes BT firmware for 8260 and 8265.
Comment 64 Jack Daniels 2017-11-27 12:41:11 UTC
Emmanuel, I replaced drivers with new one and now Bluetooth mouse retains connection after connecting to 2.4 GHz Wifi !

I cannot find btmon.log though. If you still need it please advise what I might do incorrectly in two steps mentioned above.
Comment 65 Jack Daniels 2017-11-27 12:45:36 UTC
Ah I think I understood what you meant. Log is in console output of btmon. Here is my output

< HCI Command: Intel Read Version (0x3f|0x0005) plen 0                                                                                                                                                                      [hci0] 414.242369
> HCI Event: Command Complete (0x0e) plen 13                                   
>                                                                              
>                                                               [hci0]
> 414.243044
      Intel Read Version (0x3f|0x0005) ncmd 1
        Status: Success (0x00)
        Hardware platform: 0x37
        Hardware variant: 0x0c
        Hardware revision: 1.2
        Firmware variant: 0x23
        Firmware revision: 0.1
        Firmware build: 173-45.2017
        Firmware patch: 0

Also I noticed that mouse cursor is hanging sometimes as if laptop was under high CPU load making mouse cursor jumping instead of smoothly moving.
Comment 66 Emmanuel Grumbach 2017-11-27 13:00:11 UTC
Thanks Jack.

Other users?
Comment 67 John Tanner 2017-11-27 22:13:21 UTC
Created attachment 260907 [details]
btmon and dmesg logs

Fingers crossed it looks like it is working! I've reverted back to the original iwlwifi ucode as well; let me know if you want the newer version above checking?

I've attached btmon.log and a dmesg.log as requested.

jdtanner@xiaomi13:~$ uname -r
4.14.2-041402-generic

Thanks for all of your help!
John
Comment 68 Jack Daniels 2017-11-28 07:17:14 UTC
I have to say though it's less stable than original driver. So far I noticed two things:

1) mouse lost connection after resuming system from Standby mode;
2) mouse didn't connect this morning after booting up (I never switch off mouse), but reconnected after switching on/off the mouse.

Nothing critical but suspicious :)
Comment 69 Emmanuel Grumbach 2017-11-28 08:06:13 UTC
@Jack:

The previous firmware didn't allow you to have BT connected apparently :)
So it looks like an improvement.
I am a WiFi person, so I'll stop monitoring this bug now.

There are a few other users on this bug that haven't given any feed back, but based on the current state, I'll close this bug.

I'll stay notified in case someone adds a new comment here.
Comment 70 Tim Ruffing 2017-11-28 14:30:45 UTC
I tried the pre-release from comment 63.

When bt_coex_active=0 is _not_ set, then WiFi does not work at all... There is no error message but scanning does not return any networks.

When I restore the old files, WiFi works no matter if bt_coex_active=0 or not. (Just bluetooth is much more reliable when coex is turned off). 

04:00.0 Network controller: Intel Corporation Wireless 8265 / 8275 (rev 78)
Arch Linux, Kernel 4.13.12
Comment 71 Emmanuel Grumbach 2017-11-28 14:52:46 UTC
@Tim, this is really strange.

Can you please share the dmesg output?
Do you use BT at all?
Comment 72 Martin 2017-11-28 15:10:52 UTC
First two standby-resume tests with pre-release firmware from #63 succesful.
Thx!

btmon log:
# btmon 
Bluetooth monitor ver 5.46
= Note: Linux version 4.14.2 (x86_64)                                                                                                                                                                             0.414913
= Note: Bluetooth subsystem version 2.22                                                                                                                                                                          0.414920
= New Index: 00:28:F8:C1:3A:58 (Primary,USB,hci0)                                                                                                                                                          [hci0] 0.414923
= Open Index: 00:28:F8:C1:3A:58                                                                                                                                                                            [hci0] 0.414925
= Index Info: 00:28:F8:C1:3A:58 (Intel Corp.)                                                                                                                                                              [hci0] 0.414927
@ MGMT Open: bluetoothd (privileged) version 1.14                                                                                                                                                        {0x0001} 0.414929
@ MGMT Open: btmon (privileged) version 1.14                                                                                                                                                             {0x0002} 0.415225
@ RAW Open: hcitool (privileged) version 2.22                                                                                                                                                            {0x0003} 5.329650
@ RAW Close: hcitool                                                                                                                                                                                     {0x0003} 5.329672
@ RAW Open: hcitool (privileged) version 2.22                                                                                                                                                     {0x0003} [hci0] 5.329699
< HCI Command: Intel Read Version (0x3f|0x0005) plen 0                                                                                                                                                  #1 [hci0] 5.329769
> HCI Event: Command Complete (0x0e) plen 13                                   
>                                                                              
>                                           #2 [hci0] 5.330209
      Intel Read Version (0x3f|0x0005) ncmd 1
        Status: Success (0x00)
        Hardware platform: 0x37
        Hardware variant: 0x0c
        Hardware revision: 1.2
        Firmware variant: 0x23
        Firmware revision: 0.1
        Firmware build: 103-50.2016
        Firmware patch: 0
@ RAW Close: hcitool

dmesg output sleep-wakeup cycle:
[  350.865338] wlp59s0: deauthenticating from b4:ee:xx:f9:xx:85 by local choice (Reason: 3=DEAUTH_LEAVING)
[  350.873592] wlp59s0: failed to remove key (1, ff:ff:ff:ff:ff:ff) from hardware (-22)
[  361.062537] PM: suspend entry (deep)
[  361.062538] PM: Syncing filesystems ... done.
[  361.174437] Freezing user space processes ... (elapsed 0.003 seconds) done.
[  361.178198] OOM killer disabled.
[  361.178198] Freezing remaining freezable tasks ... (elapsed 0.002 seconds) done.
[  361.180482] Suspending console(s) (use no_console_suspend to debug)
[  362.369984] ACPI: Preparing to enter system sleep state S3
[  362.374237] ACPI: EC: event blocked
[  362.374237] ACPI: EC: EC stopped
[  362.374238] PM: Saving platform NVS memory
[  362.374247] Disabling non-boot CPUs ...
[  362.382113] smpboot: CPU 1 is now offline
[  362.406217] smpboot: CPU 2 is now offline
[  362.446476] smpboot: CPU 3 is now offline
[  362.450377] ACPI: Low-level resume complete
[  362.450463] ACPI: EC: EC started
[  362.450463] PM: Restoring platform NVS memory
[  362.451367] Enabling non-boot CPUs ...
[  362.451384] x86: Booting SMP configuration:
[  362.451384] smpboot: Booting Node 0 Processor 1 APIC 0x2
[  362.453201]  cache: parent cpu1 should not be sleeping
[  362.453271] CPU1 is up
[  362.453279] smpboot: Booting Node 0 Processor 2 APIC 0x1
[  362.453891]  cache: parent cpu2 should not be sleeping
[  362.453965] CPU2 is up
[  362.453973] smpboot: Booting Node 0 Processor 3 APIC 0x3
[  362.454508]  cache: parent cpu3 should not be sleeping
[  362.454579] CPU3 is up
[  362.457281] ACPI: Waking up from system sleep state S3
[  362.505108] ACPI: EC: event unblocked
[  362.505534] iwlwifi 0000:3b:00.0: RF_KILL bit toggled to enable radio.
[  362.523237] r8169 0000:3a:00.1 enp58s0f1: link down
[  362.741136] usb 1-4: reset high-speed USB device number 2 using xhci_hcd
[  362.819939] ata1: SATA link down (SStatus 4 SControl 300)
[  363.017216] usb 1-5: reset full-speed USB device number 3 using xhci_hcd
[  363.225369] usb 1-5:1.0: rebind failed: -517
[  363.225370] usb 1-5:1.1: rebind failed: -517
[  363.225472] OOM killer enabled.
[  363.225472] Restarting tasks ... done.
[  363.233113] thermal thermal_zone3: failed to read out thermal zone (-61)
[  363.233436] PM: suspend exit
[  363.259879] [drm] RC6 on
[  363.393843] IPv6: ADDRCONF(NETDEV_UP): enp58s0f1: link is not ready
[  363.415062] r8169 0000:3a:00.1 enp58s0f1: link down
[  363.415584] IPv6: ADDRCONF(NETDEV_UP): enp58s0f1: link is not ready
[  363.416779] IPv6: ADDRCONF(NETDEV_UP): wlp59s0: link is not ready
[  363.620365] IPv6: ADDRCONF(NETDEV_UP): wlp59s0: link is not ready
[  363.759252] IPv6: ADDRCONF(NETDEV_UP): wlp59s0: link is not ready
[  363.885127] psmouse serio2: synaptics: queried max coordinates: x [..5718], y [..4908]
[  364.001384] psmouse serio2: synaptics: queried min coordinates: x [1238..], y [956..]
[  364.214806] Bluetooth: hci0: Bootloader revision 0.0 build 26 week 38 2015
[  364.215839] Bluetooth: hci0: Device revision is 16
[  364.215841] Bluetooth: hci0: Secure boot is enabled
[  364.215843] Bluetooth: hci0: OTP lock is enabled
[  364.215844] Bluetooth: hci0: API lock is enabled
[  364.215845] Bluetooth: hci0: Debug lock is disabled
[  364.215847] Bluetooth: hci0: Minimum firmware build 1 week 10 2014
[  364.223011] Bluetooth: hci0: Found device firmware: intel/ibt-12-16.sfi
[  365.801827] Bluetooth: hci0: Waiting for firmware download to complete
[  365.802796] Bluetooth: hci0: Firmware loaded in 1551767 usecs
[  365.802849] Bluetooth: hci0: Waiting for device to boot
[  365.814818] Bluetooth: hci0: Device booted in 11706 usecs
[  365.814953] Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-12-16.ddc
[  365.816818] Bluetooth: hci0: Applying Intel DDC parameters completed
[  368.988424] wlp59s0: authenticate with b4:ee:xx:f9:xx:85
[  368.995910] wlp59s0: send auth to b4:ee:xx:f9:xx:85 (try 1/3)
[  369.002853] wlp59s0: authenticated
[  369.004405] wlp59s0: associate with b4:ee:xx:f9:xx:85 (try 1/3)
[  369.008184] wlp59s0: RX AssocResp from b4:ee:xx:f9:xx:85 (capab=0x1011 status=0 aid=1)
[  369.010706] wlp59s0: associated
[  369.010773] IPv6: ADDRCONF(NETDEV_CHANGE): wlp59s0: link becomes ready
[  369.066007] wlp59s0: Limiting TX power to 27 (30 - 3) dBm as advertised by b4:ee:xx:f9:xx:85
[  378.973246] hid-generic 0005:046D:B014.0002: unknown main item tag 0x0
[  378.973643] input: Bluetooth Mouse M336/M337/M535 as /devices/pci0000:00/0000:00:14.0/usb1/1-5/1-5:1.0/bluetooth/hci0/hci0:256/0005:046D:B014.0002/input/input28
[  378.977926] hid-generic 0005:046D:B014.0002: input,hidraw0: BLUETOOTH HID v12.00 Mouse [Bluetooth Mouse M336/M337/M535] on 00:28:f8:c1:3a:58
Comment 73 Tim Ruffing 2017-11-28 16:36:05 UTC
Okay my issue is rather weird.

I ran a lot of tests... 

In the following, each line corresponds to one boot. The bit is the current setting of bt_coex_active. 
F=fails, W=works.
Sometimes, I did a full power cycle  / power off. When nothing is said, it's a soft reboot. 

0 F
1 F
1 F
0 W
0 F
0 F
1 F
--- power off
1 W
1 F
1 F
0 F
--- power off
0 W
--- power off
0 W
0 W
0 F
0 F
--- power off
0 W
1 W
1 F
1 F
--- power off
1 W
1 W
1 F
--- power off
1 W
1 W
1 F
--- revert to orig. firmware and power off
1 W
1 W
1 W
--- switch to pre-release again and power off
1 W
1 W
1 W
1 W
1 F
1 W
1 F
1 F
--- power off
1 W
1 W
1 F
--- power off
1 W

What I observe in the case of F:
rfkill switch is blocked (and my systemd boot hangs, probably trying to unblock it). I can unblock it manually, on the console, and bring the device up. But then iwlist scan just says "No networks" (or similar, I can't remember).
I haven't tried BT, maybe I should do that.
I'll attach a dmesg (F) but it doesn't show anything suspicious.

Some conclusions:
 - After a power off, WiFi works always, at least once.
 - Apart from that, the failures are somewhat random.
 - bt_coex_active seems unrelated to the issue.
 - The original firmware does not seem to have this issue. (Okay, I tested only three boots here but I've never seen this issue before and I have the machine for 6 months now. But I could test that more extensively.)

By the way: If it works, then the BT connection is great, even when bt_coex_active=1. So this is an improvement, I can confirm that.
Comment 74 Tim Ruffing 2017-11-28 16:37:27 UTC
Created attachment 260913 [details]
dmesg in the F case
Comment 75 Emmanuel Grumbach 2017-11-28 17:57:26 UTC
I can't see anything wrong in the dmesg. but nobody seems to be asking from the WiFi device.

I can't really say much here.

Can you please run sudo wpa_cli in the background after boot?

Best would be to run tracing while trying to work with the wifi device.


https://wireless.wiki.kernel.org/en/users/drivers/iwlwifi/debugging#tracing
Comment 76 Emmanuel Grumbach 2017-11-28 17:58:27 UTC
@Tim,

please open a new bug on this. Don't forget to CC linuxwifi@intel.com
Comment 77 Emmanuel Grumbach 2017-11-28 22:00:03 UTC
Closing for now. Please reopen if needed.
Comment 78 Tim Ruffing 2017-11-29 10:05:30 UTC
Okay, my issues are most probably unrelated. I assumed they're related, because 1) they appeared after the firmware upgrade and 2) I couldn't find networks manually.

However, it turns out that the reason for 2) were permission problems. And I have a reasonable idea why I had not observed the issue before the firmware upgrade, so 1) is not related either. 

Sorry for bothering you, keep up the good work.
Comment 79 Emmanuel Grumbach 2017-11-29 10:43:15 UTC
great. Thanks for letting us know.

I haven't done much besides reaching out to the BT folks :)
Comment 80 Jared Ravetch 2017-12-05 21:07:29 UTC
Emmanuel, thanks again for all your help on this issue.  Is there a ticket that tracks the bluetooth team's fix?
Comment 81 Emmanuel Grumbach 2017-12-05 21:13:44 UTC
Guess what?

https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git

It's there :)

You can now take the BT firmware from mainline and check.

You can also open a ticket on your distro so that they update the linux-firmware that they ship.
Comment 82 Jared Ravetch 2017-12-05 21:49:16 UTC
That's awesome news!  I'll test it out and let the Fedora team know.
Comment 83 Egor 2017-12-06 19:44:22 UTC
I still have bluetooth issues with active wifi connection with latest linux-firmware on 8265 chipset. When wifi connection uses 5Ghz frequency then all works fine. But when I connect to a 2.4Ghz network then bluetooth doesn't works (bt mouse lost connection). Also when I switch back to a 5Ghz wifi network and try to reconnect my bt mouse I get crash report in dmesg.

dmesg log: https://www.dropbox.com/s/vm8jpx1fup0v8ol/dmesg_bt.log?dl=0

btmon log:

        Status: Success (0x00)
        Hardware platform: 0x37
        Hardware variant: 0x0c
        Hardware revision: 1.2
        Firmware variant: 0x23
        Firmware revision: 0.1
        Firmware build: 103-50.2016
        Firmware patch: 0
Comment 84 Emmanuel Grumbach 2017-12-06 19:50:40 UTC
I think that the firmware you use is old.

Can you please do a cold reboot (poweroff and power on) and retry?
Comment 85 Egor 2017-12-06 19:57:18 UTC
Well I made cold reboot several times already (even by pressing power button for several seconds). Maybe this laptop does not power off completely. Now I am trying to drain off battery and then see what happens.
Comment 86 Emmanuel Grumbach 2017-12-06 20:01:43 UTC
ok this is strange.

You need the BT guys here. I am from the WiFi side.
Actually, I can see that your logs are the same as comment 72, so the BT firmware does look good.

You can open a new bug if you want. The BT kernel crash is clearly something that needs to be fixed by BT.
Comment 87 Egor 2017-12-06 20:05:22 UTC
Ok, thank you for your attention.

Note You need to log in before you can comment on or make changes to this bug.