Bug 210923 - ath11k: qmi failed memory request, err = -110
Summary: ath11k: qmi failed memory request, err = -110
Status: CLOSED DOCUMENTED
Alias: None
Product: Drivers
Classification: Unclassified
Component: network-wireless (show other bugs)
Hardware: Intel Linux
: P1 high
Assignee: Kalle Valo
URL:
Keywords:
: 215998 (view as bug list)
Depends on:
Blocks:
 
Reported: 2020-12-28 02:04 UTC by Ivo
Modified: 2023-07-11 09:12 UTC (History)
9 users (show)

See Also:
Kernel Version: 5.10.3
Subsystem:
Regression: Yes
Bisected commit-id:


Attachments

Description Ivo 2020-12-28 02:04:33 UTC
Hi there,

Using DELL XPS 9700 i9 and Killer(TM) AX500 WiFi card.

I was able to use it with kernel 5.10.1 but not with 5.10.2 and 5.10.3

There the the errors from the log:

dmesg|grep ath11
[    2.081002] ath11k_pci 0000:05:00.0: WARNING: ath11k PCI support is experimental!
[    2.081038] ath11k_pci 0000:05:00.0: BAR 0: assigned [mem 0xd2100000-0xd21fffff 64bit]
[    2.081067] ath11k_pci 0000:05:00.0: enabling device (0000 -> 0002)
[    2.081110] ath11k_pci 0000:05:00.0: failed to get 32 MSI vectors, only -28 available
[    2.081111] ath11k_pci 0000:05:00.0: failed to enable msi: -28
[    2.081122] ath11k_pci: probe of 0000:05:00.0 failed with error -28

Let me know if you need any more info.

Thanks
Ivo
Comment 1 Ivo 2021-01-04 02:35:37 UTC
Some more info
===============

Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz

32GB RAM

Artix Linux release
LSB_VERSION=1.4
DISTRIB_ID=Artix
DISTRIB_RELEASE=rolling
DISTRIB_DESCRIPTION="Artix Linux"
VERSION_ID=20201207
VARIANT=cinnamon-openrc
NAME="Artix Linux"
PRETTY_NAME="Artix Linux"
ID=artix
BUILD_ID=rolling

after modprobe ath11k - no messages in the kernel log
after modprobe ath11k_pci - the following is displayed.
=================
[  522.906063] ath11k_pci 0000:05:00.0: WARNING: ath11k PCI support is experimental!
[  522.906213] ath11k_pci 0000:05:00.0: BAR 0: assigned [mem 0xd2100000-0xd21fffff 64bit]
[  522.906296] ath11k_pci 0000:05:00.0: failed to get 32 MSI vectors, only -28 available
[  522.906297] ath11k_pci 0000:05:00.0: failed to enable msi: -28
[  522.906310] ath11k_pci: probe of 0000:05:00.0 failed with error -28
=====================


lsmod|grep ath11k
ath11k_pci             24576  0
mhi                    81920  1 ath11k_pci
ath11k                454656  1 ath11k_pci
qmi_helpers            32768  1 ath11k
mac80211             1110016  1 ath11k
cfg80211              987136  2 ath11k,mac80211

lspci

05:00.0 Network controller: Qualcomm Device 1101 (rev 01)
        Subsystem: Bigfoot Networks, Inc. Device a501
        Flags: fast devsel
        Memory at d2100000 (64-bit, non-prefetchable) [size=1M]
        Capabilities: <access denied>
        Kernel modules: ath11k_pci
Comment 2 Ivo 2021-01-06 23:20:09 UTC
kernel 5.10.4 also fails withe same error.
Comment 3 Ivo 2021-01-20 03:39:36 UTC
After enabling Vt-D in BIOS and upgrading it to the latest version got a different error now

[    2.267396] ath11k_pci 0000:05:00.0: WARNING: ath11k PCI support is experimental!
[    2.267435] ath11k_pci 0000:05:00.0: BAR 0: assigned [mem 0xd2100000-0xd21fffff 64bit]
[    2.267455] ath11k_pci 0000:05:00.0: enabling device (0000 -> 0002)
[    2.727171] ath11k_pci 0000:05:00.0: Respond mem req failed, result: 1, err: 0
[    2.727172] ath11k_pci 0000:05:00.0: qmi failed to respond fw mem req:-22
[    7.880850] ath11k_pci 0000:05:00.0: qmi failed memory request, err = -110
[    7.880859] ath11k_pci 0000:05:00.0: qmi failed to respond fw mem req:-110

Not sure what is causing this, Probably not connected to the MSI vectors
Comment 4 Kalle Valo 2021-01-26 15:56:20 UTC
(In reply to Ivo from comment #3)
> After enabling Vt-D in BIOS and upgrading it to the latest version got a
> different error now
> 
> [    2.267396] ath11k_pci 0000:05:00.0: WARNING: ath11k PCI support is
> experimental!
> [    2.267435] ath11k_pci 0000:05:00.0: BAR 0: assigned [mem
> 0xd2100000-0xd21fffff 64bit]
> [    2.267455] ath11k_pci 0000:05:00.0: enabling device (0000 -> 0002)
> [    2.727171] ath11k_pci 0000:05:00.0: Respond mem req failed, result: 1,
> err: 0
> [    2.727172] ath11k_pci 0000:05:00.0: qmi failed to respond fw mem req:-22
> [    7.880850] ath11k_pci 0000:05:00.0: qmi failed memory request, err = -110
> [    7.880859] ath11k_pci 0000:05:00.0: qmi failed to respond fw mem req:-110
> 
> Not sure what is causing this, Probably not connected to the MSI vectors

Firmware WLAN.HST.1.0.1-01740-QCAHSTSWPLZ_V2_TO_X86-1 has issues if host uses memory under 32M limit, I suspect that's the issue here. Try adding "memmap=12M\$20M" to your kernel:

https://lore.kernel.org/ath11k/056b74d48fb735c69a6268b18184d7e8@codeaurora.org/
Comment 5 Ivo 2021-01-27 03:43:51 UTC
Reserved some ranged of unassigned memory for use by emulated persistent memory and added it to the grub boot options with "memmap=12M\\\$20M" as with one backslash the grub interpreted the dollar sign as a shell command.

Looks OK now. I can use the Wi-Fi card and the kernel log messages are:

[    2.092063] ath11k_pci 0000:05:00.0: WARNING: ath11k PCI support is experimental!
[    2.092100] ath11k_pci 0000:05:00.0: BAR 0: assigned [mem 0xd2100000-0xd21fffff 64bit]
[    2.092120] ath11k_pci 0000:05:00.0: enabling device (0000 -> 0002)
[    2.609765] ath11k_pci 0000:05:00.0: chip_id 0x0 chip_family 0xb board_id 0xff soc_id 0xffffffff
[    2.609766] ath11k_pci 0000:05:00.0: fw_version 0x101c06cc fw_build_timestamp 2020-06-24 19:50 fw_build_id

Thank you!
Comment 6 Kalle Valo 2021-01-27 06:30:39 UTC
I'll keep this bug open until there's a fix for the firmware. Also changing the title to reflect the issue is better.
Comment 7 Owen Lynch 2021-01-31 04:17:04 UTC
I'm having similar problems with this Wi-Fi card. I added the "memmap=12M\$20M" line to the boot, and now the error I am getting is this:

https://owenlynch.org/static/ath11k_error.jpeg

Any ideas on what might be causing this?
Comment 8 Owen Lynch 2021-01-31 04:18:37 UTC
For some reason my webserver doesn't like jpegs? Here's a png:

https://owenlynch.org/static/ath11k_error.png
Comment 9 Kalle Valo 2021-02-04 10:44:15 UTC
(In reply to Owen Lynch from comment #7)
> I'm having similar problems with this Wi-Fi card. I added the
> "memmap=12M\$20M" line to the boot, and now the error I am getting is this:
> 
> https://owenlynch.org/static/ath11k_error.jpeg
> 
> Any ideas on what might be causing this?

I recommend double checking from /proc/cmdline that memmap parameter is correctly set. It should look like this:

$ cat /proc/cmdline 
[...] memmap=12M$20M [...]

Next I would make sure the firmware files are not corrupted:

$ md5sum /lib/firmware/ath11k/QCA6390/hw2.0/*
a101dc90f8e876f39383b60c9da64ec4  /lib/firmware/ath11k/QCA6390/hw2.0/amss.bin
4c0781f659d2b7d6bef10a2e3d457728  /lib/firmware/ath11k/QCA6390/hw2.0/board-2.bin
d4c912a3501a3694a3f460d13de06d28  /lib/firmware/ath11k/QCA6390/hw2.0/m3.bin
ddc7afcb118def559bf131c3ec2563af  /lib/firmware/ath11k/QCA6390/hw2.0/Notice.txt

If that doesn't help, please file a new bug. Let's keep one issue per report, please.
Comment 10 Kalle Valo 2021-02-17 09:40:49 UTC
As I'm receiving more reports, just a little more background of this issue. This was first seen in v5.10-rc1 as there was a change in mm:

7fef431be9c9 mm/page_alloc: place pages to tail in __free_pages_core()

https://git.kernel.org/linus/7fef431be9c9

Reverting that commit would also workaround this firmware issue.

In some platforms the memmap above is not enough. For example, in one x86 box I had to use memmap=14M\\\$18M. 

To make it easier to detect this issue there's now a qmi debug message which prints the buffer addresses (will be in v5.12-rc1):

https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/commit/?h=ath-next&id=d5395a548659
Comment 14 Karsten Gebbert 2021-08-17 07:07:51 UTC
I'm seeing a similar but possibly different issue on Dell XPS 15 9500 with latest Bios (1.8.1).

$ uname -a
Linux xps15 5.13.11 #1-NixOS SMP Sun Aug 15 12:01:57 UTC 2021 x86_64 GNU/Linux


$ cat /proc/cmdline
initrd=\efi\nixos\7nqynfr6b72q6ycivac4lwfs4ibvfm0v-initrd-linux-5.13.11-initrd.efi init=/nix/store/1kcalf57sypcvdz7a73ld7v6lvqhpai2-nixos-system-xps15-21.05.2501.94c989365d5/init [...] memmap=12M$20M loglevel=4

$ modinfo ath11k_pci
filename:       /run/current-system/kernel-modules/lib/modules/5.13.11/kernel/drivers/net/wireless/ath/ath11k/ath11k_pci.ko.xz
firmware:       ath11k/QCA6390/hw2.0/m3.bin
firmware:       ath11k/QCA6390/hw2.0/amss.bin
firmware:       ath11k/QCA6390/hw2.0/board-2.bin
license:        Dual BSD/GPL
description:    Driver support for Qualcomm Technologies 802.11ax WLAN PCIe devices
alias:          pci:v000017CBd00001101sv*sd*bc*sc*i*
depends:        mhi,ath11k
retpoline:      Y
intree:         Y
name:           ath11k_pci
vermagic:       5.13.11 SMP mod_unload 


Here is the log:

https://pastebin.com/YrMEictS

Should I open a new issue for this?

Thank you!
Comment 15 Ivo 2021-09-19 03:20:53 UTC
Just upgraded to kernel 5.14.4
Updated to the latest BIOS as well.

Problem reappeared with different messges:

failed to register fw indication: -110
failed to send qmi firmware indication: -110

Thanks
Ivo
Comment 16 D.F. 2021-09-19 18:02:29 UTC
This is a regression which was caused by commit in 5.14.4 and 5.13.16: 

3f38cc1ba1e289306f52cb7238406f38ecbfa21a net: qrtr: make checks in qrtr_endpoint_post() stricter

Here is the bugreport:

https://bugzilla.kernel.org/show_bug.cgi?id=214455
Comment 17 Kalle Valo 2022-01-18 13:11:14 UTC
The 32M memory limit bug is now fixed in this firmware version:

https://github.com/kvalo/ath11k-firmware/tree/master/QCA6390/hw2.0/1.0.1/WLAN.HST.1.0.1-05266-QCAHSTSWPLZ_V2_TO_X86-1

With that firmware release memmap workaround is not needed anymore. Closing the bug.
Comment 24 Len Brown 2022-05-26 22:41:12 UTC
As this broke in 5.10 and worked in earlier releases, marking as a REGRESSION.

As the fix requires new firmware, rather than a kernel "CODE_FIX" patch, marking as closed/DOCUMENTED.
Comment 25 Kalle Valo 2023-03-01 14:44:43 UTC
*** Bug 215998 has been marked as a duplicate of this bug. ***
Comment 26 Kai Mast 2023-06-12 00:39:01 UTC
Is there another reason this message could be triggered? I started seeing this in 6.3.7.

~> sudo dmesg | grep ath11k                                                                                                                                 17:31:57
[   11.441679] ath11k_pci 0000:6c:00.0: BAR 0: assigned [mem 0xb4200000-0xb42fffff 64bit]
[   11.441715] ath11k_pci 0000:6c:00.0: enabling device (0000 -> 0002)
[   11.442189] ath11k_pci 0000:6c:00.0: MSI vectors: 32
[   11.442199] ath11k_pci 0000:6c:00.0: qca6390 hw2.0
[   14.574034] ath11k_pci 0000:6c:00.0: ignore reset dev flags 0x8000
[   21.940059] ath11k_pci 0000:6c:00.0: failed to wait qmi memory request: -110
[   21.940066] ath11k_pci 0000:6c:00.0: qmi failed to respond fw mem req: -110

Firmware seems to be up to date:
 modinfo ath11k_pci                                                                                                                                       17:35:26
filename:       /lib/modules/6.3.7-arch1-1/kernel/drivers/net/wireless/ath/ath11k/ath11k_pci.ko.zst
firmware:       ath11k/QCA6390/hw2.0/m3.bin
firmware:       ath11k/QCA6390/hw2.0/amss.bin
firmware:       ath11k/QCA6390/hw2.0/board-2.bin
license:        Dual BSD/GPL
description:    Driver support for Qualcomm Technologies 802.11ax WLAN PCIe devices
srcversion:     FFC77E7DB9B98E6850D2BDA
alias:          pci:v000017CBd00001104sv*sd*bc*sc*i*
alias:          pci:v000017CBd00001103sv*sd*bc*sc*i*
alias:          pci:v000017CBd00001101sv*sd*bc*sc*i*
depends:        mhi,ath11k
retpoline:      Y
intree:         Y
name:           ath11k_pci
vermagic:       6.3.7-arch1-1 SMP preempt mod_unload
sig_id:         PKCS#7
signer:         Build time autogenerated kernel key
sig_key:        1A:AF:97:2E:13:77:23:DE:23:E6:E7:80:AD:98:5E:5C:F1:2D:C8:D7
sig_hashalgo:   sha512
signature:      30:66:02:31:00:D4:FB:48:5F:5A:4A:76:81:9E:6D:72:B7:39:54:59:
          D9:CF:FB:62:E3:AA:5C:DE:0D:0C:CF:08:44:46:A6:12:67:6D:8E:B6:
          20:B1:D8:57:FB:0B:D7:BB:72:BE:DA:E1:54:02:31:00:85:4E:F8:B6:
          E3:F1:4E:66:DF:B1:B8:57:01:FC:5E:D9:F6:45:24:08:EF:59:87:80:
          F0:91:F0:4A:A3:8B:49:B5:46:66:30:76:52:5E:5C:89:DE:13:65:05:
          2D:1F:8B:E8
Comment 27 Kalle Valo 2023-07-11 09:12:35 UTC
> Is there another reason this message could be triggered? I started seeing
> this
> in 6.3.7.

One issue per report, please. Please file a new bug.

Note You need to log in before you can comment on or make changes to this bug.