Bug 218494

Hi,
I am trying to use an RFCOMM-capable dongle under Arch Linux (kernel: 6.6.10-arch1-1).
The BT usb adapter is:
ID 0bda:8771 Realtek Semiconductor Corp. Bluetooth Radio

dmesg:
[   19.174428] Bluetooth: Core ver 2.22
[   19.174448] NET: Registered PF_BLUETOOTH protocol family
[   19.174448] Bluetooth: HCI device and connection manager initialized
[   19.174452] Bluetooth: HCI socket layer initialized
[   19.174454] Bluetooth: L2CAP socket layer initialized
[   19.174460] Bluetooth: SCO socket layer initialized
[   20.497753] Bluetooth: hci0: RTL: examining hci_ver=0a hci_rev=000b lmp_ver=0a lmp_subver=8761
[   20.500891] Bluetooth: hci0: RTL: rom_version status=0 version=1
[   20.500894] Bluetooth: hci0: RTL: loading rtl_bt/rtl8761bu_fw.bin
[   20.502433] Bluetooth: hci0: RTL: loading rtl_bt/rtl8761bu_config.bin
[   20.503140] Bluetooth: hci0: RTL: cfg_sz 6, total sz 30210
[   20.898620] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[   20.898620] Bluetooth: BNEP filters: protocol multicast
[   20.898620] Bluetooth: BNEP socket layer initialized
[   21.285074] Bluetooth: hci0: RTL: fw version 0xdfc6d922
[   21.430191] Bluetooth: MGMT ver 1.22
[   40.360736] Bluetooth: RFCOMM TTY layer initialized
[   40.360743] Bluetooth: RFCOMM socket layer initialized
[   40.360746] Bluetooth: RFCOMM ver 1.11

The rfcomm utility was not provided so I had to install:
https://archlinux.org/packages/extra/x86_64/bluez-deprecated-tools/

unfortunately the command:
rfcomm connect 0 B8:8B:15:0A:1C:AD
gives:
Can't connect RFCOMM socket: Operation now in progress

I tried also with `bind` argument - all of this gives errors.
Finally I tried python tools to directly use RFCOMM, then even a rust rfcat tool from:
https://github.com/bluez/bluer/blob/master/bluer-tools/src/rfcat.rs
All gives various errors/timeouts and cannot connect to the dongle's
serial port. I was also trying to pair/connect with bluetoothctl prior
this - no change.
In fact (see below - pairing seems not needed at all).

Finally I started very very old Asus laptop with debian.
I invoked:
rfcomm connect 0 B8:8B:15:0A:1C:AD
and this immediately gives me:
Connected /dev/rfcomm0 to B8:8B:15:0A:1C:AD on channel 1
Press CTRL-C for hangup

... and it works like a charm!!! I can connect minicom and communication is
working (no pairing/connecting with bluetoothctl needed).

dmesg there:
[  298.641307] Bluetooth: Core ver 2.22
[  298.641393] Bluetooth: HCI device and connection manager initialized
[  298.641405] Bluetooth: HCI socket layer initialized
[  298.641409] Bluetooth: L2CAP socket layer initialized
[  298.641419] Bluetooth: SCO socket layer initialized
[  299.673331] Bluetooth: hci0: unexpected event for opcode 0x0000
[  311.510413] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[  311.510422] Bluetooth: BNEP filters: protocol multicast
[  311.510435] Bluetooth: BNEP socket layer initialized
[ 1902.017008] Bluetooth: RFCOMM TTY layer initialized
[ 1902.017026] Bluetooth: RFCOMM socket layer initialized
[ 1902.017048] Bluetooth: RFCOMM ver 1.11

the adapter is:
ID 0b05:1712 ASUSTek Computer, Inc. BT-183 Bluetooth 2.0+EDR adapter

I am now trying to identify the problem/difference. I've logged all data using
btmon from the start to invoking the rfcom connect.

I can see that the difference start when my Asus laptop is showing this:
< ACL Data TX: Handle 42 flags 0x02 dlen 10                                                                        #13 [hci0] 40.891571
      L2CAP: Information Request (0x0a) ident 1 len 2
        Type: Extended features supported (0x0002)
> HCI Event: Number of Completed Packets (0x13) plen 5                         
>                                    #14 [hci0] 40.898476
        Num handles: 1
        Handle: 42 Address: B8:8B:15:0A:1C:AD (OUI B8-8B-15)
        Count: 1
> ACL Data RX: Handle 42 flags 0x02 dlen 16                                    
>                                    #15 [hci0] 40.899488
      L2CAP: Information Response (0x0b) ident 1 len 8
        Type: Extended features supported (0x0002)
        Result: Success (0x0000)
        Features: 0x00000000
< ACL Data TX: Handle 42 flags 0x02 dlen 12                                                                        #16 [hci0] 40.899554
      L2CAP: Connection Request (0x02) ident 2 len 4
        PSM: 3 (0x0003)
        Source CID: 64

I don't see any L2CAP and RFCOMM lines on the "bad" ArchLinux setup.

ps. I tested the dongle also from Windows and Android - also works OK.

update:
I've exposed the laptop's bluetooth adapter to my ArchLinux using USB/IP method.
This way on the same platform where the Realtek USB adapter is not
working I was able to also use laptop's BT adapter.

Here are the results:
❯ rfcomm -i hci0 connect 0 B8:8B:15:0A:1C:AD
Can't connect RFCOMM socket: Operation now in progress
# --> problem on realtek

❯ rfcomm -i hci1 connect 0 B8:8B:15:0A:1C:AD
Connected /dev/rfcomm0 to B8:8B:15:0A:1C:AD on channel 1
Press CTRL-C for hangup
^CDisconnected
# --> works great on Asus BT

Conclusion: the ArchLinux platform is OK (i mean the distribution,
bluetooth stack, kernel modules and utilities).

The problem seems to be the Realtek drivers/firmware for my rtl8761bu...

I am attaching full logs (called good and bad). dvbsnoop logs also
included.

update2: I also tested the initial firmware added (from 2021-06-09):
https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/log/rtl_bt/rtl8761bu_fw.bin
also doesn't work