Bug 108461
Summary: | rtl8821ae causing 100ms scheduling latency | ||
---|---|---|---|
Product: | Drivers | Reporter: | James Ettle (james) |
Component: | network-wireless | Assignee: | drivers_network-wireless (drivers_network-wireless) |
Status: | RESOLVED WILL_NOT_FIX | ||
Severity: | normal | CC: | eblanca, Larry.Finger, linville, rasasi78 |
Priority: | P1 | ||
Hardware: | All | ||
OS: | Linux | ||
Kernel Version: | 4.2.6 | Subsystem: | |
Regression: | No | Bisected commit-id: | |
Attachments: |
config file
Biggest latency list times Latency time 1 script Latency time 2 script |
Description
James Ettle
2015-11-25 20:20:09 UTC
Note - I mispasted the dev id, it's 10ec:8821. There are two things to try. Kernel 4.2.6 included a patch that reverted an earlier patch that disabled a clearing of RTL8821AE interrupts before they are set again, which led to kernel oops on boot. First, I would disable MSI interrupts with the following (as root): echo "msi=0" > /etc/modprobe.d/50-rtl8821ae.conf modprobe -rv rtl8821ae modprobe -v rtl8821ae Then run tests to see if you still have audio stuttering. If that test still fails, then do the following (again as root): echo "int_clear=0" > /etc/modprobe.d/50-rtl8821ae.conf modprobe -rv rtl8821ae modprobe -v rtl8821ae Now retest. Report both sets of results. If neither changes help, please post the exact set of perf commands you used. I have not used that utility before, and I'm having some difficulty in repeating your tests. Shouldn't that be: echo "options rtl8821ae msi=0" > /etc/modprobe.d/50-rtl8821ae.conf Testing now... I should point out that this has been happening since at least kernel 4.1.6, the version around when I got the machine. I'm not much of a perf expert either, just read enough of the man pages to try and figure out how to spot scheduling latencies. (Starting point here was the PA log reporting a scheduling delay.) The commands are as follows. First I run: # perf sched record for a few minutes with audio running, or until I hear a glitch. Press Ctrl+C to stop. Then I run # perf sched latency This gives a top-like list of the scheduling delays for various processes, their maximum delay and when that occurred. I pick out the worst (~100ms with an unconnected interface, 20-50ms when connected) and note when it happened. Finally, I run # perf sched script and search for the time I noted above. (Searching is like 'less' -- hit / key and enter a pattern.) This, as far as I understand, is a trace of what the scheduler did. I inspect what's going on just before the delay. What I generally find: Baseline with interface OFF: Maximum latencies of about 2ms. When the interface is up BUT NOT CONNECTED: I see latencies of ~100ms. This is enough to audibly glitch PA. In 'perf sched script', I see that wpa_supplicant is always scheduled just beforehand. When the interface is CONNECTED: I see latencies of around 20-50ms, so higher than the baseline. PA seems to be able to cope with this, so I don't hear a glitch. Also the scheduling pattern in the 'perf sched script' is different - I don't see wpa_supplicant being scheduled just beforehand. But I do see kworker/u8:3 a lot before, not sure if this is doing something for rtl8821ae. This happens with either no module options or msi=0. I haven't tested int_clear=0 yet. With *just* 'int_clear=0', I get the same sort of results as with no options or just 'msi=0'. OK, interrupts are not the problem. Yes, you are correct that the driver name is needed in the options line. For some reason, I will not be able to duplicate your tests. When I use perf sched record my system responds with "invalid or unsupported event: 'sched:sched_switch'". I'm still trying to solve that problem. When the wifi driver is not connected, the only usage of wpa_supplicant should be to trigger scans. What method are you using to control the network? Does 'grep scan /etc/wpa_supplicant.conf' yield any output? (In reply to Larry Finger from comment #5) > OK, interrupts are not the problem. Yes, you are correct that the driver > name is needed in the options line. > > For some reason, I will not be able to duplicate your tests. When I use > > perf sched record > > my system responds with "invalid or unsupported event: > 'sched:sched_switch'". I'm still trying to solve that problem. No idea what this means. If it helps, I'm using the stock Fedora kernel which has the following perf config options: CONFIG_PERF_EVENTS_INTEL_UNCORE=y CONFIG_CGROUP_PERF=y CONFIG_HAVE_PERF_EVENTS=y CONFIG_PERF_EVENTS=y CONFIG_HAVE_PERF_EVENTS_NMI=y CONFIG_HAVE_PERF_REGS=y CONFIG_HAVE_PERF_USER_STACK_DUMP=y > When the wifi driver is not connected, the only usage of wpa_supplicant > should be to trigger scans. What method are you using to control the network? NetworkManager-1.0.6-8.fc23.x86_64 with standard Gnome 3.18.2 UI utils. This is a pretty standard Fedora 23 install. > Does 'grep scan /etc/wpa_supplicant.conf' yield any output? Nothing at all. I should also point out: I've another two F23 machines with the same versions of everything, one using ath9k and the other iwl4965. Neither have any odd latency issues, so I'm not convinced it's anything else in the wireless stack. My configuration has CONFIG_PERF_EVENTS_INTEL_UNCORE=y CONFIG_CGROUP_PERF=y CONFIG_HAVE_PERF_EVENTS=y CONFIG_PERF_USE_VMALLOC=y CONFIG_PERF_EVENTS=y CONFIG_DEBUG_PERF_USE_VMALLOC=y CONFIG_HAVE_PERF_EVENTS_NMI=y CONFIG_HAVE_PERF_REGS=y CONFIG_HAVE_PERF_USER_STACK_DUMP=y All of the PERF options are selected. Created attachment 195661 [details]
config file
I've attached the config file for the kernel I'm currently using, but I don't know if that'll help -- I'm otherwise completely inexpert in perf and tracking down latencies.
The machine in question has an i5-5200U processor, so 2 cores... I guess if your machine has more cores on which to schedule stuff it's less likely you'd see (or hear!) the delay.
My CPU is an i7-4600M CPU @ 2.90GHz, thus I also have 2 cores plus hyperthreading. I did install Fedora 23 and tested it. One thing I do not understand is how you keep an interface unconnected. I know how to do it with the KDE applet, but not with Gnome. I went back to openSUSE 13.2 where I finally managed to get a version of perf that works. The problem was the package included with openSUSE could not handle the sched option; however, when I built and installed the version included with the kernel source, then it works. The longest scheduling delays happen when scanning. It doesn't seem to matter whether the interface is connected or not. I trigger the scans with a 'sudo iwlist <iface> scan' command. The longest delays are usually 40-50 ms and happen in the scheduling of kthreadd or one of the kernel workers. Obviously, I need to investigate what happens when scanning. (In reply to Larry Finger from comment #10) > I did install Fedora 23 and tested it. One thing I do not understand is how > you keep an interface unconnected. I know how to do it with the KDE applet, Thanks for going to all that effort! To prevent connection, assuming you've selected and set up a network: Go into WiFi Settings. There's a '*' button next to the network. Click on that and select the 'Identity' page. Uncheck 'Connect automatically' and click 'Apply'. Hello. I think I'm also running into this issue. I run debian stretch with backported kernel 4.19.12-1~bpo9+1. This is a HP Probook laptop and wireless interface is: 09:00.0 Network controller [0280]: Realtek Semiconductor Co., Ltd. RTL8723BE PCIe Wireless Network Adapter [10ec:b723] What I'm also experiencing is that when I listen to music (I use amarok) then some sound glitches can be easily appreciated only when wireless interface is enabled in network manager but it is not associated to any AP. When wireless interface is disable the sound glitches are not happening, nor they are when interface is associated with an AP. I performed the perf procedure by James on comment#3 1) perf sched record (Till the sound glitch is heard) 2) perf sched latency -s max |head -n12 |sort -nk 18 (So to get the maximum latency times) (attached perf_maxLatList) 3) Go to those times with perf sched script (attached perf_maxLat[12]) In reports by 3) there's also remarkable presence of wpa_supplincant and in one of these moments of alsa-sink Let me know if I could be of further help. HTH, Created attachment 281123 [details]
Biggest latency list times
Created attachment 281125 [details]
Latency time 1 script
Created attachment 281127 [details]
Latency time 2 script
I forgot to state module rtl8723be parameters: ant_sel:0 aspm:1 debug_level:0 debug_mask:0 disable_watchdog:N fwlps:Y ips:Y msi:N swenc:N swlps:N This should be defaults Same happening to me, both with kernel Linux cindy 4.9.0-11-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64 GNU/Linux (debian 9) and with Linux debbieb1 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1 (2020-01-26) x86_64 GNU/Linux (debian 10) My wifi adapter is a Realtek Semiconductor Co., Ltd. RTL8723BE PCIe Wireless Network Adapter handled by driver rtl8723be. Options for this driver are: $ cat /etc/modprobe.d/rtl8723be.conf options rtl8723be ant_sel=1 Up to now, my only way to stop hearing stuttering from *any* audio application (firefox, mpv, deadbeef, ...) is killing wpa_supplicant. Or, when I need wireless connectivity, avoid pulseaudio (which is subject to stuttering) and use alsa instead. I tried https://github.com/lwfinger/rtlwifi_new but with no success, latency still there. I no longer have this hardware. If there's no prospect of any progress on this it might as well be closed off as a no-fix, unless it can be transferred to one of the other stakeholders. |