Bug 12415

Summary: WARNING: at drivers/net/wireless/iwlwifi/iwl-sta.c:689
Product: Drivers Reporter: Rafael J. Wysocki (rjw)
Component: network-wirelessAssignee: drivers_network-wireless (drivers_network-wireless)
Status: CLOSED CODE_FIX    
Severity: normal CC: borntraeger, reinette.chatre
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: 2.6.28-git Subsystem:
Regression: Yes Bisected commit-id:
Bug Depends on:    
Bug Blocks: 12398    

Description Rafael J. Wysocki 2009-01-10 16:35:40 UTC
Subject    : 2.6.28-git: WARNING: at drivers/net/wireless/iwlwifi/iwl-sta.c:689
Submitter  : Christian Borntraeger <borntraeger@de.ibm.com>
Date       : 2009-01-05 10:36
References : http://marc.info/?l=linux-wireless&m=123115178019082&w=4

This entry is being used for tracking a regression from 2.6.28.  Please don't
close it until the problem is fixed in the mainline.
Comment 1 Rafael J. Wysocki 2009-01-12 04:32:25 UTC
On Monday 12 January 2009, Christian Borntraeger wrote:
> Am Sonntag 11 Januar 2009 schrieb Rafael J. Wysocki:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.28.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry   : http://bugzilla.kernel.org/show_bug.cgi?id=12415
> > Subject             : WARNING: at
> drivers/net/wireless/iwlwifi/iwl-sta.c:689
> > Submitter   : Christian Borntraeger <borntraeger@de.ibm.com>
> > Date                : 2009-01-05 10:36 (7 days old)
> > References  : http://marc.info/?l=linux-wireless&m=123115178019082&w=4
> 
> Meanwhile I have updated the device firmware and the kernel level. The 
> problems has disappeared.
Comment 2 Reinette Chatre 2009-01-19 10:14:04 UTC
Christian,

Could you please provide more detail regarding your last comment?

- to which kernel level did you update?
- which firmware did you use in your initial test, which firmware did you update to?
- does the problem go away if you update the firmware but not the kernel?

I would also like to clarify your original post. Are you saying that the issue is in released 2.6.28, but not in 2.6.28-rc9?

Thanks
Comment 3 Christian Borntraeger 2009-01-19 10:52:57 UTC
Reinette,

2.6.28 and earlier is fine. Looks like the problem was introduced after 2.6.28.
For example 2.6.28-05692-g7d3b56b was affected. I then decided to update my firmware for my 4965 (228.57.2.21 --> 228.57.2.23) but I also updated my kernel 
(to something around 2.6.29-rc1). I have not seen this problem with this configuration. The thing is, This "working" configuration had a broken suspend-to-ram.

Last weekend I finally managed to fix suspend to ram:
2.6.29-rc2 +
68564a4... work_on_cpu: don't try to get_online_cpus() in work_on_cpu.
e1d9ec6... work_on_cpu: Use our own workqueue.
6eb714c... cpufreq: use work_on_cpu in acpi-cpufreq.c for drv_read and 
from  git://git.kernel.org/pub/scm/linux/kernel/git/travis/linux-2.6-cpus4096-for-ingo.git
+ revert of a0d4922da2e4ccb0973095d8d29f36f6b1b5f703

This kernel showed the problem again. Seems that I have to go through some suspend/resume cycles to trigger the problem.

Christian
Comment 4 John W. Linville 2009-01-20 07:14:31 UTC
So to be clear, the warning only shows-up after a few successful suspend/resume cycles?
Comment 5 Christian Borntraeger 2009-01-20 07:58:17 UTC
It seems so, but I am not really sure. When S2R was broken, the system was not running very long. To get some data I looked into my /var/log/messages. It seems that there has to be at least one suspend/resume cycle before the warning appears.

As a side note: I am able to connect to the wireless network, even when the warning appears.
Comment 6 Christian Borntraeger 2009-01-20 10:39:16 UTC
The warning itself was added with before 2.6.28:

commit 40a9a8299116297429298e8fcee08235134883f7
Author: Tomas Winkler <tomas.winkler@intel.com>
Date:   Tue Nov 25 23:29:03 2008 +0200

    iwlwifi: clean key table in iwl_clear_stations_table function


but I have never seen the problem with 2.6.28
Comment 7 Christian Borntraeger 2009-01-22 00:08:55 UTC
can somebody change this bugzilla from needinfo to answered? I am not allowed to do so.
Comment 8 Reinette Chatre 2009-01-26 14:28:23 UTC
John,

Could you please help figure this out? The patch mentioned in comment #6 is in wireless-testing, but one of its hunks cannot be found in the current (2.6.29-rcX) repos. I tried git bisect, but the bad commits started going into the code where we only had 3945 and 4965 drivers. Similarly, when I tried the "history" feature of gitweb I was only able to find 591cd3e1874c7c826adb2d5c15fa72d0ab920622 as the most recent bad one and 65fa364b54604cc024c50f76a2e907831bda2cd5  the most recent good one. All commits shown in between these are a bunch of reverts.

The hunk that is missing from current code is:
diff --git a/drivers/net/wireless/iwlwifi/iwl-core.c b/drivers/net/wireless/iwlwifi/iwl-core.c
index 4c312c5..01a8458 100644
--- a/drivers/net/wireless/iwlwifi/iwl-core.c
+++ b/drivers/net/wireless/iwlwifi/iwl-core.c
@@ -290,6 +290,9 @@ void iwl_clear_stations_table(struct iwl_priv *priv)
        priv->num_stations = 0;
        memset(priv->stations, 0, sizeof(priv->stations));
 
+       /* clean ucode key table bit map */
+       priv->ucode_key_table = 0;
+
        spin_unlock_irqrestore(&priv->sta_lock, flags);
 }
 EXPORT_SYMBOL(iwl_clear_stations_table);


Please note that this code can now be found in iwl-sta.c

This bug should be resolved when we get that hunk back.

Thank you
Comment 9 Reinette Chatre 2009-01-27 14:49:54 UTC
The fix has been sent to linux-wireless (http://marc.info/?l=linux-wireless&m=123309515923154&w=2).
Comment 10 John W. Linville 2009-01-28 10:29:24 UTC
It looks like you figured this out without my help...sorry and thanks!
Comment 11 Christian Borntraeger 2009-02-01 01:06:49 UTC
The patch seems to help.

Tested-by: Christian Borntraeger <borntraeger@de.ibm.com>

Any chance to get this to Linus?
Comment 12 Reinette Chatre 2009-02-02 10:59:43 UTC
Thank you very much for testing.

Patch has been submitted for 2.6.29 on 01/28/2009 (http://marc.info/?l=linux-wireless&m=123316414222201&w=2 ). 
Comment 13 Rafael J. Wysocki 2009-02-04 17:39:32 UTC
Handled-By : Reinette Chatre <reinette.chatre@intel.com>
Patch : http://marc.info/?l=linux-wireless&m=123316414222201&w=2
Comment 14 Reinette Chatre 2009-02-10 10:39:07 UTC
The patch can now be found in Linus's tree.
Comment 15 John W. Linville 2009-02-10 10:58:15 UTC
commit 5e46882e2ecacd2ebd1bfba3caaa4a25ffbcb94d
Author: Reinette Chatre <reinette.chatre@intel.com>
Date:   Wed Jan 28 09:38:30 2009 -0800

    iwlwifi: clean key table in iwl_clear_stations_table

    Cleans uCode key table bit map iwl_clear_stations_table
    since all stations are cleared also the key table must be.

    Since the keys are not removed properly on suspend by mac80211
    this may result in exhausting key table on resume leading
    to memory corruption during removal

    Signed-off-by: Tomas Winkler <tomas.winkler@intel.com>
    Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
    Signed-off-by: John W. Linville <linville@tuxdriver.com>