Most recent kernel where this bug did not occur: 2.6.23 Distribution: Gentoo Problem Description: ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready RTNL: assertion failed at net/ipv6/addrconf.c (2164) Pid: 9, comm: events/0 Not tainted 2.6.24-rc4-git7 #1 [<78402cfb>] addrconf_notify+0x5b4/0x7b7 [<7812203a>] finish_task_switch+0x0/0x8c [<781346ff>] worker_thread+0x0/0x85 [<78438e23>] schedule+0x545/0x55f [<781408d1>] print_lock_contention_bug+0x11/0xd2 [<783bfa72>] rt_run_flush+0x43/0x8b [<783bfa93>] rt_run_flush+0x64/0x8b [<7813ac54>] notifier_call_chain+0x2a/0x52 [<7813ac9e>] raw_notifier_call_chain+0x17/0x1a [<783a3471>] netdev_state_change+0x18/0x29 [<783ac6a9>] __linkwatch_run_queue+0x150/0x17e [<783ac6f4>] linkwatch_event+0x1d/0x22 [<78133cdf>] run_workqueue+0xdb/0x1b6 [<78133c8b>] run_workqueue+0x87/0x1b6 [<783ac6d7>] linkwatch_event+0x0/0x22 [<781346ff>] worker_thread+0x0/0x85 [<78134778>] worker_thread+0x79/0x85 [<781371ad>] autoremove_wake_function+0x0/0x35 [<781370f6>] kthread+0x38/0x5e [<781370be>] kthread+0x0/0x5e [<78104baf>] kernel_thread_helper+0x7/0x10 ======================= RTNL: assertion failed at net/ipv6/addrconf.c (1610) Pid: 9, comm: events/0 Not tainted 2.6.24-rc4-git7 #1 [<78402290>] addrconf_add_dev+0x36/0x59 [<78402d27>] addrconf_notify+0x5e0/0x7b7 [<7812203a>] finish_task_switch+0x0/0x8c [<781346ff>] worker_thread+0x0/0x85 [<78438e23>] schedule+0x545/0x55f [<781408d1>] print_lock_contention_bug+0x11/0xd2 [<783bfa72>] rt_run_flush+0x43/0x8b [<783bfa93>] rt_run_flush+0x64/0x8b [<7813ac54>] notifier_call_chain+0x2a/0x52 [<7813ac9e>] raw_notifier_call_chain+0x17/0x1a [<783a3471>] netdev_state_change+0x18/0x29 [<783ac6a9>] __linkwatch_run_queue+0x150/0x17e [<783ac6f4>] linkwatch_event+0x1d/0x22 [<78133cdf>] run_workqueue+0xdb/0x1b6 [<78133c8b>] run_workqueue+0x87/0x1b6 [<783ac6d7>] linkwatch_event+0x0/0x22 [<781346ff>] worker_thread+0x0/0x85 [<78134778>] worker_thread+0x79/0x85 [<781371ad>] autoremove_wake_function+0x0/0x35 [<781370f6>] kthread+0x38/0x5e [<781370be>] kthread+0x0/0x5e [<78104baf>] kernel_thread_helper+0x7/0x10 ======================= RTNL: assertion failed at net/ipv6/addrconf.c (414) Pid: 9, comm: events/0 Not tainted 2.6.24-rc4-git7 #1 [<78402229>] ipv6_find_idev+0x36/0x67 [<78402297>] addrconf_add_dev+0x3d/0x59 [<78402d27>] addrconf_notify+0x5e0/0x7b7 [<7812203a>] finish_task_switch+0x0/0x8c [<781346ff>] worker_thread+0x0/0x85 [<78438e23>] schedule+0x545/0x55f [<781408d1>] print_lock_contention_bug+0x11/0xd2 [<783bfa72>] rt_run_flush+0x43/0x8b [<783bfa93>] rt_run_flush+0x64/0x8b [<7813ac54>] notifier_call_chain+0x2a/0x52 [<7813ac9e>] raw_notifier_call_chain+0x17/0x1a [<783a3471>] netdev_state_change+0x18/0x29 [<783ac6a9>] __linkwatch_run_queue+0x150/0x17e [<783ac6f4>] linkwatch_event+0x1d/0x22 [<78133cdf>] run_workqueue+0xdb/0x1b6 [<78133c8b>] run_workqueue+0x87/0x1b6 [<783ac6d7>] linkwatch_event+0x0/0x22 [<781346ff>] worker_thread+0x0/0x85 [<78134778>] worker_thread+0x79/0x85 [<781371ad>] autoremove_wake_function+0x0/0x35 [<781370f6>] kthread+0x38/0x5e [<781370be>] kthread+0x0/0x5e [<78104baf>] kernel_thread_helper+0x7/0x10 ======================= RTNL: assertion failed at net/ipv4/devinet.c (1055) Pid: 9, comm: events/0 Not tainted 2.6.24-rc4-git7 #1 [<783e3a93>] inetdev_event+0x56/0x465 [<7813ac54>] notifier_call_chain+0x2a/0x52 [<7813ac9e>] raw_notifier_call_chain+0x17/0x1a [<783a3471>] netdev_state_change+0x18/0x29 [<783ac6a9>] __linkwatch_run_queue+0x150/0x17e [<783ac6f4>] linkwatch_event+0x1d/0x22 [<78133cdf>] run_workqueue+0xdb/0x1b6 [<78133c8b>] run_workqueue+0x87/0x1b6 [<783ac6d7>] linkwatch_event+0x0/0x22 [<781346ff>] worker_thread+0x0/0x85 [<78134778>] worker_thread+0x79/0x85 [<781371ad>] autoremove_wake_function+0x0/0x35 [<781370f6>] kthread+0x38/0x5e [<781370be>] kthread+0x0/0x5e [<78104baf>] kernel_thread_helper+0x7/0x10 ======================= bond0: no IPv6 routers present Steps to reproduce: This happens when the system starts up.
Reply-To: akpm@linux-foundation.org On Tue, 11 Dec 2007 03:20:48 -0800 (PST) bugme-daemon@bugzilla.kernel.org wrote: > http://bugzilla.kernel.org/show_bug.cgi?id=9543 > > Summary: RTNL: assertion failed at net/ipv6/addrconf.c > (2164)/RTNL: assertion failed at net/ipv4/devinet.c > (1055) > Product: Drivers > Version: 2.5 > KernelVersion: 2.6.24-rc4-git7 > Platform: All > OS/Version: Linux > Tree: Mainline > Status: NEW > Severity: normal > Priority: P1 > Component: Network > AssignedTo: jgarzik@pobox.com > ReportedBy: olel@ans.pl > > > Most recent kernel where this bug did not occur: 2.6.23 > Distribution: Gentoo > > Problem Description: > ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready > RTNL: assertion failed at net/ipv6/addrconf.c (2164) > Pid: 9, comm: events/0 Not tainted 2.6.24-rc4-git7 #1 > [<78402cfb>] addrconf_notify+0x5b4/0x7b7 > [<7812203a>] finish_task_switch+0x0/0x8c > [<781346ff>] worker_thread+0x0/0x85 > [<78438e23>] schedule+0x545/0x55f > [<781408d1>] print_lock_contention_bug+0x11/0xd2 > [<783bfa72>] rt_run_flush+0x43/0x8b > [<783bfa93>] rt_run_flush+0x64/0x8b > [<7813ac54>] notifier_call_chain+0x2a/0x52 > [<7813ac9e>] raw_notifier_call_chain+0x17/0x1a > [<783a3471>] netdev_state_change+0x18/0x29 > [<783ac6a9>] __linkwatch_run_queue+0x150/0x17e > [<783ac6f4>] linkwatch_event+0x1d/0x22 > [<78133cdf>] run_workqueue+0xdb/0x1b6 > [<78133c8b>] run_workqueue+0x87/0x1b6 > [<783ac6d7>] linkwatch_event+0x0/0x22 > [<781346ff>] worker_thread+0x0/0x85 > [<78134778>] worker_thread+0x79/0x85 > [<781371ad>] autoremove_wake_function+0x0/0x35 > [<781370f6>] kthread+0x38/0x5e > [<781370be>] kthread+0x0/0x5e > [<78104baf>] kernel_thread_helper+0x7/0x10 > ======================= > RTNL: assertion failed at net/ipv6/addrconf.c (1610) Hopefully this is due to the bug you reported in bug #9542. Does this patch fix both issues? From: Andrew Morton <akpm@linux-foundation.org> Remove stray rtnl_unlock(). Addresses http://bugzilla.kernel.org/show_bug.cgi?id=9542 Cc: "David S. Miller" <davem@davemloft.net> Cc: Stephen Hemminger <shemminger@linux-foundation.org> Cc: Krzysztof Oledzki <olel@ans.pl> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- drivers/net/bonding/bond_sysfs.c | 2 -- 1 file changed, 2 deletions(-) diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix drivers/net/bonding/bond_sysfs.c --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix +++ a/drivers/net/bonding/bond_sysfs.c @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str out: write_unlock_bh(&bond->lock); - rtnl_unlock(); - return count; } static DEVICE_ATTR(primary, S_IRUGO | S_IWUSR, bonding_show_primary, bonding_store_primary); _
Will test ASAP, but I would like to wait for about 2h to test if 2.6.24-rc4-git7 solves bug #9182. It seems it does fix the dirty memory leak not but I need to to 100% sure.
On Tue, 11 Dec 2007, Andrew Morton wrote: > On Tue, 11 Dec 2007 03:20:48 -0800 (PST) bugme-daemon@bugzilla.kernel.org > wrote: > >> http://bugzilla.kernel.org/show_bug.cgi?id=9543 >> >> Summary: RTNL: assertion failed at net/ipv6/addrconf.c >> (2164)/RTNL: assertion failed at net/ipv4/devinet.c >> (1055) >> Product: Drivers >> Version: 2.5 >> KernelVersion: 2.6.24-rc4-git7 >> Platform: All >> OS/Version: Linux >> Tree: Mainline >> Status: NEW >> Severity: normal >> Priority: P1 >> Component: Network >> AssignedTo: jgarzik@pobox.com >> ReportedBy: olel@ans.pl >> >> >> Most recent kernel where this bug did not occur: 2.6.23 >> Distribution: Gentoo >> >> Problem Description: >> ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready >> RTNL: assertion failed at net/ipv6/addrconf.c (2164) >> Pid: 9, comm: events/0 Not tainted 2.6.24-rc4-git7 #1 >> [<78402cfb>] addrconf_notify+0x5b4/0x7b7 >> [<7812203a>] finish_task_switch+0x0/0x8c >> [<781346ff>] worker_thread+0x0/0x85 >> [<78438e23>] schedule+0x545/0x55f >> [<781408d1>] print_lock_contention_bug+0x11/0xd2 >> [<783bfa72>] rt_run_flush+0x43/0x8b >> [<783bfa93>] rt_run_flush+0x64/0x8b >> [<7813ac54>] notifier_call_chain+0x2a/0x52 >> [<7813ac9e>] raw_notifier_call_chain+0x17/0x1a >> [<783a3471>] netdev_state_change+0x18/0x29 >> [<783ac6a9>] __linkwatch_run_queue+0x150/0x17e >> [<783ac6f4>] linkwatch_event+0x1d/0x22 >> [<78133cdf>] run_workqueue+0xdb/0x1b6 >> [<78133c8b>] run_workqueue+0x87/0x1b6 >> [<783ac6d7>] linkwatch_event+0x0/0x22 >> [<781346ff>] worker_thread+0x0/0x85 >> [<78134778>] worker_thread+0x79/0x85 >> [<781371ad>] autoremove_wake_function+0x0/0x35 >> [<781370f6>] kthread+0x38/0x5e >> [<781370be>] kthread+0x0/0x5e >> [<78104baf>] kernel_thread_helper+0x7/0x10 >> ======================= >> RTNL: assertion failed at net/ipv6/addrconf.c (1610) > > > Hopefully this is due to the bug you reported in bug #9542. > > Does this patch fix both issues? Unfortunately not. I just updated bugzilla. Best regards, Krzysztof Ol
Reply-To: akpm@linux-foundation.org On Tue, 11 Dec 2007 16:04:27 +0100 (CET) Krzysztof Oledzki <olel@ans.pl> wrote: > >> [<783bfa72>] rt_run_flush+0x43/0x8b > >> [<783bfa93>] rt_run_flush+0x64/0x8b > >> [<7813ac54>] notifier_call_chain+0x2a/0x52 > >> [<7813ac9e>] raw_notifier_call_chain+0x17/0x1a > >> [<783a3471>] netdev_state_change+0x18/0x29 > >> [<783ac6a9>] __linkwatch_run_queue+0x150/0x17e > >> [<783ac6f4>] linkwatch_event+0x1d/0x22 > >> [<78133cdf>] run_workqueue+0xdb/0x1b6 > >> [<78133c8b>] run_workqueue+0x87/0x1b6 > >> [<783ac6d7>] linkwatch_event+0x0/0x22 > >> [<781346ff>] worker_thread+0x0/0x85 > >> [<78134778>] worker_thread+0x79/0x85 > >> [<781371ad>] autoremove_wake_function+0x0/0x35 > >> [<781370f6>] kthread+0x38/0x5e > >> [<781370be>] kthread+0x0/0x5e > >> [<78104baf>] kernel_thread_helper+0x7/0x10 > >> ======================= > >> RTNL: assertion failed at net/ipv6/addrconf.c (1610) > > > > > > Hopefully this is due to the bug you reported in bug #9542. > > > > Does this patch fix both issues? > > Unfortunately not. I just updated bugzilla. argh. Please conduct these discussions via emailed reply-to-all, not via the bugzilla web interface. I think my patch _did_ fix http://bugzilla.kernel.org/show_bug.cgi?id=9542 But then you hit a new bonding locking bug. Rafael, please track 9542 and 9543 as post-2.6.23 regressions, thanks.
Andrew Morton <akpm@linux-foundation.org> wrote: > > From: Andrew Morton <akpm@linux-foundation.org> > > Remove stray rtnl_unlock(). > > Addresses http://bugzilla.kernel.org/show_bug.cgi?id=9542 Adnrew, please cc Jay Vosburgh <fubar@us.ibm.com> on bonding issues. > diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > drivers/net/bonding/bond_sysfs.c > --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > +++ a/drivers/net/bonding/bond_sysfs.c > @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str > out: > write_unlock_bh(&bond->lock); > > - rtnl_unlock(); > - Looking at the changeset that added this perhaps the intention is to hold the lock? If so we should add an rtnl_lock to the start of the function. Thanks,
Herbert Xu <herbert@gondor.apana.org.au> wrote: >> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >> drivers/net/bonding/bond_sysfs.c >> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >> +++ a/drivers/net/bonding/bond_sysfs.c >> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str >> out: >> write_unlock_bh(&bond->lock); >> >> - rtnl_unlock(); >> - > >Looking at the changeset that added this perhaps the intention >is to hold the lock? If so we should add an rtnl_lock to the start >of the function. Yes, this function needs to hold locks, and more than just what's there now. I believe the following should be correct; I haven't tested it, though (I'm supposedly on vacation right now). The following change should be correct for the bonding_store_primary case discussed in this thread, and also corrects the bonding_store_active case which performs similar functions. The bond_change_active_slave and bond_select_active_slave functions both require rtnl, bond->lock for read and curr_slave_lock for write_bh, and no other locks. This is so that the lower level mode-specific functions can release locks down to just rtnl in order to call, e.g., dev_set_mac_address with the locks it expects (rtnl only). Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> diff --git a/drivers/net/bonding/bond_sysfs.c b/drivers/net/bonding/bond_sysfs.c index 11b76b3..28a2d80 100644 --- a/drivers/net/bonding/bond_sysfs.c +++ b/drivers/net/bonding/bond_sysfs.c @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device *d, struct slave *slave; struct bonding *bond = to_bond(d); - write_lock_bh(&bond->lock); + rtnl_lock(); + read_lock(&bond->lock); + write_lock_bh(&bond->curr_slave_lock); + if (!USES_PRIMARY(bond->params.mode)) { printk(KERN_INFO DRV_NAME ": %s: Unable to set primary slave; %s is in mode %d\n", @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device *d, } } out: - write_unlock_bh(&bond->lock); - + write_unlock_bh(&bond->curr_slave_lock); + read_unlock(&bond->lock); rtnl_unlock(); return count; @@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct device *d, struct bonding *bond = to_bond(d); rtnl_lock(); - write_lock_bh(&bond->lock); + read_lock(&bond->lock); + write_lock_bh(&bond->curr_slave_lock); if (!USES_PRIMARY(bond->params.mode)) { printk(KERN_INFO DRV_NAME @@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct device *d, } } out: - write_unlock_bh(&bond->lock); + write_unlock_bh(&bond->curr_slave_lock); + read_unlock(&bond->lock); rtnl_unlock(); return count;
Reply-To: andy@greyhouse.net On Wed, Dec 12, 2007 at 09:46:55AM -0800, Jay Vosburgh wrote: > Herbert Xu <herbert@gondor.apana.org.au> wrote: > > >> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > drivers/net/bonding/bond_sysfs.c > >> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > >> +++ a/drivers/net/bonding/bond_sysfs.c > >> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str > >> out: > >> write_unlock_bh(&bond->lock); > >> > >> - rtnl_unlock(); > >> - > > > >Looking at the changeset that added this perhaps the intention > >is to hold the lock? If so we should add an rtnl_lock to the start > >of the function. > > Yes, this function needs to hold locks, and more than just > what's there now. I believe the following should be correct; I haven't > tested it, though (I'm supposedly on vacation right now). > > The following change should be correct for the > bonding_store_primary case discussed in this thread, and also corrects > the bonding_store_active case which performs similar functions. > > The bond_change_active_slave and bond_select_active_slave > functions both require rtnl, bond->lock for read and curr_slave_lock for > write_bh, and no other locks. This is so that the lower level > mode-specific functions can release locks down to just rtnl in order to > call, e.g., dev_set_mac_address with the locks it expects (rtnl only). > > Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> This looks good to me as well.... Signed-off-by: Andy Gospodarek <andy@greyhouse.net>
On Wed, 12 Dec 2007, bugme-daemon@bugzilla.kernel.org wrote: > http://bugzilla.kernel.org/show_bug.cgi?id=9543 > > > > > > ------- Comment #6 from fubar@us.ibm.com 2007-12-12 09:47 ------- > Herbert Xu <herbert@gondor.apana.org.au> wrote: > >>> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >>> drivers/net/bonding/bond_sysfs.c >>> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >>> +++ a/drivers/net/bonding/bond_sysfs.c >>> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str >>> out: >>> write_unlock_bh(&bond->lock); >>> >>> - rtnl_unlock(); >>> - >> >> Looking at the changeset that added this perhaps the intention >> is to hold the lock? If so we should add an rtnl_lock to the start >> of the function. > > Yes, this function needs to hold locks, and more than just > what's there now. I believe the following should be correct; I haven't > tested it, though (I'm supposedly on vacation right now). > > The following change should be correct for the > bonding_store_primary case discussed in this thread, and also corrects > the bonding_store_active case which performs similar functions. > > The bond_change_active_slave and bond_select_active_slave > functions both require rtnl, bond->lock for read and curr_slave_lock for > write_bh, and no other locks. This is so that the lower level > mode-specific functions can release locks down to just rtnl in order to > call, e.g., dev_set_mac_address with the locks it expects (rtnl only). > > Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> > > diff --git a/drivers/net/bonding/bond_sysfs.c > b/drivers/net/bonding/bond_sysfs.c > index 11b76b3..28a2d80 100644 > --- a/drivers/net/bonding/bond_sysfs.c > +++ b/drivers/net/bonding/bond_sysfs.c > @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device *d, > struct slave *slave; > struct bonding *bond = to_bond(d); > > - write_lock_bh(&bond->lock); > + rtnl_lock(); > + read_lock(&bond->lock); > + write_lock_bh(&bond->curr_slave_lock); > + > if (!USES_PRIMARY(bond->params.mode)) { > printk(KERN_INFO DRV_NAME > ": %s: Unable to set primary slave; %s is in mode > %d\n", > @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device *d, > } > } > out: > - write_unlock_bh(&bond->lock); > - > + write_unlock_bh(&bond->curr_slave_lock); > + read_unlock(&bond->lock); > rtnl_unlock(); > > return count; > @@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct device > *d, > struct bonding *bond = to_bond(d); > > rtnl_lock(); > - write_lock_bh(&bond->lock); > + read_lock(&bond->lock); > + write_lock_bh(&bond->curr_slave_lock); > > if (!USES_PRIMARY(bond->params.mode)) { > printk(KERN_INFO DRV_NAME > @@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct device > *d, > } > } > out: > - write_unlock_bh(&bond->lock); > + write_unlock_bh(&bond->curr_slave_lock); > + read_unlock(&bond->lock); > rtnl_unlock(); > > return count; > Any chances for undamaged patch so I can test it? This one has tabs converted into spaces and long lines splitted. :( Best regards, Krzysztof Ol
On Wed, 12 Dec 2007, Jay Vosburgh wrote: > Herbert Xu <herbert@gondor.apana.org.au> wrote: > >>> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >>> drivers/net/bonding/bond_sysfs.c >>> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >>> +++ a/drivers/net/bonding/bond_sysfs.c >>> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str >>> out: >>> write_unlock_bh(&bond->lock); >>> >>> - rtnl_unlock(); >>> - >> >> Looking at the changeset that added this perhaps the intention >> is to hold the lock? If so we should add an rtnl_lock to the start >> of the function. > > Yes, this function needs to hold locks, and more than just > what's there now. I believe the following should be correct; I haven't > tested it, though (I'm supposedly on vacation right now). > > The following change should be correct for the > bonding_store_primary case discussed in this thread, and also corrects > the bonding_store_active case which performs similar functions. > > The bond_change_active_slave and bond_select_active_slave > functions both require rtnl, bond->lock for read and curr_slave_lock for > write_bh, and no other locks. This is so that the lower level > mode-specific functions can release locks down to just rtnl in order to > call, e.g., dev_set_mac_address with the locks it expects (rtnl only). > > Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> > > diff --git a/drivers/net/bonding/bond_sysfs.c > b/drivers/net/bonding/bond_sysfs.c > index 11b76b3..28a2d80 100644 > --- a/drivers/net/bonding/bond_sysfs.c > +++ b/drivers/net/bonding/bond_sysfs.c > @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device *d, > struct slave *slave; > struct bonding *bond = to_bond(d); > > - write_lock_bh(&bond->lock); > + rtnl_lock(); > + read_lock(&bond->lock); > + write_lock_bh(&bond->curr_slave_lock); > + > if (!USES_PRIMARY(bond->params.mode)) { > printk(KERN_INFO DRV_NAME > ": %s: Unable to set primary slave; %s is in mode %d\n", > @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device *d, > } > } > out: > - write_unlock_bh(&bond->lock); > - > + write_unlock_bh(&bond->curr_slave_lock); > + read_unlock(&bond->lock); > rtnl_unlock(); > > return count; > @@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct device > *d, > struct bonding *bond = to_bond(d); > > rtnl_lock(); > - write_lock_bh(&bond->lock); > + read_lock(&bond->lock); > + write_lock_bh(&bond->curr_slave_lock); > > if (!USES_PRIMARY(bond->params.mode)) { > printk(KERN_INFO DRV_NAME > @@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct device > *d, > } > } > out: > - write_unlock_bh(&bond->lock); > + write_unlock_bh(&bond->curr_slave_lock); > + read_unlock(&bond->lock); > rtnl_unlock(); > > return count; Vanilla 2.6.24-rc5 plus this patch: ========================================================= [ INFO: possible irq lock inversion dependency detected ] 2.6.24-rc5 #1 --------------------------------------------------------- events/0/9 just changed the state of lock: (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb but this lock took another, soft-read-irq-unsafe lock in the past: (&bond->lock){-.--} and interrupts could create inverse lock ordering between them. other info that might help us debug this: 4 locks held by events/0/9: #0: (events){--..}, at: [<c0133c57>] run_workqueue+0x87/0x1b6 #1: ((linkwatch_work).work){--..}, at: [<c0133c57>] run_workqueue+0x87/0x1b6 #2: (rtnl_mutex){--..}, at: [<c03abd50>] linkwatch_event+0x5/0x22 #3: (&ndev->lock){-.-+}, at: [<c0411b61>] mld_ifc_timer_expire+0x17/0x1fb the first lock's dependencies: -> (&mc->mca_lock){-+..} ops: 10 { initial-use at: [<c0104ee2>] dump_trace+0x83/0x8d [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0109ef2>] save_stack_trace+0x20/0x3a [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c [<c0143062>] lock_acquire+0x79/0x93 [<c04120d6>] igmp6_group_added+0x18/0x11d [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c04120d6>] igmp6_group_added+0x18/0x11d [<c04120d6>] igmp6_group_added+0x18/0x11d [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c05c5b40>] addrconf_init+0x13/0x193 [<c0199f63>] proc_net_fops_create+0x10/0x21 [<c0419b38>] ip6_flowlabel_init+0x1e/0x20 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff in-softirq-W at: [<c0142822>] __lock_acquire+0x440/0xc07 [<c013e0f3>] clockevents_program_event+0xe0/0xee [<c0143062>] lock_acquire+0x79/0x93 [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb [<c012df52>] run_timer_softirq+0xfa/0x15d [<c012a8a6>] __do_softirq+0x56/0xdb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c012a8b8>] __do_softirq+0x68/0xdb [<c012a961>] do_softirq+0x36/0x51 [<c012ae4a>] local_bh_enable_ip+0xad/0xed [<c03bf107>] rt_run_flush+0x64/0x8b [<c03e9296>] fib_netdev_event+0x61/0x65 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a2ae5>] netdev_state_change+0x18/0x29 [<c03abd1d>] __linkwatch_run_queue+0x150/0x17e [<c03abd68>] linkwatch_event+0x1d/0x22 [<c0133cab>] run_workqueue+0xdb/0x1b6 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c03abd4b>] linkwatch_event+0x0/0x22 [<c01346cb>] worker_thread+0x0/0x85 [<c0134744>] worker_thread+0x79/0x85 [<c0137179>] autoremove_wake_function+0x0/0x35 [<c01370c2>] kthread+0x38/0x5e [<c013708a>] kthread+0x0/0x5e [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c01417ee>] find_usage_backwards+0xbb/0xe2 [<c0104ee2>] dump_trace+0x83/0x8d [<c014286a>] __lock_acquire+0x488/0xc07 [<c0109ef2>] save_stack_trace+0x20/0x3a [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c [<c0143062>] lock_acquire+0x79/0x93 [<c04120d6>] igmp6_group_added+0x18/0x11d [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c04120d6>] igmp6_group_added+0x18/0x11d [<c04120d6>] igmp6_group_added+0x18/0x11d [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c05c5b40>] addrconf_init+0x13/0x193 [<c0199f63>] proc_net_fops_create+0x10/0x21 [<c0419b38>] ip6_flowlabel_init+0x1e/0x20 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c087e2d8>] __key.30798+0x0/0x8 -> (_xmit_ETHER){-...} ops: 8 { initial-use at: [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c0141986>] mark_lock+0x64/0x451 [<c014286a>] __lock_acquire+0x488/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0 ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff -> (&bonding_netdev_xmit_lock_key){-...} ops: 6 { initial-use at: [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c0141986>] mark_lock+0x64/0x451 [<c014286a>] __lock_acquire+0x488/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c0877804>] bonding_netdev_xmit_lock_key+0x0/0x8 -> (&bond->lock){-.--} ops: 99 { initial-use at: [<c013fe35>] put_lock_stats+0xa/0x1e [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c0439eee>] _read_lock_bh+0x3b/0x64 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563 [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf [<c03aaa26>] rtnetlink_event+0x3f/0x42 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a31de>] register_netdevice+0x2a7/0x2e7 [<c02ed862>] bond_create+0x1f2/0x26a [<c05bedcd>] bonding_init+0x761/0x7ea [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c014286a>] __lock_acquire+0x488/0xc07 [<c012093c>] try_to_wake_up+0x2ce/0x2d8 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c0439e25>] _write_lock_bh+0x3b/0x64 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c013fe35>] put_lock_stats+0xa/0x1e [<c03a14db>] __dev_set_rx_mode+0x7b/0x7d [<c03a1675>] dev_set_rx_mode+0x23/0x36 [<c03a3d50>] dev_open+0x5e/0x77 [<c03a2a1f>] dev_change_flags+0x9d/0x14b [<c03a1823>] __dev_get_by_name+0x68/0x73 [<c03e3850>] devinet_ioctl+0x22b/0x536 [<c03a3b45>] dev_ioctl+0x46f/0x5b7 [<c0399c78>] sock_ioctl+0x167/0x18b [<c0399b11>] sock_ioctl+0x0/0x18b [<c01725f7>] do_ioctl+0x1f/0x62 [<c0172867>] vfs_ioctl+0x22d/0x23f [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c01728ac>] sys_ioctl+0x33/0x4b [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff softirq-on-R at: [<c0141986>] mark_lock+0x64/0x451 [<c013575e>] __kernel_text_address+0x5/0xe [<c0104ee2>] dump_trace+0x83/0x8d [<c0142889>] __lock_acquire+0x4a7/0xc07 [<c013fc76>] save_trace+0x37/0x89 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c0143062>] lock_acquire+0x79/0x93 [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c0439f4d>] _read_lock+0x36/0x5f [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c0133cab>] run_workqueue+0xdb/0x1b6 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c02eee44>] bond_mii_monitor+0x0/0x85 [<c01346cb>] worker_thread+0x0/0x85 [<c0134744>] worker_thread+0x79/0x85 [<c0137179>] autoremove_wake_function+0x0/0x35 [<c01370c2>] kthread+0x38/0x5e [<c013708a>] kthread+0x0/0x5e [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-R at: [<c013fe0a>] get_lock_stats+0xd/0x2e [<c013fe35>] put_lock_stats+0xa/0x1e [<c0142844>] __lock_acquire+0x462/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c0439eee>] _read_lock_bh+0x3b/0x64 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563 [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf [<c03aaa26>] rtnetlink_event+0x3f/0x42 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a31de>] register_netdevice+0x2a7/0x2e7 [<c02ed862>] bond_create+0x1f2/0x26a [<c05bedcd>] bonding_init+0x761/0x7ea [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c08777d0>] __key.32969+0x0/0x8 -> (_xmit_ETHER){-...} ops: 8 { initial-use at: [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c0141986>] mark_lock+0x64/0x451 [<c014286a>] __lock_acquire+0x488/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0 ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0109ef2>] save_stack_trace+0x20/0x3a [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c02ee492>] bond_change_active_slave+0x1a9/0x3bf [<c02ec7c3>] bond_update_speed_duplex+0x26/0x65 [<c02ee9af>] bond_select_active_slave+0x95/0xcd [<c02ed22b>] bond_compute_features+0x45/0x84 [<c02ef9be>] bond_enslave+0x6a7/0x884 [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb [<c02ce8d7>] dev_attr_store+0x27/0x2c [<c019bcb9>] sysfs_write_file+0xad/0xe0 [<c019bc0c>] sysfs_write_file+0x0/0xe0 [<c0168ddc>] vfs_write+0x8a/0x10c [<c0118566>] do_page_fault+0x0/0x54a [<c0169361>] sys_write+0x41/0x67 [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff -> (lweventlist_lock){.+..} ops: 10 { initial-use at: [<c0141986>] mark_lock+0x64/0x451 [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c02e365c>] e1000_read_phy_reg+0x1c7/0x1d3 [<c02e348b>] e1000_write_phy_reg+0xb9/0xc3 [<c024a7de>] delay_tsc+0x25/0x3b [<c0143062>] lock_acquire+0x79/0x93 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abbbb>] linkwatch_fire_event+0x25/0x37 [<c02e1c43>] e1000_probe+0xad1/0xbe8 [<c0257f3f>] pci_device_probe+0x36/0x57 [<c02d0e5f>] driver_probe_device+0xe1/0x15f [<c043a2f9>] _spin_unlock+0x25/0x3b [<c04375b2>] klist_next+0x58/0x6d [<c02d0f6f>] __driver_attach+0x0/0x7f [<c02d0fb8>] __driver_attach+0x49/0x7f [<c02d0403>] bus_for_each_dev+0x36/0x58 [<c02d0cb7>] driver_attach+0x16/0x18 [<c02d0f6f>] __driver_attach+0x0/0x7f [<c02d06fa>] bus_add_driver+0x6d/0x18d [<c0258089>] __pci_register_driver+0x53/0x7f [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff in-softirq-W at: [<c011d20a>] __wake_up_common+0x32/0x5c [<c0142822>] __lock_acquire+0x440/0xc07 [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c02dff01>] e1000_watchdog+0x0/0x5c9 [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abbbb>] linkwatch_fire_event+0x25/0x37 [<c03aeb42>] netif_carrier_on+0x16/0x27 [<c02e0156>] e1000_watchdog+0x255/0x5c9 [<c02dff01>] e1000_watchdog+0x0/0x5c9 [<c012df52>] run_timer_softirq+0xfa/0x15d [<c012a8a6>] __do_softirq+0x56/0xdb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c012a8b8>] __do_softirq+0x68/0xdb [<c012a961>] do_softirq+0x36/0x51 [<c012ab07>] irq_exit+0x43/0x4e [<c0114122>] smp_apic_timer_interrupt+0x74/0x80 [<c0104a01>] apic_timer_interrupt+0x29/0x38 [<c0104a0b>] apic_timer_interrupt+0x33/0x38 [<c01600d8>] sys_swapon+0x29c/0x9aa [<c01021a6>] mwait_idle_with_hints+0x3b/0x3f [<c0102447>] mwait_idle+0x0/0xf [<c0102581>] cpu_idle+0x99/0xc6 [<c05a98c7>] start_kernel+0x2c7/0x2cf [<c05a90e0>] unknown_bootoption+0x0/0x195 [<ffffffff>] 0xffffffff } ... key at: [<c058a194>] lweventlist_lock+0x14/0x40 ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abbbb>] linkwatch_fire_event+0x25/0x37 [<c03aeb42>] netif_carrier_on+0x16/0x27 [<c02ede2c>] bond_set_carrier+0x31/0x55 [<c02ee9b6>] bond_select_active_slave+0x9c/0xcd [<c02ed22b>] bond_compute_features+0x45/0x84 [<c02ef9be>] bond_enslave+0x6a7/0x884 [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb [<c02ce8d7>] dev_attr_store+0x27/0x2c [<c019bcb9>] sysfs_write_file+0xad/0xe0 [<c019bc0c>] sysfs_write_file+0x0/0xe0 [<c0168ddc>] vfs_write+0x8a/0x10c [<c0118566>] do_page_fault+0x0/0x54a [<c0169361>] sys_write+0x41/0x67 [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c0143062>] lock_acquire+0x79/0x93 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c0439e25>] _write_lock_bh+0x3b/0x64 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c013fe35>] put_lock_stats+0xa/0x1e [<c03a14db>] __dev_set_rx_mode+0x7b/0x7d [<c03a1675>] dev_set_rx_mode+0x23/0x36 [<c03a3d50>] dev_open+0x5e/0x77 [<c03a2a1f>] dev_change_flags+0x9d/0x14b [<c03a1823>] __dev_get_by_name+0x68/0x73 [<c03e3850>] devinet_ioctl+0x22b/0x536 [<c03a3b45>] dev_ioctl+0x46f/0x5b7 [<c0399c78>] sock_ioctl+0x167/0x18b [<c0399b11>] sock_ioctl+0x0/0x18b [<c01725f7>] do_ioctl+0x1f/0x62 [<c0172867>] vfs_ioctl+0x22d/0x23f [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c01728ac>] sys_ioctl+0x33/0x4b [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff the second lock's dependencies: -> (&bond->lock){-.--} ops: 99 { initial-use at: [<c013fe35>] put_lock_stats+0xa/0x1e [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c0439eee>] _read_lock_bh+0x3b/0x64 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563 [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf [<c03aaa26>] rtnetlink_event+0x3f/0x42 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a31de>] register_netdevice+0x2a7/0x2e7 [<c02ed862>] bond_create+0x1f2/0x26a [<c05bedcd>] bonding_init+0x761/0x7ea [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c014286a>] __lock_acquire+0x488/0xc07 [<c012093c>] try_to_wake_up+0x2ce/0x2d8 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c0439e25>] _write_lock_bh+0x3b/0x64 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c013fe35>] put_lock_stats+0xa/0x1e [<c03a14db>] __dev_set_rx_mode+0x7b/0x7d [<c03a1675>] dev_set_rx_mode+0x23/0x36 [<c03a3d50>] dev_open+0x5e/0x77 [<c03a2a1f>] dev_change_flags+0x9d/0x14b [<c03a1823>] __dev_get_by_name+0x68/0x73 [<c03e3850>] devinet_ioctl+0x22b/0x536 [<c03a3b45>] dev_ioctl+0x46f/0x5b7 [<c0399c78>] sock_ioctl+0x167/0x18b [<c0399b11>] sock_ioctl+0x0/0x18b [<c01725f7>] do_ioctl+0x1f/0x62 [<c0172867>] vfs_ioctl+0x22d/0x23f [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c01728ac>] sys_ioctl+0x33/0x4b [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff softirq-on-R at: [<c0141986>] mark_lock+0x64/0x451 [<c013575e>] __kernel_text_address+0x5/0xe [<c0104ee2>] dump_trace+0x83/0x8d [<c0142889>] __lock_acquire+0x4a7/0xc07 [<c013fc76>] save_trace+0x37/0x89 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c0143062>] lock_acquire+0x79/0x93 [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c0439f4d>] _read_lock+0x36/0x5f [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c0133cab>] run_workqueue+0xdb/0x1b6 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c02eee44>] bond_mii_monitor+0x0/0x85 [<c01346cb>] worker_thread+0x0/0x85 [<c0134744>] worker_thread+0x79/0x85 [<c0137179>] autoremove_wake_function+0x0/0x35 [<c01370c2>] kthread+0x38/0x5e [<c013708a>] kthread+0x0/0x5e [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-R at: [<c013fe0a>] get_lock_stats+0xd/0x2e [<c013fe35>] put_lock_stats+0xa/0x1e [<c0142844>] __lock_acquire+0x462/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c0439eee>] _read_lock_bh+0x3b/0x64 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563 [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf [<c03aaa26>] rtnetlink_event+0x3f/0x42 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a31de>] register_netdevice+0x2a7/0x2e7 [<c02ed862>] bond_create+0x1f2/0x26a [<c05bedcd>] bonding_init+0x761/0x7ea [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c08777d0>] __key.32969+0x0/0x8 -> (_xmit_ETHER){-...} ops: 8 { initial-use at: [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c0141986>] mark_lock+0x64/0x451 [<c014286a>] __lock_acquire+0x488/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0 ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0109ef2>] save_stack_trace+0x20/0x3a [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c02ee492>] bond_change_active_slave+0x1a9/0x3bf [<c02ec7c3>] bond_update_speed_duplex+0x26/0x65 [<c02ee9af>] bond_select_active_slave+0x95/0xcd [<c02ed22b>] bond_compute_features+0x45/0x84 [<c02ef9be>] bond_enslave+0x6a7/0x884 [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb [<c02ce8d7>] dev_attr_store+0x27/0x2c [<c019bcb9>] sysfs_write_file+0xad/0xe0 [<c019bc0c>] sysfs_write_file+0x0/0xe0 [<c0168ddc>] vfs_write+0x8a/0x10c [<c0118566>] do_page_fault+0x0/0x54a [<c0169361>] sys_write+0x41/0x67 [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff -> (lweventlist_lock){.+..} ops: 10 { initial-use at: [<c0141986>] mark_lock+0x64/0x451 [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c02e365c>] e1000_read_phy_reg+0x1c7/0x1d3 [<c02e348b>] e1000_write_phy_reg+0xb9/0xc3 [<c024a7de>] delay_tsc+0x25/0x3b [<c0143062>] lock_acquire+0x79/0x93 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abbbb>] linkwatch_fire_event+0x25/0x37 [<c02e1c43>] e1000_probe+0xad1/0xbe8 [<c0257f3f>] pci_device_probe+0x36/0x57 [<c02d0e5f>] driver_probe_device+0xe1/0x15f [<c043a2f9>] _spin_unlock+0x25/0x3b [<c04375b2>] klist_next+0x58/0x6d [<c02d0f6f>] __driver_attach+0x0/0x7f [<c02d0fb8>] __driver_attach+0x49/0x7f [<c02d0403>] bus_for_each_dev+0x36/0x58 [<c02d0cb7>] driver_attach+0x16/0x18 [<c02d0f6f>] __driver_attach+0x0/0x7f [<c02d06fa>] bus_add_driver+0x6d/0x18d [<c0258089>] __pci_register_driver+0x53/0x7f [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff in-softirq-W at: [<c011d20a>] __wake_up_common+0x32/0x5c [<c0142822>] __lock_acquire+0x440/0xc07 [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c02dff01>] e1000_watchdog+0x0/0x5c9 [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abbbb>] linkwatch_fire_event+0x25/0x37 [<c03aeb42>] netif_carrier_on+0x16/0x27 [<c02e0156>] e1000_watchdog+0x255/0x5c9 [<c02dff01>] e1000_watchdog+0x0/0x5c9 [<c012df52>] run_timer_softirq+0xfa/0x15d [<c012a8a6>] __do_softirq+0x56/0xdb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c012a8b8>] __do_softirq+0x68/0xdb [<c012a961>] do_softirq+0x36/0x51 [<c012ab07>] irq_exit+0x43/0x4e [<c0114122>] smp_apic_timer_interrupt+0x74/0x80 [<c0104a01>] apic_timer_interrupt+0x29/0x38 [<c0104a0b>] apic_timer_interrupt+0x33/0x38 [<c01600d8>] sys_swapon+0x29c/0x9aa [<c01021a6>] mwait_idle_with_hints+0x3b/0x3f [<c0102447>] mwait_idle+0x0/0xf [<c0102581>] cpu_idle+0x99/0xc6 [<c05a98c7>] start_kernel+0x2c7/0x2cf [<c05a90e0>] unknown_bootoption+0x0/0x195 [<ffffffff>] 0xffffffff } ... key at: [<c058a194>] lweventlist_lock+0x14/0x40 ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abbbb>] linkwatch_fire_event+0x25/0x37 [<c03aeb42>] netif_carrier_on+0x16/0x27 [<c02ede2c>] bond_set_carrier+0x31/0x55 [<c02ee9b6>] bond_select_active_slave+0x9c/0xcd [<c02ed22b>] bond_compute_features+0x45/0x84 [<c02ef9be>] bond_enslave+0x6a7/0x884 [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb [<c02ce8d7>] dev_attr_store+0x27/0x2c [<c019bcb9>] sysfs_write_file+0xad/0xe0 [<c019bc0c>] sysfs_write_file+0x0/0xe0 [<c0168ddc>] vfs_write+0x8a/0x10c [<c0118566>] do_page_fault+0x0/0x54a [<c0169361>] sys_write+0x41/0x67 [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff stack backtrace: Pid: 9, comm: events/0 Not tainted 2.6.24-rc5 #1 [<c0140b38>] print_irq_inversion_bug+0x108/0x112 [<c014191d>] check_usage_forwards+0x3c/0x41 [<c0141b09>] mark_lock+0x1e7/0x451 [<c0142822>] __lock_acquire+0x440/0xc07 [<c013e0f3>] clockevents_program_event+0xe0/0xee [<c0143062>] lock_acquire+0x79/0x93 [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb [<c012df52>] run_timer_softirq+0xfa/0x15d [<c012a8a6>] __do_softirq+0x56/0xdb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c012a8b8>] __do_softirq+0x68/0xdb [<c012a961>] do_softirq+0x36/0x51 [<c012ae4a>] local_bh_enable_ip+0xad/0xed [<c03bf107>] rt_run_flush+0x64/0x8b [<c03e9296>] fib_netdev_event+0x61/0x65 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a2ae5>] netdev_state_change+0x18/0x29 [<c03abd1d>] __linkwatch_run_queue+0x150/0x17e [<c03abd68>] linkwatch_event+0x1d/0x22 [<c0133cab>] run_workqueue+0xdb/0x1b6 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c03abd4b>] linkwatch_event+0x0/0x22 [<c01346cb>] worker_thread+0x0/0x85 [<c0134744>] worker_thread+0x79/0x85 [<c0137179>] autoremove_wake_function+0x0/0x35 [<c01370c2>] kthread+0x38/0x5e [<c013708a>] kthread+0x0/0x5e [<c0104baf>] kernel_thread_helper+0x7/0x10 ======================= Best regards, Krzysztof Ol
Reply-To: andy@greyhouse.net On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote: > > > On Wed, 12 Dec 2007, Jay Vosburgh wrote: > > >Herbert Xu <herbert@gondor.apana.org.au> wrote: > > > >>>diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > >>>drivers/net/bonding/bond_sysfs.c > >>>--- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > >>>+++ a/drivers/net/bonding/bond_sysfs.c > >>>@@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str > >>>out: > >>> write_unlock_bh(&bond->lock); > >>> > >>>- rtnl_unlock(); > >>>- > >> > >>Looking at the changeset that added this perhaps the intention > >>is to hold the lock? If so we should add an rtnl_lock to the start > >>of the function. > > > > Yes, this function needs to hold locks, and more than just > >what's there now. I believe the following should be correct; I haven't > >tested it, though (I'm supposedly on vacation right now). > > > > The following change should be correct for the > >bonding_store_primary case discussed in this thread, and also corrects > >the bonding_store_active case which performs similar functions. > > > > The bond_change_active_slave and bond_select_active_slave > >functions both require rtnl, bond->lock for read and curr_slave_lock for > >write_bh, and no other locks. This is so that the lower level > >mode-specific functions can release locks down to just rtnl in order to > >call, e.g., dev_set_mac_address with the locks it expects (rtnl only). > > > >Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> > > > >diff --git a/drivers/net/bonding/bond_sysfs.c > >b/drivers/net/bonding/bond_sysfs.c > >index 11b76b3..28a2d80 100644 > >--- a/drivers/net/bonding/bond_sysfs.c > >+++ b/drivers/net/bonding/bond_sysfs.c > >@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device > >*d, > > struct slave *slave; > > struct bonding *bond = to_bond(d); > > > >- write_lock_bh(&bond->lock); > >+ rtnl_lock(); > >+ read_lock(&bond->lock); > >+ write_lock_bh(&bond->curr_slave_lock); > >+ > > if (!USES_PRIMARY(bond->params.mode)) { > > printk(KERN_INFO DRV_NAME > > ": %s: Unable to set primary slave; %s is in mode > > %d\n", > >@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device > >*d, > > } > > } > >out: > >- write_unlock_bh(&bond->lock); > >- > >+ write_unlock_bh(&bond->curr_slave_lock); > >+ read_unlock(&bond->lock); > > rtnl_unlock(); > > > > return count; > >@@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct > >device *d, > > struct bonding *bond = to_bond(d); > > > > rtnl_lock(); > >- write_lock_bh(&bond->lock); > >+ read_lock(&bond->lock); > >+ write_lock_bh(&bond->curr_slave_lock); > > > > if (!USES_PRIMARY(bond->params.mode)) { > > printk(KERN_INFO DRV_NAME > >@@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct > >device *d, > > } > > } > >out: > >- write_unlock_bh(&bond->lock); > >+ write_unlock_bh(&bond->curr_slave_lock); > >+ read_unlock(&bond->lock); > > rtnl_unlock(); > > > > return count; > > Vanilla 2.6.24-rc5 plus this patch: > > ========================================================= > [ INFO: possible irq lock inversion dependency detected ] > 2.6.24-rc5 #1 > --------------------------------------------------------- > events/0/9 just changed the state of lock: > (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb > but this lock took another, soft-read-irq-unsafe lock in the past: > (&bond->lock){-.--} > > and interrupts could create inverse lock ordering between them. > > Grrr, I should have seen that -- sorry. Try your luck with this instead: diff --git a/drivers/net/bonding/bond_sysfs.c b/drivers/net/bonding/bond_sysfs.c index 11b76b3..0694254 100644 --- a/drivers/net/bonding/bond_sysfs.c +++ b/drivers/net/bonding/bond_sysfs.c @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device *d, struct slave *slave; struct bonding *bond = to_bond(d); - write_lock_bh(&bond->lock); + rtnl_lock(); + read_lock_bh(&bond->lock); + write_lock_bh(&bond->curr_slave_lock); + if (!USES_PRIMARY(bond->params.mode)) { printk(KERN_INFO DRV_NAME ": %s: Unable to set primary slave; %s is in mode %d\n", @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device *d, } } out: - write_unlock_bh(&bond->lock); - + write_unlock_bh(&bond->curr_slave_lock); + read_unlock_bh(&bond->lock); rtnl_unlock(); return count; @@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct device *d, struct bonding *bond = to_bond(d); rtnl_lock(); - write_lock_bh(&bond->lock); + read_lock_bh(&bond->lock); + write_lock_bh(&bond->curr_slave_lock); if (!USES_PRIMARY(bond->params.mode)) { printk(KERN_INFO DRV_NAME @@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct device *d, } } out: - write_unlock_bh(&bond->lock); + write_unlock_bh(&bond->curr_slave_lock); + read_unlock_bh(&bond->lock); rtnl_unlock(); return count;
On Fri, 14 Dec 2007, Andy Gospodarek wrote: > On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote: >> >> >> On Wed, 12 Dec 2007, Jay Vosburgh wrote: >> >>> Herbert Xu <herbert@gondor.apana.org.au> wrote: >>> >>>>> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >>>>> drivers/net/bonding/bond_sysfs.c >>>>> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >>>>> +++ a/drivers/net/bonding/bond_sysfs.c >>>>> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str >>>>> out: >>>>> write_unlock_bh(&bond->lock); >>>>> >>>>> - rtnl_unlock(); >>>>> - >>>> >>>> Looking at the changeset that added this perhaps the intention >>>> is to hold the lock? If so we should add an rtnl_lock to the start >>>> of the function. >>> >>> Yes, this function needs to hold locks, and more than just >>> what's there now. I believe the following should be correct; I haven't >>> tested it, though (I'm supposedly on vacation right now). >>> >>> The following change should be correct for the >>> bonding_store_primary case discussed in this thread, and also corrects >>> the bonding_store_active case which performs similar functions. >>> >>> The bond_change_active_slave and bond_select_active_slave >>> functions both require rtnl, bond->lock for read and curr_slave_lock for >>> write_bh, and no other locks. This is so that the lower level >>> mode-specific functions can release locks down to just rtnl in order to >>> call, e.g., dev_set_mac_address with the locks it expects (rtnl only). >>> >>> Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> >>> >>> diff --git a/drivers/net/bonding/bond_sysfs.c >>> b/drivers/net/bonding/bond_sysfs.c >>> index 11b76b3..28a2d80 100644 >>> --- a/drivers/net/bonding/bond_sysfs.c >>> +++ b/drivers/net/bonding/bond_sysfs.c >>> @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device >>> *d, >>> struct slave *slave; >>> struct bonding *bond = to_bond(d); >>> >>> - write_lock_bh(&bond->lock); >>> + rtnl_lock(); >>> + read_lock(&bond->lock); >>> + write_lock_bh(&bond->curr_slave_lock); >>> + >>> if (!USES_PRIMARY(bond->params.mode)) { >>> printk(KERN_INFO DRV_NAME >>> ": %s: Unable to set primary slave; %s is in mode >>> %d\n", >>> @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device >>> *d, >>> } >>> } >>> out: >>> - write_unlock_bh(&bond->lock); >>> - >>> + write_unlock_bh(&bond->curr_slave_lock); >>> + read_unlock(&bond->lock); >>> rtnl_unlock(); >>> >>> return count; >>> @@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct >>> device *d, >>> struct bonding *bond = to_bond(d); >>> >>> rtnl_lock(); >>> - write_lock_bh(&bond->lock); >>> + read_lock(&bond->lock); >>> + write_lock_bh(&bond->curr_slave_lock); >>> >>> if (!USES_PRIMARY(bond->params.mode)) { >>> printk(KERN_INFO DRV_NAME >>> @@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct >>> device *d, >>> } >>> } >>> out: >>> - write_unlock_bh(&bond->lock); >>> + write_unlock_bh(&bond->curr_slave_lock); >>> + read_unlock(&bond->lock); >>> rtnl_unlock(); >>> >>> return count; >> >> Vanilla 2.6.24-rc5 plus this patch: >> >> ========================================================= >> [ INFO: possible irq lock inversion dependency detected ] >> 2.6.24-rc5 #1 >> --------------------------------------------------------- >> events/0/9 just changed the state of lock: >> (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb >> but this lock took another, soft-read-irq-unsafe lock in the past: >> (&bond->lock){-.--} >> >> and interrupts could create inverse lock ordering between them. >> >> > > Grrr, I should have seen that -- sorry. Try your luck with this instead: <CUT> No luck. bonding: bond0: setting mode to active-backup (1). bonding: bond0: Setting MII monitoring interval to 100. ADDRCONF(NETDEV_UP): bond0: link is not ready bonding: bond0: Adding slave eth0. e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX bonding: bond0: making interface eth0 the new active one. bonding: bond0: first active interface up! bonding: bond0: enslaving eth0 as an active interface with an up link. bonding: bond0: Adding slave eth1. ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready ========================================================= [ INFO: possible irq lock inversion dependency detected ] 2.6.24-rc5 #1 --------------------------------------------------------- events/0/9 just changed the state of lock: (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb but this lock took another, soft-read-irq-unsafe lock in the past: (&bond->lock){-.--} and interrupts could create inverse lock ordering between them. other info that might help us debug this: 4 locks held by events/0/9: #0: (events){--..}, at: [<c0133c57>] run_workqueue+0x87/0x1b6 #1: ((linkwatch_work).work){--..}, at: [<c0133c57>] run_workqueue+0x87/0x1b6 #2: (rtnl_mutex){--..}, at: [<c03abd50>] linkwatch_event+0x5/0x22 #3: (&ndev->lock){-.-+}, at: [<c0411b61>] mld_ifc_timer_expire+0x17/0x1fb the first lock's dependencies: -> (&mc->mca_lock){-+..} ops: 10 { initial-use at: [<c0104ee2>] dump_trace+0x83/0x8d [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0109ef2>] save_stack_trace+0x20/0x3a [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c [<c0143062>] lock_acquire+0x79/0x93 [<c04120d6>] igmp6_group_added+0x18/0x11d [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c04120d6>] igmp6_group_added+0x18/0x11d [<c04120d6>] igmp6_group_added+0x18/0x11d [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c05c5b40>] addrconf_init+0x13/0x193 [<c0199f63>] proc_net_fops_create+0x10/0x21 [<c0419b38>] ip6_flowlabel_init+0x1e/0x20 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff in-softirq-W at: [<c0142822>] __lock_acquire+0x440/0xc07 [<c013fe0a>] get_lock_stats+0xd/0x2e [<c013fe35>] put_lock_stats+0xa/0x1e [<c0143062>] lock_acquire+0x79/0x93 [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb [<c012df52>] run_timer_softirq+0xfa/0x15d [<c012a8a6>] __do_softirq+0x56/0xdb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c012a8b8>] __do_softirq+0x68/0xdb [<c012a961>] do_softirq+0x36/0x51 [<c012ae4a>] local_bh_enable_ip+0xad/0xed [<c03bf107>] rt_run_flush+0x64/0x8b [<c03e9296>] fib_netdev_event+0x61/0x65 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a2ae5>] netdev_state_change+0x18/0x29 [<c03abd1d>] __linkwatch_run_queue+0x150/0x17e [<c03abd68>] linkwatch_event+0x1d/0x22 [<c0133cab>] run_workqueue+0xdb/0x1b6 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c03abd4b>] linkwatch_event+0x0/0x22 [<c01346cb>] worker_thread+0x0/0x85 [<c0134744>] worker_thread+0x79/0x85 [<c0137179>] autoremove_wake_function+0x0/0x35 [<c01370c2>] kthread+0x38/0x5e [<c013708a>] kthread+0x0/0x5e [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c01417ee>] find_usage_backwards+0xbb/0xe2 [<c0104ee2>] dump_trace+0x83/0x8d [<c014286a>] __lock_acquire+0x488/0xc07 [<c0109ef2>] save_stack_trace+0x20/0x3a [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c [<c0143062>] lock_acquire+0x79/0x93 [<c04120d6>] igmp6_group_added+0x18/0x11d [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c04120d6>] igmp6_group_added+0x18/0x11d [<c04120d6>] igmp6_group_added+0x18/0x11d [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c05c5b40>] addrconf_init+0x13/0x193 [<c0199f63>] proc_net_fops_create+0x10/0x21 [<c0419b38>] ip6_flowlabel_init+0x1e/0x20 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c087e2d8>] __key.30798+0x0/0x8 -> (_xmit_ETHER){-...} ops: 8 { initial-use at: [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c0141986>] mark_lock+0x64/0x451 [<c014286a>] __lock_acquire+0x488/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0 ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff -> (&bonding_netdev_xmit_lock_key){-...} ops: 6 { initial-use at: [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c0141986>] mark_lock+0x64/0x451 [<c014286a>] __lock_acquire+0x488/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c0877804>] bonding_netdev_xmit_lock_key+0x0/0x8 -> (&bond->lock){-.--} ops: 98 { initial-use at: [<c013fe35>] put_lock_stats+0xa/0x1e [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c0439eee>] _read_lock_bh+0x3b/0x64 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563 [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf [<c03aaa26>] rtnetlink_event+0x3f/0x42 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a31de>] register_netdevice+0x2a7/0x2e7 [<c02ed862>] bond_create+0x1f2/0x26a [<c05bedcd>] bonding_init+0x761/0x7ea [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c014286a>] __lock_acquire+0x488/0xc07 [<c012093c>] try_to_wake_up+0x2ce/0x2d8 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c0439e25>] _write_lock_bh+0x3b/0x64 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c013fe35>] put_lock_stats+0xa/0x1e [<c03a14db>] __dev_set_rx_mode+0x7b/0x7d [<c03a1675>] dev_set_rx_mode+0x23/0x36 [<c03a3d50>] dev_open+0x5e/0x77 [<c03a2a1f>] dev_change_flags+0x9d/0x14b [<c03a1823>] __dev_get_by_name+0x68/0x73 [<c03e3850>] devinet_ioctl+0x22b/0x536 [<c03a3b45>] dev_ioctl+0x46f/0x5b7 [<c0399c78>] sock_ioctl+0x167/0x18b [<c0399b11>] sock_ioctl+0x0/0x18b [<c01725f7>] do_ioctl+0x1f/0x62 [<c0172867>] vfs_ioctl+0x22d/0x23f [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c01728ac>] sys_ioctl+0x33/0x4b [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff softirq-on-R at: [<c0141986>] mark_lock+0x64/0x451 [<c013575e>] __kernel_text_address+0x5/0xe [<c0104ee2>] dump_trace+0x83/0x8d [<c0142889>] __lock_acquire+0x4a7/0xc07 [<c013fc76>] save_trace+0x37/0x89 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c0143062>] lock_acquire+0x79/0x93 [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c0439f4d>] _read_lock+0x36/0x5f [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c0133cab>] run_workqueue+0xdb/0x1b6 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c02eee44>] bond_mii_monitor+0x0/0x85 [<c01346cb>] worker_thread+0x0/0x85 [<c0134744>] worker_thread+0x79/0x85 [<c0137179>] autoremove_wake_function+0x0/0x35 [<c01370c2>] kthread+0x38/0x5e [<c013708a>] kthread+0x0/0x5e [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-R at: [<c013fe0a>] get_lock_stats+0xd/0x2e [<c013fe35>] put_lock_stats+0xa/0x1e [<c0142844>] __lock_acquire+0x462/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c0439eee>] _read_lock_bh+0x3b/0x64 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563 [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf [<c03aaa26>] rtnetlink_event+0x3f/0x42 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a31de>] register_netdevice+0x2a7/0x2e7 [<c02ed862>] bond_create+0x1f2/0x26a [<c05bedcd>] bonding_init+0x761/0x7ea [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c08777d0>] __key.32969+0x0/0x8 -> (_xmit_ETHER){-...} ops: 8 { initial-use at: [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c0141986>] mark_lock+0x64/0x451 [<c014286a>] __lock_acquire+0x488/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0 ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0109ef2>] save_stack_trace+0x20/0x3a [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c02ee492>] bond_change_active_slave+0x1a9/0x3bf [<c02ec7c3>] bond_update_speed_duplex+0x26/0x65 [<c02ee9af>] bond_select_active_slave+0x95/0xcd [<c02ed22b>] bond_compute_features+0x45/0x84 [<c02ef9be>] bond_enslave+0x6a7/0x884 [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb [<c02ce8d7>] dev_attr_store+0x27/0x2c [<c019bcb9>] sysfs_write_file+0xad/0xe0 [<c019bc0c>] sysfs_write_file+0x0/0xe0 [<c0168ddc>] vfs_write+0x8a/0x10c [<c0118566>] do_page_fault+0x0/0x54a [<c0169361>] sys_write+0x41/0x67 [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff -> (lweventlist_lock){.+..} ops: 10 { initial-use at: [<c0141986>] mark_lock+0x64/0x451 [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c02e365c>] e1000_read_phy_reg+0x1c7/0x1d3 [<c02e348b>] e1000_write_phy_reg+0xb9/0xc3 [<c024a7de>] delay_tsc+0x25/0x3b [<c0143062>] lock_acquire+0x79/0x93 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abbbb>] linkwatch_fire_event+0x25/0x37 [<c02e1c43>] e1000_probe+0xad1/0xbe8 [<c0257f3f>] pci_device_probe+0x36/0x57 [<c02d0e5f>] driver_probe_device+0xe1/0x15f [<c043a2f9>] _spin_unlock+0x25/0x3b [<c04375b2>] klist_next+0x58/0x6d [<c02d0f6f>] __driver_attach+0x0/0x7f [<c02d0fb8>] __driver_attach+0x49/0x7f [<c02d0403>] bus_for_each_dev+0x36/0x58 [<c02d0cb7>] driver_attach+0x16/0x18 [<c02d0f6f>] __driver_attach+0x0/0x7f [<c02d06fa>] bus_add_driver+0x6d/0x18d [<c0258089>] __pci_register_driver+0x53/0x7f [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff in-softirq-W at: [<c011d20a>] __wake_up_common+0x32/0x5c [<c0142822>] __lock_acquire+0x440/0xc07 [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c02dff01>] e1000_watchdog+0x0/0x5c9 [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abbbb>] linkwatch_fire_event+0x25/0x37 [<c03aeb42>] netif_carrier_on+0x16/0x27 [<c02e0156>] e1000_watchdog+0x255/0x5c9 [<c02dff01>] e1000_watchdog+0x0/0x5c9 [<c012df52>] run_timer_softirq+0xfa/0x15d [<c012a8a6>] __do_softirq+0x56/0xdb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c012a8b8>] __do_softirq+0x68/0xdb [<c012a961>] do_softirq+0x36/0x51 [<c012ab07>] irq_exit+0x43/0x4e [<c0114122>] smp_apic_timer_interrupt+0x74/0x80 [<c0104a01>] apic_timer_interrupt+0x29/0x38 [<c0104a0b>] apic_timer_interrupt+0x33/0x38 [<c01600d8>] sys_swapon+0x29c/0x9aa [<c01021a6>] mwait_idle_with_hints+0x3b/0x3f [<c0102447>] mwait_idle+0x0/0xf [<c0102581>] cpu_idle+0x99/0xc6 [<c05a98c7>] start_kernel+0x2c7/0x2cf [<c05a90e0>] unknown_bootoption+0x0/0x195 [<ffffffff>] 0xffffffff } ... key at: [<c058a194>] lweventlist_lock+0x14/0x40 ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abbbb>] linkwatch_fire_event+0x25/0x37 [<c03aeb42>] netif_carrier_on+0x16/0x27 [<c02ede2c>] bond_set_carrier+0x31/0x55 [<c02ee9b6>] bond_select_active_slave+0x9c/0xcd [<c02ed22b>] bond_compute_features+0x45/0x84 [<c02ef9be>] bond_enslave+0x6a7/0x884 [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb [<c02ce8d7>] dev_attr_store+0x27/0x2c [<c019bcb9>] sysfs_write_file+0xad/0xe0 [<c019bc0c>] sysfs_write_file+0x0/0xe0 [<c0168ddc>] vfs_write+0x8a/0x10c [<c0118566>] do_page_fault+0x0/0x54a [<c0169361>] sys_write+0x41/0x67 [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c0143062>] lock_acquire+0x79/0x93 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c0439e25>] _write_lock_bh+0x3b/0x64 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c013fe35>] put_lock_stats+0xa/0x1e [<c03a14db>] __dev_set_rx_mode+0x7b/0x7d [<c03a1675>] dev_set_rx_mode+0x23/0x36 [<c03a3d50>] dev_open+0x5e/0x77 [<c03a2a1f>] dev_change_flags+0x9d/0x14b [<c03a1823>] __dev_get_by_name+0x68/0x73 [<c03e3850>] devinet_ioctl+0x22b/0x536 [<c03a3b45>] dev_ioctl+0x46f/0x5b7 [<c0399c78>] sock_ioctl+0x167/0x18b [<c0399b11>] sock_ioctl+0x0/0x18b [<c01725f7>] do_ioctl+0x1f/0x62 [<c0172867>] vfs_ioctl+0x22d/0x23f [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c01728ac>] sys_ioctl+0x33/0x4b [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff the second lock's dependencies: -> (&bond->lock){-.--} ops: 98 { initial-use at: [<c013fe35>] put_lock_stats+0xa/0x1e [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c0439eee>] _read_lock_bh+0x3b/0x64 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563 [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf [<c03aaa26>] rtnetlink_event+0x3f/0x42 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a31de>] register_netdevice+0x2a7/0x2e7 [<c02ed862>] bond_create+0x1f2/0x26a [<c05bedcd>] bonding_init+0x761/0x7ea [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c014286a>] __lock_acquire+0x488/0xc07 [<c012093c>] try_to_wake_up+0x2ce/0x2d8 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c0439e25>] _write_lock_bh+0x3b/0x64 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c013fe35>] put_lock_stats+0xa/0x1e [<c03a14db>] __dev_set_rx_mode+0x7b/0x7d [<c03a1675>] dev_set_rx_mode+0x23/0x36 [<c03a3d50>] dev_open+0x5e/0x77 [<c03a2a1f>] dev_change_flags+0x9d/0x14b [<c03a1823>] __dev_get_by_name+0x68/0x73 [<c03e3850>] devinet_ioctl+0x22b/0x536 [<c03a3b45>] dev_ioctl+0x46f/0x5b7 [<c0399c78>] sock_ioctl+0x167/0x18b [<c0399b11>] sock_ioctl+0x0/0x18b [<c01725f7>] do_ioctl+0x1f/0x62 [<c0172867>] vfs_ioctl+0x22d/0x23f [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c01728ac>] sys_ioctl+0x33/0x4b [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff softirq-on-R at: [<c0141986>] mark_lock+0x64/0x451 [<c013575e>] __kernel_text_address+0x5/0xe [<c0104ee2>] dump_trace+0x83/0x8d [<c0142889>] __lock_acquire+0x4a7/0xc07 [<c013fc76>] save_trace+0x37/0x89 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c0143062>] lock_acquire+0x79/0x93 [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c0439f4d>] _read_lock+0x36/0x5f [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c0133cab>] run_workqueue+0xdb/0x1b6 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c02eee44>] bond_mii_monitor+0x0/0x85 [<c01346cb>] worker_thread+0x0/0x85 [<c0134744>] worker_thread+0x79/0x85 [<c0137179>] autoremove_wake_function+0x0/0x35 [<c01370c2>] kthread+0x38/0x5e [<c013708a>] kthread+0x0/0x5e [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-R at: [<c013fe0a>] get_lock_stats+0xd/0x2e [<c013fe35>] put_lock_stats+0xa/0x1e [<c0142844>] __lock_acquire+0x462/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c0439eee>] _read_lock_bh+0x3b/0x64 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563 [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf [<c03aaa26>] rtnetlink_event+0x3f/0x42 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a31de>] register_netdevice+0x2a7/0x2e7 [<c02ed862>] bond_create+0x1f2/0x26a [<c05bedcd>] bonding_init+0x761/0x7ea [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c08777d0>] __key.32969+0x0/0x8 -> (_xmit_ETHER){-...} ops: 8 { initial-use at: [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c0141986>] mark_lock+0x64/0x451 [<c014286a>] __lock_acquire+0x488/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0412114>] igmp6_group_added+0x56/0x11d [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0xde/0x138 [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c [<c0401834>] ipv6_add_dev+0x21c/0x24b [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401e17>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0439066>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f4d>] register_netdevice_notifier+0xe/0x126 [<c03a3f88>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0 ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0109ef2>] save_stack_trace+0x20/0x3a [<c0143062>] lock_acquire+0x79/0x93 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c03a5aa1>] dev_mc_add+0x1a/0x6a [<c02ee492>] bond_change_active_slave+0x1a9/0x3bf [<c02ec7c3>] bond_update_speed_duplex+0x26/0x65 [<c02ee9af>] bond_select_active_slave+0x95/0xcd [<c02ed22b>] bond_compute_features+0x45/0x84 [<c02ef9be>] bond_enslave+0x6a7/0x884 [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb [<c02ce8d7>] dev_attr_store+0x27/0x2c [<c019bcb9>] sysfs_write_file+0xad/0xe0 [<c019bc0c>] sysfs_write_file+0x0/0xe0 [<c0168ddc>] vfs_write+0x8a/0x10c [<c0118566>] do_page_fault+0x0/0x54a [<c0169361>] sys_write+0x41/0x67 [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff -> (lweventlist_lock){.+..} ops: 10 { initial-use at: [<c0141986>] mark_lock+0x64/0x451 [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c02e365c>] e1000_read_phy_reg+0x1c7/0x1d3 [<c02e348b>] e1000_write_phy_reg+0xb9/0xc3 [<c024a7de>] delay_tsc+0x25/0x3b [<c0143062>] lock_acquire+0x79/0x93 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abbbb>] linkwatch_fire_event+0x25/0x37 [<c02e1c43>] e1000_probe+0xad1/0xbe8 [<c0257f3f>] pci_device_probe+0x36/0x57 [<c02d0e5f>] driver_probe_device+0xe1/0x15f [<c043a2f9>] _spin_unlock+0x25/0x3b [<c04375b2>] klist_next+0x58/0x6d [<c02d0f6f>] __driver_attach+0x0/0x7f [<c02d0fb8>] __driver_attach+0x49/0x7f [<c02d0403>] bus_for_each_dev+0x36/0x58 [<c02d0cb7>] driver_attach+0x16/0x18 [<c02d0f6f>] __driver_attach+0x0/0x7f [<c02d06fa>] bus_add_driver+0x6d/0x18d [<c0258089>] __pci_register_driver+0x53/0x7f [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff in-softirq-W at: [<c011d20a>] __wake_up_common+0x32/0x5c [<c0142822>] __lock_acquire+0x440/0xc07 [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c02dff01>] e1000_watchdog+0x0/0x5c9 [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abbbb>] linkwatch_fire_event+0x25/0x37 [<c03aeb42>] netif_carrier_on+0x16/0x27 [<c02e0156>] e1000_watchdog+0x255/0x5c9 [<c02dff01>] e1000_watchdog+0x0/0x5c9 [<c012df52>] run_timer_softirq+0xfa/0x15d [<c012a8a6>] __do_softirq+0x56/0xdb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c012a8b8>] __do_softirq+0x68/0xdb [<c012a961>] do_softirq+0x36/0x51 [<c012ab07>] irq_exit+0x43/0x4e [<c0114122>] smp_apic_timer_interrupt+0x74/0x80 [<c0104a01>] apic_timer_interrupt+0x29/0x38 [<c0104a0b>] apic_timer_interrupt+0x33/0x38 [<c01600d8>] sys_swapon+0x29c/0x9aa [<c01021a6>] mwait_idle_with_hints+0x3b/0x3f [<c0102447>] mwait_idle+0x0/0xf [<c0102581>] cpu_idle+0x99/0xc6 [<c05a98c7>] start_kernel+0x2c7/0x2cf [<c05a90e0>] unknown_bootoption+0x0/0x195 [<ffffffff>] 0xffffffff } ... key at: [<c058a194>] lweventlist_lock+0x14/0x40 ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abaf7>] linkwatch_add_event+0xd/0x2c [<c03abbbb>] linkwatch_fire_event+0x25/0x37 [<c03aeb42>] netif_carrier_on+0x16/0x27 [<c02ede2c>] bond_set_carrier+0x31/0x55 [<c02ee9b6>] bond_select_active_slave+0x9c/0xcd [<c02ed22b>] bond_compute_features+0x45/0x84 [<c02ef9be>] bond_enslave+0x6a7/0x884 [<c0439084>] mutex_lock_nested+0x2a4/0x2ac [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb [<c02ce8d7>] dev_attr_store+0x27/0x2c [<c019bcb9>] sysfs_write_file+0xad/0xe0 [<c019bc0c>] sysfs_write_file+0x0/0xe0 [<c0168ddc>] vfs_write+0x8a/0x10c [<c0118566>] do_page_fault+0x0/0x54a [<c0169361>] sys_write+0x41/0x67 [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff stack backtrace: Pid: 9, comm: events/0 Not tainted 2.6.24-rc5 #1 [<c0140b38>] print_irq_inversion_bug+0x108/0x112 [<c014191d>] check_usage_forwards+0x3c/0x41 [<c0141b09>] mark_lock+0x1e7/0x451 [<c0142822>] __lock_acquire+0x440/0xc07 [<c013fe0a>] get_lock_stats+0xd/0x2e [<c013fe35>] put_lock_stats+0xa/0x1e [<c0143062>] lock_acquire+0x79/0x93 [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb [<c0439d62>] _spin_lock_bh+0x3b/0x64 [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb [<c012df52>] run_timer_softirq+0xfa/0x15d [<c012a8a6>] __do_softirq+0x56/0xdb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c012a8b8>] __do_softirq+0x68/0xdb [<c012a961>] do_softirq+0x36/0x51 [<c012ae4a>] local_bh_enable_ip+0xad/0xed [<c03bf107>] rt_run_flush+0x64/0x8b [<c03e9296>] fib_netdev_event+0x61/0x65 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a2ae5>] netdev_state_change+0x18/0x29 [<c03abd1d>] __linkwatch_run_queue+0x150/0x17e [<c03abd68>] linkwatch_event+0x1d/0x22 [<c0133cab>] run_workqueue+0xdb/0x1b6 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c03abd4b>] linkwatch_event+0x0/0x22 [<c01346cb>] worker_thread+0x0/0x85 [<c0134744>] worker_thread+0x79/0x85 [<c0137179>] autoremove_wake_function+0x0/0x35 [<c01370c2>] kthread+0x38/0x5e [<c013708a>] kthread+0x0/0x5e [<c0104baf>] kernel_thread_helper+0x7/0x10 ======================= bonding: bond0: enslaving eth1 as a backup interface with a down link. bonding: bond0: Setting eth0 as primary slave. bond0: no IPv6 routers present Best regards, Krzysztof Ol
Reply-To: andy@greyhouse.net On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote: > > > On Fri, 14 Dec 2007, Andy Gospodarek wrote: > > >On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote: > >> > >> > >>On Wed, 12 Dec 2007, Jay Vosburgh wrote: > >> > >>>Herbert Xu <herbert@gondor.apana.org.au> wrote: > >>> > >>>>>diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > >>>>>drivers/net/bonding/bond_sysfs.c > >>>>>--- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > >>>>>+++ a/drivers/net/bonding/bond_sysfs.c > >>>>>@@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str > >>>>>out: > >>>>> write_unlock_bh(&bond->lock); > >>>>> > >>>>>- rtnl_unlock(); > >>>>>- > >>>> > >>>>Looking at the changeset that added this perhaps the intention > >>>>is to hold the lock? If so we should add an rtnl_lock to the start > >>>>of the function. > >>> > >>> Yes, this function needs to hold locks, and more than just > >>>what's there now. I believe the following should be correct; I haven't > >>>tested it, though (I'm supposedly on vacation right now). > >>> > >>> The following change should be correct for the > >>>bonding_store_primary case discussed in this thread, and also corrects > >>>the bonding_store_active case which performs similar functions. > >>> > >>> The bond_change_active_slave and bond_select_active_slave > >>>functions both require rtnl, bond->lock for read and curr_slave_lock for > >>>write_bh, and no other locks. This is so that the lower level > >>>mode-specific functions can release locks down to just rtnl in order to > >>>call, e.g., dev_set_mac_address with the locks it expects (rtnl only). > >>> > >>>Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> > >>> > >>>diff --git a/drivers/net/bonding/bond_sysfs.c > >>>b/drivers/net/bonding/bond_sysfs.c > >>>index 11b76b3..28a2d80 100644 > >>>--- a/drivers/net/bonding/bond_sysfs.c > >>>+++ b/drivers/net/bonding/bond_sysfs.c > >>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device > >>>*d, > >>> struct slave *slave; > >>> struct bonding *bond = to_bond(d); > >>> > >>>- write_lock_bh(&bond->lock); > >>>+ rtnl_lock(); > >>>+ read_lock(&bond->lock); > >>>+ write_lock_bh(&bond->curr_slave_lock); F > >>>+ > >>> if (!USES_PRIMARY(bond->params.mode)) { > >>> printk(KERN_INFO DRV_NAME > >>> ": %s: Unable to set primary slave; %s is in mode > >>> %d\n", > >>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device > >>>*d, > >>> } > >>> } > >>>out: > >>>- write_unlock_bh(&bond->lock); > >>>- > >>>+ write_unlock_bh(&bond->curr_slave_lock); > >>>+ read_unlock(&bond->lock); > >>> rtnl_unlock(); > >>> > >>> return count; > >>>@@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct > >>>device *d, > >>> struct bonding *bond = to_bond(d); > >>> > >>> rtnl_lock(); > >>>- write_lock_bh(&bond->lock); > >>>+ read_lock(&bond->lock); > >>>+ write_lock_bh(&bond->curr_slave_lock); > >>> > >>> if (!USES_PRIMARY(bond->params.mode)) { > >>> printk(KERN_INFO DRV_NAME > >>>@@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct > >>>device *d, > >>> } > >>> } > >>>out: > >>>- write_unlock_bh(&bond->lock); > >>>+ write_unlock_bh(&bond->curr_slave_lock); > >>>+ read_unlock(&bond->lock); > >>> rtnl_unlock(); > >>> > >>> return count; > >> > >>Vanilla 2.6.24-rc5 plus this patch: > >> > >>========================================================= > >>[ INFO: possible irq lock inversion dependency detected ] > >>2.6.24-rc5 #1 > >>--------------------------------------------------------- > >>events/0/9 just changed the state of lock: > >> (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb > >>but this lock took another, soft-read-irq-unsafe lock in the past: > >> (&bond->lock){-.--} > >> > >>and interrupts could create inverse lock ordering between them. > >> > >> > > > >Grrr, I should have seen that -- sorry. Try your luck with this instead: > <CUT> > > No luck. > > bonding: bond0: setting mode to active-backup (1). > bonding: bond0: Setting MII monitoring interval to 100. > ADDRCONF(NETDEV_UP): bond0: link is not ready > bonding: bond0: Adding slave eth0. > e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow > Control: RX/TX > bonding: bond0: making interface eth0 the new active one. > bonding: bond0: first active interface up! > bonding: bond0: enslaving eth0 as an active interface with an up link. > bonding: bond0: Adding slave eth1. > ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready <SNIP> > bonding: bond0: enslaving eth1 as a backup interface with a down link. > bonding: bond0: Setting eth0 as primary slave. > bond0: no IPv6 routers present > Based on the console log, I'm guessing your initialization scripts use sysfs to set eth0 as the primary interface for bond0? Can you confirm? If you did somehow use sysfs to set the primary device as eth0, I'm guessing you never see this issue without that line or without this patch. Please confirm this as well. Thanks, -andy
On Fri, 14 Dec 2007, Andy Gospodarek wrote: > On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote: >> >> >> On Fri, 14 Dec 2007, Andy Gospodarek wrote: >> >>> On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote: >>>> >>>> >>>> On Wed, 12 Dec 2007, Jay Vosburgh wrote: >>>> >>>>> Herbert Xu <herbert@gondor.apana.org.au> wrote: >>>>> >>>>>>> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >>>>>>> drivers/net/bonding/bond_sysfs.c >>>>>>> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >>>>>>> +++ a/drivers/net/bonding/bond_sysfs.c >>>>>>> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str >>>>>>> out: >>>>>>> write_unlock_bh(&bond->lock); >>>>>>> >>>>>>> - rtnl_unlock(); >>>>>>> - >>>>>> >>>>>> Looking at the changeset that added this perhaps the intention >>>>>> is to hold the lock? If so we should add an rtnl_lock to the start >>>>>> of the function. >>>>> >>>>> Yes, this function needs to hold locks, and more than just >>>>> what's there now. I believe the following should be correct; I haven't >>>>> tested it, though (I'm supposedly on vacation right now). >>>>> >>>>> The following change should be correct for the >>>>> bonding_store_primary case discussed in this thread, and also corrects >>>>> the bonding_store_active case which performs similar functions. >>>>> >>>>> The bond_change_active_slave and bond_select_active_slave >>>>> functions both require rtnl, bond->lock for read and curr_slave_lock for >>>>> write_bh, and no other locks. This is so that the lower level >>>>> mode-specific functions can release locks down to just rtnl in order to >>>>> call, e.g., dev_set_mac_address with the locks it expects (rtnl only). >>>>> >>>>> Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> >>>>> >>>>> diff --git a/drivers/net/bonding/bond_sysfs.c >>>>> b/drivers/net/bonding/bond_sysfs.c >>>>> index 11b76b3..28a2d80 100644 >>>>> --- a/drivers/net/bonding/bond_sysfs.c >>>>> +++ b/drivers/net/bonding/bond_sysfs.c >>>>> @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device >>>>> *d, >>>>> struct slave *slave; >>>>> struct bonding *bond = to_bond(d); >>>>> >>>>> - write_lock_bh(&bond->lock); >>>>> + rtnl_lock(); >>>>> + read_lock(&bond->lock); >>>>> + write_lock_bh(&bond->curr_slave_lock); > F >>>>> + >>>>> if (!USES_PRIMARY(bond->params.mode)) { >>>>> printk(KERN_INFO DRV_NAME >>>>> ": %s: Unable to set primary slave; %s is in mode >>>>> %d\n", >>>>> @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device >>>>> *d, >>>>> } >>>>> } >>>>> out: >>>>> - write_unlock_bh(&bond->lock); >>>>> - >>>>> + write_unlock_bh(&bond->curr_slave_lock); >>>>> + read_unlock(&bond->lock); >>>>> rtnl_unlock(); >>>>> >>>>> return count; >>>>> @@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct >>>>> device *d, >>>>> struct bonding *bond = to_bond(d); >>>>> >>>>> rtnl_lock(); >>>>> - write_lock_bh(&bond->lock); >>>>> + read_lock(&bond->lock); >>>>> + write_lock_bh(&bond->curr_slave_lock); >>>>> >>>>> if (!USES_PRIMARY(bond->params.mode)) { >>>>> printk(KERN_INFO DRV_NAME >>>>> @@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct >>>>> device *d, >>>>> } >>>>> } >>>>> out: >>>>> - write_unlock_bh(&bond->lock); >>>>> + write_unlock_bh(&bond->curr_slave_lock); >>>>> + read_unlock(&bond->lock); >>>>> rtnl_unlock(); >>>>> >>>>> return count; >>>> >>>> Vanilla 2.6.24-rc5 plus this patch: >>>> >>>> ========================================================= >>>> [ INFO: possible irq lock inversion dependency detected ] >>>> 2.6.24-rc5 #1 >>>> --------------------------------------------------------- >>>> events/0/9 just changed the state of lock: >>>> (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb >>>> but this lock took another, soft-read-irq-unsafe lock in the past: >>>> (&bond->lock){-.--} >>>> >>>> and interrupts could create inverse lock ordering between them. >>>> >>>> >>> >>> Grrr, I should have seen that -- sorry. Try your luck with this instead: >> <CUT> >> >> No luck. >> >> bonding: bond0: setting mode to active-backup (1). >> bonding: bond0: Setting MII monitoring interval to 100. >> ADDRCONF(NETDEV_UP): bond0: link is not ready >> bonding: bond0: Adding slave eth0. >> e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow >> Control: RX/TX >> bonding: bond0: making interface eth0 the new active one. >> bonding: bond0: first active interface up! >> bonding: bond0: enslaving eth0 as an active interface with an up link. >> bonding: bond0: Adding slave eth1. >> ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready > > <SNIP> > >> bonding: bond0: enslaving eth1 as a backup interface with a down link. >> bonding: bond0: Setting eth0 as primary slave. >> bond0: no IPv6 routers present >> > > Based on the console log, I'm guessing your initialization scripts use > sysfs to set eth0 as the primary interface for bond0? Can you confirm? Yep, that's correct: postup() { if [[ ${IFACE} == "bond0" ]] ; then echo -n +eth0 > /sys/class/net/${IFACE}/bonding/slaves echo -n +eth1 > /sys/class/net/${IFACE}/bonding/slaves echo -n eth0 > /sys/class/net/${IFACE}/bonding/primary fi } > If you did somehow use sysfs to set the primary device as eth0, I'm > guessing you never see this issue without that line or without this > patch. Please confirm this as well. Without this patch I get another error, of course. Unfortunately checking what happens without using sysfs to set the primary device as eth0, have to wait, as I am not able to test it before Monday. Best regards, Krzysztof Ol
Reply-To: andy@greyhouse.net On Fri, Dec 14, 2007 at 11:11:15PM +0100, Krzysztof Oledzki wrote: > > > On Fri, 14 Dec 2007, Andy Gospodarek wrote: > > >On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote: > >> > >> > >>On Fri, 14 Dec 2007, Andy Gospodarek wrote: > >> > >>>On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote: > >>>> > >>>> > >>>>On Wed, 12 Dec 2007, Jay Vosburgh wrote: > >>>> > >>>>>Herbert Xu <herbert@gondor.apana.org.au> wrote: > >>>>> > >>>>>>>diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > >>>>>>>drivers/net/bonding/bond_sysfs.c > >>>>>>>--- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > >>>>>>>+++ a/drivers/net/bonding/bond_sysfs.c > >>>>>>>@@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str > >>>>>>>out: > >>>>>>> write_unlock_bh(&bond->lock); > >>>>>>> > >>>>>>>- rtnl_unlock(); > >>>>>>>- > >>>>>> > >>>>>>Looking at the changeset that added this perhaps the intention > >>>>>>is to hold the lock? If so we should add an rtnl_lock to the start > >>>>>>of the function. > >>>>> > >>>>> Yes, this function needs to hold locks, and more than just > >>>>>what's there now. I believe the following should be correct; I haven't > >>>>>tested it, though (I'm supposedly on vacation right now). > >>>>> > >>>>> The following change should be correct for the > >>>>>bonding_store_primary case discussed in this thread, and also corrects > >>>>>the bonding_store_active case which performs similar functions. > >>>>> > >>>>> The bond_change_active_slave and bond_select_active_slave > >>>>>functions both require rtnl, bond->lock for read and curr_slave_lock > >>>>>for > >>>>>write_bh, and no other locks. This is so that the lower level > >>>>>mode-specific functions can release locks down to just rtnl in order to > >>>>>call, e.g., dev_set_mac_address with the locks it expects (rtnl only). > >>>>> > >>>>>Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> > >>>>> > >>>>>diff --git a/drivers/net/bonding/bond_sysfs.c > >>>>>b/drivers/net/bonding/bond_sysfs.c > >>>>>index 11b76b3..28a2d80 100644 > >>>>>--- a/drivers/net/bonding/bond_sysfs.c > >>>>>+++ b/drivers/net/bonding/bond_sysfs.c > >>>>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct > >>>>>device > >>>>>*d, > >>>>> struct slave *slave; > >>>>> struct bonding *bond = to_bond(d); > >>>>> > >>>>>- write_lock_bh(&bond->lock); > >>>>>+ rtnl_lock(); > >>>>>+ read_lock(&bond->lock); > >>>>>+ write_lock_bh(&bond->curr_slave_lock); > >F > >>>>>+ > >>>>> if (!USES_PRIMARY(bond->params.mode)) { > >>>>> printk(KERN_INFO DRV_NAME > >>>>> ": %s: Unable to set primary slave; %s is in mode > >>>>> %d\n", > >>>>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct > >>>>>device > >>>>>*d, > >>>>> } > >>>>> } > >>>>>out: > >>>>>- write_unlock_bh(&bond->lock); > >>>>>- > >>>>>+ write_unlock_bh(&bond->curr_slave_lock); > >>>>>+ read_unlock(&bond->lock); > >>>>> rtnl_unlock(); > >>>>> > >>>>> return count; > >>>>>@@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct > >>>>>device *d, > >>>>> struct bonding *bond = to_bond(d); > >>>>> > >>>>> rtnl_lock(); > >>>>>- write_lock_bh(&bond->lock); > >>>>>+ read_lock(&bond->lock); > >>>>>+ write_lock_bh(&bond->curr_slave_lock); > >>>>> > >>>>> if (!USES_PRIMARY(bond->params.mode)) { > >>>>> printk(KERN_INFO DRV_NAME > >>>>>@@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct > >>>>>device *d, > >>>>> } > >>>>> } > >>>>>out: > >>>>>- write_unlock_bh(&bond->lock); > >>>>>+ write_unlock_bh(&bond->curr_slave_lock); > >>>>>+ read_unlock(&bond->lock); > >>>>> rtnl_unlock(); > >>>>> > >>>>> return count; > >>>> > >>>>Vanilla 2.6.24-rc5 plus this patch: > >>>> > >>>>========================================================= > >>>>[ INFO: possible irq lock inversion dependency detected ] > >>>>2.6.24-rc5 #1 > >>>>--------------------------------------------------------- > >>>>events/0/9 just changed the state of lock: > >>>>(&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb > >>>>but this lock took another, soft-read-irq-unsafe lock in the past: > >>>>(&bond->lock){-.--} > >>>> > >>>>and interrupts could create inverse lock ordering between them. > >>>> > >>>> > >>> > >>>Grrr, I should have seen that -- sorry. Try your luck with this instead: > >><CUT> > >> > >>No luck. > >> > >>bonding: bond0: setting mode to active-backup (1). > >>bonding: bond0: Setting MII monitoring interval to 100. > >>ADDRCONF(NETDEV_UP): bond0: link is not ready > >>bonding: bond0: Adding slave eth0. > >>e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow > >>Control: RX/TX > >>bonding: bond0: making interface eth0 the new active one. > >>bonding: bond0: first active interface up! > >>bonding: bond0: enslaving eth0 as an active interface with an up link. > >>bonding: bond0: Adding slave eth1. > >>ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready > > > ><SNIP> > > > >>bonding: bond0: enslaving eth1 as a backup interface with a down link. > >>bonding: bond0: Setting eth0 as primary slave. > >>bond0: no IPv6 routers present > >> > > > >Based on the console log, I'm guessing your initialization scripts use > >sysfs to set eth0 as the primary interface for bond0? Can you confirm? > > Yep, that's correct: > > postup() { > if [[ ${IFACE} == "bond0" ]] ; then > echo -n +eth0 > /sys/class/net/${IFACE}/bonding/slaves > echo -n +eth1 > /sys/class/net/${IFACE}/bonding/slaves > echo -n eth0 > /sys/class/net/${IFACE}/bonding/primary > fi > } > Good. Thanks for the confirmation. > >If you did somehow use sysfs to set the primary device as eth0, I'm > >guessing you never see this issue without that line or without this > >patch. Please confirm this as well. > > Without this patch I get another error, of course. Unfortunately checking :-/ > what happens without using sysfs to set the primary device as eth0, have > to wait, as I am not able to test it before Monday. > No problem, thanks for the feedback and quick response.
Reply-To: andy@greyhouse.net On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote: > > > On Fri, 14 Dec 2007, Andy Gospodarek wrote: > > >On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote: > >> > >> > >>On Wed, 12 Dec 2007, Jay Vosburgh wrote: > >> > >>>Herbert Xu <herbert@gondor.apana.org.au> wrote: > >>> > >>>>>diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > >>>>>drivers/net/bonding/bond_sysfs.c > >>>>>--- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > >>>>>+++ a/drivers/net/bonding/bond_sysfs.c > >>>>>@@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str > >>>>>out: > >>>>> write_unlock_bh(&bond->lock); > >>>>> > >>>>>- rtnl_unlock(); > >>>>>- > >>>> > >>>>Looking at the changeset that added this perhaps the intention > >>>>is to hold the lock? If so we should add an rtnl_lock to the start > >>>>of the function. > >>> > >>> Yes, this function needs to hold locks, and more than just > >>>what's there now. I believe the following should be correct; I haven't > >>>tested it, though (I'm supposedly on vacation right now). > >>> > >>> The following change should be correct for the > >>>bonding_store_primary case discussed in this thread, and also corrects > >>>the bonding_store_active case which performs similar functions. > >>> > >>> The bond_change_active_slave and bond_select_active_slave > >>>functions both require rtnl, bond->lock for read and curr_slave_lock for > >>>write_bh, and no other locks. This is so that the lower level > >>>mode-specific functions can release locks down to just rtnl in order to > >>>call, e.g., dev_set_mac_address with the locks it expects (rtnl only). > >>> > >>>Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> > >>> > >>>diff --git a/drivers/net/bonding/bond_sysfs.c > >>>b/drivers/net/bonding/bond_sysfs.c > >>>index 11b76b3..28a2d80 100644 > >>>--- a/drivers/net/bonding/bond_sysfs.c > >>>+++ b/drivers/net/bonding/bond_sysfs.c > >>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device > >>>*d, > >>> struct slave *slave; > >>> struct bonding *bond = to_bond(d); > >>> > >>>- write_lock_bh(&bond->lock); > >>>+ rtnl_lock(); > >>>+ read_lock(&bond->lock); > >>>+ write_lock_bh(&bond->curr_slave_lock); > >>>+ > >>> if (!USES_PRIMARY(bond->params.mode)) { > >>> printk(KERN_INFO DRV_NAME > >>> ": %s: Unable to set primary slave; %s is in mode > >>> %d\n", > >>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device > >>>*d, > >>> } > >>> } > >>>out: > >>>- write_unlock_bh(&bond->lock); > >>>- > >>>+ write_unlock_bh(&bond->curr_slave_lock); > >>>+ read_unlock(&bond->lock); > >>> rtnl_unlock(); > >>> > >>> return count; > >>>@@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct > >>>device *d, > >>> struct bonding *bond = to_bond(d); > >>> > >>> rtnl_lock(); > >>>- write_lock_bh(&bond->lock); > >>>+ read_lock(&bond->lock); > >>>+ write_lock_bh(&bond->curr_slave_lock); > >>> > >>> if (!USES_PRIMARY(bond->params.mode)) { > >>> printk(KERN_INFO DRV_NAME > >>>@@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct > >>>device *d, > >>> } > >>> } > >>>out: > >>>- write_unlock_bh(&bond->lock); > >>>+ write_unlock_bh(&bond->curr_slave_lock); > >>>+ read_unlock(&bond->lock); > >>> rtnl_unlock(); > >>> > >>> return count; > >> > >>Vanilla 2.6.24-rc5 plus this patch: > >> > >>========================================================= > >>[ INFO: possible irq lock inversion dependency detected ] > >>2.6.24-rc5 #1 > >>--------------------------------------------------------- > >>events/0/9 just changed the state of lock: > >> (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb > >>but this lock took another, soft-read-irq-unsafe lock in the past: > >> (&bond->lock){-.--} > >> > >>and interrupts could create inverse lock ordering between them. > >> > >> > > > >Grrr, I should have seen that -- sorry. Try your luck with this instead: > <CUT> > > No luck. > I'm guessing if we go back to using a write-lock for bond->lock this will go back to working again, but I'm not totally convinced since there are plenty of places where we used a read-lock with it. diff --git a/drivers/net/bonding/bond_sysfs.c b/drivers/net/bonding/bond_sysfs.c index 11b76b3..635b857 100644 --- a/drivers/net/bonding/bond_sysfs.c +++ b/drivers/net/bonding/bond_sysfs.c @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device *d, struct slave *slave; struct bonding *bond = to_bond(d); + rtnl_lock(); write_lock_bh(&bond->lock); + write_lock_bh(&bond->curr_slave_lock); + if (!USES_PRIMARY(bond->params.mode)) { printk(KERN_INFO DRV_NAME ": %s: Unable to set primary slave; %s is in mode %d\n", @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device *d, } } out: + write_unlock_bh(&bond->curr_slave_lock); write_unlock_bh(&bond->lock); - rtnl_unlock(); return count; @@ -1191,6 +1194,7 @@ static ssize_t bonding_store_active_slave(struct device *d, rtnl_lock(); write_lock_bh(&bond->lock); + write_lock_bh(&bond->curr_slave_lock); if (!USES_PRIMARY(bond->params.mode)) { printk(KERN_INFO DRV_NAME @@ -1247,6 +1251,7 @@ static ssize_t bonding_store_active_slave(struct device *d, } } out: + write_unlock_bh(&bond->curr_slave_lock); write_unlock_bh(&bond->lock); rtnl_unlock();
On Fri, Dec 14, 2007 at 05:47:22PM -0500, Andy Gospodarek wrote: > > I'm guessing if we go back to using a write-lock for bond->lock this > will go back to working again, but I'm not totally convinced since there > are plenty of places where we used a read-lock with it. Sorry I forgot to cc you earlier Andy. But to fix this you need make sure that all read locks on bond->lock in process context disable BH. This is because at least one write lock can be taken from BH context. You don't need to turn the read locks into write locks however. This is also something that we can undo once the set_multicast interface has been fixed to not take the tx lock. Cheers,
Reply-To: andy@greyhouse.net On Dec 14, 2007 11:10 PM, Herbert Xu <herbert@gondor.apana.org.au> wrote: > On Fri, Dec 14, 2007 at 05:47:22PM -0500, Andy Gospodarek wrote: > > > > I'm guessing if we go back to using a write-lock for bond->lock this > > will go back to working again, but I'm not totally convinced since there > > are plenty of places where we used a read-lock with it. > > Sorry I forgot to cc you earlier Andy. > > But to fix this you need make sure that all read locks on bond->lock > in process context disable BH. This is because at least one write > lock can be taken from BH context. I agree with you completely, Herbet, which is why I was surprised that my first apparently did not resolve the issue. I felt it should have.... > You don't need to turn the read locks into write locks however. > > This is also something that we can undo once the set_multicast > interface has been fixed to not take the tx lock. > > Cheers, > -- > Visit Openswan at http://www.openswan.org/ > Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au> > Home Page: http://gondor.apana.org.au/~herbert/ > PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt >
Andy Gospodarek <andy@greyhouse.net> wrote: > > I agree with you completely, Herbet, which is why I was surprised that > my first apparently did not resolve the issue. I felt it should > have.... Did it change all occurrences of read_lock(&bond->lock) to read_lock_bh? If so I better look at the lockdep output again. Cheers,
Reply-To: andy@greyhouse.net On Dec 15, 2007 9:27 PM, Herbert Xu <herbert@gondor.apana.org.au> wrote: > Andy Gospodarek <andy@greyhouse.net> wrote: > > > > I agree with you completely, Herbet, which is why I was surprised that > > my first apparently did not resolve the issue. I felt it should > > have.... > > Did it change all occurrences of read_lock(&bond->lock) to > read_lock_bh? If so I better look at the lockdep output again. > Not all of them in the bonding code, but all two of them in the small patch.
On Sat, Dec 15, 2007 at 10:17:35PM -0500, Andy Gospodarek wrote: > > Not all of them in the bonding code, but all two of them in the small patch. OK, we need to change all of the ones that may be called from process context with BH on. Cheers,
On Fri, 14 Dec 2007, Andy Gospodarek wrote: > On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote: >> >> >> On Fri, 14 Dec 2007, Andy Gospodarek wrote: >> >>> On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote: >>>> >>>> >>>> On Wed, 12 Dec 2007, Jay Vosburgh wrote: >>>> >>>>> Herbert Xu <herbert@gondor.apana.org.au> wrote: >>>>> >>>>>>> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >>>>>>> drivers/net/bonding/bond_sysfs.c >>>>>>> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >>>>>>> +++ a/drivers/net/bonding/bond_sysfs.c >>>>>>> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str >>>>>>> out: >>>>>>> write_unlock_bh(&bond->lock); >>>>>>> >>>>>>> - rtnl_unlock(); >>>>>>> - >>>>>> >>>>>> Looking at the changeset that added this perhaps the intention >>>>>> is to hold the lock? If so we should add an rtnl_lock to the start >>>>>> of the function. >>>>> >>>>> Yes, this function needs to hold locks, and more than just >>>>> what's there now. I believe the following should be correct; I haven't >>>>> tested it, though (I'm supposedly on vacation right now). >>>>> >>>>> The following change should be correct for the >>>>> bonding_store_primary case discussed in this thread, and also corrects >>>>> the bonding_store_active case which performs similar functions. >>>>> >>>>> The bond_change_active_slave and bond_select_active_slave >>>>> functions both require rtnl, bond->lock for read and curr_slave_lock for >>>>> write_bh, and no other locks. This is so that the lower level >>>>> mode-specific functions can release locks down to just rtnl in order to >>>>> call, e.g., dev_set_mac_address with the locks it expects (rtnl only). >>>>> >>>>> Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> >>>>> >>>>> diff --git a/drivers/net/bonding/bond_sysfs.c >>>>> b/drivers/net/bonding/bond_sysfs.c >>>>> index 11b76b3..28a2d80 100644 >>>>> --- a/drivers/net/bonding/bond_sysfs.c >>>>> +++ b/drivers/net/bonding/bond_sysfs.c >>>>> @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device >>>>> *d, >>>>> struct slave *slave; >>>>> struct bonding *bond = to_bond(d); >>>>> >>>>> - write_lock_bh(&bond->lock); >>>>> + rtnl_lock(); >>>>> + read_lock(&bond->lock); >>>>> + write_lock_bh(&bond->curr_slave_lock); > F >>>>> + >>>>> if (!USES_PRIMARY(bond->params.mode)) { >>>>> printk(KERN_INFO DRV_NAME >>>>> ": %s: Unable to set primary slave; %s is in mode >>>>> %d\n", >>>>> @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device >>>>> *d, >>>>> } >>>>> } >>>>> out: >>>>> - write_unlock_bh(&bond->lock); >>>>> - >>>>> + write_unlock_bh(&bond->curr_slave_lock); >>>>> + read_unlock(&bond->lock); >>>>> rtnl_unlock(); >>>>> >>>>> return count; >>>>> @@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct >>>>> device *d, >>>>> struct bonding *bond = to_bond(d); >>>>> >>>>> rtnl_lock(); >>>>> - write_lock_bh(&bond->lock); >>>>> + read_lock(&bond->lock); >>>>> + write_lock_bh(&bond->curr_slave_lock); >>>>> >>>>> if (!USES_PRIMARY(bond->params.mode)) { >>>>> printk(KERN_INFO DRV_NAME >>>>> @@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct >>>>> device *d, >>>>> } >>>>> } >>>>> out: >>>>> - write_unlock_bh(&bond->lock); >>>>> + write_unlock_bh(&bond->curr_slave_lock); >>>>> + read_unlock(&bond->lock); >>>>> rtnl_unlock(); >>>>> >>>>> return count; >>>> >>>> Vanilla 2.6.24-rc5 plus this patch: >>>> >>>> ========================================================= >>>> [ INFO: possible irq lock inversion dependency detected ] >>>> 2.6.24-rc5 #1 >>>> --------------------------------------------------------- >>>> events/0/9 just changed the state of lock: >>>> (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb >>>> but this lock took another, soft-read-irq-unsafe lock in the past: >>>> (&bond->lock){-.--} >>>> >>>> and interrupts could create inverse lock ordering between them. >>>> >>>> >>> >>> Grrr, I should have seen that -- sorry. Try your luck with this instead: >> <CUT> >> >> No luck. >> >> bonding: bond0: setting mode to active-backup (1). >> bonding: bond0: Setting MII monitoring interval to 100. >> ADDRCONF(NETDEV_UP): bond0: link is not ready >> bonding: bond0: Adding slave eth0. >> e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow >> Control: RX/TX >> bonding: bond0: making interface eth0 the new active one. >> bonding: bond0: first active interface up! >> bonding: bond0: enslaving eth0 as an active interface with an up link. >> bonding: bond0: Adding slave eth1. >> ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready > > <SNIP> > >> bonding: bond0: enslaving eth1 as a backup interface with a down link. >> bonding: bond0: Setting eth0 as primary slave. >> bond0: no IPv6 routers present >> > > Based on the console log, I'm guessing your initialization scripts use > sysfs to set eth0 as the primary interface for bond0? Can you confirm? > > If you did somehow use sysfs to set the primary device as eth0, I'm > guessing you never see this issue without that line or without this > patch. Please confirm this as well. Unpatched 2.6.24-rc5 with "echo -n eth0 > /sys/class/net/${IFACE}/bonding/primary" removed, still has this issue. I also removed: # echo -n 1 > /sys/class/net/${IFACE}/bonding/mode # echo -n 100 > /sys/class/net/${IFACE}/bonding/miimon but this did not help: bonding: bond0: setting mode to active-backup (1). bonding: bond0: Setting MII monitoring interval to 100. ADDRCONF(NETDEV_UP): bond0: link is not ready bonding: bond0: Adding slave eth0. e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX bonding: bond0: making interface eth0 the new active one. bonding: bond0: first active interface up! bonding: bond0: enslaving eth0 as an active interface with an up link. bonding: bond0: Adding slave eth1. ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready ========================================================= [ INFO: possible irq lock inversion dependency detected ] 2.6.24-rc5 #1 --------------------------------------------------------- events/0/9 just changed the state of lock: (&mc->mca_lock){-+..}, at: [<c0411c52>] mld_ifc_timer_expire+0x130/0x1fb but this lock took another, soft-read-irq-unsafe lock in the past: (&bond->lock){-.--} and interrupts could create inverse lock ordering between them. other info that might help us debug this: 4 locks held by events/0/9: #0: (events){--..}, at: [<c0133c57>] run_workqueue+0x87/0x1b6 #1: ((linkwatch_work).work){--..}, at: [<c0133c57>] run_workqueue+0x87/0x1b6 #2: (rtnl_mutex){--..}, at: [<c03abd28>] linkwatch_event+0x5/0x22 #3: (&ndev->lock){-.-+}, at: [<c0411b39>] mld_ifc_timer_expire+0x17/0x1fb the first lock's dependencies: -> (&mc->mca_lock){-+..} ops: 10 { initial-use at: [<c0104ee2>] dump_trace+0x83/0x8d [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0109ef2>] save_stack_trace+0x20/0x3a [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c041242a>] ipv6_dev_mc_inc+0x24d/0x31c [<c0143062>] lock_acquire+0x79/0x93 [<c04120ae>] igmp6_group_added+0x18/0x11d [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c04120ae>] igmp6_group_added+0x18/0x11d [<c04120ae>] igmp6_group_added+0x18/0x11d [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c [<c041242a>] ipv6_dev_mc_inc+0x24d/0x31c [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c [<c040180c>] ipv6_add_dev+0x21c/0x24b [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c05c5b40>] addrconf_init+0x13/0x193 [<c0199f63>] proc_net_fops_create+0x10/0x21 [<c0419b10>] ip6_flowlabel_init+0x1e/0x20 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff in-softirq-W at: [<c0142822>] __lock_acquire+0x440/0xc07 [<c011d686>] update_curr+0x52/0xc4 [<c013f175>] tick_sched_timer+0x129/0x165 [<c0143062>] lock_acquire+0x79/0x93 [<c0411c52>] mld_ifc_timer_expire+0x130/0x1fb [<c0411b22>] mld_ifc_timer_expire+0x0/0x1fb [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c0411c52>] mld_ifc_timer_expire+0x130/0x1fb [<c0411c52>] mld_ifc_timer_expire+0x130/0x1fb [<c0411b22>] mld_ifc_timer_expire+0x0/0x1fb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c0411b22>] mld_ifc_timer_expire+0x0/0x1fb [<c012df52>] run_timer_softirq+0xfa/0x15d [<c012a8a6>] __do_softirq+0x56/0xdb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c012a8b8>] __do_softirq+0x68/0xdb [<c012a961>] do_softirq+0x36/0x51 [<c012ae4a>] local_bh_enable_ip+0xad/0xed [<c03bf0df>] rt_run_flush+0x64/0x8b [<c03e926e>] fib_netdev_event+0x61/0x65 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a2abd>] netdev_state_change+0x18/0x29 [<c03abcf5>] __linkwatch_run_queue+0x150/0x17e [<c03abd40>] linkwatch_event+0x1d/0x22 [<c0133cab>] run_workqueue+0xdb/0x1b6 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c03abd23>] linkwatch_event+0x0/0x22 [<c01346cb>] worker_thread+0x0/0x85 [<c0134744>] worker_thread+0x79/0x85 [<c0137179>] autoremove_wake_function+0x0/0x35 [<c01370c2>] kthread+0x38/0x5e [<c013708a>] kthread+0x0/0x5e [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c01417ee>] find_usage_backwards+0xbb/0xe2 [<c0104ee2>] dump_trace+0x83/0x8d [<c014286a>] __lock_acquire+0x488/0xc07 [<c0109ef2>] save_stack_trace+0x20/0x3a [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c041242a>] ipv6_dev_mc_inc+0x24d/0x31c [<c0143062>] lock_acquire+0x79/0x93 [<c04120ae>] igmp6_group_added+0x18/0x11d [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c04120ae>] igmp6_group_added+0x18/0x11d [<c04120ae>] igmp6_group_added+0x18/0x11d [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c [<c041242a>] ipv6_dev_mc_inc+0x24d/0x31c [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c [<c040180c>] ipv6_add_dev+0x21c/0x24b [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c05c5b40>] addrconf_init+0x13/0x193 [<c0199f63>] proc_net_fops_create+0x10/0x21 [<c0419b10>] ip6_flowlabel_init+0x1e/0x20 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c087e2d8>] __key.30798+0x0/0x8 -> (_xmit_ETHER){-...} ops: 8 { initial-use at: [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c04120ec>] igmp6_group_added+0x56/0x11d [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0x106/0x138 [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c [<c040180c>] ipv6_add_dev+0x21c/0x24b [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401def>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043903e>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f60>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c0141986>] mark_lock+0x64/0x451 [<c014286a>] __lock_acquire+0x488/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c04120ec>] igmp6_group_added+0x56/0x11d [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0x106/0x138 [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c [<c040180c>] ipv6_add_dev+0x21c/0x24b [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401def>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043903e>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f60>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0 ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c0143062>] lock_acquire+0x79/0x93 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c04120ec>] igmp6_group_added+0x56/0x11d [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0x106/0x138 [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c [<c040180c>] ipv6_add_dev+0x21c/0x24b [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401def>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043903e>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f60>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff -> (&bonding_netdev_xmit_lock_key){-...} ops: 6 { initial-use at: [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c04120ec>] igmp6_group_added+0x56/0x11d [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0x106/0x138 [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c [<c040180c>] ipv6_add_dev+0x21c/0x24b [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401def>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043903e>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f60>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c0141986>] mark_lock+0x64/0x451 [<c014286a>] __lock_acquire+0x488/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c04120ec>] igmp6_group_added+0x56/0x11d [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0x106/0x138 [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c [<c040180c>] ipv6_add_dev+0x21c/0x24b [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401def>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043903e>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f60>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c0877804>] bonding_netdev_xmit_lock_key+0x0/0x8 -> (&bond->lock){-.--} ops: 99 { initial-use at: [<c013fe35>] put_lock_stats+0xa/0x1e [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c0439ec6>] _read_lock_bh+0x3b/0x64 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c03aa3bf>] rtnl_fill_ifinfo+0x2bf/0x563 [<c03aa93d>] rtmsg_ifinfo+0x5d/0xdf [<c03aa9fe>] rtnetlink_event+0x3f/0x42 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a31b6>] register_netdevice+0x2a7/0x2e7 [<c02ed862>] bond_create+0x1f2/0x26a [<c05bedcd>] bonding_init+0x761/0x7ea [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c014286a>] __lock_acquire+0x488/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043a3d5>] _spin_unlock_irq+0x20/0x41 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c043a3e0>] _spin_unlock_irq+0x2b/0x41 [<c012208a>] finish_task_switch+0x50/0x8c [<c0143062>] lock_acquire+0x79/0x93 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c0439dfd>] _write_lock_bh+0x3b/0x64 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c013fe35>] put_lock_stats+0xa/0x1e [<c03a14b3>] __dev_set_rx_mode+0x7b/0x7d [<c03a164d>] dev_set_rx_mode+0x23/0x36 [<c03a3d28>] dev_open+0x5e/0x77 [<c03a29f7>] dev_change_flags+0x9d/0x14b [<c03a17fb>] __dev_get_by_name+0x68/0x73 [<c03e3828>] devinet_ioctl+0x22b/0x536 [<c03a3b1d>] dev_ioctl+0x46f/0x5b7 [<c0399c50>] sock_ioctl+0x167/0x18b [<c0399ae9>] sock_ioctl+0x0/0x18b [<c01725f7>] do_ioctl+0x1f/0x62 [<c0172867>] vfs_ioctl+0x22d/0x23f [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c01728ac>] sys_ioctl+0x33/0x4b [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff softirq-on-R at: [<c0141986>] mark_lock+0x64/0x451 [<c013575e>] __kernel_text_address+0x5/0xe [<c0104ee2>] dump_trace+0x83/0x8d [<c0142889>] __lock_acquire+0x4a7/0xc07 [<c013fc76>] save_trace+0x37/0x89 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c0143062>] lock_acquire+0x79/0x93 [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c0439f25>] _read_lock+0x36/0x5f [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c0133cab>] run_workqueue+0xdb/0x1b6 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c02eee44>] bond_mii_monitor+0x0/0x85 [<c01346cb>] worker_thread+0x0/0x85 [<c0134744>] worker_thread+0x79/0x85 [<c0137179>] autoremove_wake_function+0x0/0x35 [<c01370c2>] kthread+0x38/0x5e [<c013708a>] kthread+0x0/0x5e [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-R at: [<c013fe0a>] get_lock_stats+0xd/0x2e [<c013fe35>] put_lock_stats+0xa/0x1e [<c0142844>] __lock_acquire+0x462/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c0439ec6>] _read_lock_bh+0x3b/0x64 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c03aa3bf>] rtnl_fill_ifinfo+0x2bf/0x563 [<c03aa93d>] rtmsg_ifinfo+0x5d/0xdf [<c03aa9fe>] rtnetlink_event+0x3f/0x42 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a31b6>] register_netdevice+0x2a7/0x2e7 [<c02ed862>] bond_create+0x1f2/0x26a [<c05bedcd>] bonding_init+0x761/0x7ea [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c08777d0>] __key.32969+0x0/0x8 -> (_xmit_ETHER){-...} ops: 8 { initial-use at: [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c04120ec>] igmp6_group_added+0x56/0x11d [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0x106/0x138 [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c [<c040180c>] ipv6_add_dev+0x21c/0x24b [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401def>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043903e>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f60>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c0141986>] mark_lock+0x64/0x451 [<c014286a>] __lock_acquire+0x488/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c04120ec>] igmp6_group_added+0x56/0x11d [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0x106/0x138 [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c [<c040180c>] ipv6_add_dev+0x21c/0x24b [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401def>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043903e>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f60>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0 ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58 [<c0109ef2>] save_stack_trace+0x20/0x3a [<c0143062>] lock_acquire+0x79/0x93 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c02ee492>] bond_change_active_slave+0x1a9/0x3bf [<c02ec7c3>] bond_update_speed_duplex+0x26/0x65 [<c02ee9af>] bond_select_active_slave+0x95/0xcd [<c02ed22b>] bond_compute_features+0x45/0x84 [<c02ef9be>] bond_enslave+0x6a7/0x884 [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c02f616d>] bonding_store_slaves+0x1ae/0x2fb [<c02f5fbf>] bonding_store_slaves+0x0/0x2fb [<c02ce8d7>] dev_attr_store+0x27/0x2c [<c019bcb9>] sysfs_write_file+0xad/0xe0 [<c019bc0c>] sysfs_write_file+0x0/0xe0 [<c0168ddc>] vfs_write+0x8a/0x10c [<c0118566>] do_page_fault+0x0/0x54a [<c0169361>] sys_write+0x41/0x67 [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff -> (lweventlist_lock){.+..} ops: 10 { initial-use at: [<c0141986>] mark_lock+0x64/0x451 [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c02e365c>] e1000_read_phy_reg+0x1c7/0x1d3 [<c02e348b>] e1000_write_phy_reg+0xb9/0xc3 [<c024a7de>] delay_tsc+0x25/0x3b [<c0143062>] lock_acquire+0x79/0x93 [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c043a057>] _spin_lock_irqsave+0x3f/0x6c [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c03abb93>] linkwatch_fire_event+0x25/0x37 [<c02e1c43>] e1000_probe+0xad1/0xbe8 [<c0257f3f>] pci_device_probe+0x36/0x57 [<c02d0e5f>] driver_probe_device+0xe1/0x15f [<c043a2d1>] _spin_unlock+0x25/0x3b [<c043758a>] klist_next+0x58/0x6d [<c02d0f6f>] __driver_attach+0x0/0x7f [<c02d0fb8>] __driver_attach+0x49/0x7f [<c02d0403>] bus_for_each_dev+0x36/0x58 [<c02d0cb7>] driver_attach+0x16/0x18 [<c02d0f6f>] __driver_attach+0x0/0x7f [<c02d06fa>] bus_add_driver+0x6d/0x18d [<c0258089>] __pci_register_driver+0x53/0x7f [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff in-softirq-W at: [<c011d20a>] __wake_up_common+0x32/0x5c [<c0142822>] __lock_acquire+0x440/0xc07 [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c02dff01>] e1000_watchdog+0x0/0x5c9 [<c043a057>] _spin_lock_irqsave+0x3f/0x6c [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c03abb93>] linkwatch_fire_event+0x25/0x37 [<c03aeb1a>] netif_carrier_on+0x16/0x27 [<c02e0156>] e1000_watchdog+0x255/0x5c9 [<c02dff01>] e1000_watchdog+0x0/0x5c9 [<c012df52>] run_timer_softirq+0xfa/0x15d [<c012a8a6>] __do_softirq+0x56/0xdb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c012a8b8>] __do_softirq+0x68/0xdb [<c012a961>] do_softirq+0x36/0x51 [<c012ae4a>] local_bh_enable_ip+0xad/0xed [<c03bf0df>] rt_run_flush+0x64/0x8b [<c03e7924>] ip_mc_inc_group+0x184/0x1c1 [<c03e79a2>] ip_mc_up+0x41/0x59 [<c03e32dc>] inetdev_event+0x257/0x465 [<c03aa8da>] rtnl_notify+0x3a/0x40 [<c03aa997>] rtmsg_ifinfo+0xb7/0xdf [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a3d3b>] dev_open+0x71/0x77 [<c02ef626>] bond_enslave+0x30f/0x884 [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c02f6161>] bonding_store_slaves+0x1a2/0x2fb [<c02f616d>] bonding_store_slaves+0x1ae/0x2fb [<c02f5fbf>] bonding_store_slaves+0x0/0x2fb [<c02ce8d7>] dev_attr_store+0x27/0x2c [<c019bcb9>] sysfs_write_file+0xad/0xe0 [<c019bc0c>] sysfs_write_file+0x0/0xe0 [<c0168ddc>] vfs_write+0x8a/0x10c [<c0118566>] do_page_fault+0x0/0x54a [<c0169361>] sys_write+0x41/0x67 [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff } ... key at: [<c058a194>] lweventlist_lock+0x14/0x40 ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c043a057>] _spin_lock_irqsave+0x3f/0x6c [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c03abb93>] linkwatch_fire_event+0x25/0x37 [<c03aeb1a>] netif_carrier_on+0x16/0x27 [<c02ede2c>] bond_set_carrier+0x31/0x55 [<c02ee9b6>] bond_select_active_slave+0x9c/0xcd [<c02ed22b>] bond_compute_features+0x45/0x84 [<c02ef9be>] bond_enslave+0x6a7/0x884 [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c02f616d>] bonding_store_slaves+0x1ae/0x2fb [<c02f5fbf>] bonding_store_slaves+0x0/0x2fb [<c02ce8d7>] dev_attr_store+0x27/0x2c [<c019bcb9>] sysfs_write_file+0xad/0xe0 [<c019bc0c>] sysfs_write_file+0x0/0xe0 [<c0168ddc>] vfs_write+0x8a/0x10c [<c0118566>] do_page_fault+0x0/0x54a [<c0169361>] sys_write+0x41/0x67 [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c0141dac>] mark_held_locks+0x39/0x53 [<c0143062>] lock_acquire+0x79/0x93 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c0439dfd>] _write_lock_bh+0x3b/0x64 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c013fe35>] put_lock_stats+0xa/0x1e [<c03a14b3>] __dev_set_rx_mode+0x7b/0x7d [<c03a164d>] dev_set_rx_mode+0x23/0x36 [<c03a3d28>] dev_open+0x5e/0x77 [<c03a29f7>] dev_change_flags+0x9d/0x14b [<c03a17fb>] __dev_get_by_name+0x68/0x73 [<c03e3828>] devinet_ioctl+0x22b/0x536 [<c03a3b1d>] dev_ioctl+0x46f/0x5b7 [<c0399c50>] sock_ioctl+0x167/0x18b [<c0399ae9>] sock_ioctl+0x0/0x18b [<c01725f7>] do_ioctl+0x1f/0x62 [<c0172867>] vfs_ioctl+0x22d/0x23f [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c01728ac>] sys_ioctl+0x33/0x4b [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c0143062>] lock_acquire+0x79/0x93 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c04120ec>] igmp6_group_added+0x56/0x11d [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0x106/0x138 [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c [<c040180c>] ipv6_add_dev+0x21c/0x24b [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401def>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043903e>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f60>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff the second lock's dependencies: -> (&bond->lock){-.--} ops: 99 { initial-use at: [<c013fe35>] put_lock_stats+0xa/0x1e [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c0439ec6>] _read_lock_bh+0x3b/0x64 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c03aa3bf>] rtnl_fill_ifinfo+0x2bf/0x563 [<c03aa93d>] rtmsg_ifinfo+0x5d/0xdf [<c03aa9fe>] rtnetlink_event+0x3f/0x42 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a31b6>] register_netdevice+0x2a7/0x2e7 [<c02ed862>] bond_create+0x1f2/0x26a [<c05bedcd>] bonding_init+0x761/0x7ea [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c014286a>] __lock_acquire+0x488/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043a3d5>] _spin_unlock_irq+0x20/0x41 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c043a3e0>] _spin_unlock_irq+0x2b/0x41 [<c012208a>] finish_task_switch+0x50/0x8c [<c0143062>] lock_acquire+0x79/0x93 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c0439dfd>] _write_lock_bh+0x3b/0x64 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c02eda75>] bond_set_multicast_list+0x1d/0x241 [<c013fe35>] put_lock_stats+0xa/0x1e [<c03a14b3>] __dev_set_rx_mode+0x7b/0x7d [<c03a164d>] dev_set_rx_mode+0x23/0x36 [<c03a3d28>] dev_open+0x5e/0x77 [<c03a29f7>] dev_change_flags+0x9d/0x14b [<c03a17fb>] __dev_get_by_name+0x68/0x73 [<c03e3828>] devinet_ioctl+0x22b/0x536 [<c03a3b1d>] dev_ioctl+0x46f/0x5b7 [<c0399c50>] sock_ioctl+0x167/0x18b [<c0399ae9>] sock_ioctl+0x0/0x18b [<c01725f7>] do_ioctl+0x1f/0x62 [<c0172867>] vfs_ioctl+0x22d/0x23f [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c01728ac>] sys_ioctl+0x33/0x4b [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff softirq-on-R at: [<c0141986>] mark_lock+0x64/0x451 [<c013575e>] __kernel_text_address+0x5/0xe [<c0104ee2>] dump_trace+0x83/0x8d [<c0142889>] __lock_acquire+0x4a7/0xc07 [<c013fc76>] save_trace+0x37/0x89 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c0143062>] lock_acquire+0x79/0x93 [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c0439f25>] _read_lock+0x36/0x5f [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c02eee5d>] bond_mii_monitor+0x19/0x85 [<c0133cab>] run_workqueue+0xdb/0x1b6 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c02eee44>] bond_mii_monitor+0x0/0x85 [<c01346cb>] worker_thread+0x0/0x85 [<c0134744>] worker_thread+0x79/0x85 [<c0137179>] autoremove_wake_function+0x0/0x35 [<c01370c2>] kthread+0x38/0x5e [<c013708a>] kthread+0x0/0x5e [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-R at: [<c013fe0a>] get_lock_stats+0xd/0x2e [<c013fe35>] put_lock_stats+0xa/0x1e [<c0142844>] __lock_acquire+0x462/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c0439ec6>] _read_lock_bh+0x3b/0x64 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c02edcc1>] bond_get_stats+0x28/0xd0 [<c03aa3bf>] rtnl_fill_ifinfo+0x2bf/0x563 [<c03aa93d>] rtmsg_ifinfo+0x5d/0xdf [<c03aa9fe>] rtnetlink_event+0x3f/0x42 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a31b6>] register_netdevice+0x2a7/0x2e7 [<c02ed862>] bond_create+0x1f2/0x26a [<c05bedcd>] bonding_init+0x761/0x7ea [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c08777d0>] __key.32969+0x0/0x8 -> (_xmit_ETHER){-...} ops: 8 { initial-use at: [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c04120ec>] igmp6_group_added+0x56/0x11d [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0x106/0x138 [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c [<c040180c>] ipv6_add_dev+0x21c/0x24b [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401def>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043903e>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f60>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff hardirq-on-W at: [<c0141986>] mark_lock+0x64/0x451 [<c014286a>] __lock_acquire+0x488/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c04120ec>] igmp6_group_added+0x56/0x11d [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c [<c0410100>] igmp6_mc_seq_start+0x106/0x138 [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c [<c040180c>] ipv6_add_dev+0x21c/0x24b [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef [<c0401def>] addrconf_notify+0x60/0x7b7 [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0141dac>] mark_held_locks+0x39/0x53 [<c043903e>] mutex_lock_nested+0x286/0x2ac [<c0141f9f>] trace_hardirqs_on+0x122/0x14c [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f25>] register_netdevice_notifier+0xe/0x126 [<c03a3f60>] register_netdevice_notifier+0x49/0x126 [<c05c5bda>] addrconf_init+0xad/0x193 [<c05c5b48>] addrconf_init+0x1b/0x193 [<c05c5a20>] inet6_init+0x1f0/0x2ad [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff } ... key at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0 ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58 [<c0109ef2>] save_stack_trace+0x20/0x3a [<c0143062>] lock_acquire+0x79/0x93 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c03a5a79>] dev_mc_add+0x1a/0x6a [<c02ee492>] bond_change_active_slave+0x1a9/0x3bf [<c02ec7c3>] bond_update_speed_duplex+0x26/0x65 [<c02ee9af>] bond_select_active_slave+0x95/0xcd [<c02ed22b>] bond_compute_features+0x45/0x84 [<c02ef9be>] bond_enslave+0x6a7/0x884 [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c02f616d>] bonding_store_slaves+0x1ae/0x2fb [<c02f5fbf>] bonding_store_slaves+0x0/0x2fb [<c02ce8d7>] dev_attr_store+0x27/0x2c [<c019bcb9>] sysfs_write_file+0xad/0xe0 [<c019bc0c>] sysfs_write_file+0x0/0xe0 [<c0168ddc>] vfs_write+0x8a/0x10c [<c0118566>] do_page_fault+0x0/0x54a [<c0169361>] sys_write+0x41/0x67 [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff -> (lweventlist_lock){.+..} ops: 10 { initial-use at: [<c0141986>] mark_lock+0x64/0x451 [<c014289c>] __lock_acquire+0x4ba/0xc07 [<c02e365c>] e1000_read_phy_reg+0x1c7/0x1d3 [<c02e348b>] e1000_write_phy_reg+0xb9/0xc3 [<c024a7de>] delay_tsc+0x25/0x3b [<c0143062>] lock_acquire+0x79/0x93 [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c043a057>] _spin_lock_irqsave+0x3f/0x6c [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c03abb93>] linkwatch_fire_event+0x25/0x37 [<c02e1c43>] e1000_probe+0xad1/0xbe8 [<c0257f3f>] pci_device_probe+0x36/0x57 [<c02d0e5f>] driver_probe_device+0xe1/0x15f [<c043a2d1>] _spin_unlock+0x25/0x3b [<c043758a>] klist_next+0x58/0x6d [<c02d0f6f>] __driver_attach+0x0/0x7f [<c02d0fb8>] __driver_attach+0x49/0x7f [<c02d0403>] bus_for_each_dev+0x36/0x58 [<c02d0cb7>] driver_attach+0x16/0x18 [<c02d0f6f>] __driver_attach+0x0/0x7f [<c02d06fa>] bus_add_driver+0x6d/0x18d [<c0258089>] __pci_register_driver+0x53/0x7f [<c05be635>] e1000_init_module+0x45/0x7c [<c05a9499>] kernel_init+0x150/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c05a9349>] kernel_init+0x0/0x2b7 [<c0104baf>] kernel_thread_helper+0x7/0x10 [<ffffffff>] 0xffffffff in-softirq-W at: [<c011d20a>] __wake_up_common+0x32/0x5c [<c0142822>] __lock_acquire+0x440/0xc07 [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58 [<c0143062>] lock_acquire+0x79/0x93 [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c02dff01>] e1000_watchdog+0x0/0x5c9 [<c043a057>] _spin_lock_irqsave+0x3f/0x6c [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c03abb93>] linkwatch_fire_event+0x25/0x37 [<c03aeb1a>] netif_carrier_on+0x16/0x27 [<c02e0156>] e1000_watchdog+0x255/0x5c9 [<c02dff01>] e1000_watchdog+0x0/0x5c9 [<c012df52>] run_timer_softirq+0xfa/0x15d [<c012a8a6>] __do_softirq+0x56/0xdb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c012a8b8>] __do_softirq+0x68/0xdb [<c012a961>] do_softirq+0x36/0x51 [<c012ae4a>] local_bh_enable_ip+0xad/0xed [<c03bf0df>] rt_run_flush+0x64/0x8b [<c03e7924>] ip_mc_inc_group+0x184/0x1c1 [<c03e79a2>] ip_mc_up+0x41/0x59 [<c03e32dc>] inetdev_event+0x257/0x465 [<c03aa8da>] rtnl_notify+0x3a/0x40 [<c03aa997>] rtmsg_ifinfo+0xb7/0xdf [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a3d3b>] dev_open+0x71/0x77 [<c02ef626>] bond_enslave+0x30f/0x884 [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c02f6161>] bonding_store_slaves+0x1a2/0x2fb [<c02f616d>] bonding_store_slaves+0x1ae/0x2fb [<c02f5fbf>] bonding_store_slaves+0x0/0x2fb [<c02ce8d7>] dev_attr_store+0x27/0x2c [<c019bcb9>] sysfs_write_file+0xad/0xe0 [<c019bc0c>] sysfs_write_file+0x0/0xe0 [<c0168ddc>] vfs_write+0x8a/0x10c [<c0118566>] do_page_fault+0x0/0x54a [<c0169361>] sys_write+0x41/0x67 [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff } ... key at: [<c058a194>] lweventlist_lock+0x14/0x40 ... acquired at: [<c0142dff>] __lock_acquire+0xa1d/0xc07 [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c0142fa1>] __lock_acquire+0xbbf/0xc07 [<c0143062>] lock_acquire+0x79/0x93 [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c043a057>] _spin_lock_irqsave+0x3f/0x6c [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c03abacf>] linkwatch_add_event+0xd/0x2c [<c03abb93>] linkwatch_fire_event+0x25/0x37 [<c03aeb1a>] netif_carrier_on+0x16/0x27 [<c02ede2c>] bond_set_carrier+0x31/0x55 [<c02ee9b6>] bond_select_active_slave+0x9c/0xcd [<c02ed22b>] bond_compute_features+0x45/0x84 [<c02ef9be>] bond_enslave+0x6a7/0x884 [<c043905c>] mutex_lock_nested+0x2a4/0x2ac [<c02f616d>] bonding_store_slaves+0x1ae/0x2fb [<c02f5fbf>] bonding_store_slaves+0x0/0x2fb [<c02ce8d7>] dev_attr_store+0x27/0x2c [<c019bcb9>] sysfs_write_file+0xad/0xe0 [<c019bc0c>] sysfs_write_file+0x0/0xe0 [<c0168ddc>] vfs_write+0x8a/0x10c [<c0118566>] do_page_fault+0x0/0x54a [<c0169361>] sys_write+0x41/0x67 [<c0103e92>] sysenter_past_esp+0x5f/0xa5 [<ffffffff>] 0xffffffff stack backtrace: Pid: 9, comm: events/0 Not tainted 2.6.24-rc5 #1 [<c0140b38>] print_irq_inversion_bug+0x108/0x112 [<c014191d>] check_usage_forwards+0x3c/0x41 [<c0141b09>] mark_lock+0x1e7/0x451 [<c0142822>] __lock_acquire+0x440/0xc07 [<c011d686>] update_curr+0x52/0xc4 [<c013f175>] tick_sched_timer+0x129/0x165 [<c0143062>] lock_acquire+0x79/0x93 [<c0411c52>] mld_ifc_timer_expire+0x130/0x1fb [<c0411b22>] mld_ifc_timer_expire+0x0/0x1fb [<c0439d3a>] _spin_lock_bh+0x3b/0x64 [<c0411c52>] mld_ifc_timer_expire+0x130/0x1fb [<c0411c52>] mld_ifc_timer_expire+0x130/0x1fb [<c0411b22>] mld_ifc_timer_expire+0x0/0x1fb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c0411b22>] mld_ifc_timer_expire+0x0/0x1fb [<c012df52>] run_timer_softirq+0xfa/0x15d [<c012a8a6>] __do_softirq+0x56/0xdb [<c0141f89>] trace_hardirqs_on+0x10c/0x14c [<c012a8b8>] __do_softirq+0x68/0xdb [<c012a961>] do_softirq+0x36/0x51 [<c012ae4a>] local_bh_enable_ip+0xad/0xed [<c03bf0df>] rt_run_flush+0x64/0x8b [<c03e926e>] fib_netdev_event+0x61/0x65 [<c013ac20>] notifier_call_chain+0x2a/0x52 [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a [<c03a2abd>] netdev_state_change+0x18/0x29 [<c03abcf5>] __linkwatch_run_queue+0x150/0x17e [<c03abd40>] linkwatch_event+0x1d/0x22 [<c0133cab>] run_workqueue+0xdb/0x1b6 [<c0133c57>] run_workqueue+0x87/0x1b6 [<c03abd23>] linkwatch_event+0x0/0x22 [<c01346cb>] worker_thread+0x0/0x85 [<c0134744>] worker_thread+0x79/0x85 [<c0137179>] autoremove_wake_function+0x0/0x35 [<c01370c2>] kthread+0x38/0x5e [<c013708a>] kthread+0x0/0x5e [<c0104baf>] kernel_thread_helper+0x7/0x10 ======================= bonding: bond0: enslaving eth1 as a backup interface with a down link. bond0: no IPv6 routers present Best regards, Krzysztof Ol
On Fri, 14 Dec 2007, Andy Gospodarek wrote: > On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote: >> >> >> On Fri, 14 Dec 2007, Andy Gospodarek wrote: >> >>> On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote: >>>> >>>> >>>> On Wed, 12 Dec 2007, Jay Vosburgh wrote: >>>> >>>>> Herbert Xu <herbert@gondor.apana.org.au> wrote: >>>>> >>>>>>> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >>>>>>> drivers/net/bonding/bond_sysfs.c >>>>>>> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >>>>>>> +++ a/drivers/net/bonding/bond_sysfs.c >>>>>>> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str >>>>>>> out: >>>>>>> write_unlock_bh(&bond->lock); >>>>>>> >>>>>>> - rtnl_unlock(); >>>>>>> - >>>>>> >>>>>> Looking at the changeset that added this perhaps the intention >>>>>> is to hold the lock? If so we should add an rtnl_lock to the start >>>>>> of the function. >>>>> >>>>> Yes, this function needs to hold locks, and more than just >>>>> what's there now. I believe the following should be correct; I haven't >>>>> tested it, though (I'm supposedly on vacation right now). >>>>> >>>>> The following change should be correct for the >>>>> bonding_store_primary case discussed in this thread, and also corrects >>>>> the bonding_store_active case which performs similar functions. >>>>> >>>>> The bond_change_active_slave and bond_select_active_slave >>>>> functions both require rtnl, bond->lock for read and curr_slave_lock for >>>>> write_bh, and no other locks. This is so that the lower level >>>>> mode-specific functions can release locks down to just rtnl in order to >>>>> call, e.g., dev_set_mac_address with the locks it expects (rtnl only). >>>>> >>>>> Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> >>>>> >>>>> diff --git a/drivers/net/bonding/bond_sysfs.c >>>>> b/drivers/net/bonding/bond_sysfs.c >>>>> index 11b76b3..28a2d80 100644 >>>>> --- a/drivers/net/bonding/bond_sysfs.c >>>>> +++ b/drivers/net/bonding/bond_sysfs.c >>>>> @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device >>>>> *d, >>>>> struct slave *slave; >>>>> struct bonding *bond = to_bond(d); >>>>> >>>>> - write_lock_bh(&bond->lock); >>>>> + rtnl_lock(); >>>>> + read_lock(&bond->lock); >>>>> + write_lock_bh(&bond->curr_slave_lock); >>>>> + >>>>> if (!USES_PRIMARY(bond->params.mode)) { >>>>> printk(KERN_INFO DRV_NAME >>>>> ": %s: Unable to set primary slave; %s is in mode >>>>> %d\n", >>>>> @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device >>>>> *d, >>>>> } >>>>> } >>>>> out: >>>>> - write_unlock_bh(&bond->lock); >>>>> - >>>>> + write_unlock_bh(&bond->curr_slave_lock); >>>>> + read_unlock(&bond->lock); >>>>> rtnl_unlock(); >>>>> >>>>> return count; >>>>> @@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct >>>>> device *d, >>>>> struct bonding *bond = to_bond(d); >>>>> >>>>> rtnl_lock(); >>>>> - write_lock_bh(&bond->lock); >>>>> + read_lock(&bond->lock); >>>>> + write_lock_bh(&bond->curr_slave_lock); >>>>> >>>>> if (!USES_PRIMARY(bond->params.mode)) { >>>>> printk(KERN_INFO DRV_NAME >>>>> @@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct >>>>> device *d, >>>>> } >>>>> } >>>>> out: >>>>> - write_unlock_bh(&bond->lock); >>>>> + write_unlock_bh(&bond->curr_slave_lock); >>>>> + read_unlock(&bond->lock); >>>>> rtnl_unlock(); >>>>> >>>>> return count; >>>> >>>> Vanilla 2.6.24-rc5 plus this patch: >>>> >>>> ========================================================= >>>> [ INFO: possible irq lock inversion dependency detected ] >>>> 2.6.24-rc5 #1 >>>> --------------------------------------------------------- >>>> events/0/9 just changed the state of lock: >>>> (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb >>>> but this lock took another, soft-read-irq-unsafe lock in the past: >>>> (&bond->lock){-.--} >>>> >>>> and interrupts could create inverse lock ordering between them. >>>> >>>> >>> >>> Grrr, I should have seen that -- sorry. Try your luck with this instead: >> <CUT> >> >> No luck. >> > > > I'm guessing if we go back to using a write-lock for bond->lock this > will go back to working again, but I'm not totally convinced since there > are plenty of places where we used a read-lock with it. Should I check this patch or rather, based on a future discussion, wait for another version? > > diff --git a/drivers/net/bonding/bond_sysfs.c > b/drivers/net/bonding/bond_sysfs.c > index 11b76b3..635b857 100644 > --- a/drivers/net/bonding/bond_sysfs.c > +++ b/drivers/net/bonding/bond_sysfs.c > @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device *d, > struct slave *slave; > struct bonding *bond = to_bond(d); > > + rtnl_lock(); > write_lock_bh(&bond->lock); > + write_lock_bh(&bond->curr_slave_lock); > + > if (!USES_PRIMARY(bond->params.mode)) { > printk(KERN_INFO DRV_NAME > ": %s: Unable to set primary slave; %s is in mode %d\n", > @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device *d, > } > } > out: > + write_unlock_bh(&bond->curr_slave_lock); > write_unlock_bh(&bond->lock); > - > rtnl_unlock(); > > return count; > @@ -1191,6 +1194,7 @@ static ssize_t bonding_store_active_slave(struct device > *d, > > rtnl_lock(); > write_lock_bh(&bond->lock); > + write_lock_bh(&bond->curr_slave_lock); > > if (!USES_PRIMARY(bond->params.mode)) { > printk(KERN_INFO DRV_NAME > @@ -1247,6 +1251,7 @@ static ssize_t bonding_store_active_slave(struct device > *d, > } > } > out: > + write_unlock_bh(&bond->curr_slave_lock); > write_unlock_bh(&bond->lock); > rtnl_unlock(); > Best regards, Krzysztof Ol
Reply-To: andy@greyhouse.net On Tue, Dec 18, 2007 at 08:53:39PM +0100, Krzysztof Oledzki wrote: > > > On Fri, 14 Dec 2007, Andy Gospodarek wrote: > > >On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote: > >> > >> > >>On Fri, 14 Dec 2007, Andy Gospodarek wrote: > >> > >>>On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote: > >>>> > >>>> > >>>>On Wed, 12 Dec 2007, Jay Vosburgh wrote: > >>>> > >>>>>Herbert Xu <herbert@gondor.apana.org.au> wrote: > >>>>> > >>>>>>>diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > >>>>>>>drivers/net/bonding/bond_sysfs.c > >>>>>>>--- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > >>>>>>>+++ a/drivers/net/bonding/bond_sysfs.c > >>>>>>>@@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str > >>>>>>>out: > >>>>>>> write_unlock_bh(&bond->lock); > >>>>>>> > >>>>>>>- rtnl_unlock(); > >>>>>>>- > >>>>>> > >>>>>>Looking at the changeset that added this perhaps the intention > >>>>>>is to hold the lock? If so we should add an rtnl_lock to the start > >>>>>>of the function. > >>>>> > >>>>> Yes, this function needs to hold locks, and more than just > >>>>>what's there now. I believe the following should be correct; I haven't > >>>>>tested it, though (I'm supposedly on vacation right now). > >>>>> > >>>>> The following change should be correct for the > >>>>>bonding_store_primary case discussed in this thread, and also corrects > >>>>>the bonding_store_active case which performs similar functions. > >>>>> > >>>>> The bond_change_active_slave and bond_select_active_slave > >>>>>functions both require rtnl, bond->lock for read and curr_slave_lock > >>>>>for > >>>>>write_bh, and no other locks. This is so that the lower level > >>>>>mode-specific functions can release locks down to just rtnl in order to > >>>>>call, e.g., dev_set_mac_address with the locks it expects (rtnl only). > >>>>> > >>>>>Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> > >>>>> > >>>>>diff --git a/drivers/net/bonding/bond_sysfs.c > >>>>>b/drivers/net/bonding/bond_sysfs.c > >>>>>index 11b76b3..28a2d80 100644 > >>>>>--- a/drivers/net/bonding/bond_sysfs.c > >>>>>+++ b/drivers/net/bonding/bond_sysfs.c > >>>>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct > >>>>>device > >>>>>*d, > >>>>> struct slave *slave; > >>>>> struct bonding *bond = to_bond(d); > >>>>> > >>>>>- write_lock_bh(&bond->lock); > >>>>>+ rtnl_lock(); > >>>>>+ read_lock(&bond->lock); > >>>>>+ write_lock_bh(&bond->curr_slave_lock); > >>>>>+ > >>>>> if (!USES_PRIMARY(bond->params.mode)) { > >>>>> printk(KERN_INFO DRV_NAME > >>>>> ": %s: Unable to set primary slave; %s is in mode > >>>>> %d\n", > >>>>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct > >>>>>device > >>>>>*d, > >>>>> } > >>>>> } > >>>>>out: > >>>>>- write_unlock_bh(&bond->lock); > >>>>>- > >>>>>+ write_unlock_bh(&bond->curr_slave_lock); > >>>>>+ read_unlock(&bond->lock); > >>>>> rtnl_unlock(); > >>>>> > >>>>> return count; > >>>>>@@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct > >>>>>device *d, > >>>>> struct bonding *bond = to_bond(d); > >>>>> > >>>>> rtnl_lock(); > >>>>>- write_lock_bh(&bond->lock); > >>>>>+ read_lock(&bond->lock); > >>>>>+ write_lock_bh(&bond->curr_slave_lock); > >>>>> > >>>>> if (!USES_PRIMARY(bond->params.mode)) { > >>>>> printk(KERN_INFO DRV_NAME > >>>>>@@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct > >>>>>device *d, > >>>>> } > >>>>> } > >>>>>out: > >>>>>- write_unlock_bh(&bond->lock); > >>>>>+ write_unlock_bh(&bond->curr_slave_lock); > >>>>>+ read_unlock(&bond->lock); > >>>>> rtnl_unlock(); > >>>>> > >>>>> return count; > >>>> > >>>>Vanilla 2.6.24-rc5 plus this patch: > >>>> > >>>>========================================================= > >>>>[ INFO: possible irq lock inversion dependency detected ] > >>>>2.6.24-rc5 #1 > >>>>--------------------------------------------------------- > >>>>events/0/9 just changed the state of lock: > >>>>(&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb > >>>>but this lock took another, soft-read-irq-unsafe lock in the past: > >>>>(&bond->lock){-.--} > >>>> > >>>>and interrupts could create inverse lock ordering between them. > >>>> > >>>> > >>> > >>>Grrr, I should have seen that -- sorry. Try your luck with this instead: > >><CUT> > >> > >>No luck. > >> > > > > > >I'm guessing if we go back to using a write-lock for bond->lock this > >will go back to working again, but I'm not totally convinced since there > >are plenty of places where we used a read-lock with it. > > Should I check this patch or rather, based on a future discussion, wait > for another version? > > > > >diff --git a/drivers/net/bonding/bond_sysfs.c > >b/drivers/net/bonding/bond_sysfs.c > >index 11b76b3..635b857 100644 > >--- a/drivers/net/bonding/bond_sysfs.c > >+++ b/drivers/net/bonding/bond_sysfs.c > >@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device > >*d, > > struct slave *slave; > > struct bonding *bond = to_bond(d); > > > >+ rtnl_lock(); > > write_lock_bh(&bond->lock); > >+ write_lock_bh(&bond->curr_slave_lock); > >+ > > if (!USES_PRIMARY(bond->params.mode)) { > > printk(KERN_INFO DRV_NAME > > ": %s: Unable to set primary slave; %s is in mode > > %d\n", > >@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device > >*d, > > } > > } > >out: > >+ write_unlock_bh(&bond->curr_slave_lock); > > write_unlock_bh(&bond->lock); > >- > > rtnl_unlock(); > > > > return count; > >@@ -1191,6 +1194,7 @@ static ssize_t bonding_store_active_slave(struct > >device *d, > > > > rtnl_lock(); > > write_lock_bh(&bond->lock); > >+ write_lock_bh(&bond->curr_slave_lock); > > > > if (!USES_PRIMARY(bond->params.mode)) { > > printk(KERN_INFO DRV_NAME > >@@ -1247,6 +1251,7 @@ static ssize_t bonding_store_active_slave(struct > >device *d, > > } > > } > >out: > >+ write_unlock_bh(&bond->curr_slave_lock); > > write_unlock_bh(&bond->lock); > > rtnl_unlock(); > > > > > Best regards, > > Krzysztof Olędzki For now, I prefer Jay's original patch -- with the read_locks (rather than read/write_lock_bh) and the added rtnl_lock. There is still a lockdep issue that we need to sort-out, but this patch is needed first.
Any chance to push the patch upstream for 2.6.24?
On Wed, 19 Dec 2007, Andy Gospodarek wrote: > On Tue, Dec 18, 2007 at 08:53:39PM +0100, Krzysztof Oledzki wrote: >> >> >> On Fri, 14 Dec 2007, Andy Gospodarek wrote: >> >>> On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote: >>>> >>>> >>>> On Fri, 14 Dec 2007, Andy Gospodarek wrote: >>>> >>>>> On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote: >>>>>> >>>>>> >>>>>> On Wed, 12 Dec 2007, Jay Vosburgh wrote: >>>>>> >>>>>>> Herbert Xu <herbert@gondor.apana.org.au> wrote: >>>>>>> >>>>>>>>> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >>>>>>>>> drivers/net/bonding/bond_sysfs.c >>>>>>>>> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >>>>>>>>> +++ a/drivers/net/bonding/bond_sysfs.c >>>>>>>>> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str >>>>>>>>> out: >>>>>>>>> write_unlock_bh(&bond->lock); >>>>>>>>> >>>>>>>>> - rtnl_unlock(); >>>>>>>>> - >>>>>>>> >>>>>>>> Looking at the changeset that added this perhaps the intention >>>>>>>> is to hold the lock? If so we should add an rtnl_lock to the start >>>>>>>> of the function. >>>>>>> >>>>>>> Yes, this function needs to hold locks, and more than just >>>>>>> what's there now. I believe the following should be correct; I haven't >>>>>>> tested it, though (I'm supposedly on vacation right now). >>>>>>> >>>>>>> The following change should be correct for the >>>>>>> bonding_store_primary case discussed in this thread, and also corrects >>>>>>> the bonding_store_active case which performs similar functions. >>>>>>> >>>>>>> The bond_change_active_slave and bond_select_active_slave >>>>>>> functions both require rtnl, bond->lock for read and curr_slave_lock >>>>>>> for >>>>>>> write_bh, and no other locks. This is so that the lower level >>>>>>> mode-specific functions can release locks down to just rtnl in order to >>>>>>> call, e.g., dev_set_mac_address with the locks it expects (rtnl only). >>>>>>> >>>>>>> Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> >>>>>>> >>>>>>> diff --git a/drivers/net/bonding/bond_sysfs.c >>>>>>> b/drivers/net/bonding/bond_sysfs.c >>>>>>> index 11b76b3..28a2d80 100644 >>>>>>> --- a/drivers/net/bonding/bond_sysfs.c >>>>>>> +++ b/drivers/net/bonding/bond_sysfs.c >>>>>>> @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct >>>>>>> device >>>>>>> *d, >>>>>>> struct slave *slave; >>>>>>> struct bonding *bond = to_bond(d); >>>>>>> >>>>>>> - write_lock_bh(&bond->lock); >>>>>>> + rtnl_lock(); >>>>>>> + read_lock(&bond->lock); >>>>>>> + write_lock_bh(&bond->curr_slave_lock); >>>>>>> + >>>>>>> if (!USES_PRIMARY(bond->params.mode)) { >>>>>>> printk(KERN_INFO DRV_NAME >>>>>>> ": %s: Unable to set primary slave; %s is in >>>>>>> mode >>>>>>> %d\n", >>>>>>> @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct >>>>>>> device >>>>>>> *d, >>>>>>> } >>>>>>> } >>>>>>> out: >>>>>>> - write_unlock_bh(&bond->lock); >>>>>>> - >>>>>>> + write_unlock_bh(&bond->curr_slave_lock); >>>>>>> + read_unlock(&bond->lock); >>>>>>> rtnl_unlock(); >>>>>>> >>>>>>> return count; >>>>>>> @@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct >>>>>>> device *d, >>>>>>> struct bonding *bond = to_bond(d); >>>>>>> >>>>>>> rtnl_lock(); >>>>>>> - write_lock_bh(&bond->lock); >>>>>>> + read_lock(&bond->lock); >>>>>>> + write_lock_bh(&bond->curr_slave_lock); >>>>>>> >>>>>>> if (!USES_PRIMARY(bond->params.mode)) { >>>>>>> printk(KERN_INFO DRV_NAME >>>>>>> @@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct >>>>>>> device *d, >>>>>>> } >>>>>>> } >>>>>>> out: >>>>>>> - write_unlock_bh(&bond->lock); >>>>>>> + write_unlock_bh(&bond->curr_slave_lock); >>>>>>> + read_unlock(&bond->lock); >>>>>>> rtnl_unlock(); >>>>>>> >>>>>>> return count; >>>>>> >>>>>> Vanilla 2.6.24-rc5 plus this patch: >>>>>> >>>>>> ========================================================= >>>>>> [ INFO: possible irq lock inversion dependency detected ] >>>>>> 2.6.24-rc5 #1 >>>>>> --------------------------------------------------------- >>>>>> events/0/9 just changed the state of lock: >>>>>> (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb >>>>>> but this lock took another, soft-read-irq-unsafe lock in the past: >>>>>> (&bond->lock){-.--} >>>>>> >>>>>> and interrupts could create inverse lock ordering between them. >>>>>> >>>>>> >>>>> >>>>> Grrr, I should have seen that -- sorry. Try your luck with this instead: >>>> <CUT> >>>> >>>> No luck. >>>> >>> >>> >>> I'm guessing if we go back to using a write-lock for bond->lock this >>> will go back to working again, but I'm not totally convinced since there >>> are plenty of places where we used a read-lock with it. >> >> Should I check this patch or rather, based on a future discussion, wait >> for another version? >> >>> >>> diff --git a/drivers/net/bonding/bond_sysfs.c >>> b/drivers/net/bonding/bond_sysfs.c >>> index 11b76b3..635b857 100644 >>> --- a/drivers/net/bonding/bond_sysfs.c >>> +++ b/drivers/net/bonding/bond_sysfs.c >>> @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device >>> *d, >>> struct slave *slave; >>> struct bonding *bond = to_bond(d); >>> >>> + rtnl_lock(); >>> write_lock_bh(&bond->lock); >>> + write_lock_bh(&bond->curr_slave_lock); >>> + >>> if (!USES_PRIMARY(bond->params.mode)) { >>> printk(KERN_INFO DRV_NAME >>> ": %s: Unable to set primary slave; %s is in mode >>> %d\n", >>> @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device >>> *d, >>> } >>> } >>> out: >>> + write_unlock_bh(&bond->curr_slave_lock); >>> write_unlock_bh(&bond->lock); >>> - >>> rtnl_unlock(); >>> >>> return count; >>> @@ -1191,6 +1194,7 @@ static ssize_t bonding_store_active_slave(struct >>> device *d, >>> >>> rtnl_lock(); >>> write_lock_bh(&bond->lock); >>> + write_lock_bh(&bond->curr_slave_lock); >>> >>> if (!USES_PRIMARY(bond->params.mode)) { >>> printk(KERN_INFO DRV_NAME >>> @@ -1247,6 +1251,7 @@ static ssize_t bonding_store_active_slave(struct >>> device *d, >>> } >>> } >>> out: >>> + write_unlock_bh(&bond->curr_slave_lock); >>> write_unlock_bh(&bond->lock); >>> rtnl_unlock(); >>> >> >> >> Best regards, >> >> Krzysztof Ol
Reply-To: andy@greyhouse.net On Mon, Jan 07, 2008 at 06:57:25PM +0100, Krzysztof Oledzki wrote: > > > On Wed, 19 Dec 2007, Andy Gospodarek wrote: > > >On Tue, Dec 18, 2007 at 08:53:39PM +0100, Krzysztof Oledzki wrote: > >> > >> > >>On Fri, 14 Dec 2007, Andy Gospodarek wrote: > >> > >>>On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote: > >>>> > >>>> > >>>>On Fri, 14 Dec 2007, Andy Gospodarek wrote: > >>>> > >>>>>On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote: > >>>>>> > >>>>>> > >>>>>>On Wed, 12 Dec 2007, Jay Vosburgh wrote: > >>>>>> > >>>>>>>Herbert Xu <herbert@gondor.apana.org.au> wrote: > >>>>>>> > >>>>>>>>>diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > >>>>>>>>>drivers/net/bonding/bond_sysfs.c > >>>>>>>>>--- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix > >>>>>>>>>+++ a/drivers/net/bonding/bond_sysfs.c > >>>>>>>>>@@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str > >>>>>>>>>out: > >>>>>>>>> write_unlock_bh(&bond->lock); > >>>>>>>>> > >>>>>>>>>- rtnl_unlock(); > >>>>>>>>>- > >>>>>>>> > >>>>>>>>Looking at the changeset that added this perhaps the intention > >>>>>>>>is to hold the lock? If so we should add an rtnl_lock to the start > >>>>>>>>of the function. > >>>>>>> > >>>>>>> Yes, this function needs to hold locks, and more than just > >>>>>>>what's there now. I believe the following should be correct; I > >>>>>>>haven't > >>>>>>>tested it, though (I'm supposedly on vacation right now). > >>>>>>> > >>>>>>> The following change should be correct for the > >>>>>>>bonding_store_primary case discussed in this thread, and also > >>>>>>>corrects > >>>>>>>the bonding_store_active case which performs similar functions. > >>>>>>> > >>>>>>> The bond_change_active_slave and bond_select_active_slave > >>>>>>>functions both require rtnl, bond->lock for read and curr_slave_lock > >>>>>>>for > >>>>>>>write_bh, and no other locks. This is so that the lower level > >>>>>>>mode-specific functions can release locks down to just rtnl in order > >>>>>>>to > >>>>>>>call, e.g., dev_set_mac_address with the locks it expects (rtnl > >>>>>>>only). > >>>>>>> > >>>>>>>Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> > >>>>>>> > >>>>>>>diff --git a/drivers/net/bonding/bond_sysfs.c > >>>>>>>b/drivers/net/bonding/bond_sysfs.c > >>>>>>>index 11b76b3..28a2d80 100644 > >>>>>>>--- a/drivers/net/bonding/bond_sysfs.c > >>>>>>>+++ b/drivers/net/bonding/bond_sysfs.c > >>>>>>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct > >>>>>>>device > >>>>>>>*d, > >>>>>>> struct slave *slave; > >>>>>>> struct bonding *bond = to_bond(d); > >>>>>>> > >>>>>>>- write_lock_bh(&bond->lock); > >>>>>>>+ rtnl_lock(); > >>>>>>>+ read_lock(&bond->lock); > >>>>>>>+ write_lock_bh(&bond->curr_slave_lock); > >>>>>>>+ > >>>>>>> if (!USES_PRIMARY(bond->params.mode)) { > >>>>>>> printk(KERN_INFO DRV_NAME > >>>>>>> ": %s: Unable to set primary slave; %s is in > >>>>>>> mode > >>>>>>> %d\n", > >>>>>>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct > >>>>>>>device > >>>>>>>*d, > >>>>>>> } > >>>>>>> } > >>>>>>>out: > >>>>>>>- write_unlock_bh(&bond->lock); > >>>>>>>- > >>>>>>>+ write_unlock_bh(&bond->curr_slave_lock); > >>>>>>>+ read_unlock(&bond->lock); > >>>>>>> rtnl_unlock(); > >>>>>>> > >>>>>>> return count; > >>>>>>>@@ -1190,7 +1193,8 @@ static ssize_t > >>>>>>>bonding_store_active_slave(struct > >>>>>>>device *d, > >>>>>>> struct bonding *bond = to_bond(d); > >>>>>>> > >>>>>>> rtnl_lock(); > >>>>>>>- write_lock_bh(&bond->lock); > >>>>>>>+ read_lock(&bond->lock); > >>>>>>>+ write_lock_bh(&bond->curr_slave_lock); > >>>>>>> > >>>>>>> if (!USES_PRIMARY(bond->params.mode)) { > >>>>>>> printk(KERN_INFO DRV_NAME > >>>>>>>@@ -1247,7 +1251,8 @@ static ssize_t > >>>>>>>bonding_store_active_slave(struct > >>>>>>>device *d, > >>>>>>> } > >>>>>>> } > >>>>>>>out: > >>>>>>>- write_unlock_bh(&bond->lock); > >>>>>>>+ write_unlock_bh(&bond->curr_slave_lock); > >>>>>>>+ read_unlock(&bond->lock); > >>>>>>> rtnl_unlock(); > >>>>>>> > >>>>>>> return count; > >>>>>> > >>>>>>Vanilla 2.6.24-rc5 plus this patch: > >>>>>> > >>>>>>========================================================= > >>>>>>[ INFO: possible irq lock inversion dependency detected ] > >>>>>>2.6.24-rc5 #1 > >>>>>>--------------------------------------------------------- > >>>>>>events/0/9 just changed the state of lock: > >>>>>>(&mc->mca_lock){-+..}, at: [<c0411c7a>] > >>>>>>mld_ifc_timer_expire+0x130/0x1fb > >>>>>>but this lock took another, soft-read-irq-unsafe lock in the past: > >>>>>>(&bond->lock){-.--} > >>>>>> > >>>>>>and interrupts could create inverse lock ordering between them. > >>>>>> > >>>>>> > >>>>> > >>>>>Grrr, I should have seen that -- sorry. Try your luck with this > >>>>>instead: > >>>><CUT> > >>>> > >>>>No luck. > >>>> > >>> > >>> > >>>I'm guessing if we go back to using a write-lock for bond->lock this > >>>will go back to working again, but I'm not totally convinced since there > >>>are plenty of places where we used a read-lock with it. > >> > >>Should I check this patch or rather, based on a future discussion, wait > >>for another version? > >> > >>> > >>>diff --git a/drivers/net/bonding/bond_sysfs.c > >>>b/drivers/net/bonding/bond_sysfs.c > >>>index 11b76b3..635b857 100644 > >>>--- a/drivers/net/bonding/bond_sysfs.c > >>>+++ b/drivers/net/bonding/bond_sysfs.c > >>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device > >>>*d, > >>> struct slave *slave; > >>> struct bonding *bond = to_bond(d); > >>> > >>>+ rtnl_lock(); > >>> write_lock_bh(&bond->lock); > >>>+ write_lock_bh(&bond->curr_slave_lock); > >>>+ > >>> if (!USES_PRIMARY(bond->params.mode)) { > >>> printk(KERN_INFO DRV_NAME > >>> ": %s: Unable to set primary slave; %s is in mode > >>> %d\n", > >>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device > >>>*d, > >>> } > >>> } > >>>out: > >>>+ write_unlock_bh(&bond->curr_slave_lock); > >>> write_unlock_bh(&bond->lock); > >>>- > >>> rtnl_unlock(); > >>> > >>> return count; > >>>@@ -1191,6 +1194,7 @@ static ssize_t bonding_store_active_slave(struct > >>>device *d, > >>> > >>> rtnl_lock(); > >>> write_lock_bh(&bond->lock); > >>>+ write_lock_bh(&bond->curr_slave_lock); > >>> > >>> if (!USES_PRIMARY(bond->params.mode)) { > >>> printk(KERN_INFO DRV_NAME > >>>@@ -1247,6 +1251,7 @@ static ssize_t bonding_store_active_slave(struct > >>>device *d, > >>> } > >>> } > >>>out: > >>>+ write_unlock_bh(&bond->curr_slave_lock); > >>> write_unlock_bh(&bond->lock); > >>> rtnl_unlock(); > >>> > >> > >> > >>Best regards, > >> > >> Krzysztof Olędzki > > > >For now, I prefer Jay's original patch -- with the read_locks (rather > >than read/write_lock_bh) and the added rtnl_lock. There is still a > >lockdep issue that we need to sort-out, but this patch is needed first. > > This bug has not been fixed yet as it still exists in 2.6.24-rc7. Any > chances to cure it before 2.6.24-final? > > Best regards, > > Krzysztof Olędzki Krzysztof, I doubt the lockdep issue will be fixed, but the patch Jay posted and I acked needs to be included in 2.6.24. I played around with the locking when setting the multicast list and I can make the lockdep issue go away, but I need to be sure that it's OK to switch it to a read-lock from a write-lock (and I don't really think it is). -andy
Andy Gospodarek <andy@greyhouse.net> wrote: >On Mon, Jan 07, 2008 at 06:57:25PM +0100, Krzysztof Oledzki wrote: >> >> >> On Wed, 19 Dec 2007, Andy Gospodarek wrote: >> >> >On Tue, Dec 18, 2007 at 08:53:39PM +0100, Krzysztof Oledzki wrote: >> >> >> >> >> >>On Fri, 14 Dec 2007, Andy Gospodarek wrote: >> >> >> >>>On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote: >> >>>> >> >>>> >> >>>>On Fri, 14 Dec 2007, Andy Gospodarek wrote: >> >>>> >> >>>>>On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote: >> >>>>>> >> >>>>>> >> >>>>>>On Wed, 12 Dec 2007, Jay Vosburgh wrote: >> >>>>>> >> >>>>>>>Herbert Xu <herbert@gondor.apana.org.au> wrote: >> >>>>>>> >> >>>>>>>>>diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >> >>>>>>>>>drivers/net/bonding/bond_sysfs.c >> >>>>>>>>>--- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix >> >>>>>>>>>+++ a/drivers/net/bonding/bond_sysfs.c >> >>>>>>>>>@@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str >> >>>>>>>>>out: >> >>>>>>>>> write_unlock_bh(&bond->lock); >> >>>>>>>>> >> >>>>>>>>>- rtnl_unlock(); >> >>>>>>>>>- >> >>>>>>>> >> >>>>>>>>Looking at the changeset that added this perhaps the intention >> >>>>>>>>is to hold the lock? If so we should add an rtnl_lock to the start >> >>>>>>>>of the function. >> >>>>>>> >> >>>>>>> Yes, this function needs to hold locks, and more than just >> >>>>>>>what's there now. I believe the following should be correct; I >> >>>>>>>haven't >> >>>>>>>tested it, though (I'm supposedly on vacation right now). >> >>>>>>> >> >>>>>>> The following change should be correct for the >> >>>>>>>bonding_store_primary case discussed in this thread, and also >> >>>>>>>corrects >> >>>>>>>the bonding_store_active case which performs similar functions. >> >>>>>>> >> >>>>>>> The bond_change_active_slave and bond_select_active_slave >> >>>>>>>functions both require rtnl, bond->lock for read and curr_slave_lock >> >>>>>>>for >> >>>>>>>write_bh, and no other locks. This is so that the lower level >> >>>>>>>mode-specific functions can release locks down to just rtnl in order >> >>>>>>>to >> >>>>>>>call, e.g., dev_set_mac_address with the locks it expects (rtnl >> >>>>>>>only). >> >>>>>>> >> >>>>>>>Signed-off-by: Jay Vosburgh <fubar@us.ibm.com> >> >>>>>>> >> >>>>>>>diff --git a/drivers/net/bonding/bond_sysfs.c >> >>>>>>>b/drivers/net/bonding/bond_sysfs.c >> >>>>>>>index 11b76b3..28a2d80 100644 >> >>>>>>>--- a/drivers/net/bonding/bond_sysfs.c >> >>>>>>>+++ b/drivers/net/bonding/bond_sysfs.c >> >>>>>>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct >> >>>>>>>device >> >>>>>>>*d, >> >>>>>>> struct slave *slave; >> >>>>>>> struct bonding *bond = to_bond(d); >> >>>>>>> >> >>>>>>>- write_lock_bh(&bond->lock); >> >>>>>>>+ rtnl_lock(); >> >>>>>>>+ read_lock(&bond->lock); >> >>>>>>>+ write_lock_bh(&bond->curr_slave_lock); >> >>>>>>>+ >> >>>>>>> if (!USES_PRIMARY(bond->params.mode)) { >> >>>>>>> printk(KERN_INFO DRV_NAME >> >>>>>>> ": %s: Unable to set primary slave; %s is in >> >>>>>>> mode >> >>>>>>> %d\n", >> >>>>>>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct >> >>>>>>>device >> >>>>>>>*d, >> >>>>>>> } >> >>>>>>> } >> >>>>>>>out: >> >>>>>>>- write_unlock_bh(&bond->lock); >> >>>>>>>- >> >>>>>>>+ write_unlock_bh(&bond->curr_slave_lock); >> >>>>>>>+ read_unlock(&bond->lock); >> >>>>>>> rtnl_unlock(); >> >>>>>>> >> >>>>>>> return count; >> >>>>>>>@@ -1190,7 +1193,8 @@ static ssize_t >> >>>>>>>bonding_store_active_slave(struct >> >>>>>>>device *d, >> >>>>>>> struct bonding *bond = to_bond(d); >> >>>>>>> >> >>>>>>> rtnl_lock(); >> >>>>>>>- write_lock_bh(&bond->lock); >> >>>>>>>+ read_lock(&bond->lock); >> >>>>>>>+ write_lock_bh(&bond->curr_slave_lock); >> >>>>>>> >> >>>>>>> if (!USES_PRIMARY(bond->params.mode)) { >> >>>>>>> printk(KERN_INFO DRV_NAME >> >>>>>>>@@ -1247,7 +1251,8 @@ static ssize_t >> >>>>>>>bonding_store_active_slave(struct >> >>>>>>>device *d, >> >>>>>>> } >> >>>>>>> } >> >>>>>>>out: >> >>>>>>>- write_unlock_bh(&bond->lock); >> >>>>>>>+ write_unlock_bh(&bond->curr_slave_lock); >> >>>>>>>+ read_unlock(&bond->lock); >> >>>>>>> rtnl_unlock(); >> >>>>>>> >> >>>>>>> return count; >> >>>>>> >> >>>>>>Vanilla 2.6.24-rc5 plus this patch: >> >>>>>> >> >>>>>>========================================================= >> >>>>>>[ INFO: possible irq lock inversion dependency detected ] >> >>>>>>2.6.24-rc5 #1 >> >>>>>>--------------------------------------------------------- >> >>>>>>events/0/9 just changed the state of lock: >> >>>>>>(&mc->mca_lock){-+..}, at: [<c0411c7a>] >> >>>>>>mld_ifc_timer_expire+0x130/0x1fb >> >>>>>>but this lock took another, soft-read-irq-unsafe lock in the past: >> >>>>>>(&bond->lock){-.--} >> >>>>>> >> >>>>>>and interrupts could create inverse lock ordering between them. >> >>>>>> >> >>>>>> >> >>>>> >> >>>>>Grrr, I should have seen that -- sorry. Try your luck with this >> >>>>>instead: >> >>>><CUT> >> >>>> >> >>>>No luck. >> >>>> >> >>> >> >>> >> >>>I'm guessing if we go back to using a write-lock for bond->lock this >> >>>will go back to working again, but I'm not totally convinced since there >> >>>are plenty of places where we used a read-lock with it. >> >> >> >>Should I check this patch or rather, based on a future discussion, wait >> >>for another version? >> >> >> >>> >> >>>diff --git a/drivers/net/bonding/bond_sysfs.c >> >>>b/drivers/net/bonding/bond_sysfs.c >> >>>index 11b76b3..635b857 100644 >> >>>--- a/drivers/net/bonding/bond_sysfs.c >> >>>+++ b/drivers/net/bonding/bond_sysfs.c >> >>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device >> >>>*d, >> >>> struct slave *slave; >> >>> struct bonding *bond = to_bond(d); >> >>> >> >>>+ rtnl_lock(); >> >>> write_lock_bh(&bond->lock); >> >>>+ write_lock_bh(&bond->curr_slave_lock); >> >>>+ >> >>> if (!USES_PRIMARY(bond->params.mode)) { >> >>> printk(KERN_INFO DRV_NAME >> >>> ": %s: Unable to set primary slave; %s is in mode >> >>> %d\n", >> >>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device >> >>>*d, >> >>> } >> >>> } >> >>>out: >> >>>+ write_unlock_bh(&bond->curr_slave_lock); >> >>> write_unlock_bh(&bond->lock); >> >>>- >> >>> rtnl_unlock(); >> >>> >> >>> return count; >> >>>@@ -1191,6 +1194,7 @@ static ssize_t bonding_store_active_slave(struct >> >>>device *d, >> >>> >> >>> rtnl_lock(); >> >>> write_lock_bh(&bond->lock); >> >>>+ write_lock_bh(&bond->curr_slave_lock); >> >>> >> >>> if (!USES_PRIMARY(bond->params.mode)) { >> >>> printk(KERN_INFO DRV_NAME >> >>>@@ -1247,6 +1251,7 @@ static ssize_t bonding_store_active_slave(struct >> >>>device *d, >> >>> } >> >>> } >> >>>out: >> >>>+ write_unlock_bh(&bond->curr_slave_lock); >> >>> write_unlock_bh(&bond->lock); >> >>> rtnl_unlock(); >> >>> >> >> >> >> >> >>Best regards, >> >> >> >> Krzysztof Olędzki >> > >> >For now, I prefer Jay's original patch -- with the read_locks (rather >> >than read/write_lock_bh) and the added rtnl_lock. There is still a >> >lockdep issue that we need to sort-out, but this patch is needed first. >> >> This bug has not been fixed yet as it still exists in 2.6.24-rc7. Any >> chances to cure it before 2.6.24-final? >> >> Best regards, >> >> Krzysztof Olędzki > >Krzysztof, > >I doubt the lockdep issue will be fixed, but the patch Jay posted and I >acked needs to be included in 2.6.24. I'm (finally) back from vacation and am working on the lock problem right now; there are a couple of other changes that need to go in (in addition to what was posted previously). One is a spurious RTNL warning, the other is a similar 'wrong lock' type of problem that arises during module unload. I should have a patch set for this posted in a couple of hours. >I played around with the locking when setting the multicast list and I >can make the lockdep issue go away, but I need to be sure that it's OK >to switch it to a read-lock from a write-lock (and I don't really think >it is). I haven't looked at the lockdep problem yet. If you want to be brave and post your working patch for the lockdep thing, I might be able to crush your hopes that it's ok. -J --- -Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com
No longer exists in 2.6.24-final. AFAIK it was fixed by: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=e934dd7862e7f613b2ce9730d548a0a70913c8f7