Latest working kernel version: 2.6.27 Earliest failing kernel version: 2.6.28 Distribution: gentoo Hardware Environment: AMD64 (laptop - lenovo thinkpad sl500) Software Environment: linux 64 bit Problem Description: 2.6.28 kernel boots fine until the firewall script will be started (when iptables will be loaded by shorewall initscript). The responsible command for the oops is: iptables -A fooX1594 -m set --set fooX1594 src -j ACCEPT The problem seems to be timing sensitive according to Daniel Drake Jan 12 23:23:18 lasan ip_tables: (C) 2000-2006 Netfilter Core Team Jan 12 23:23:21 lasan Netfilter messages via NETLINK v0.30. Jan 12 23:23:21 lasan nf_conntrack version 0.5.0 (16384 buckets, 65536 max) Jan 12 23:23:21 lasan CONFIG_NF_CT_ACCT is deprecated and will be removed soon. Please use Jan 12 23:23:21 lasan nf_conntrack.acct=1 kernel paramater, acct=1 nf_conntrack module option or Jan 12 23:23:21 lasan sysctl net.netfilter.nf_conntrack_acct=1 to enable it. Jan 12 23:23:21 lasan ctnetlink v0.93: registering with nfnetlink. Jan 12 23:23:22 lasan ClusterIP Version 0.8 loaded successfully Jan 12 23:23:23 lasan BUG: unable to handle kernel NULL pointer dereference at 0000000000000000 Jan 12 23:23:23 lasan IP: [<ffffffffa056201c>] 0xffffffffa056201c Jan 12 23:23:23 lasan PGD 13b3d7067 PUD 139d6e067 PMD 0 Jan 12 23:23:23 lasan Oops: 0000 [#1] SMP Jan 12 23:23:23 lasan last sysfs file: /sys/class/power_supply/BAT0/energy_full Jan 12 23:23:23 lasan CPU 0 Jan 12 23:23:23 lasan Modules linked in: iptable_raw xt_recent xt_policy ipt_ULOG ipt_TTL ipt_ttl ipt_set ipt_REJECT ipt_REDIRECT ipt_NETMAP ipt_MASQUERADE ipt_LOG ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah ipt_addrtype nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_proto_sctp nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp ip_set_portmap ip_set_macipmap ip_set_ipmap ip_set_iphash ip_set xt_tcpmss xt_pkttype xt_physdev xt_owner xt_NFQUEUE xt_NFLOG xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_iprange xt_helper xt_hashlimit xt_DSCP xt_dscp xt_dccp xt_conntrack xt_CONNMARK xt_connmark xt_CLASSIFY xt_tcpudp xt_state iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle nfnetlink iptable_filter ip_tables x_tables aes_x86_64 aes_generic bridge stp llc snd_pcm_oss snd_mixer_oss snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device ipv6 coretemp btusb bluetooth uvcvideo compat_ioctl32 videodev v4l1_compat snd_hda_intel arc4 ecb firewire_ohci firewire_core iwlagn ehci_hcd uhci_hcd snd_pcm usbcore nvidiafb ohci1394 iwlcore intel_agp ieee1394 snd_timer sdhci_pci sdhci mac80211 sg snd_page_alloc snd_hwdep mmc_core snd cfg80211 r8169 fb_ddc i2c_algo_bit pcspkr vgastate video mii output joydev serio_raw i2c_core button battery ac Jan 12 23:23:23 lasan Pid: 5167, comm: iptables Not tainted 2.6.28 #1 Jan 12 23:23:23 lasan RIP: 0010:[<ffffffffa056201c>] [<ffffffffa056201c>] 0xffffffffa056201c Jan 12 23:23:23 lasan RSP: 0018:ffff880139cd7c98 EFLAGS: 00010292 Jan 12 23:23:27 lasan RAX: ffffffffa0562010 RBX: 0000000000000000 RCX: 0000000000000000 Jan 12 23:23:27 lasan RDX: 0000000000000020 RSI: 0000000000000020 RDI: ffff880139cd7d48 Jan 12 23:23:27 lasan RBP: ffff880139cd7ca8 R08: 0000000000000000 R09: 00000000fffffffe Jan 12 23:23:27 lasan R10: 0000000000000130 R11: 0000000000000000 R12: 0000000000000000 Jan 12 23:23:27 lasan R13: ffff880139cd7d48 R14: ffffc20013c6a152 R15: 0000000000000000 Jan 12 23:23:27 lasan FS: 00007fb014ad36f0(0000) GS:ffffffff80864d80(0000) knlGS:0000000000000000 Jan 12 23:23:27 lasan CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jan 12 23:23:27 lasan CR2: 0000000000000000 CR3: 0000000139cfb000 CR4: 00000000000006e0 Jan 12 23:23:27 lasan DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Jan 12 23:23:27 lasan DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Jan 12 23:23:27 lasan Process iptables (pid: 5167, threadinfo ffff880139cd6000, task ffff880137c01610) Jan 12 23:23:27 lasan Stack: Jan 12 23:23:27 lasan ffff880139cd7ce8 ffffffffa05624a0 ffff880139cd7ce8 ffffffffa0365a95 Jan 12 23:23:27 lasan 00ff88013a0048b0 ffff88013a236300 ffffc20013c6a150 ffffc20013c69000 Jan 12 23:23:27 lasan ffffc20013c6a152 0000000000000070 ffff880139cd7da8 ffffffffa037064e Jan 12 23:23:27 lasan Call Trace: Jan 12 23:23:27 lasan [<ffffffffa0365a95>] xt_check_match+0xf5/0x180 [x_tables] Jan 12 23:23:27 lasan [<ffffffffa037064e>] translate_table+0x30e/0x680 [ip_tables] Jan 12 23:23:27 lasan [<ffffffffa0371924>] do_ipt_set_ctl+0x124/0x1a0 [ip_tables] Jan 12 23:23:27 lasan [<ffffffff805b5f00>] nf_sockopt+0x60/0xa0 Jan 12 23:23:28 lasan [<ffffffff805b5f7c>] nf_setsockopt+0x1c/0x20 Jan 12 23:23:28 lasan [<ffffffff805c3622>] ip_setsockopt+0xa2/0xb0 Jan 12 23:23:28 lasan [<ffffffff805dff35>] raw_setsockopt+0x25/0x70 Jan 12 23:23:28 lasan [<ffffffff8058a08f>] sock_common_setsockopt+0xf/0x20 Jan 12 23:23:28 lasan [<ffffffff80587bbb>] sys_setsockopt+0x7b/0xe0 Jan 12 23:23:28 lasan [<ffffffff8020c2db>] system_call_fastpath+0x16/0x1b Jan 12 23:23:28 lasan Code: <0f> b7 39 e8 6c a0 ef ff 66 ff c0 74 13 8b 53 1c b8 01 00 00 00 85 Jan 12 23:23:28 lasan RIP [<ffffffffa056201c>] 0xffffffffa056201c Jan 12 23:23:28 lasan RSP <ffff880139cd7c98> Jan 12 23:23:28 lasan CR2: 0000000000000000 Jan 12 23:23:28 lasan ---[ end trace e6a1812c0f88b8a1 ]---
The Gentoo bugzilla link for this bug, containing various useful information about Jochen's setup is: http://bugs.gentoo.org/show_bug.cgi?id=254207
This appears to be caused by the out-of-tree set match, which hasn't been adjusted to API changes in 2.6.28. Please retry without it to confirm. Thanks.
sorry for the delay. perhaps I totally misunderstand Patrick but the oops happens also with a vanilla kernel and the stored stack trace is from the vanilla kernel - so what does this "out-of-tree set match" mean? I will retry it when I know what I should do
The set match is not in the vanilla kernel. It has not been adjusted to the current API and therefore crashes.
(In reply to comment #4) > The set match is not in the vanilla kernel. It has not been adjusted to the > current API and therefore crashes. > Could you be a more detailed and clear on how Jochen can 'retry without the out-of-tree set match' to confirm that the bug is caused by it? Thanks :)
ok, now it seems to be clear for me. It was a vanilla-kernel but what I forgot was that I have also installed the ipset package apart from the iptables package on this host. next step is then to deinstall the ipset package and see what happens. and according to Patrick's comment the oops should disappear
(In reply to comment #6) > ok, now it seems to be clear for me. > It was a vanilla-kernel but what I forgot was that I have also installed the > ipset package apart from the iptables package on this host. > > next step is then to deinstall the ipset package and see what happens. and > according to Patrick's comment the oops should disappear > > > I see. Alright, we'll be waiting for the results then :)
verified !! after deinstalling ipset package completely it's no longer possible to reproduce the oops. thanx
out-of-tree module bug