Bug 28862

Created attachment 47232 [details]
Output of iptables -nvL and dmesg

Here is how I can reproduce the case:

1. Stop iptables and remove all iptable and conntrack modules (ip_* & nf_*)
2. Start iptables and empty rules

- Here is are the rules at this step:

[root@laptop /home/dobedo]$ iptables -nvL
Chain INPUT (policy ACCEPT 1446 packets, 1194K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 1463 packets, 234K bytes)
 pkts bytes target     prot opt in     out     source               destination 

- Here is the list of modules at this step:

[root@laptop /home/dobedo]$ lsmod | grep -E "ip_|nf_"
nf_conntrack_netlink    15541  0 
nf_conntrack           48578  1 nf_conntrack_netlink
ip_tables               9108  1 iptable_filter
ip_queue                4194  0 
nfnetlink               2302  2 nf_conntrack_netlink,nfnetlink_queue
nf_defrag_ipv4          1007  0 
x_tables               11530  10 iptable_filter,ip_tables,xt_CLASSIFY,xt_TCPMSS,xt_NFQUEUE,xt_mac,xt_mark,xt_limit,xt_length,xt_tcpudp

- conntrack is not yet loaded at this step:

[root@laptop /home/dobedo]$ cat /proc/net/ip_conntrack
cat: /proc/net/ip_conntrack: No such file or directory

3. Load conntrack modules, which trigger the creation of the file in /proc

[root@laptop /home/dobedo]$ modprobe nf_conntrack_ipv4
[root@laptop /home/dobedo]$ cat /proc/net/ip_conntrack
cat: /proc/net/ip_conntrack: No space left on device

4. Empty the conntrack table:

[root@laptop /home/dobedo]$ conntrack -F
conntrack v0.9.15 (conntrack-tools): connection tracking table has been emptied.
[root@laptop /home/dobedo]$ cat /proc/net/ip_conntrack

5. Display e.g. a website in my web browser

6. Try again

[root@laptop /home/dobedo]$ cat /proc/net/ip_conntrack
cat: /proc/net/ip_conntrack: No space left on device

...also, this is what I get with conntrack tool:

[root@laptop /home/dobedo]$ conntrack -E
WARNING: We have hit ENOBUFS! We are losing events.
This message means that the current netlink socket buffer size is too small.
Please, check --buffer-size in conntrack(8) manpage.
conntrack v0.9.15 (conntrack-tools): Operation failed: No buffer space available

...and I forgot to mention also the dmesg result after case reproduced:

ctnetlink: unregistering from nfnetlink.
ip_tables: (C) 2000-2006 Netfilter Core Team
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
ctnetlink v0.93: registering with nfnetlink.
------------[ cut here ]------------
WARNING: at mm/page_alloc.c:1990 __alloc_pages_nodemask+0x4be/0x600()
Hardware name: Inspiron 1520                   
Modules linked in: nf_conntrack_ipv4 nf_conntrack_netlink nf_conntrack iptable_filter ip_tables rfcomm sco bnep l2cap crc16 fuse xt_CLASSIFY xt_TCPMSS xt_NFQUEUE xt_mac xt_mark xt_limit xt_length xt_tcpudp nfnetlink_queue ip_queue nfnetlink nf_defrag_ipv4 x_tables coretemp nvidia(P) arc4 ecb iwl3945 snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_hda_codec_idt iwlcore snd_pcm_oss snd_mixer_oss mac80211 cfg80211 cpufreq_ondemand joydev dell_wmi sparse_keymap snd_hda_intel snd_hda_codec snd_hwdep snd_pcm b44 acpi_cpufreq intel_agp snd_timer firewire_ohci uvcvideo freq_table ssb firewire_core pcmcia snd sdhci_pci intel_gtt videodev i2c_i801 dell_laptop pcmcia_core sdhci soundcore mii mmc_core crc_itu_t btusb v4l1_compat snd_page_alloc bluetooth usbhid video psmouse wmi i2c_core agpgart output hid shpchp ac battery rfkill thermal iTCO_wdt iTCO_vendor_support button dcdbas processor pci_hotplug sg serio_raw mperf evdev kvm_intel kvm ext3 jbd mbcache uhci_hcd ehci_hcd usbcore sr_mod cdrom sd_mod ata_piix ahci libahci ata_generic pata_acpi libata scsi_mod [last unloaded: ipt_REJECT]
Pid: 7990, comm: xchat Tainted: P            2.6.37-ARCH #1
Call Trace:
 [<c1044b7d>] warn_slowpath_common+0x6d/0xa0
 [<c10ca2ee>] ? __alloc_pages_nodemask+0x4be/0x600
 [<c10ca2ee>] ? __alloc_pages_nodemask+0x4be/0x600
 [<c1044bcd>] warn_slowpath_null+0x1d/0x20
 [<c10ca2ee>] __alloc_pages_nodemask+0x4be/0x600
 [<c10ca447>] __get_free_pages+0x17/0x30
 [<c10fa1d8>] __kmalloc_track_caller+0x178/0x200
 [<c10f92b9>] ? __kmalloc+0x159/0x200
 [<c103db8e>] ? dequeue_task_fair+0x3e/0x50
 [<c1272f19>] ? __alloc_skb+0x29/0x100
 [<fb84e03b>] ? ctnetlink_conntrack_event+0xeb/0x790 [nf_conntrack_netlink]
 [<c1272f43>] __alloc_skb+0x53/0x100
 [<fb84e03b>] ctnetlink_conntrack_event+0xeb/0x790 [nf_conntrack_netlink]
 [<c1315388>] ? _raw_spin_unlock_bh+0x18/0x20
 [<fb835c16>] nf_ct_deliver_cached_events+0x86/0xd0 [nf_conntrack]
 [<fb86f408>] ipv4_confirm+0xd8/0x170 [nf_conntrack_ipv4]
 [<c129f13d>] nf_iterate+0x4d/0x80
 [<c12a97b0>] ? ip_finish_output+0x0/0x300
 [<c12a97b0>] ? ip_finish_output+0x0/0x300
 [<c129f215>] nf_hook_slow+0xa5/0x100
 [<c12a97b0>] ? ip_finish_output+0x0/0x300
 [<c12aa4e2>] ip_output+0xc2/0xe0
 [<c12a97b0>] ? ip_finish_output+0x0/0x300
 [<c12a88f0>] ? dst_output+0x0/0x10
 [<c12a9bfb>] ip_local_out+0x1b/0x20
 [<c12a9d4c>] ip_queue_xmit+0x14c/0x3f0
 [<c11129d0>] ? __pollwait+0x0/0xd0
 [<c12c1d47>] ? __tcp_v4_send_check+0x47/0xe0
 [<c12be643>] tcp_transmit_skb+0x363/0x800
 [<c1112aa0>] ? pollwake+0x0/0x60
 [<c12bf0ba>] tcp_write_xmit+0xfa/0x950
 [<c12bf977>] __tcp_push_pending_frames+0x27/0x80
 [<c12b2b05>] tcp_sendmsg+0x7b5/0xa80
 [<c12d22f7>] inet_sendmsg+0x57/0xb0
 [<c126accf>] sock_sendmsg+0xef/0x110
 [<c126c358>] sys_sendto+0x108/0x150
 [<c110302c>] ? do_sync_read+0x9c/0xd0
 [<c126c3d6>] sys_send+0x36/0x40
 [<c126ce48>] sys_socketcall+0x1d8/0x2a0
 [<c11aa65e>] ? copy_to_user+0x2e/0x50
 [<c100391f>] sysenter_do_call+0x12/0x28
 [<c1310000>] ? generic_processor_info+0x18/0x14f
---[ end trace a73667eead5b71e9 ]---

Am 11.02.2011 21:59, schrieb bugzilla-daemon@bugzilla.kernel.org:
> https://bugzilla.kernel.org/show_bug.cgi?id=28862

Does this patch fix the issue?

Created attachment 47892 [details]
Fix return values of netfilter sysfs

This one, apparently bugzilla doesn't add attachments to the report.

Hi,

The patch helps partially.

Now, I can read the pseudo file /proc/net/ip_conntrack => no ENOSPC anymore (good) but the data read doesn't contain newlines anymore (not good). It's all on one line.

Maybe the ENOSPC is used in one way or another to trigger the newline. I'll try to check that further tonight.

Cheers

On 17.02.2011 10:06, bugzilla-daemon@bugzilla.kernel.org wrote:
> https://bugzilla.kernel.org/show_bug.cgi?id=28862
> 
> 
> --- Comment #6 from valere monseur <valere.monseur@ymail.com>  2011-02-17
> 09:06:37 ---
> Hi,
> 
> The patch helps partially.
> 
> Now, I can read the pseudo file /proc/net/ip_conntrack => no ENOSPC anymore
> (good) but the data read doesn't contain newlines anymore (not good). It's
> all
> on one line.

I'll look into this. Please post a few lines of the output you're getting.

Created attachment 48172 [details]
output of /proc/net/ip_conntrack after patch (no newline)

Am 17.02.2011 20:00, schrieb bugzilla-daemon@bugzilla.kernel.org:
> https://bugzilla.kernel.org/show_bug.cgi?id=28862
> 
> 
> 
> 
> 
> --- Comment #8 from valere monseur <valere.monseur@ymail.com>  2011-02-17
> 19:00:31 ---
> Created an attachment (id=48172)
>  --> (https://bugzilla.kernel.org/attachment.cgi?id=48172)
> output of /proc/net/ip_conntrack after patch (no newline)
> 

There's something very odd about your setup, apparently the space
provided to the conntrack seq-file callbacks isn't even enough
for a single entry. The "use=" part should follow "mark=", but
not even the first entry contains this.

Are you running a mainline kernel or does it include any patches?

Created attachment 48202 [details]
Things done

Well, 2 things:
- First, I'm using 2.6.37 to which the current archlinux patches have been applied. Location: ftp://ftp.archlinux.org/other/kernel26/patch-2.6.37-4-ARCH.bz2

- Second, in fact I've been unable to apply exactly the patch you provide me so I've made the changes "by hand" in the 2 files...probably a bad idea I know. I've attached the file mypatch.tar.gz containing the changes I've made if you want to have a look.

Have I broken the stuff ?
Do you want me to use 2.6.37 with only your patch ?
Something special to do to apply it ?

Thanks

Am 17.02.2011 20:50, schrieb bugzilla-daemon@bugzilla.kernel.org:
> Well, 2 things:
> - First, I'm using 2.6.37 to which the current archlinux patches have been
> applied. Location:
> ftp://ftp.archlinux.org/other/kernel26/patch-2.6.37-4-ARCH.bz2

That one looks fine.

> - Second, in fact I've been unable to apply exactly the patch you provide me
> so
> I've made the changes "by hand" in the 2 files...probably a bad idea I know.
> I've attached the file mypatch.tar.gz containing the changes I've made if you
> want to have a look.

That patch is hard to read because of reindentation, but the relevant
parts looks fine.

> Have I broken the stuff ?
> Do you want me to use 2.6.37 with only your patch ?
> Something special to do to apply it ?

That would be useful I think. I think the patch should apply
cleanly, but in case it doesn't, just change the seq file
functions to unconditionally return 0.

Created attachment 48342 [details]
/proc/net/ip_conntrack output

Hi,
Sorry for the reindentation, I turned it off in my editor now.

I've recompiled 2.6.37 with only your patch, which applies correctly btw (when I take correct actions).

Please, find the result in attach. Result is still the same: no newlines.

Any idea?
Thanks

I'll run a test now, keep you posted.

Created attachment 48382 [details]
test: where does it 'goto release' ?

Hi, I've made a quick and ugly test case to know where it does the 'goto release', which seems to be the reason why there is no newline (indeed, the newline is only written at the very last seq_printf, meaning that any 'goto release' performed before will cause the generation of a line without newline).

The attachment contains these files: the 2 modified sources, the .config file used (which is the one of the distro - unmodified), the outpur result of /proc/net/ip_conntrack.

What we can see is that there is this call performed from ct_seq_show function:

	if (ct_show_secctx(s, ct))
		goto release;

and then this call is done from ct_show_secctx function:

	ret = security_secid_to_secctx(ct->secmark, &secctx, &len);
	if (ret)
		return ret;

...and it returns! Meaning that back into ct_seq_show function if will perform the 'goto release' and then the newline is not written at all.

Ok, now my stupid patch doesn't tell me the return value of security_secid_to_secctx but I could modify the patch to get it, if needed.

Do you have any idea why security_secid_to_secctx returns a value and what does it mean ? is it a kernel bug or a misconfiguration ?

Thanks in advance
Cheers

Damned, this looks quite similar to what I have here: http://gitorious.org/sched_deadline/linux-26/commit/cba85b532e4aabdb97f44c18987d45141fd93faa as I have also no security module but NF_CONNTRACK_SECMARK enabled.

Can you check and let me know ?

Closing as obsolete, if this is still seen with modern kernels please re-open and update