Bug 209427 - Incorrect timestamp cause packet to be dropped
Summary: Incorrect timestamp cause packet to be dropped
Status: NEW
Alias: None
Product: Networking
Classification: Unclassified
Component: Other (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: Stephen Hemminger
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-09-29 17:15 UTC by ep4eg
Modified: 2020-09-29 17:15 UTC (History)
0 users

See Also:
Kernel Version: 5.8.10
Subsystem:
Regression: No
Bisected commit-id:


Attachments

Description ep4eg 2020-09-29 17:15:23 UTC
After upgrading from my 3.10 to 5.8.10 I found out some of my packets are getting dropped by ipvlan interface (I'm using qdisk fq). Debugging session led me to the place where this happens

net/sched/sch_fq.c:464
...
	if (fq_packet_beyond_horizon(skb, q)) {
		if (q->horizon_drop) {
			q->stat_horizon_drops++;
			return qdisc_drop(skb, sch, to_free);
		}
		q->stat_horizon_caps++;
		skb->tstamp = q->ktime_cache + q->horizon;
	}
...

then I noticed that in some cases skb->tstamp is equal to real ts whereas in the regular cases where a packet pass through it's time since kernel boot. This doesn't make any sense for me as this condition is satisfied constantly

net/sched/sch_fq.c:439
static bool fq_packet_beyond_horizon(const struct sk_buff *skb,
				    const struct fq_sched_data *q)
{
	return unlikely((s64)skb->tstamp > (s64)(q->ktime_cache + q->horizon));
}

Any ideas on what it can be?

some outputs:
    [Tue Sep 29 14:59:06 2020] DBG: TIME: trid: -1081131982. all: q->ktime_cache 328453964122, skb->tstamp 1601391546982793177 <<
    [Tue Sep 29 14:59:06 2020] DBG: TIME: trid: -1485308564. all: q->ktime_cache 334998110463, skb->tstamp 335012588783
    ...
    [Tue Sep 29 14:59:06 2020] DBG: TIME: trid: -1010372082. all: q->ktime_cache 335873778729, skb->tstamp 335741726080
    [Tue Sep 29 14:59:06 2020] DBG: TIME: trid: 192888327. all: q->ktime_cache 335860696387, skb->tstamp 335870531339
    [Tue Sep 29 14:59:07 2020] DBG: TIME: trid: -1463571809. all: q->ktime_cache 335305774517, skb->tstamp 1601391548109319017 <<

  perf trace: 
    curl 32613 [014]  1546.957467: skb:kfree_skb: skbaddr=0xffff888f57159ae0 protocol=2048 location=0xffffffff817bfdad
                  9bfd56 kfree_skb (/lib/modules/5.8.10/build/vmlinux)
                  9bfd56 kfree_skb (/lib/modules/5.8.10/build/vmlinux)
                  9bfdad kfree_skb_list (/lib/modules/5.8.10/build/vmlinux)
                  9da2b1 __dev_queue_xmit (/lib/modules/5.8.10/build/vmlinux)
                  9da380 dev_queue_xmit (/lib/modules/5.8.10/build/vmlinux)
                    1f39 vlan_dev_open ([8021q])
                  9d9851 dev_hard_start_xmit (/lib/modules/5.8.10/build/vmlinux)
                  9d9d05 __dev_queue_xmit (/lib/modules/5.8.10/build/vmlinux)
                  9da380 dev_queue_xmit (/lib/modules/5.8.10/build/vmlinux)
                  9d9d05 __dev_queue_xmit (/lib/modules/5.8.10/build/vmlinux)
                  9da380 dev_queue_xmit (/lib/modules/5.8.10/build/vmlinux)
                  9e32c3 neigh_connected_output (/lib/modules/5.8.10/build/vmlinux)
                  a68cdb ip_finish_output2 (/lib/modules/5.8.10/build/vmlinux)
                  a69b77 __ip_finish_output (/lib/modules/5.8.10/build/vmlinux)
                  a69cad ip_finish_output (/lib/modules/5.8.10/build/vmlinux)
                  a6b65a ip_output (/lib/modules/5.8.10/build/vmlinux)
                    f2e4 ip_vs_tunnel_xmit ([ip_vs])
                    536a ip_vs_nat_icmp ([ip_vs])
                    5913 ip_vs_out_icmp ([ip_vs])
                    59c6 ip_vs_out_icmp ([ip_vs])
                  a58255 nf_hook_slow (/lib/modules/5.8.10/build/vmlinux)
                  a656f1 ip_local_deliver (/lib/modules/5.8.10/build/vmlinux)
                  a64e07 ip_rcv_finish (/lib/modules/5.8.10/build/vmlinux)
                  b717bb ip_sabotage_in (/lib/modules/5.8.10/build/vmlinux)
                  a58255 nf_hook_slow (/lib/modules/5.8.10/build/vmlinux)
                  a657a0 ip_rcv (/lib/modules/5.8.10/build/vmlinux)
                  9db608 __netif_receive_skb_one_core (/lib/modules/5.8.10/build/vmlinux)
                  9db658 __netif_receive_skb (/lib/modules/5.8.10/build/vmlinux)
                  9db6e9 netif_receive_skb (/lib/modules/5.8.10/build/vmlinux)
                  b5c5b6 br_netif_receive_skb (/lib/modules/5.8.10/build/vmlinux)
                  b5c7d3 br_pass_frame_up (/lib/modules/5.8.10/build/vmlinux)
                  b5c94d br_handle_frame_finish (/lib/modules/5.8.10/build/vmlinux)
                  b7253b br_nf_hook_thresh (/lib/modules/5.8.10/build/vmlinux)
                  b72fb0 br_nf_pre_routing_finish (/lib/modules/5.8.10/build/vmlinux)
                  b733bc br_nf_pre_routing (/lib/modules/5.8.10/build/vmlinux)
                  b5cdf1 br_handle_frame (/lib/modules/5.8.10/build/vmlinux)
                  9da8c8 __netif_receive_skb_core (/lib/modules/5.8.10/build/vmlinux)
                  9db5bf __netif_receive_skb_one_core (/lib/modules/5.8.10/build/vmlinux)
                  9db658 __netif_receive_skb (/lib/modules/5.8.10/build/vmlinux)
                  9db8a9 process_backlog (/lib/modules/5.8.10/build/vmlinux)
                  9dd337 net_rx_action (/lib/modules/5.8.10/build/vmlinux)
                  e000e1 __do_softirq (/lib/modules/5.8.10/build/vmlinux)
                  c010c2 asm_call_on_stack (/lib/modules/5.8.10/build/vmlinux)
                  235bef do_softirq_own_stack (/lib/modules/5.8.10/build/vmlinux)
                  297f86 do_softirq.part.0 (/lib/modules/5.8.10/build/vmlinux)
                  297fe0 __local_bh_enable_ip (/lib/modules/5.8.10/build/vmlinux)
                  a68cef ip_finish_output2 (/lib/modules/5.8.10/build/vmlinux)
                  a69b77 __ip_finish_output (/lib/modules/5.8.10/build/vmlinux)
                  a69cad ip_finish_output (/lib/modules/5.8.10/build/vmlinux)
                  a6b65a ip_output (/lib/modules/5.8.10/build/vmlinux)
                  a6ad4d ip_local_out (/lib/modules/5.8.10/build/vmlinux)
                  a6b097 __ip_queue_xmit (/lib/modules/5.8.10/build/vmlinux)
                  a8f740 ip_queue_xmit (/lib/modules/5.8.10/build/vmlinux)
                  a89746 __tcp_transmit_skb (/lib/modules/5.8.10/build/vmlinux)
                  a8a5de tcp_connect (/lib/modules/5.8.10/build/vmlinux)
                  a90f73 tcp_v4_connect (/lib/modules/5.8.10/build/vmlinux)
                  aad59a __inet_stream_connect (/lib/modules/5.8.10/build/vmlinux)
                  aad88b inet_stream_connect (/lib/modules/5.8.10/build/vmlinux)
                  9b3e3f __sys_connect_file (/lib/modules/5.8.10/build/vmlinux)
                  9b3ef1 __sys_connect (/lib/modules/5.8.10/build/vmlinux)
                  9b3f3a __x64_sys_connect (/lib/modules/5.8.10/build/vmlinux)
                  b896b2 do_syscall_64 (/lib/modules/5.8.10/build/vmlinux)
                  c0008c entry_SYSCALL_64 (/lib/modules/5.8.10/build/vmlinux)
                   53878 [unknown] (/lib/ld-musl-x86_64.so.1)
                       0 [unknown] ([unknown])
                       0 [unknown] ([unknown])

Note You need to log in before you can comment on or make changes to this bug.