linux/net/sched
Eric Dumazet b60fa1c5d0 net_sched: remove need_resched() from qdisc_run()
The introduction of this schedule point was done in commit
2ba2506ca7 ("[NET]: Add preemption point in qdisc_run")
at a time the loop was not bounded.

Then later in commit d5b8aa1d24 ("net_sched: fix dequeuer fairness")
we added a limit on the number of packets.

Now is the time to remove the schedule point, since the default
limit of 64 packets matches the number of packets a typical NAPI
poll can process in a row.

This solves a latency problem for most TCP receivers under moderate load :

1) host receives a packet.
   NET_RX_SOFTIRQ is raised by NIC hard IRQ handler

2) __do_softirq() does its first loop, handling NET_RX_SOFTIRQ
   and calling the driver napi->loop() function

3) TCP stores the skb in socket receive queue:

4) TCP calls sk->sk_data_ready() and wakeups a user thread
   waiting for EPOLLIN (as a result, need_resched() might now be true)

5) TCP cooks an ACK and sends it.

6) qdisc_run() processes one packet from qdisc, and sees need_resched(),
   this raises NET_TX_SOFTIRQ (even if there are no more packets in
   the qdisc)

Then we go back to the __do_softirq() in 2), and we see that new
softirqs were raised. Since need_resched() is true, we end up waking
ksoftirqd in this path :

    if (pending) {
            if (time_before(jiffies, end) && !need_resched() &&
                --max_restart)
                    goto restart;

            wakeup_softirqd();
    }

So we have many wakeups of ksoftirqd kernel threads,
and more calls to qdisc_run() with associated lock overhead.

Note that another way to solve the issue would be to change TCP
to first send the ACK packet, then signal the EPOLLIN,
but this changes P99 latencies, as sending the ACK packet
can add a long delay.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-10-02 14:26:33 -07:00
..
act_api.c net_sched: add policy validation for action attributes 2019-09-21 19:33:13 -07:00
act_bpf.c net_sched: fix a NULL pointer deref in ipt action 2019-08-27 15:05:58 -07:00
act_connmark.c net_sched: fix a NULL pointer deref in ipt action 2019-08-27 15:05:58 -07:00
act_csum.c net_sched: fix a NULL pointer deref in ipt action 2019-08-27 15:05:58 -07:00
act_ct.c netfilter: update include directives. 2019-09-13 12:33:06 +02:00
act_ctinfo.c net_sched: fix a NULL pointer deref in ipt action 2019-08-27 15:05:58 -07:00
act_gact.c net_sched: fix a NULL pointer deref in ipt action 2019-08-27 15:05:58 -07:00
act_ife.c net_sched: fix a NULL pointer deref in ipt action 2019-08-27 15:05:58 -07:00
act_ipt.c net_sched: fix a NULL pointer deref in ipt action 2019-08-27 15:05:58 -07:00
act_meta_mark.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
act_meta_skbprio.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
act_meta_skbtcindex.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
act_mirred.c net: sched: use get_dev() action API in flow_action infra 2019-09-16 09:18:03 +02:00
act_mpls.c net_sched: fix a NULL pointer deref in ipt action 2019-08-27 15:05:58 -07:00
act_nat.c net_sched: fix a NULL pointer deref in ipt action 2019-08-27 15:05:58 -07:00
act_pedit.c net_sched: fix a NULL pointer deref in ipt action 2019-08-27 15:05:58 -07:00
act_police.c net_sched: act_police: add 2 new attributes to support police 64bit rate and peakrate 2019-09-06 15:02:16 +02:00
act_sample.c net/sched: act_sample: don't push mac header on ip6gre ingress 2019-09-20 17:01:59 -07:00
act_simple.c net_sched: fix a NULL pointer deref in ipt action 2019-08-27 15:05:58 -07:00
act_skbedit.c net_sched: fix a NULL pointer deref in ipt action 2019-08-27 15:05:58 -07:00
act_skbmod.c net_sched: fix a NULL pointer deref in ipt action 2019-08-27 15:05:58 -07:00
act_tunnel_key.c net_sched: fix a NULL pointer deref in ipt action 2019-08-27 15:05:58 -07:00
act_vlan.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2019-09-02 11:20:17 -07:00
cls_api.c net: sched: fix possible crash in tcf_action_destroy() 2019-09-24 16:33:57 +02:00
cls_basic.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
cls_bpf.c net: sched: refactor block offloads counter usage 2019-08-26 14:17:43 -07:00
cls_cgroup.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
cls_flow.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
cls_flower.c net: sched: flower: don't take rtnl lock for cls hw offloads API 2019-08-26 14:17:43 -07:00
cls_fw.c net: sched: remove NET_CLS_IND config option 2019-06-15 14:06:13 -07:00
cls_matchall.c net: sched: cls_matchall: cleanup flow_action before deallocating 2019-08-30 15:12:05 -07:00
cls_route.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
cls_rsvp6.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
cls_rsvp.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
cls_rsvp.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
cls_tcindex.c treewide: Add SPDX license identifier for more missed files 2019-05-21 10:50:45 +02:00
cls_u32.c net: sched: refactor block offloads counter usage 2019-08-26 14:17:43 -07:00
em_canid.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 11 2019-05-21 11:28:45 +02:00
em_cmp.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
em_ipset.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
em_ipt.c net: sched: em_ipt: add support for addrtype matching 2019-06-29 11:15:12 -07:00
em_meta.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
em_nbyte.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
em_text.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
em_u32.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
ematch.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
Kconfig net/sched: Set default of CONFIG_NET_TC_SKB_EXT to N 2019-09-27 20:08:28 +02:00
Makefile net/sched: Introduce action ct 2019-07-09 12:11:59 -07:00
sch_api.c net_sched: add max len check for TCA_KIND 2019-09-21 19:18:51 -07:00
sch_atm.c treewide: Add SPDX license identifier for more missed files 2019-05-21 10:50:45 +02:00
sch_blackhole.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
sch_cake.c netlink: make validation more configurable for future strictness 2019-04-27 17:07:21 -04:00
sch_cbq.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
sch_cbs.c net/sched: cbs: Fix not adding cbs instance to list 2019-09-26 09:03:03 +02:00
sch_choke.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
sch_codel.c net: sched: Fix a possible null-pointer dereference in dequeue_func() 2019-07-29 09:46:58 -07:00
sch_drr.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
sch_dsmark.c treewide: Add SPDX license identifier for more missed files 2019-05-21 10:50:45 +02:00
sch_etf.c etf: Add skip_sock_check 2019-06-28 14:45:33 -07:00
sch_fifo.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
sch_fq_codel.c fq_codel: remove set but not used variables 'prev_ecn_mark' and 'prev_drop_count' 2019-08-08 22:32:19 -07:00
sch_fq.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
sch_generic.c net_sched: remove need_resched() from qdisc_run() 2019-10-02 14:26:33 -07:00
sch_gred.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
sch_hfsc.c netlink: make validation more configurable for future strictness 2019-04-27 17:07:21 -04:00
sch_hhf.c sch_hhf: ensure quantum and hhf_non_hh_weight are non-zero 2019-09-10 18:31:00 +01:00
sch_htb.c net: sched: sch_htb: don't call qdisc_put() while holding tree lock 2019-09-27 12:13:55 +02:00
sch_ingress.c net: flow_offload: rename TCF_BLOCK_BINDER_TYPE_* to FLOW_BLOCK_BINDER_TYPE_* 2019-07-09 14:38:50 -07:00
sch_mq.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
sch_mqprio.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
sch_multiq.c net: sched: multiq: don't call qdisc_put() while holding tree lock 2019-09-27 12:13:55 +02:00
sch_netem.c sch_netem: fix rcu splat in netem_enqueue() 2019-09-27 10:29:11 +02:00
sch_pie.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 235 2019-06-19 17:09:07 +02:00
sch_plug.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
sch_prio.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
sch_qfq.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
sch_red.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
sch_sfb.c net: sched: sch_sfb: don't call qdisc_put() while holding tree lock 2019-09-27 12:13:55 +02:00
sch_sfq.c net_sched: unset TCQ_F_CAN_BYPASS when adding filters 2019-07-17 13:34:09 -07:00
sch_skbprio.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
sch_taprio.c taprio: Add support for hardware offloading 2019-09-16 21:32:57 +02:00
sch_tbf.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
sch_teql.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00