iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
a9744f7ca2
linux/include/net
History
Magnus Karlsson a9744f7ca2 xsk: fix potential race in SKB TX completion code 
There is a potential race in the TX completion code for the SKB
case. One process enters the sendmsg code of an AF_XDP socket in order
to send a frame. The execution eventually trickles down to the driver
that is told to send the packet. However, it decides to drop the
packet due to some error condition (e.g., rings full) and frees the
SKB. This will trigger the SKB destructor and a completion will be
sent to the AF_XDP user space through its
single-producer/single-consumer queues.

At the same time a TX interrupt has fired on another core and it
dispatches the TX completion code in the driver. It does its HW
specific things and ends up freeing the SKB associated with the
transmitted packet. This will trigger the SKB destructor and a
completion will be sent to the AF_XDP user space through its
single-producer/single-consumer queues. With a pseudo call stack, it
would look like this:

Core 1:
sendmsg() being called in the application
  netdev_start_xmit()
    Driver entered through ndo_start_xmit
      Driver decides to free the SKB for some reason (e.g., rings full)
        Destructor of SKB called
          xskq_produce_addr() is called to signal completion to user space

Core 2:
TX completion irq
  NAPI loop
    Driver irq handler for TX completions
      Frees the SKB
        Destructor of SKB called
          xskq_produce_addr() is called to signal completion to user space

We now have a violation of the single-producer/single-consumer
principle for our queues as there are two threads trying to produce at
the same time on the same queue.

Fixed by introducing a spin_lock in the destructor. In regards to the
performance, I get around 1.74 Mpps for txonly before and after the
introduction of the spinlock. There is of course some impact due to
the spin lock but it is in the less significant digits that are too
noisy for me to measure. But let us say that the version without the
spin lock got 1.745 Mpps in the best case and the version with 1.735
Mpps in the worst case, then that would mean a maximum drop in
performance of 0.5%.

Fixes: 35fcde7f8d ("xsk: support for Tx")
Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2018-07-02 18:37:12 -07:00
..
9p 9p: Implement show_options 2017-07-11 06:08:58 -04:00
bluetooth Revert changes to convert to ->poll_mask() and aio IOCB_CMD_POLL 2018-06-28 10:40:47 -07:00
caif caif: reduce stack size with KASAN 2018-01-19 14:02:12 -05:00
iucv Revert changes to convert to ->poll_mask() and aio IOCB_CMD_POLL 2018-06-28 10:40:47 -07:00
netfilter netfilter: nf_conncount: Fix garbage collection with zones 2018-06-12 20:07:07 +02:00
netns netfilter: ipv6: nf_defrag: reduce struct net memory waste 2018-06-18 14:13:25 +02:00
nfc
phonet proc: introduce proc_create_net{,_data} 2018-05-16 07:24:30 +02:00
sctp Revert changes to convert to ->poll_mask() and aio IOCB_CMD_POLL 2018-06-28 10:40:47 -07:00
tc_act net/sched: act_csum: don't use spinlock in the fast path 2018-01-23 19:51:46 -05:00
6lowpan.h
act_api.h net/sched: remove tcf_idr_cleanup() 2018-03-23 21:52:19 -04:00
addrconf.h net/ipv6: Add support for specifying metric of connected routes 2018-05-29 10:12:45 -04:00
af_ieee802154.h
af_rxrpc.h rxrpc, afs: Use debug_ids rather than pointers in traces 2018-03-27 23:03:00 +01:00
af_unix.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
af_vsock.h VSOCK: use TCP state constants for sk_state 2017-10-05 18:44:17 -07:00
ah.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
arp.h ipv4: Make neigh lookup keys for loopback/point-to-point devices be INADDR_ANY 2018-01-15 14:53:43 -05:00
atmclip.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ax25.h proc: introduce proc_create_seq{,_data} 2018-05-16 07:23:35 +02:00
ax88796.h net-next: ax88796: add interrupt status callback to platform data 2018-04-19 16:11:11 -04:00
bond_3ad.h
bond_alb.h
bond_options.h bonding: Prevent duplicate userspace notification 2017-05-27 18:51:41 -04:00
bonding.h bonding: allow use of tx hashing in balance-alb 2018-05-16 12:15:11 -04:00
busy_poll.h net: refactor socket_poll 2018-05-26 09:16:44 +02:00
calipso.h net, calipso: convert calipso_doi.refcount from atomic_t to refcount_t 2017-07-04 22:35:16 +01:00
cfg80211-wext.h
cfg80211.h nl80211: Update ERP info using NL80211_CMD_UPDATE_CONNECT_PARAMS 2018-05-23 11:21:35 +02:00
cfg802154.h
checksum.h
cipso_ipv4.h net, ipv4: convert cipso_v4_doi.refcount from atomic_t to refcount_t 2017-07-04 01:29:04 -07:00
cls_cgroup.h
codel_impl.h
codel_qdisc.h
codel.h
compat.h net: remove compat_sys_*() prototypes from net/compat.h 2018-04-02 20:16:17 +02:00
datalink.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
dcbevent.h
dcbnl.h net/dcb: Add dcbnl buffer attribute 2018-05-24 14:22:59 -07:00
devlink.h devlink: Add extack to reload and port_{un, }split operations 2018-06-05 12:32:37 -04:00
dn_dev.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
dn_fib.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
dn_neigh.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
dn_nsp.h net/decnet: Convert timers to use timer_setup() 2017-10-18 12:39:36 +01:00
dn_route.h decnet: Move dn_next into decnet route structure. 2017-11-30 09:54:25 -05:00
dn.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2017-11-04 09:26:51 +09:00
dsa.h net: dsa: Plug in PHYLINK support 2018-05-11 12:03:06 -04:00
dsfield.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
dst_cache.h net: core: dst_cache_set_ip6: Rename 'addr' parameter to 'saddr' for consistency 2018-03-05 12:52:45 -05:00
dst_metadata.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2017-11-04 09:26:51 +09:00
dst_ops.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
dst.h net: core: dst: Add kernel-doc for 'net' parameter 2018-03-05 12:52:45 -05:00
erspan.h erspan: set bso bit based on mirrored packet's len 2018-05-20 18:31:42 -04:00
esp.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ethoc.h inet: whitespace cleanup 2018-02-28 11:43:28 -05:00
failover.h net: Introduce generic failover module 2018-05-28 22:59:54 -04:00
fib_notifier.h net: Add extack to fib_notifier_info 2017-11-01 11:50:43 +09:00
fib_rules.h net: fib_rules: add extack support 2018-04-23 10:21:24 -04:00
firewire.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
flow_dissector.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2018-05-11 20:53:22 -04:00
flow.h net: Remove unused get_hash_from_flow functions 2018-03-04 13:04:23 -05:00
fou.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
fq_impl.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2017-10-30 21:09:24 +09:00
fq.h fq: support filtering a given tin 2017-10-11 09:49:34 +02:00
garp.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
gen_stats.h net: sched: add support for TCQ_F_NOLOCK subqueues to sch_mq 2017-12-08 13:32:26 -05:00
genetlink.h genetlink: fix genlmsg_nlhdr() 2017-11-16 10:49:00 +09:00
geneve.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
gre.h net: GRE: Add is_gretap_dev, is_ip6gretap_dev 2018-02-27 14:46:26 -05:00
gro_cells.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
gtp.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
gue.h fou: fix some member types in guehdr 2017-12-11 14:10:06 -05:00
hwbm.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
icmp.h
ieee80211_radiotap.h mac80211: support reporting A-MPDU EOF bit value/known 2018-02-22 21:13:02 +01:00
ieee802154_netdev.h
if_inet6.h net/ipv6: Add support for specifying metric of connected routes 2018-05-29 10:12:45 -04:00
ife.h net: sched: ife: handle malformed tlv length 2018-04-22 21:12:00 -04:00
ila.h
inet6_connection_sock.h
inet6_hashtables.h net: ipv6: add second dif to inet6 socket lookups 2017-08-07 11:39:22 -07:00
inet_common.h net: Introduce __inet_bind() and __inet6_bind 2018-03-31 02:15:43 +02:00
inet_connection_sock.h net: ipv4: remove define INET_CSK_DEBUG and unnecessary EXPORT_SYMBOL 2018-05-10 17:43:55 -04:00
inet_ecn.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2017-11-04 09:26:51 +09:00
inet_frag.h inet: frags: reorganize struct netns_frags 2018-03-31 23:25:39 -04:00
inet_hashtables.h inet: Add a 2nd listener hashtable (port+addr) 2017-12-03 10:18:28 -05:00
inet_sock.h udp: generate gso with UDP_SEGMENT 2018-04-26 15:08:04 -04:00
inet_timewait_sock.h net-tcp: remove useless tw_timeout field 2018-06-05 10:45:24 -04:00
inetpeer.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ip6_checksum.h
ip6_fib.h net/ipv6: respect rcu grace period before freeing fib6_info 2018-06-20 07:57:23 +09:00
ip6_route.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2018-05-24 22:20:51 -04:00
ip6_tunnel.h ip6_gre: add erspan v2 support 2017-12-15 12:34:00 -05:00
ip_fib.h net/ipv4: Add helper to return path MTU based on fib result 2018-05-22 10:51:09 +02:00
ip_tunnels.h net/ipv4: Update ip_tunnel_metadata_cnt static key to modern api 2018-05-10 15:13:33 -04:00
ip_vs.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf 2018-06-11 14:24:32 -07:00
ip.h ipv4: support sport, dport and ip_proto in RTM_GETROUTE 2018-05-23 15:14:12 -04:00
ipcomp.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ipconfig.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ipv6.h ipv6: omit traffic class when calculating flow hash 2018-06-04 13:21:18 -04:00
ipx.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
iw_handler.h net: Spelling s/stucture/structure/ 2018-03-27 09:51:23 +02:00
kcm.h
l3mdev.h
lapb.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
lib80211.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
llc_c_ac.h net: LLC: Convert timers to use timer_setup() 2017-10-25 12:06:25 +09:00
llc_c_ev.h
llc_c_st.h
llc_conn.h llc: delete timers synchronously in llc_sk_free() 2018-04-22 14:55:03 -04:00
llc_if.h
llc_pdu.h
llc_s_ac.h
llc_s_ev.h
llc_s_st.h
llc_sap.h
llc.h net, llc: convert llc_sap.refcnt from atomic_t to refcount_t 2017-07-04 22:35:15 +01:00
lwtunnel.h net: Move ipv4 set_lwt_redirect helper to lwtunnel 2018-02-14 14:43:32 -05:00
mac80211.h mac80211: Support adding duration for prepare_tx() callback 2018-05-23 11:06:10 +02:00
mac802154.h
mip6.h
mld.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
mpls_iptunnel.h
mpls.h
mrp.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ncsi.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ndisc.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
neighbour.h neighbour: support for NTF_EXT_LEARNED flag 2018-04-25 13:19:59 -04:00
net_failover.h net: Introduce net_failover driver 2018-05-28 22:59:54 -04:00
net_namespace.h netfilter: ipv6: nf_defrag: reduce struct net memory waste 2018-06-18 14:13:25 +02:00
net_ratelimit.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
netevent.h net/ipv6: Add support for path selection using hash of 5-tuple 2018-03-04 13:04:23 -05:00
netlabel.h net: convert netlbl_lsm_cache.refcount from atomic_t to refcount_t 2017-07-01 07:39:09 -07:00
netlink.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
netprio_cgroup.h
netrom.h proc: introduce proc_create_seq{,_data} 2018-05-16 07:23:35 +02:00
nexthop.h net: fix rtnh_ok() 2018-04-07 22:32:31 -04:00
nl802154.h
nsh.h openvswitch: enable NSH support 2017-11-08 16:12:33 +09:00
p8022.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
page_pool.h xdp: introduce xdp_return_frame_rx_napi 2018-05-24 18:36:15 -07:00
ping.h ipv{4,6}/ping: simplify proc file creation 2018-05-16 07:23:35 +02:00
pkt_cls.h nfp: reject binding to shared blocks 2018-06-27 10:46:43 +09:00
pkt_sched.h net: remove prototype of qdisc_lookup_class() 2018-01-16 14:56:54 -05:00
pptp.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
protocol.h IPv4: early demux can return an error code 2017-10-01 03:55:47 +01:00
psample.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
psnap.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
raw.h ipv{4,6}/raw: simplify ѕeq_file code 2018-05-16 07:23:35 +02:00
rawv6.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
red.h net_sched: red: Avoid illegal values 2017-12-05 14:37:13 -05:00
regulatory.h cfg80211: read wmm rules from regulatory database 2018-03-29 11:11:40 +02:00
request_sock.h tcp: socket option to set TCP fast open key 2017-10-20 13:21:36 +01:00
rose.h proc: introduce proc_create_seq{,_data} 2018-05-16 07:23:35 +02:00
route.h net/ipv4: Add support for specifying metric of connected routes 2018-05-29 10:12:45 -04:00
rsi_91x.h Bluetooth: btrsi: add new rsi bluetooth driver 2018-03-13 18:37:02 +02:00
rtnetlink.h rtnetlink: Remove VLA usage 2018-05-31 22:48:46 -04:00
sch_generic.h net: sched: add qstats.qlen to qlen 2018-05-29 09:49:15 -04:00
scm.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
secure_seq.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
seg6_hmac.h
seg6_local.h bpf: Add IPv6 Segment Routing helpers 2018-05-24 11:57:35 +02:00
seg6.h ipv6: sr: export function lookup_nexthop 2018-05-24 11:57:35 +02:00
slhc_vj.h slip: Check if rstate is initialized before uncompressing 2018-04-11 10:33:46 -04:00
smc.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
snmp.h
sock_reuseport.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
sock.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next 2018-06-06 18:39:49 -07:00
Space.h net/mac89x0: Convert to platform_driver 2018-03-01 21:21:36 -05:00
stp.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
strparser.h strparser: Add __strp_unpause and use it in ktls. 2018-06-06 14:07:53 -04:00
switchdev.h switchdev: Add fdb.added_by_user to switchdev notifications 2018-05-03 13:46:47 -04:00
tcp_states.h tcp: remove the hardcode in the definition of TCPF Macro 2018-02-21 15:06:05 -05:00
tcp.h Revert changes to convert to ->poll_mask() and aio IOCB_CMD_POLL 2018-06-28 10:40:47 -07:00
timewait_sock.h
tipc.h flow_dissector: do not rely on implicit casts 2018-05-08 00:02:41 -04:00
tls.h Revert changes to convert to ->poll_mask() and aio IOCB_CMD_POLL 2018-06-28 10:40:47 -07:00
transp_v6.h udp: fix rx queue len reported by diag and proc interface 2018-06-08 19:55:15 -04:00
tso.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
tun_proto.h vxlan: factor out VXLAN-GPE next protocol 2017-08-29 15:16:52 -07:00
udp_tunnel.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
udp.h Revert changes to convert to ->poll_mask() and aio IOCB_CMD_POLL 2018-06-28 10:40:47 -07:00
udplite.h udplite: fix partial checksum initialization 2018-02-16 15:57:42 -05:00
vsock_addr.h
vxlan.h vxlan: add ttl inherit support 2018-04-17 13:53:13 -04:00
wext.h lift handling of SIOCIW... out of dev_ioctl() 2018-01-24 19:13:45 -05:00
wimax.h
x25.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
x25device.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
xdp_sock.h xsk: fix potential race in SKB TX completion code 2018-07-02 18:37:12 -07:00
xdp.h xdp: add MEM_TYPE_ZERO_COPY 2018-06-05 15:46:26 +02:00
xfrm.h audit/stable-4.18 PR 20180605 2018-06-06 16:34:00 -07:00