linux/net/netfilter
David S. Miller bae97d8410 Merge git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf-next
Pablo Neira Ayuso says:

====================
Netfilter updates for net-next

A final pull request, I know it's very late but this time I think it's worth a
bit of rush.

The following patchset contains Netfilter/nf_tables updates for net-next, more
specifically concatenation support and dynamic stateful expression
instantiation.

This also comes with a couple of small patches. One to fix the ebtables.h
userspace header and another to get rid of an obsolete example file in tree
that describes a nf_tables expression.

This time, I decided to paste the original descriptions. This will result in a
rather large commit description, but I think these bytes to keep.

Patrick McHardy says:

====================
netfilter: nf_tables: concatenation support

The following patches add support for concatenations, which allow multi
dimensional exact matches in O(1).

The basic idea is to split the data registers, currently consisting of
4 registers of 16 bytes each, into smaller units, 16 registers of 4
bytes each, and making sure each register store always leaves the
full 32 bit in a well defined state, meaning smaller stores will
zero the remaining bits.

Based on that, we can load multiple adjacent registers with different
values, thereby building a concatenated bigger value, and use that
value for set lookups.

Sets are changed to use variable sized extensions for their key and
data values, removing the fixed limit of 16 bytes while saving memory
if less space is needed.

As a side effect, these patches will allow some nice optimizations in
the future, like using jhash2 in nft_hash, removing the masking in
nft_cmp_fast, optimized data comparison using 32 bit word size etc.
These are not done so far however.

The patches are split up as follows:

 * the first five patches add length validation to register loads and
   stores to make sure we stay within bounds and prepare the validation
   functions for the new addressing mode

 * the next patches prepare for changing to 32 bit addressing by
   introducing a struct nft_regs, which holds the verdict register as
   well as the data registers. The verdict members are moved to a new
   struct nft_verdict to allow to pull struct nft_data out of the stack.

 * the next patches contain preparatory conversions of expressions and
   sets to use 32 bit addressing

 * the next patch introduces so far unused register conversion helpers
   for parsing and dumping register numbers over netlink

 * following is the real conversion to 32 bit addressing, consisting of
   replacing struct nft_data in struct nft_regs by an array of u32s and
   actually translating and validating the new register numbers.

 * the final two patches add support for variable sized data items and
   variable sized keys / data in set elements

The patches have been verified to work correctly with nft binaries using
both old and new addressing.
====================

Patrick McHardy says:

====================
netfilter: nf_tables: dynamic stateful expression instantiation

The following patches are the grand finale of my nf_tables set work,
using all the building blocks put in place by the previous patches
to support something like iptables hashlimit, but a lot more powerful.

Sets are extended to allow attaching expressions to set elements.
The dynset expression dynamically instantiates these expressions
based on a template when creating new set elements and evaluates
them for all new or updated set members.

In combination with concatenations this effectively creates state
tables for arbitrary combinations of keys, using the existing
expression types to maintain that state. Regular set GC takes care
of purging expired states.

We currently support two different stateful expressions, counter
and limit. Using limit as a template we can express the functionality
of hashlimit, but completely unrestricted in the combination of keys.
Using counter we can perform accounting for arbitrary flows.

The following examples from patch 5/5 show some possibilities.
Userspace syntax is still WIP, especially the listing of state
tables will most likely be seperated from normal set listings
and use a more structured format:

1. Limit the rate of new SSH connections per host, similar to iptables
   hashlimit:

        flow ip saddr timeout 60s \
        limit 10/second \
        accept

2. Account network traffic between each set of /24 networks:

        flow ip saddr & 255.255.255.0 . ip daddr & 255.255.255.0 \
        counter

3. Account traffic to each host per user:

        flow skuid . ip daddr \
        counter

4. Account traffic for each combination of source address and TCP flags:

        flow ip saddr . tcp flags \
        counter

The resulting set content after a Xmas-scan look like this:

{
        192.168.122.1 . fin | psh | urg : counter packets 1001 bytes 40040,
        192.168.122.1 . ack : counter packets 74 bytes 3848,
        192.168.122.1 . psh | ack : counter packets 35 bytes 3144
}

In the future the "expressions attached to elements" will be extended
to also support user created non-stateful expressions to allow to
efficiently select beween a set of parameter sets, f.i. a set of log
statements with different prefixes based on the interface, which currently
require one rule each. This will most likely have to wait until the next
kernel version though.
====================

Signed-off-by: David S. Miller <davem@davemloft.net>
2015-04-14 18:51:19 -04:00
..
ipset netfilter: bridge: add helpers for fetching physin/outdev 2015-04-08 16:49:08 +02:00
ipvs netfilter: Pass socket pointer down through okfn(). 2015-04-07 15:25:55 -04:00
core.c netfilter: Make nf_hookfn use nf_hook_state. 2015-04-04 12:31:38 -04:00
Kconfig netfilter: nf_tables: nft_queue does not depend on x_tables 2015-03-25 12:09:39 +01:00
Makefile netfilter: nf_tables: add support for dynamic set updates 2015-04-08 16:58:27 +02:00
nf_conntrack_acct.c netfilter: Remove uses of seq_<foo> return values 2015-03-18 10:51:35 +01:00
nf_conntrack_amanda.c net: Remove state argument from skb_find_text() 2015-02-22 15:59:54 -05:00
nf_conntrack_broadcast.c
nf_conntrack_core.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf-next 2015-01-15 01:50:25 -05:00
nf_conntrack_ecache.c netfilter: conntrack: remove timer from ecache extension 2014-06-25 19:15:38 +02:00
nf_conntrack_expect.c netfilter: Remove uses of seq_<foo> return values 2015-03-18 10:51:35 +01:00
nf_conntrack_extend.c
nf_conntrack_ftp.c netfilter: replace strnicmp with strncasecmp 2014-10-14 02:18:24 +02:00
nf_conntrack_h323_asn1.c
nf_conntrack_h323_main.c netfilter: nf_conntrack_h323: lookup route from proper net namespace 2014-11-17 12:47:14 +01:00
nf_conntrack_h323_types.c
nf_conntrack_helper.c netfilter: fix spelling errors 2014-10-30 17:35:30 +01:00
nf_conntrack_irc.c netfilter: add my copyright statements 2013-04-18 20:27:55 +02:00
nf_conntrack_l3proto_generic.c netfilter: Convert print_tuple functions to return void 2014-11-05 14:10:33 -05:00
nf_conntrack_labels.c netfilter: connlabels: remove unneeded includes 2013-07-31 16:39:18 +02:00
nf_conntrack_netbios_ns.c
nf_conntrack_netlink.c netfilter: conntrack: Flush connections with a given mark 2015-01-08 12:14:20 +01:00
nf_conntrack_pptp.c netfilter: nf_conntrack: flush net_gre->keymap_list only from gre helper 2014-04-08 10:56:12 +02:00
nf_conntrack_proto_dccp.c netfilter: Convert print_tuple functions to return void 2014-11-05 14:10:33 -05:00
nf_conntrack_proto_generic.c netfilter: Convert print_tuple functions to return void 2014-11-05 14:10:33 -05:00
nf_conntrack_proto_gre.c netfilter: Convert print_tuple functions to return void 2014-11-05 14:10:33 -05:00
nf_conntrack_proto_sctp.c netfilter: Convert print_tuple functions to return void 2014-11-05 14:10:33 -05:00
nf_conntrack_proto_tcp.c Merge branch 'iov_iter' into for-next 2014-12-08 20:39:29 -05:00
nf_conntrack_proto_udp.c netfilter: Convert print_tuple functions to return void 2014-11-05 14:10:33 -05:00
nf_conntrack_proto_udplite.c netfilter: Convert print_tuple functions to return void 2014-11-05 14:10:33 -05:00
nf_conntrack_proto.c netfilter: nf_conntrack: remove dead code 2014-01-03 23:41:37 +01:00
nf_conntrack_sane.c netfilter: nf_ct_helper: better logging for dropped packets 2013-02-19 02:48:05 +01:00
nf_conntrack_seqadj.c netfilter: nf_ct_seqadj: print ack seq in the right host byte order 2015-01-05 13:52:20 +01:00
nf_conntrack_sip.c netfilter: replace strnicmp with strncasecmp 2014-10-14 02:18:24 +02:00
nf_conntrack_snmp.c netfilter: nf_ct_snmp: add include file 2013-01-18 00:28:18 +01:00
nf_conntrack_standalone.c netfilter: Remove checks of seq_printf() return values 2014-11-05 14:11:02 -05:00
nf_conntrack_tftp.c netfilter: add my copyright statements 2013-04-18 20:27:55 +02:00
nf_conntrack_timeout.c netfilter: nf_ct_timeout: move initialization out of pernet_operations 2013-01-23 12:56:02 +01:00
nf_conntrack_timestamp.c netfilter: nf_ct_timestamp: Fix BUG_ON after netns deletion 2013-12-20 14:58:29 +01:00
nf_internals.h netfilter: Create and use nf_hook_state. 2015-04-04 12:17:40 -04:00
nf_log_common.c netfilter: bridge: add helpers for fetching physin/outdev 2015-04-08 16:49:08 +02:00
nf_log.c netfilter: restore rule tracing via nfnetlink_log 2015-03-19 11:14:48 +01:00
nf_nat_amanda.c netfilter: add my copyright statements 2013-04-18 20:27:55 +02:00
nf_nat_core.c net: use reciprocal_scale() helper 2014-08-23 12:21:21 -07:00
nf_nat_ftp.c netfilter: nf_ct_helper: better logging for dropped packets 2013-02-19 02:48:05 +01:00
nf_nat_helper.c netfilter: nf_conntrack: make sequence number adjustments usuable without NAT 2013-08-28 00:26:48 +02:00
nf_nat_irc.c netfilter: nf_nat: fix access to uninitialized buffer in IRC NAT helper 2014-01-06 14:17:17 +01:00
nf_nat_proto_common.c netfilter: use IS_ENABLED() macro 2014-06-30 11:38:03 +02:00
nf_nat_proto_dccp.c netfilter: use IS_ENABLED() macro 2014-06-30 11:38:03 +02:00
nf_nat_proto_sctp.c netfilter: use IS_ENABLED() macro 2014-06-30 11:38:03 +02:00
nf_nat_proto_tcp.c netfilter: use IS_ENABLED() macro 2014-06-30 11:38:03 +02:00
nf_nat_proto_udp.c netfilter: use IS_ENABLED() macro 2014-06-30 11:38:03 +02:00
nf_nat_proto_udplite.c netfilter: use IS_ENABLED() macro 2014-06-30 11:38:03 +02:00
nf_nat_proto_unknown.c
nf_nat_redirect.c netfilter: combine IPv4 and IPv6 nf_nat_redirect code in one module 2014-11-27 13:08:42 +01:00
nf_nat_sip.c netfilter: replace strnicmp with strncasecmp 2014-10-14 02:18:24 +02:00
nf_nat_tftp.c netfilter: nf_ct_helper: better logging for dropped packets 2013-02-19 02:48:05 +01:00
nf_queue.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next 2015-04-08 18:30:21 +02:00
nf_sockopt.c netfilter: don't use mutex_lock_interruptible() 2014-08-08 16:47:23 +02:00
nf_synproxy_core.c netfilter: nf_conntrack: don't release a conntrack with non-zero refcnt 2014-02-05 17:46:06 +01:00
nf_tables_api.c netfilter: nf_tables: add flag to indicate set contains expressions 2015-04-13 20:12:32 +02:00
nf_tables_core.c netfilter: nf_tables: switch registers to 32 bit addressing 2015-04-13 17:17:29 +02:00
nf_tables_inet.c netfilter: nf_tables: fix error path in the init functions 2014-01-09 23:25:48 +01:00
nfnetlink_acct.c netfilter: nfnetlink_acct: add filter support to nfacct counter list/reset 2014-08-26 21:36:19 +02:00
nfnetlink_cthelper.c netfilter: Zero the tuple in nfnl_cthelper_parse_tuple() 2015-03-12 13:07:36 +01:00
nfnetlink_cttimeout.c netfilter: cttimeout: allow to set/get default protocol timeouts 2013-10-01 13:17:39 +02:00
nfnetlink_log.c netfilter: Fix format string of nfnetlink_log proc file 2015-04-13 16:35:17 -04:00
nfnetlink_queue_core.c netfilter: Fix format string of nfnetlink_queue proc file 2015-04-13 16:35:16 -04:00
nfnetlink_queue_ct.c netfilter: nf_conntrack: make sequence number adjustments usuable without NAT 2013-08-28 00:26:48 +02:00
nfnetlink.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf-next 2015-01-15 01:50:25 -05:00
nft_bitwise.c netfilter: nf_tables: support variable sized data in nft_data_init() 2015-04-13 17:17:30 +02:00
nft_byteorder.c netfilter: nf_tables: switch registers to 32 bit addressing 2015-04-13 17:17:29 +02:00
nft_cmp.c netfilter: nf_tables: support variable sized data in nft_data_init() 2015-04-13 17:17:30 +02:00
nft_compat.c netfilter: nf_tables: get rid of NFT_REG_VERDICT usage 2015-04-13 17:17:07 +02:00
nft_counter.c netfilter: nf_tables: mark stateful expressions 2015-04-13 20:12:31 +02:00
nft_ct.c netfilter: nf_tables: switch registers to 32 bit addressing 2015-04-13 17:17:29 +02:00
nft_dynset.c netfilter: nft_dynset: dynamic stateful expression instantiation 2015-04-13 20:19:55 +02:00
nft_exthdr.c netfilter: nf_tables: switch registers to 32 bit addressing 2015-04-13 17:17:29 +02:00
nft_hash.c netfilter: nf_tables: variable sized set element keys / data 2015-04-13 17:17:31 +02:00
nft_immediate.c netfilter: nf_tables: support variable sized data in nft_data_init() 2015-04-13 17:17:30 +02:00
nft_limit.c netfilter: nf_tables: mark stateful expressions 2015-04-13 20:12:31 +02:00
nft_log.c netfilter: nf_tables: get rid of NFT_REG_VERDICT usage 2015-04-13 17:17:07 +02:00
nft_lookup.c netfilter: nf_tables: add flag to indicate set contains expressions 2015-04-13 20:12:32 +02:00
nft_masq.c netfilter: nf_tables: validate hooks in NAT expressions 2015-01-19 14:52:39 +01:00
nft_meta.c netfilter: nf_tables: switch registers to 32 bit addressing 2015-04-13 17:17:29 +02:00
nft_nat.c netfilter: nf_tables: switch registers to 32 bit addressing 2015-04-13 17:17:29 +02:00
nft_payload.c netfilter: nf_tables: switch registers to 32 bit addressing 2015-04-13 17:17:29 +02:00
nft_queue.c netfilter: nf_tables: get rid of NFT_REG_VERDICT usage 2015-04-13 17:17:07 +02:00
nft_rbtree.c netfilter: nf_tables: variable sized set element keys / data 2015-04-13 17:17:31 +02:00
nft_redir.c netfilter: nf_tables: add register parsing/dumping helpers 2015-04-13 17:17:28 +02:00
nft_reject_inet.c netfilter: nf_tables: get rid of NFT_REG_VERDICT usage 2015-04-13 17:17:07 +02:00
nft_reject.c netfilter: fix wrong arithmetics regarding NFT_REJECT_ICMPX_MAX 2014-10-07 20:16:31 +02:00
x_tables.c netfilter: Remove checks of seq_printf() return values 2014-11-05 14:11:02 -05:00
xt_addrtype.c netfilter: xt_addrtype: fix trivial typo 2013-07-31 16:36:25 +02:00
xt_AUDIT.c netfilter: Convert uses of __constant_<foo> to <foo> 2014-03-13 14:13:19 +01:00
xt_bpf.c net: filter: split 'struct sk_filter' into socket and bpf parts 2014-08-02 15:03:58 -07:00
xt_cgroup.c netfilter: x_tables: fix cgroup matching on non-full sks 2015-04-01 11:26:42 +02:00
xt_CHECKSUM.c
xt_CLASSIFY.c
xt_cluster.c net: use reciprocal_scale() helper 2014-08-23 12:21:21 -07:00
xt_comment.c
xt_connbytes.c netfilter: Convert pr_warning to pr_warn 2014-09-10 12:40:10 -07:00
xt_connlabel.c netfilter: add connlabel conntrack extension 2013-01-18 00:28:15 +01:00
xt_connlimit.c netfilter: xt_connlimit: honor conntrack zone if available 2014-11-17 12:44:20 +01:00
xt_connmark.c netfilter: Fix FSF address in file headers 2013-12-06 12:37:57 -05:00
xt_CONNSECMARK.c
xt_conntrack.c netfilter: add my copyright statements 2013-04-18 20:27:55 +02:00
xt_cpu.c
xt_CT.c netfilter: nf_conntrack: don't release a conntrack with non-zero refcnt 2014-02-05 17:46:06 +01:00
xt_dccp.c
xt_devgroup.c
xt_dscp.c
xt_DSCP.c netfilter: fix various sparse warnings 2014-11-13 12:14:42 +01:00
xt_ecn.c
xt_esp.c
xt_hashlimit.c netfilter: Remove checks of seq_printf() return values 2014-11-05 14:11:02 -05:00
xt_helper.c
xt_hl.c
xt_HL.c
xt_HMARK.c net: use reciprocal_scale() helper 2014-08-23 12:21:21 -07:00
xt_IDLETIMER.c
xt_ipcomp.c netfilter: xt_ipcomp: Use ntohs to ease sparse warning 2014-02-19 11:41:25 +01:00
xt_iprange.c
xt_ipvs.c
xt_l2tp.c netfilter: introduce l2tp match extension 2014-01-09 21:36:39 +01:00
xt_LED.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2014-08-05 18:46:26 -07:00
xt_length.c
xt_limit.c netfilter: add my copyright statements 2013-04-18 20:27:55 +02:00
xt_LOG.c netfilter: xt_LOG: add missing string format in nf_log_packet() 2014-06-28 18:50:35 +02:00
xt_mac.c
xt_mark.c
xt_multiport.c
xt_nat.c
xt_NETMAP.c
xt_nfacct.c netfilter: nfnetlink_acct: Adding quota support to accounting framework 2014-04-29 18:25:14 +02:00
xt_NFLOG.c netfilter: log: netns NULL ptr bug when calling from conntrack 2013-05-15 14:11:07 +02:00
xt_NFQUEUE.c netfilter: xt_NFQUEUE: separate reusable code 2013-12-07 23:20:45 +01:00
xt_osf.c netfilter: xt_osf: Use continue to reduce indentation 2014-12-23 14:20:10 +01:00
xt_owner.c
xt_physdev.c netfilter: physdev: use helpers 2015-04-08 16:49:09 +02:00
xt_pkttype.c
xt_policy.c
xt_quota.c
xt_rateest.c net_sched: add 64bit rate estimators 2013-06-11 02:51:03 -07:00
xt_RATEEST.c net: sched: make bstats per cpu and estimator RCU safe 2014-09-30 01:02:26 -04:00
xt_realm.c
xt_recent.c netfilter: xt_recent: don't reject rule if new hitcount exceeds table max 2015-02-16 17:00:47 +01:00
xt_REDIRECT.c netfilter: combine IPv4 and IPv6 nf_nat_redirect code in one module 2014-11-27 13:08:42 +01:00
xt_repldata.h net: netfilter: LLVMLinux: vlais-netfilter 2014-06-07 11:44:39 -07:00
xt_sctp.c
xt_SECMARK.c
xt_set.c netfilter: ipset: fix boolreturn.cocci warnings 2015-02-11 16:13:30 +01:00
xt_socket.c netfilter: x_tables: don't extract flow keys on early demuxed sks in socket match 2015-04-08 16:47:49 +02:00
xt_state.c
xt_statistic.c net: replace macros net_random and net_srandom with direct calls to prandom 2014-01-14 15:15:25 -08:00
xt_string.c net: Remove state argument from skb_find_text() 2015-02-22 15:59:54 -05:00
xt_tcpmss.c
xt_TCPMSS.c netfilter: xt_TCPMSS: lookup route from proper net namespace 2013-09-27 16:18:23 +02:00
xt_TCPOPTSTRIP.c netfilter: xt_TCPOPTSTRIP: fix possible off by one access 2013-08-01 11:45:15 +02:00
xt_tcpudp.c
xt_TEE.c net: pass info struct via netdevice notifier 2013-05-28 13:11:01 -07:00
xt_time.c
xt_TPROXY.c tcp/dccp: get rid of central timewait timer 2015-04-13 16:40:05 -04:00
xt_TRACE.c
xt_u32.c