linux/net/tipc
Jon Paul Maloy c4282ca76c tipc: correct error in node fsm
commit 88e8ac7000 ("tipc: reduce transmission rate of reset messages
when link is down") revealed a flaw in the node FSM, as defined in
the log of commit 66996b6c47 ("tipc: extend node FSM").

We see the following scenario:
1: Node B receives a RESET message from node A before its link endpoint
   is fully up, i.e., the node FSM is in state SELF_UP_PEER_COMING. This
   event will not change the node FSM state, but the (distinct) link FSM
   will move to state RESETTING.
2: As an effect of the previous event, the local endpoint on B will
   declare node A lost, and post the event SELF_DOWN to the its node
   FSM. This moves the FSM state to SELF_DOWN_PEER_LEAVING, meaning
   that no messages will be accepted from A until it receives another
   RESET message that confirms that A's endpoint has been reset. This
   is  wasteful, since we know this as a fact already from the first
   received RESET, but worse is that the link instance's FSM has not
   wasted this information, but instead moved on to state ESTABLISHING,
   meaning that it repeatedly sends out ACTIVATE messages to the reset
   peer A.
3: Node A will receive one of the ACTIVATE messages, move its link FSM
   to state ESTABLISHED, and start repeatedly sending out STATE messages
   to node B.
4: Node B will consistently drop these messages, since it can only accept
   accept a RESET according to its node FSM.
5: After four lost STATE messages node A will reset its link and start
   repeatedly sending out RESET messages to B.
6: Because of the reduced send rate for RESET messages, it is very
   likely that A will receive an ACTIVATE (which is sent out at a much
   higher frequency) before it gets the chance to send a RESET, and A
   may hence quickly move back to state ESTABLISHED and continue sending
   out STATE messages, which will again be dropped by B.
7: GOTO 5.
8: After having repeated the cycle 5-7 a number of times, node A will
   by chance get in between with sending a RESET, and the situation is
   resolved.

Unfortunately, we have seen that it may take a substantial amount of
time before this vicious loop is broken, sometimes in the order of
minutes.

We correct this by making a small correction to the node FSM: When a
node in state SELF_UP_PEER_COMING receives a SELF_DOWN event, it now
moves directly back to state SELF_DOWN_PEER_DOWN, instead of as now
SELF_DOWN_PEER_LEAVING. This is logically consistent, since we don't
need to wait for RESET confirmation from of an endpoint that we alread
know has been reset. It also means that node B in the scenario above
will not be dropping incoming STATE messages, and the link can come up
immediately.

Finally, a symmetry comparison reveals that the  FSM has a similar
error when receiving the event PEER_DOWN in state PEER_UP_SELF_COMING.
Instead of moving to PERR_DOWN_SELF_LEAVING, it should move directly
to SELF_DOWN_PEER_DOWN. Although we have never seen any negative effect
of this logical error, we choose fix this one, too.

The node FSM looks as follows after those changes:

                           +----------------------------------------+
                           |                           PEER_DOWN_EVT|
                           |                                        |
  +------------------------+----------------+                       |
  |SELF_DOWN_EVT           |                |                       |
  |                        |                |                       |
  |              +-----------+          +-----------+               |
  |              |NODE_      |          |NODE_      |               |
  |   +----------|FAILINGOVER|<---------|SYNCHING   |-----------+   |
  |   |SELF_     +-----------+ FAILOVER_+-----------+   PEER_   |   |
  |   |DOWN_EVT   |          A BEGIN_EVT  A         |   DOWN_EVT|   |
  |   |           |          |            |         |           |   |
  |   |           |          |            |         |           |   |
  |   |           |FAILOVER_ |FAILOVER_   |SYNCH_   |SYNCH_     |   |
  |   |           |END_EVT   |BEGIN_EVT   |BEGIN_EVT|END_EVT    |   |
  |   |           |          |            |         |           |   |
  |   |           |          |            |         |           |   |
  |   |           |         +--------------+        |           |   |
  |   |           +-------->|   SELF_UP_   |<-------+           |   |
  |   |   +-----------------|   PEER_UP    |----------------+   |   |
  |   |   |SELF_DOWN_EVT    +--------------+   PEER_DOWN_EVT|   |   |
  |   |   |                    A        A                   |   |   |
  |   |   |                    |        |                   |   |   |
  |   |   |         PEER_UP_EVT|        |SELF_UP_EVT        |   |   |
  |   |   |                    |        |                   |   |   |
  V   V   V                    |        |                   V   V   V
+------------+       +-----------+    +-----------+       +------------+
|SELF_DOWN_  |       |SELF_UP_   |    |PEER_UP_   |       |PEER_DOWN   |
|PEER_LEAVING|       |PEER_COMING|    |SELF_COMING|       |SELF_LEAVING|
+------------+       +-----------+    +-----------+       +------------+
       |               |       A        A       |                |
       |               |       |        |       |                |
       |       SELF_   |       |SELF_   |PEER_  |PEER_           |
       |       DOWN_EVT|       |UP_EVT  |UP_EVT |DOWN_EVT        |
       |               |       |        |       |                |
       |               |       |        |       |                |
       |               |    +--------------+    |                |
       |PEER_DOWN_EVT  +--->|  SELF_DOWN_  |<---+   SELF_DOWN_EVT|
       +------------------->|  PEER_DOWN   |<--------------------+
                            +--------------+

Acked-by: Ying Xue <ying.xue@windriver.com>
Signed-off-by: Jon Maloy <jon.maloy@ericsson.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-06-08 11:27:01 -07:00
..
addr.c tipc: simplify include dependencies 2015-05-14 12:24:45 -04:00
addr.h tipc: simplify include dependencies 2015-05-14 12:24:45 -04:00
bcast.c tipc: remove pre-allocated message header in link struct 2016-03-06 23:01:20 -05:00
bcast.h tipc: remove pre-allocated message header in link struct 2016-03-06 23:01:20 -05:00
bearer.c tipc: stricter filtering of packets in bearer layer 2016-04-07 17:00:13 -04:00
bearer.h tipc: remove remnants of old broadcast code 2016-04-13 17:49:11 -04:00
core.c tipc: redesign connection-level flow control 2016-05-03 15:51:16 -04:00
core.h tipc: make dist queue pernet 2016-04-11 15:22:20 -04:00
discover.c tipc: eliminate buffer leak in bearer layer 2016-04-07 17:00:13 -04:00
discover.h tipc: eliminate buffer leak in bearer layer 2016-04-07 17:00:13 -04:00
eth_media.c tipc: make media address offset a common define 2015-02-27 18:18:48 -05:00
ib_media.c tipc: rename media/msg related definitions 2015-02-27 18:18:48 -05:00
Kconfig tipc: add ip/udp media type 2015-03-05 22:08:42 -05:00
link.c tipc: fix stale links after re-enabling bearer 2016-04-24 14:35:07 -04:00
link.h tipc: let first message on link be a state message 2016-04-15 16:09:06 -04:00
Makefile tipc: add ip/udp media type 2015-03-05 22:08:42 -05:00
msg.c tipc: let broadcast packet reception use new link receive function 2015-10-24 06:56:37 -07:00
msg.h tipc: redesign connection-level flow control 2016-05-03 15:51:16 -04:00
name_distr.c tipc: purge deferred updates from dead nodes 2016-04-11 15:22:20 -04:00
name_distr.h tipc: reduce code dependency between binding table and node layer 2015-11-20 14:06:10 -05:00
name_table.c tipc: move netlink policies to netlink.c 2016-03-07 14:56:41 -05:00
name_table.h tipc: convert legacy nl name table dump to nl compat 2015-02-09 13:20:48 -08:00
net.c tipc: move netlink policies to netlink.c 2016-03-07 14:56:41 -05:00
net.h tipc: make tipc node table aware of net namespace 2015-01-12 16:24:32 -05:00
netlink_compat.c tipc: fix potential null pointer dereferences in some compat functions 2016-05-25 12:33:52 -07:00
netlink.c tipc: move netlink policies to netlink.c 2016-03-07 14:56:41 -05:00
netlink.h tipc: move netlink policies to netlink.c 2016-03-07 14:56:41 -05:00
node.c tipc: correct error in node fsm 2016-06-08 11:27:01 -07:00
node.h tipc: redesign connection-level flow control 2016-05-03 15:51:16 -04:00
server.c tipc: block BH in TCP callbacks 2016-05-19 11:36:49 -07:00
server.h tipc: fix a race condition leading to subscriber refcnt bug 2016-04-14 16:46:46 -04:00
socket.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial 2016-05-17 17:05:30 -07:00
socket.h tipc: redesign connection-level flow control 2016-05-03 15:51:16 -04:00
subscr.c tipc: remove an unnecessary NULL check 2016-04-28 16:54:12 -04:00
subscr.h tipc: remove struct tipc_name_seq from struct tipc_subscription 2016-02-06 03:40:43 -05:00
sysctl.c tipc: add name distributor resiliency queue 2014-09-01 17:51:48 -07:00
udp_media.c tipc: make sure IPv6 header fits in skb headroom 2016-03-14 12:23:12 -04:00