42 Commits

Author SHA1 Message Date
Jakub Kicinski
f2ba1e5e22 tools: ynl-gen: stop generating common notification handlers
Common notification handler was supposed to be a way for the user
to parse the notifications from a socket synchronously.
I don't think we'll end up using it, ynl_ntf_check() works for
all known use cases.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
e4ea3cc684 tools: ynl-gen: get attr type outside of if()
Reading attr type with mnl_attr_get_type() for each condition
leads to most conditions being longer than 80 chars.
Avoid this by reading the type to a variable on the stack.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
2c0f146686 tools: ynl-gen: combine else with closing bracket
Code gen currently prints:

  }
  else if (...

This is really ugly. Fix it by delaying printing of closing
brackets in anticipation of else coming along.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
820343ccbb tools: ynl-gen: complete the C keyword list
C keywords need to be avoided when naming things.
Complete the list (ethtool has at least one thing called "auto").

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
30b5c720e1 tools: ynl-gen: cleanup user space header includes
Bots started screaming that we're including stdlib.h twice.
While at it move string.h into a common spot and drop stdio.h
which we don't need.

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=5464
Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=5466
Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=5467
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
168dea20ec tools: ynl-gen: don't generate forward declarations for policies
Now that all nested types have structs and are sorted topologically
there should be no need to generate forward declarations for policies.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
eae7af21bd tools: ynl-gen: walk nested types in depth
So far we had only created structures for nested types nested
directly in messages (second level of attrs so to speak).
Walk types in depth to support deeper nesting.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
37487f93b1 tools: ynl-gen: inherit struct use info
We only render parse and netlink generation helpers as needed,
to avoid generating dead code. Propagate the information from
first- and second-layer attribute sets onto all children.
Otherwise devlink won't work, it has a lot more levels of nesting.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
6afaa0ef9b tools: ynl-gen: try to sort the types more intelligently
We need to sort the structures to avoid the need for forward
declarations. While at it remove the sort of structs when
rendering, it doesn't do anything.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
ff6db4b58c tools: ynl-gen: enable code gen for directional specs
I think that user space code gen for directional specs
works after recent changes. Let them through.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
6f115d4575 tools: ynl-gen: refactor strmap helper generation
Move generating strmap lookup function to a helper.
No functional changes.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
9858bfc271 tools: ynl-gen: use enum names in op strmap more carefully
In preparation for supporting families which use different msg
ids to and from the kernel - make sure the ids in op strmap
are correct. The map is expected to be used mostly for notifications,
don't generate a separate map for the "to kernel" direction.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
58da455b31 tools: ynl-gen: improve unwind on parsing errors
When parsing multi-attr we count the objects and then allocate
an array to hold the parsed objects. If an attr space has multiple
multi-attr objects, however, if parsing the first array fails
we'll leave the object count for the second even tho the second
array was never allocated.

This may cause crashes when freeing objects on error.

Count attributes to a variable on the stack and only set the count
in the object once the memory was allocated.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-07 12:53:10 -07:00
Jakub Kicinski
2cc9671a82 tools: ynl-gen: fill in support for MultiAttr scalars
The handshake family needs support for MultiAttr scalars.
Right now we only support code gen for MultiAttr nested
types.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-07 12:53:10 -07:00
Jakub Kicinski
a99bfdf647 tools: ynl-gen: clean up stray new lines at the end of reply-less requests
Do not print empty lines before closing brackets.

Reviewed-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-06 12:31:31 -07:00
Jakub Kicinski
59d814f0f2 tools: ynl-gen: generate static descriptions of notifications
Notifications may come in at any time. The family must be always
ready to parse a random incoming notification. Generate notification
table for parsing and tell YNL which request we're processing
to distinguish responses from notifications.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
8cb6afb335 tools: ynl-gen: switch to family struct
We'll want to store static info about the family soon.
Generate a struct. This changes creation from, e.g.:

	 ys = ynl_sock_create("netdev", &yerr);
to:
	 ys = ynl_sock_create(&ynl_netdev_family, &yerr);

on user's side.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
5d58f911c7 tools: ynl-gen: generate alloc and free helpers for req
We expect user to allocate requests with calloc(),
make things a bit more consistent and provide helpers.
Generate free calls, too.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
dc0956c98f tools: ynl-gen: move the response reading logic into YNL
We generate send() and recv() calls and all msg handling for
each operation. It's a lot of repeated code and will only grow
with notification handling. Call back to a helper YNL lib instead.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
21b6e30278 tools: ynl-gen: generate enum-to-string helpers
It's sometimes useful to print the name of an enum value,
flag or name of the op. Python can do it, add C helper
code gen for getting names of things.

Example:

  static const char * const netdev_xdp_act_strmap[] = {
	[0] = "basic",
	[1] = "redirect",
	[2] = "ndo-xmit",
	[3] = "xsk-zerocopy",
	[4] = "hw-offload",
	[5] = "rx-sg",
	[6] = "ndo-xmit-sg",
  };

  const char *netdev_xdp_act_str(enum netdev_xdp_act value)
  {
	value = ffs(value) - 1;
	if (value < 0 || value >= (int)MNL_ARRAY_SIZE(netdev_xdp_act_strmap))
		return NULL;
	return netdev_xdp_act_strmap[value];
  }

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
eef9b794ea tools: ynl-gen: add error checking for nested structs
Parsing nested types may return an error, propagate it.
Not marking as a fix, because nothing uses YNL upstream.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
5605f10237 tools: ynl-gen: loosen type consistency check for events
Both event and notify types are always consistent. Rewrite
the condition checking if we can reuse reply types to be
less picky and let notify thru.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
67c65ce762 tools: ynl-gen: don't override pure nested struct
For pure structs (parsed nested attributes) we track what
forms of the struct exist in request and reply directions.
Make sure we don't overwrite the recorded struct each time,
otherwise the information is lost.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:46 -07:00
Jakub Kicinski
6ad49839ba tools: ynl-gen: fix unused / pad attribute handling
Unused and Pad attributes don't carry information.
Unused should never exist, and be rejected.
Pad should be silently skipped.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:46 -07:00
Jakub Kicinski
91dfaef243 tools: ynl-gen: add extra headers for user space
Make sure all relevant headers are included, we allocate memory,
use memcpy() and Linux types without including the headers.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:46 -07:00
Jakub Kicinski
9b66ee06e5 net: ynl: prefix uAPI header include with uapi/
To keep things simple we used to include the uAPI header
in the kernel in the #include <linux/$family.h> format.
This works well enough, most of the genl families should
have headers in include/net/ so linux/$family.h ends up
referring to the uAPI header, anyway. And if it doesn't
no big deal, we'll just include more info than we need.

Unless that is there is a naming conflict. Someone recently
created include/linux/psp.h which will be a problem when
supporting the PSP protocol. (I'm talking about
work-in-progress patches, but it's just a proof that assuming
lack of name conflicts was overly optimistic.)

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2023-05-26 10:30:14 +01:00
Jakub Kicinski
4c6170d1ae tools: ynl: default to treating enums as flags for mask generation
I was a bit too optimistic in commit bf51d27704c9 ("tools: ynl: fix
get_mask utility routine"), not every mask we use is necessarily
coming from an enum of type "flags". We also allow flipping an
enum into flags on per-attribute basis. That's done by
the 'enum-as-flags' property of an attribute.

Restore this functionality, it's not currently used by any in-tree
family.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2023-03-27 08:56:04 +01:00
Jakub Kicinski
56c874f7db tools: ynl: skip the explicit op array size when not needed
Jiri suggests it reads more naturally to skip the explicit
array size when possible. When we export the symbol we want
to make sure that the size is right but for statics its
not needed.

Link: https://lore.kernel.org/r/20230321044159.1031040-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-03-21 21:45:31 -07:00
Jakub Kicinski
cfab77c0b5 ynl: make the tooling check the license
The (only recently documented) expectation is that all specs
are under a certain license, but we don't actually enforce it.
What's worse we then go ahead and assume the license was right,
outputting the expected license into generated files.

Fixes: 37d9df224d1e ("ynl: re-license uniformly under GPL-2.0 OR BSD-3-Clause")
Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-03-16 21:22:44 -07:00
Jakub Kicinski
4e16b6a748 ynl: broaden the license even more
I relicensed Netlink spec code to GPL-2.0 OR BSD-3-Clause but
we still put a slightly different license on the uAPI header
than the rest of the code. Use the Linux-syscall-note on all
the specs and all generated code. It's moot for kernel code,
but should not hurt. This way the licenses match everywhere.

Cc: Chuck Lever <chuck.lever@oracle.com>
Fixes: 37d9df224d1e ("ynl: re-license uniformly under GPL-2.0 OR BSD-3-Clause")
Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-03-16 21:20:32 -07:00
Lorenzo Bianconi
8f76a4f80f tools: ynl: fix render-max for flags definition
Properly manage render-max property for flags definition type
introducing mask value and setting it to (last_element << 1) - 1
instead of adding max value set to last_element + 1

Fixes: be5bea1cc0bf ("net: add basic C code generators for Netlink")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-03-10 21:33:47 -08:00
Jakub Kicinski
6517a60b03 tools: ynl: move the enum classes to shared code
Move bulk of the EnumSet and EnumEntry code to shared
code for reuse by cli.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-03-08 23:28:21 -08:00
Jakub Kicinski
37d9df224d ynl: re-license uniformly under GPL-2.0 OR BSD-3-Clause
I was intending to make all the Netlink Spec code BSD-3-Clause
to ease the adoption but it appears that:
 - I fumbled the uAPI and used "GPL WITH uAPI note" there
 - it gives people pause as they expect GPL in the kernel
As suggested by Chuck re-license under dual. This gives us benefit
of full BSD freedom while fulfilling the broad "kernel is under GPL"
expectations.

Link: https://lore.kernel.org/all/20230304120108.05dd44c5@kernel.org/
Link: https://lore.kernel.org/r/20230306200457.3903854-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-03-07 13:44:30 -08:00
Jakub Kicinski
ad4fafcde5 tools: ynl: use 1 as the default for first entry in attrs/ops
Pretty much all families use value: 1 or reserve as unspec
the first entry in attribute set and the first operation.
Make this the default. Update documentation (the doc for
values of operations just refers back to doc for attrs
so updating only attrs).

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2023-03-03 08:22:39 +00:00
Jakub Kicinski
f7cf644796 tools: ynl-gen: fix single attribute structs with attr 0 only
Chuck run into an issue with a single-element attr-set which
only has an attr with value of 0. The search for max attr in
a struct records attrs with value larger than 0 only (max_val
is set to 0 at the start). Adjust the comparison, alternatively
max_val could be init'ed to -1. Somehow picking the last attr
of a value seems like a good idea in general.

Reported-by: Chuck Lever III <chuck.lever@oracle.com>
Fixes: be5bea1cc0bf ("net: add basic C code generators for Netlink")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-02-24 11:55:47 -08:00
Jakub Kicinski
981cbcb030 tools: net: use python3 explicitly
The scripts require Python 3 and some distros are dropping
Python 2 support.

Reported-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-01-31 20:36:03 -08:00
Jakub Kicinski
30a5c6c810 tools: ynl: use the common YAML loading and validation code
Adapt the common object hierarchy in code gen and CLI.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-01-31 20:36:03 -08:00
Jakub Kicinski
eaf317e7d2 tools: ynl-gen: prevent do / dump reordering
An earlier fix tried to address generated code jumping around
one code-gen run to another. Turns out dict()s are already
ordered since Python 3.7, the problem is that we iterate over
operation modes using a set(). Sets are unordered in Python.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-01-31 20:36:03 -08:00
Jakub Kicinski
3a43ded081 tools: ynl: store ops in ordered dict to avoid random ordering
When rendering code we should walk the ops in the order in which
they are declared in the spec. This is both more intuitive and
prevents code from jumping around when hashing in the dict changes.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-01-26 16:32:41 -08:00
Jakub Kicinski
b49c34e217 tools: ynl: rename ops_list -> msg_list
ops_list contains all the operations, but the main iteration use
case is to walk only ops which define attrs. Rename ops_list to
msg_list, because now it looks like the contents are the same,
just the format is different. While at it convert from tuple
to just keys, none of the users care about the name of the op.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-01-26 16:32:41 -08:00
Jakub Kicinski
66fa34b9c2 tools: ynl: support kdocs for flags in code generation
Lorenzo reports that after switching from enum to flags netdev
family lost ability to render kdoc (and the enum contents got
generally garbled).

Combine the flags and enum handling in uAPI handling.

Reported-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-01-26 16:32:41 -08:00
Jakub Kicinski
be5bea1cc0 net: add basic C code generators for Netlink
Code generators to turn Netlink specs into C code.
I'm definitely not proud of it.

The main generator is in Python, there's a bash script
to regen all code-gen'ed files in tree after making
spec changes.

Acked-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
2023-01-24 10:58:11 +01:00