linux/Documentation/ABI
Jarkko Sakkinen 50468e4313 x86/sgx: Add an attribute for the amount of SGX memory in a NUMA node
== Problem ==

The amount of SGX memory on a system is determined by the BIOS and it
varies wildly between systems.  It can be as small as dozens of MB's
and as large as many GB's on servers.  Just like how applications need
to know how much regular RAM is available, enclave builders need to
know how much SGX memory an enclave can consume.

== Solution ==

Introduce a new sysfs file:

	/sys/devices/system/node/nodeX/x86/sgx_total_bytes

to enumerate the amount of SGX memory available in each NUMA node.
This serves the same function for SGX as /proc/meminfo or
/sys/devices/system/node/nodeX/meminfo does for normal RAM.

'sgx_total_bytes' is needed today to help drive the SGX selftests.
SGX-specific swap code is exercised by creating overcommitted enclaves
which are larger than the physical SGX memory on the system.  They
currently use a CPUID-based approach which can diverge from the actual
amount of SGX memory available.  'sgx_total_bytes' ensures that the
selftests can work efficiently and do not attempt stupid things like
creating a 100,000 MB enclave on a system with 128 MB of SGX memory.

== Implementation Details ==

Introduce CONFIG_HAVE_ARCH_NODE_DEV_GROUP opt-in flag to expose an
arch specific attribute group, and add an attribute for the amount of
SGX memory in bytes to each NUMA node:

== ABI Design Discussion ==

As opposed to the per-node ABI, a single, global ABI was considered.
However, this would prevent enclaves from being able to size
themselves so that they fit on a single NUMA node.  Essentially, a
single value would rule out NUMA optimizations for enclaves.

Create a new "x86/" directory inside each "nodeX/" sysfs directory.
'sgx_total_bytes' is expected to be the first of at least a few
sgx-specific files to be placed in the new directory.  Just scanning
/proc/meminfo, these are the no-brainers that we have for RAM, but we
need for SGX:

	MemTotal:       xxxx kB // sgx_total_bytes (implemented here)
	MemFree:        yyyy kB // sgx_free_bytes
	SwapTotal:      zzzz kB // sgx_swapped_bytes

So, at *least* three.  I think we will eventually end up needing
something more along the lines of a dozen.  A new directory (as
opposed to being in the nodeX/ "root") directory avoids cluttering the
root with several "sgx_*" files.

Place the new file in a new "nodeX/x86/" directory because SGX is
highly x86-specific.  It is very unlikely that any other architecture
(or even non-Intel x86 vendor) will ever implement SGX.  Using "sgx/"
as opposed to "x86/" was also considered.  But, there is a real chance
this can get used for other arch-specific purposes.

[ dhansen: rewrite changelog ]

Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Borislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20211116162116.93081-2-jarkko@kernel.org
2021-12-09 07:02:22 -08:00
..
obsolete ABI: obsolete/sysfs-bus-iio: add some missing blank lines 2021-09-28 12:48:15 +02:00
removed This was a reasonably active cycle for documentation; this pull includes: 2021-06-28 16:53:05 -07:00
stable x86/sgx: Add an attribute for the amount of SGX memory in a NUMA node 2021-12-09 07:02:22 -08:00
testing f2fs-for-5.16-rc1 2021-11-13 11:20:22 -08:00
README docs: ABI: README: specify that files should be ReST compatible 2020-10-30 13:07:01 +01:00

This directory attempts to document the ABI between the Linux kernel and
userspace, and the relative stability of these interfaces.  Due to the
everchanging nature of Linux, and the differing maturity levels, these
interfaces should be used by userspace programs in different ways.

We have four different levels of ABI stability, as shown by the four
different subdirectories in this location.  Interfaces may change levels
of stability according to the rules described below.

The different levels of stability are:

  stable/
	This directory documents the interfaces that the developer has
	defined to be stable.  Userspace programs are free to use these
	interfaces with no restrictions, and backward compatibility for
	them will be guaranteed for at least 2 years.  Most interfaces
	(like syscalls) are expected to never change and always be
	available.

  testing/
	This directory documents interfaces that are felt to be stable,
	as the main development of this interface has been completed.
	The interface can be changed to add new features, but the
	current interface will not break by doing this, unless grave
	errors or security problems are found in them.  Userspace
	programs can start to rely on these interfaces, but they must be
	aware of changes that can occur before these interfaces move to
	be marked stable.  Programs that use these interfaces are
	strongly encouraged to add their name to the description of
	these interfaces, so that the kernel developers can easily
	notify them if any changes occur (see the description of the
	layout of the files below for details on how to do this.)

  obsolete/
	This directory documents interfaces that are still remaining in
	the kernel, but are marked to be removed at some later point in
	time.  The description of the interface will document the reason
	why it is obsolete and when it can be expected to be removed.

  removed/
	This directory contains a list of the old interfaces that have
	been removed from the kernel.

Every file in these directories will contain the following information:

What:		Short description of the interface
Date:		Date created
KernelVersion:	Kernel version this feature first showed up in.
Contact:	Primary contact for this interface (may be a mailing list)
Description:	Long description of the interface and how to use it.
Users:		All users of this interface who wish to be notified when
		it changes.  This is very important for interfaces in
		the "testing" stage, so that kernel developers can work
		with userspace developers to ensure that things do not
		break in ways that are unacceptable.  It is also
		important to get feedback for these interfaces to make
		sure they are working in a proper way and do not need to
		be changed further.


Note:
   The fields should be use a simple notation, compatible with ReST markup.
   Also, the file **should not** have a top-level index, like::

	===
	foo
	===

How things move between levels:

Interfaces in stable may move to obsolete, as long as the proper
notification is given.

Interfaces may be removed from obsolete and the kernel as long as the
documented amount of time has gone by.

Interfaces in the testing state can move to the stable state when the
developers feel they are finished.  They cannot be removed from the
kernel tree without going through the obsolete state first.

It's up to the developer to place their interfaces in the category they
wish for it to start out in.


Notable bits of non-ABI, which should not under any circumstances be considered
stable:

- Kconfig.  Userspace should not rely on the presence or absence of any
  particular Kconfig symbol, in /proc/config.gz, in the copy of .config
  commonly installed to /boot, or in any invocation of the kernel build
  process.

- Kernel-internal symbols.  Do not rely on the presence, absence, location, or
  type of any kernel symbol, either in System.map files or the kernel binary
  itself.  See Documentation/process/stable-api-nonsense.rst.