mirror of
https://github.com/systemd/systemd.git
synced 2024-10-30 14:55:37 +03:00
ed1de7108e
This makes it easier for people than just recommending the syscall.
482 lines
28 KiB
Markdown
482 lines
28 KiB
Markdown
---
|
|
title: Control Group APIs and Delegation
|
|
category: Interfaces
|
|
layout: default
|
|
---
|
|
|
|
# Control Group APIs and Delegation
|
|
|
|
*Intended audience: hackers working on userspace subsystems that require direct
|
|
cgroup access, such as container managers and similar.*
|
|
|
|
So you are wondering about resource management with systemd, you know Linux
|
|
control groups (cgroups) a bit and are trying to integrate your software with
|
|
what systemd has to offer there. Here's a bit of documentation about the
|
|
concepts and interfaces involved with this.
|
|
|
|
What's described here has been part of systemd and documented since v205
|
|
times. However, it has been updated and improved substantially, even
|
|
though the concepts stayed mostly the same. This is an attempt to provide more
|
|
comprehensive up-to-date information about all this, particular in light of the
|
|
poor implementations of the components interfacing with systemd of current
|
|
container managers.
|
|
|
|
Before you read on, please make sure you read the low-level kernel
|
|
documentation about the
|
|
[unified cgroup hierarchy](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html).
|
|
This document then adds in the higher-level view from systemd.
|
|
|
|
This document augments the existing documentation we already have:
|
|
|
|
* [The New Control Group Interfaces](https://www.freedesktop.org/wiki/Software/systemd/ControlGroupInterface/)
|
|
* [Writing VM and Container Managers](https://www.freedesktop.org/wiki/Software/systemd/writing-vm-managers/)
|
|
|
|
These wiki documents are not as up to date as they should be, currently, but
|
|
the basic concepts still fully apply. You should read them too, if you do something
|
|
with cgroups and systemd, in particular as they shine more light on the various
|
|
D-Bus APIs provided. (That said, sooner or later we should probably fold that
|
|
wiki documentation into this very document, too.)
|
|
|
|
## Two Key Design Rules
|
|
|
|
Much of the philosophy behind these concepts is based on a couple of basic
|
|
design ideas of cgroup v2 (which we however try to adapt as far as we can to
|
|
cgroup v1 too). Specifically two cgroup v2 rules are the most relevant:
|
|
|
|
1. The **no-processes-in-inner-nodes** rule: this means that it's not permitted
|
|
to have processes directly attached to a cgroup that also has child cgroups and
|
|
vice versa. A cgroup is either an inner node or a leaf node of the tree, and if
|
|
it's an inner node it may not contain processes directly, and if it's a leaf
|
|
node then it may not have child cgroups. (Note that there are some minor
|
|
exceptions to this rule, though. E.g. the root cgroup is special and allows
|
|
both processes and children — which is used in particular to maintain kernel
|
|
threads.)
|
|
|
|
2. The **single-writer** rule: this means that each cgroup only has a single
|
|
writer, i.e. a single process managing it. It's OK if different cgroups have
|
|
different processes managing them. However, only a single process should own a
|
|
specific cgroup, and when it does that ownership is exclusive, and nothing else
|
|
should manipulate it at the same time. This rule ensures that various pieces of
|
|
software don't step on each other's toes constantly.
|
|
|
|
These two rules have various effects. For example, one corollary of this is: if
|
|
your container manager creates and manages cgroups in the system's root cgroup
|
|
you violate rule #2, as the root cgroup is managed by systemd and hence off
|
|
limits to everybody else.
|
|
|
|
Note that rule #1 is generally enforced by the kernel if cgroup v2 is used: as
|
|
soon as you add a process to a cgroup it is ensured the rule is not
|
|
violated. On cgroup v1 this rule didn't exist, and hence isn't enforced, even
|
|
though it's a good thing to follow it then too. Rule #2 is not enforced on
|
|
either cgroup v1 nor cgroup v2 (this is UNIX after all, in the general case
|
|
root can do anything, modulo SELinux and friends), but if you ignore it you'll
|
|
be in constant pain as various pieces of software will fight over cgroup
|
|
ownership.
|
|
|
|
Note that cgroup v1 is currently the most deployed implementation, even though
|
|
it's semantically broken in many ways, and in many cases doesn't actually do
|
|
what people think it does. cgroup v2 is where things are going, and most new
|
|
kernel features in this area are only added to cgroup v2, and not cgroup v1
|
|
anymore. For example cgroup v2 provides proper cgroup-empty notifications, has
|
|
support for all kinds of per-cgroup BPF magic, supports secure delegation of
|
|
cgroup trees to less privileged processes and so on, which all are not
|
|
available on cgroup v1.
|
|
|
|
## Three Different Tree Setups 🌳
|
|
|
|
systemd supports three different modes how cgroups are set up. Specifically:
|
|
|
|
1. **Unified** — this is the simplest mode, and exposes a pure cgroup v2
|
|
logic. In this mode `/sys/fs/cgroup` is the only mounted cgroup API file system
|
|
and all available controllers are exclusively exposed through it.
|
|
|
|
2. **Legacy** — this is the traditional cgroup v1 mode. In this mode the
|
|
various controllers each get their own cgroup file system mounted to
|
|
`/sys/fs/cgroup/<controller>/`. On top of that systemd manages its own cgroup
|
|
hierarchy for managing purposes as `/sys/fs/cgroup/systemd/`.
|
|
|
|
3. **Hybrid** — this is a hybrid between the unified and legacy mode. It's set
|
|
up mostly like legacy, except that there's also an additional hierarchy
|
|
`/sys/fs/cgroup/unified/` that contains the cgroup v2 hierarchy. (Note that in
|
|
this mode the unified hierarchy won't have controllers attached, the
|
|
controllers are all mounted as separate hierarchies as in legacy mode,
|
|
i.e. `/sys/fs/cgroup/unified/` is purely and exclusively about core cgroup v2
|
|
functionality and not about resource management.) In this mode compatibility
|
|
with cgroup v1 is retained while some cgroup v2 features are available
|
|
too. This mode is a stopgap. Don't bother with this too much unless you have
|
|
too much free time.
|
|
|
|
To say this clearly, legacy and hybrid modes have no future. If you develop
|
|
software today and don't focus on the unified mode, then you are writing
|
|
software for yesterday, not tomorrow. They are primarily supported for
|
|
compatibility reasons and will not receive new features. Sorry.
|
|
|
|
Superficially, in legacy and hybrid modes it might appear that the parallel
|
|
cgroup hierarchies for each controller are orthogonal from each other. In
|
|
systemd they are not: the hierarchies of all controllers are always kept in
|
|
sync (at least mostly: sub-trees might be suppressed in certain hierarchies if
|
|
no controller usage is required for them). The fact that systemd keeps these
|
|
hierarchies in sync means that the legacy and hybrid hierarchies are
|
|
conceptually very close to the unified hierarchy. In particular this allows us
|
|
to talk of one specific cgroup and actually mean the same cgroup in all
|
|
available controller hierarchies. E.g. if we talk about the cgroup `/foo/bar/`
|
|
then we actually mean `/sys/fs/cgroup/cpu/foo/bar/` as well as
|
|
`/sys/fs/cgroup/memory/foo/bar/`, `/sys/fs/cgroup/pids/foo/bar/`, and so on.
|
|
Note that in cgroup v2 the controller hierarchies aren't orthogonal, hence
|
|
thinking about them as orthogonal won't help you in the long run anyway.
|
|
|
|
If you wonder how to detect which of these three modes is currently used, use
|
|
`statfs()` on `/sys/fs/cgroup/`. If it reports `CGROUP2_SUPER_MAGIC` in its
|
|
`.f_type` field, then you are in unified mode. If it reports `TMPFS_MAGIC` then
|
|
you are either in legacy or hybrid mode. To distinguish these two cases, run
|
|
`statfs()` again on `/sys/fs/cgroup/unified/`. If that succeeds and reports
|
|
`CGROUP2_SUPER_MAGIC` you are in hybrid mode, otherwise not.
|
|
From a shell, you can use check the `Type` in `stat -f /sys/fs/cgroup` and
|
|
`stat -f /sys/fs/cgroup/unified`.
|
|
|
|
## systemd's Unit Types
|
|
|
|
The low-level kernel cgroups feature is exposed in systemd in three different
|
|
"unit" types. Specifically:
|
|
|
|
1. 💼 The `.service` unit type. This unit type is for units encapsulating
|
|
processes systemd itself starts. Units of these types have cgroups that are
|
|
the leaves of the cgroup tree the systemd instance manages (though possibly
|
|
they might contain a sub-tree of their own managed by something else, made
|
|
possible by the concept of delegation, see below). Service units are usually
|
|
instantiated based on a unit file on disk that describes the command line to
|
|
invoke and other properties of the service. However, service units may also
|
|
be declared and started programmatically at runtime through a D-Bus API
|
|
(which is called *transient* services).
|
|
|
|
2. 👓 The `.scope` unit type. This is very similar to `.service`. The main
|
|
difference: the processes the units of this type encapsulate are forked off
|
|
by some unrelated manager process, and that manager asked systemd to expose
|
|
them as a unit. Unlike services, scopes can only be declared and started
|
|
programmatically, i.e. are always transient. That's because they encapsulate
|
|
processes forked off by something else, i.e. existing runtime objects, and
|
|
hence cannot really be defined fully in 'offline' concepts such as unit
|
|
files.
|
|
|
|
3. 🔪 The `.slice` unit type. Units of this type do not directly contain any
|
|
processes. Units of this type are the inner nodes of part of the cgroup tree
|
|
the systemd instance manages. Much like services, slices can be defined
|
|
either on disk with unit files or programmatically as transient units.
|
|
|
|
Slices expose the trunk and branches of a tree, and scopes and services are
|
|
attached to those branches as leaves. The idea is that scopes and services can
|
|
be moved around though, i.e. assigned to a different slice if needed.
|
|
|
|
The naming of slice units directly maps to the cgroup tree path. This is not
|
|
the case for service and scope units however. A slice named `foo-bar-baz.slice`
|
|
maps to a cgroup `/foo.slice/foo-bar.slice/foo-bar-baz.slice/`. A service
|
|
`quux.service` which is attached to the slice `foo-bar-baz.slice` maps to the
|
|
cgroup `/foo.slice/foo-bar.slice/foo-bar-baz.slice/quux.service/`.
|
|
|
|
By default systemd sets up four slice units:
|
|
|
|
1. `-.slice` is the root slice. i.e. the parent of everything else. On the host
|
|
system it maps directly to the top-level directory of cgroup v2.
|
|
|
|
2. `system.slice` is where system services are by default placed, unless
|
|
configured otherwise.
|
|
|
|
3. `user.slice` is where user sessions are placed. Each user gets a slice of
|
|
its own below that.
|
|
|
|
4. `machines.slice` is where VMs and containers are supposed to be
|
|
placed. `systemd-nspawn` makes use of this by default, and you're very welcome
|
|
to place your containers and VMs there too if you hack on managers for those.
|
|
|
|
Users may define any amount of additional slices they like though, the four
|
|
above are just the defaults.
|
|
|
|
## Delegation
|
|
|
|
Container managers and suchlike often want to control cgroups directly using
|
|
the raw kernel APIs. That's entirely fine and supported, as long as proper
|
|
*delegation* is followed. Delegation is a concept we inherited from cgroup v2,
|
|
but we expose it on cgroup v1 too. Delegation means that some parts of the
|
|
cgroup tree may be managed by different managers than others. As long as it is
|
|
clear which manager manages which part of the tree each one can do within its
|
|
sub-graph of the tree whatever it wants.
|
|
|
|
Only sub-trees can be delegated (though whoever decides to request a sub-tree
|
|
can delegate sub-sub-trees further to somebody else if they like). Delegation
|
|
takes place at a specific cgroup: in systemd there's a `Delegate=` property you
|
|
can set for a service or scope unit. If you do, it's the cut-off point for
|
|
systemd's cgroup management: the unit itself is managed by systemd, i.e. all
|
|
its attributes are managed exclusively by systemd, however your program may
|
|
create/remove sub-cgroups inside it freely, and those then become exclusive
|
|
property of your program, systemd won't touch them — all attributes of *those*
|
|
sub-cgroups can be manipulated freely and exclusively by your program.
|
|
|
|
By turning on the `Delegate=` property for a scope or service you get a few
|
|
guarantees:
|
|
|
|
1. systemd won't fiddle with your sub-tree of the cgroup tree anymore. It won't
|
|
change attributes of any cgroups below it, nor will it create or remove any
|
|
cgroups thereunder, nor migrate processes across the boundaries of that
|
|
sub-tree as it deems useful anymore.
|
|
|
|
2. If your service makes use of the `User=` functionality, then the sub-tree
|
|
will be `chown()`ed to the indicated user so that it can correctly create
|
|
cgroups below it. Note however that systemd will do that only in the unified
|
|
hierarchy (in unified and hybrid mode) as well as on systemd's own private
|
|
hierarchy (in legacy and hybrid mode). It won't pass ownership of the legacy
|
|
controller hierarchies. Delegation to less privileges processes is not safe
|
|
in cgroup v1 (as a limitation of the kernel), hence systemd won't facilitate
|
|
access to it.
|
|
|
|
3. Any BPF IP filter programs systemd installs will be installed with
|
|
`BPF_F_ALLOW_MULTI` so that your program can install additional ones.
|
|
|
|
In unit files the `Delegate=` property is superficially exposed as
|
|
boolean. However, since v236 it optionally takes a list of controller names
|
|
instead. If so, delegation is requested for listed controllers
|
|
specifically. Note that this only encodes a request. Depending on various
|
|
parameters it might happen that your service actually will get fewer
|
|
controllers delegated (for example, because the controller is not available on
|
|
the current kernel or was turned off) or more. If no list is specified
|
|
(i.e. the property simply set to `yes`) then all available controllers are
|
|
delegated.
|
|
|
|
Let's stress one thing: delegation is available on scope and service units
|
|
only. It's expressly not available on slice units. Why? Because slice units are
|
|
our *inner* nodes of the cgroup trees and we freely attach service and scopes
|
|
to them. If we'd allow delegation on slice units then this would mean that
|
|
both systemd and your own manager would create/delete cgroups below the slice
|
|
unit and that conflicts with the single-writer rule.
|
|
|
|
So, if you want to do your own raw cgroups kernel level access, then allocate a
|
|
scope unit, or a service unit (or just use the service unit you already have
|
|
for your service code), and turn on delegation for it.
|
|
|
|
(OK, here's one caveat: if you turn on delegation for a service, and that
|
|
service has `ExecStartPost=`, `ExecReload=`, `ExecStop=` or `ExecStopPost=`
|
|
set, then these commands will be executed within the `.control/` sub-cgroup of
|
|
your service's cgroup. This is necessary because by turning on delegation we
|
|
have to assume that the cgroup delegated to your service is now an *inner*
|
|
cgroup, which means that it may not directly contain any processes. Hence, if
|
|
your service has any of these four settings set, you must be prepared that a
|
|
`.control/` subcgroup might appear, managed by the service manager. This also
|
|
means that your service code should have moved itself further down the cgroup
|
|
tree by the time it notifies the service manager about start-up readiness, so
|
|
that the service's main cgroup is definitely an inner node by the time the
|
|
service manager might start `ExecStartPost=`.)
|
|
|
|
## Three Scenarios
|
|
|
|
Let's say you write a container manager, and you wonder what to do regarding
|
|
cgroups for it, as you want your manager to be able to run on systemd systems.
|
|
|
|
You basically have three options:
|
|
|
|
1. 😊 The *integration-is-good* option. For this, you register each container
|
|
you have either as a systemd service (i.e. let systemd invoke the executor
|
|
binary for you) or a systemd scope (i.e. your manager executes the binary
|
|
directly, but then tells systemd about it. In this mode the administrator
|
|
can use the usual systemd resource management and reporting commands
|
|
individually on those containers. By turning on `Delegate=` for these scopes
|
|
or services you make it possible to run cgroup-enabled programs in your
|
|
containers, for example a nested systemd instance. This option has two
|
|
sub-options:
|
|
|
|
a. You transiently register the service or scope by directly contacting
|
|
systemd via D-Bus. In this case systemd will just manage the unit for you
|
|
and nothing else.
|
|
|
|
b. Instead you register the service or scope through `systemd-machined`
|
|
(also via D-Bus). This mini-daemon is basically just a proxy for the same
|
|
operations as in a. The main benefit of this: this way you let the system
|
|
know that what you are registering is a container, and this opens up
|
|
certain additional integration points. For example, `journalctl -M` can
|
|
then be used to directly look into any container's journal logs (should
|
|
the container run systemd inside), or `systemctl -M` can be used to
|
|
directly invoke systemd operations inside the containers. Moreover tools
|
|
like "ps" can then show you to which container a process belongs (`ps -eo
|
|
pid,comm,machine`), and even gnome-system-monitor supports it.
|
|
|
|
2. 🙁 The *i-like-islands* option. If all you care about is your own cgroup tree,
|
|
and you want to have to do as little as possible with systemd and no
|
|
interest in integration with the rest of the system, then this is a valid
|
|
option. For this all you have to do is turn on `Delegate=` for your main
|
|
manager daemon. Then figure out the cgroup systemd placed your daemon in:
|
|
you can now freely create sub-cgroups beneath it. Don't forget the
|
|
*no-processes-in-inner-nodes* rule however: you have to move your main
|
|
daemon process out of that cgroup (and into a sub-cgroup) before you can
|
|
start further processes in any of your sub-cgroups.
|
|
|
|
3. 🙁 The *i-like-continents* option. In this option you'd leave your manager
|
|
daemon where it is, and would not turn on delegation on its unit. However,
|
|
as first thing you register a new scope unit with systemd, and that scope
|
|
unit would have `Delegate=` turned on, and then you place all your
|
|
containers underneath it. From systemd's PoV there'd be two units: your
|
|
manager service and the big scope that contains all your containers in one.
|
|
|
|
BTW: if for whatever reason you say "I hate D-Bus, I'll never call any D-Bus
|
|
API, kthxbye", then options #1 and #3 are not available, as they generally
|
|
involve talking to systemd from your program code, via D-Bus. You still have
|
|
option #2 in that case however, as you can simply set `Delegate=` in your
|
|
service's unit file and you are done and have your own sub-tree. In fact, #2 is
|
|
the one option that allows you to completely ignore systemd's existence: you
|
|
can entirely generically follow the single rule that you just use the cgroup
|
|
you are started in, and everything below it, whatever that might be. That said,
|
|
maybe if you dislike D-Bus and systemd that much, the better approach might be
|
|
to work on that, and widen your horizon a bit. You are welcome.
|
|
|
|
## Controller Support
|
|
|
|
systemd supports a number of controllers (but not all). Specifically, supported
|
|
are:
|
|
|
|
* on cgroup v1: `cpu`, `cpuacct`, `blkio`, `memory`, `devices`, `pids`
|
|
* on cgroup v2: `cpu`, `io`, `memory`, `pids`
|
|
|
|
It is our intention to natively support all cgroup v2 controllers as they are
|
|
added to the kernel. However, regarding cgroup v1: at this point we will not
|
|
add support for any other controllers anymore. This means systemd currently
|
|
does not and will never manage the following controllers on cgroup v1:
|
|
`freezer`, `cpuset`, `net_cls`, `perf_event`, `net_prio`, `hugetlb`. Why not?
|
|
Depending on the case, either their API semantics or implementations aren't
|
|
really usable, or it's very clear they have no future on cgroup v2, and we
|
|
won't add new code for stuff that clearly has no future.
|
|
|
|
Effectively this means that all those mentioned cgroup v1 controllers are up
|
|
for grabs: systemd won't manage them, and hence won't delegate them to your
|
|
code (however, systemd will still mount their hierarchies, simply because it
|
|
mounts all controller hierarchies it finds available in the kernel). If you
|
|
decide to use them, then that's fine, but systemd won't help you with it (but
|
|
also not interfere with it). To be nice to other tenants it might be wise to
|
|
replicate the cgroup hierarchies of the other controllers in them too however,
|
|
but of course that's between you and those other tenants, and systemd won't
|
|
care. Replicating the cgroup hierarchies in those unsupported controllers would
|
|
mean replicating the full cgroup paths in them, and hence the prefixing
|
|
`.slice` components too, otherwise the hierarchies will start being orthogonal
|
|
after all, and that's not really desirable. On more thing: systemd will clean
|
|
up after you in the hierarchies it manages: if your daemon goes down, its
|
|
cgroups will be removed too. You basically get the guarantee that you start
|
|
with a pristine cgroup sub-tree for your service or scope whenever it is
|
|
started. This is not the case however in the hierarchies systemd doesn't
|
|
manage. This means that your programs should be ready to deal with left-over
|
|
cgroups in them — from previous runs, and be extra careful with them as they
|
|
might still carry settings that might not be valid anymore.
|
|
|
|
Note a particular asymmetry here: if your systemd version doesn't support a
|
|
specific controller on cgroup v1 you can still make use of it for delegation,
|
|
by directly fiddling with its hierarchy and replicating the cgroup tree there
|
|
as necessary (as suggested above). However, on cgroup v2 this is different:
|
|
separately mounted hierarchies are not available, and delegation has always to
|
|
happen through systemd itself. This means: when you update your kernel and it
|
|
adds a new, so far unseen controller, and you want to use it for delegation,
|
|
then you also need to update systemd to a version that groks it.
|
|
|
|
## systemd as Container Payload
|
|
|
|
systemd can happily run as a container payload's PID 1. Note that systemd
|
|
unconditionally needs write access to the cgroup tree however, hence you need
|
|
to delegate a sub-tree to it. Note that there's nothing too special you have to
|
|
do beyond that: just invoke systemd as PID 1 inside the root of the delegated
|
|
cgroup sub-tree, and it will figure out the rest: it will determine the cgroup
|
|
it is running in and take possession of it. It won't interfere with any cgroup
|
|
outside of the sub-tree it was invoked in. Use of `CLONE_NEWCGROUP` is hence
|
|
optional (but of course wise).
|
|
|
|
Note one particular asymmetry here though: systemd will try to take possession
|
|
of the root cgroup you pass to it *in* *full*, i.e. it will not only
|
|
create/remove child cgroups below it, it will also attempt to manage the
|
|
attributes of it. OTOH as mentioned above, when delegating a cgroup tree to
|
|
somebody else it only passes the rights to create/remove sub-cgroups, but will
|
|
insist on managing the delegated cgroup tree's top-level attributes. Or in
|
|
other words: systemd is *greedy* when accepting delegated cgroup trees and also
|
|
*greedy* when delegating them to others: it insists on managing attributes on
|
|
the specific cgroup in both cases. A container manager that is itself a payload
|
|
of a host systemd which wants to run a systemd as its own container payload
|
|
instead hence needs to insert an extra level in the hierarchy in between, so
|
|
that the systemd on the host and the one in the container won't fight for the
|
|
attributes. That said, you likely should do that anyway, due to the
|
|
no-processes-in-inner-cgroups rule, see below.
|
|
|
|
When systemd runs as container payload it will make use of all hierarchies it
|
|
has write access to. For legacy mode you need to make at least
|
|
`/sys/fs/cgroup/systemd/` available, all other hierarchies are optional. For
|
|
hybrid mode you need to add `/sys/fs/cgroup/unified/`. Finally, for fully
|
|
unified you (of course, I guess) need to provide only `/sys/fs/cgroup/` itself.
|
|
|
|
## Some Dos
|
|
|
|
1. ⚡ If you go for implementation option 1a or 1b (as in the list above), then
|
|
each of your containers will have its own systemd-managed unit and hence
|
|
cgroup with possibly further sub-cgroups below. Typically the first process
|
|
running in that unit will be some kind of executor program, which will in
|
|
turn fork off the payload processes of the container. In this case don't
|
|
forget that there are two levels of delegation involved: first, systemd
|
|
delegates a group sub-tree to your executor. And then your executor should
|
|
delegate a sub-tree further down to the container payload. Oh, and because
|
|
of the no-process-in-inner-nodes rule, your executor needs to migrate itself
|
|
to a sub-cgroup of the cgroup it got delegated, too. Most likely you hence
|
|
want a two-pronged approach: below the cgroup you got started in, you want
|
|
one cgroup maybe called `supervisor/` where your manager runs in and then
|
|
for each container a sibling cgroup of that maybe called `payload-xyz/`.
|
|
|
|
2. ⚡ Don't forget that the cgroups you create have to have names that are
|
|
suitable as UNIX file names, and that they live in the same namespace as the
|
|
various kernel attribute files. Hence, when you want to allow the user
|
|
arbitrary naming, you might need to escape some of the names (for example,
|
|
you really don't want to create a cgroup named `tasks`, just because the
|
|
user created a container by that name, because `tasks` after all is a magic
|
|
attribute in cgroup v1, and your `mkdir()` will hence fail with `EEXIST`. In
|
|
systemd we do escaping by prefixing names that might collide with a kernel
|
|
attribute name with an underscore. You might want to do the same, but this
|
|
is really up to you how you do it. Just do it, and be careful.
|
|
|
|
## Some Don'ts
|
|
|
|
1. 🚫 Never create your own cgroups below arbitrary cgroups systemd manages, i.e
|
|
cgroups you haven't set `Delegate=` in. Specifically: 🔥 don't create your
|
|
own cgroups below the root cgroup 🔥. That's owned by systemd, and you will
|
|
step on systemd's toes if you ignore that, and systemd will step on
|
|
yours. Get your own delegated sub-tree, you may create as many cgroups there
|
|
as you like. Seriously, if you create cgroups directly in the cgroup root,
|
|
then all you do is ask for trouble.
|
|
|
|
2. 🚫 Don't attempt to set `Delegate=` in slice units, and in particular not in
|
|
`-.slice`. It's not supported, and will generate an error.
|
|
|
|
3. 🚫 Never *write* to any of the attributes of a cgroup systemd created for
|
|
you. It's systemd's private property. You are welcome to manipulate the
|
|
attributes of cgroups you created in your own delegated sub-tree, but the
|
|
cgroup tree of systemd itself is out of limits for you. It's fine to *read*
|
|
from any attribute you like however. That's totally OK and welcome.
|
|
|
|
4. 🚫 When not using `CLONE_NEWCGROUP` when delegating a sub-tree to a
|
|
container payload running systemd, then don't get the idea that you can bind
|
|
mount only a sub-tree of the host's cgroup tree into the container. Part of
|
|
the cgroup API is that `/proc/$PID/cgroup` reports the cgroup path of every
|
|
process, and hence any path below `/sys/fs/cgroup/` needs to match what
|
|
`/proc/$PID/cgroup` of the payload processes reports. What you can do safely
|
|
however, is mount the upper parts of the cgroup tree read-only (or even
|
|
replace the middle bits with an intermediary `tmpfs` — but be careful not to
|
|
break the `statfs()` detection logic discussed above), as long as the path
|
|
to the delegated sub-tree remains accessible as-is.
|
|
|
|
5. ⚡ Currently, the algorithm for mapping between slice/scope/service unit
|
|
naming and their cgroup paths is not considered public API of systemd, and
|
|
may change in future versions. This means: it's best to avoid implementing a
|
|
local logic of translating cgroup paths to slice/scope/service names in your
|
|
program, or vice versa — it's likely going to break sooner or later. Use the
|
|
appropriate D-Bus API calls for that instead, so that systemd translates
|
|
this for you. (Specifically: each Unit object has a `ControlGroup` property
|
|
to get the cgroup for a unit. The method `GetUnitByControlGroup()` may be
|
|
used to get the unit for a cgroup.)
|
|
|
|
6. ⚡ Think twice before delegating cgroup v1 controllers to less privileged
|
|
containers. It's not safe, you basically allow your containers to freeze the
|
|
system with that and worse. Delegation is a strongpoint of cgroup v2 though,
|
|
and there it's safe to treat delegation boundaries as privilege boundaries.
|
|
|
|
And that's it for now. If you have further questions, refer to the systemd
|
|
mailing list.
|
|
|
|
— Berlin, 2018-04-20
|