IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
This will warn if fake flexible arrays are re-introduced. I'm not using
-Werror=… because we may still get warnings when compiling against old kernel
headers. We can crank this up to error later.
-fstrict-flex-arrays means that the compiler doesn't have to assume that any
trailing array is a flex array. I.e. unless the array is declared without a
specified size, only indices in the declared range are valid.
-Warray-bounds turns on the warnings about out-of-bounds array accesses.
-Warray-bounds=2 does some more warnings, with higher false positive rate. But
it doesn't seem to yield any false positives in our codebase, so enable it.
clang supports -Warray-bounds, but not -Warray-bounds=2.
gcc supports both.
gcc-13 supports -fstrict-flex-arrays.
See https://people.kernel.org/kees/bounded-flexible-arrays-in-c for a long
discussion of use in the kernel.
Config options are -Ddefault-timeout-sec= and -Ddefault-user-timeout-sec=.
Existing -Dupdate-helper-user-timeout= is renamed to -Dupdate-helper-user-timeout-sec=
for consistency. All three options take an integer value in seconds. The
renaming and type-change of the option is a small compat break, but it's just
at compile time and result in a clear error message. I also doubt that anyone was
actually using the option.
This commit separates the user manager timeouts, but keeps them unchanged at 90 s.
The timeout for the user manager is set to 4/3*user-timeout, which means that it
is still 120 s.
Fedora wants to experiment with lower timeouts, but doing this via a patch would
be annoying and more work than necessary. Let's make this easy to configure.
The unlink command removes an entry from the ESP including
referenced files that are not referenced in other entries. That is
useful eg to have multiple entries that use the same kernel with
different options.
The cleanup command removes all files that are not referenced by any
entry.
These options allow measuring the volume key used for unlocking the
volume to a TPM2 PCR. This is ideally used for the volume key of the
root file system and can then be used to bind other resources to the
root file system volume in a secure way.
See: #24503
We converted to not using #ifdef for most of our defines because the syntax is
nicer and we are protected against typos and can set -Werror=undef. Let's do
the same for SD_BOOT. The define is nicely hidden in build.h for normal builds,
and for EFI builds we were already setting SD_BOOT on the commandline.
Some gymnastics were needed to import ukify as a module. Before the file
was templated, this was trivial: insert the directory in sys.path, call import.
But it's a real pain to import the unsuffixed file after processing. Instead,
the untemplated file is imported, which works well enough for tests and is
very simple.
The tests can be called via pytest:
PATH=build/:$PATH pytest -v src/ukify/test/test_ukify.py
or directly:
PATH=build/:$PATH src/ukify/test/test_ukify.py
or via the meson test machinery output:
meson test -C build test-ukify -v
or without verbose output:
meson test -C build test-ukify
Zekret files are obfuscated using base64.
The option is added because we have a similar one for kernel-install. This
program requires python, and some people might want to skip it because of this.
The tool is installed in /usr/lib/systemd for now, since the interface might
change.
A template file is used, but there is no .in suffix.
The problem is that we'll later want to import the file as a module
for tests, but recent Python versions make it annoyingly hard to import
a module from a file without a .py suffix. imp.load_sources() works, but it
is deprecated and throws warnings.
importlib.machinery.SourceFileLoader().load_module() works, but is also
deprecated. And the documented replacements are a maze of twisted little
callbacks that result in an empty module.
So let's take the easy way out, and skip the suffix which makes it easy
to import the template as a module after adding the directory to sys.path.
I'd like to use this as a basis for an exitrd:
When compiled with -Dstandalone-binaries=true -Db_lto=true -Dbuildtype=release,
the new file is 800k. It's more than I'd like, but still quite a bit less
than libsystemd-shared.so, which is 3800k.
systemd-cryptenroll complains (but succeeds!) upon binding to a signed PCR
policy:
$ systemd-cryptenroll --unlock-key-file=/tmp/passphrase --tpm2-device=auto
--tpm2-public-key=... --tpm2-signature=..." /tmp/tmp.img
ERROR:esys:src/tss2-esys/esys_iutil.c:394:iesys_handle_to_tpm_handle() Error: Esys invalid ESAPI handle (40000001).
WARNING:esys:src/tss2-esys/esys_iutil.c:415:iesys_is_platform_handle() Convert handle from TPM2_RH to ESYS_TR, got: 0x40000001
ERROR:esys:src/tss2-esys/esys_iutil.c:394:iesys_handle_to_tpm_handle() Error: Esys invalid ESAPI handle (40000001).
WARNING:esys:src/tss2-esys/esys_iutil.c:415:iesys_is_platform_handle() Convert handle from TPM2_RH to ESYS_TR, got: 0x4000000
New TPM2 token enrolled as key slot 1.
The problem seems to be that Esys_LoadExternal() function from tpm2-tss
expects a 'ESYS_TR_RH*' constant specifying the requested hierarchy and not
a 'TPM2_RH_*' one (see Esys_LoadExternal() -> Esys_LoadExternal_Async() ->
iesys_handle_to_tpm_handle() call chain).
It all works because Esys_LoadExternal_Async() falls back to using the
supplied values when iesys_handle_to_tpm_handle() fails:
r = iesys_handle_to_tpm_handle(hierarchy, &tpm_hierarchy);
if (r != TSS2_RC_SUCCESS) {
...
tpm_hierarchy = hierarchy;
}
Note, TPM2_RH_OWNER was used on purpose to support older tpm2-tss versions
(pre https://github.com/tpm2-software/tpm2-tss/pull/1531), use meson magic
to preserve compatibility.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Offline encryption can be done without mounting the luks device. For
now we still use loop devices to split out the partition we want to
write to but in a later commit we'll replace this with a regular file.
For offline encryption, we need to keep 2x the luks header size space
free at the end of the partition, so this means our encrypted partitions
will be 16M larger than before.
Due to "historical reasons" both gcc and clang treat *all* trailing
arrays members as flexible arrays, this has an evil side effect
of inhibiting bounds checks on such members as __builtin_object_size
cannot say for sure that:
struct {
...
type foo[3];
}
has a trailing foo member of fixed size rather than unspecified.
Ideally we should use -fstrict-flex-arrays as is, but we have to
tolerate kernel uapi headers that use [0] and third party libraries
written in c89 that may use [1] like curl.
Follow-up to c47511da7e.
```
archlinux_systemd_ci: In file included from ../build/src/dissect/dissect.c:15:
archlinux_systemd_ci: ../build/src/basic/build.h:4:10: fatal error: version.h: No such file or directory
archlinux_systemd_ci: 4 | #include "version.h"
archlinux_systemd_ci: | ^~~~~~~~~~~
archlinux_systemd_ci: compilation terminated.
```
```
archlinux_systemd_ci: In file included from ../build/src/journal/cat.c:13:
archlinux_systemd_ci: ../build/src/basic/build.h:4:10: fatal error: 'version.h' file not found
archlinux_systemd_ci: #include "version.h"
archlinux_systemd_ci: ^~~~~~~~~~~
archlinux_systemd_ci: 1 error generated.
```
```
archlinux_systemd_ci: In file included from ../build/src/sysext/sysext.c:10:
archlinux_systemd_ci: ../build/src/basic/build.h:4:10: fatal error: version.h: No such file or directory
archlinux_systemd_ci: 4 | #include "version.h"
archlinux_systemd_ci: | ^~~~~~~~~~~
archlinux_systemd_ci: compilation terminated.
archlinux_systemd_ci: FAILED: systemd-inhibit.p/src_login_inhibit.c.o
```
```
archlinux_systemd_ci: In file included from ../build/src/login/inhibit.c:12:
archlinux_systemd_ci: ../build/src/basic/build.h:4:10: fatal error: version.h: No such file or directory
archlinux_systemd_ci: 4 | #include "version.h"
archlinux_systemd_ci: | ^~~~~~~~~~~
archlinux_systemd_ci: compilation terminated.
```
...
version.h can be generated after compilation starts, creating a race condition
between compilation of various .c files and creation of version.h. Let's add it
as a dependency to more build targets that require version.h or build.h.
So far we played whack'a'mole by adding versiondep whenever compilation failed.
In principle any target which includes compilation (i.e. any that has .c
sources directly), could require this. I don't understand why we didn't see
more failures… But it seems reasonable to just add the dependency more widely.
In the Xen case, it's the hypervisor which manages kexec. We thus
have to ask it whether a kernel is loaded, instead of relying on
/sys/kernel/kexec_loaded.
Using fsopen()/fsconfig(), we can check if hidepid/subset are supported to
avoid the noisy logs from the kernel if they aren't supported. This works
on centos/redhat 8 as well since they've backported fsopen()/fsconfig().
Repart is growing into an important tool on its own, and users might
want to install newer versions on systems that have older systemd. Let's
make this easier by providing a standalone binary.
Related to https://github.com/systemd/mkosi/issues/1228.
Semi-quoting https://github.com/systemd/systemd/issues/25057:
clang-16 has made the choice to turn on -Werror=implicit-function-declaration,implicit-int.
(See Gentoo's tracker bug https://bugs.gentoo.org/870412).
Added in commit 132c73b57a, systemd now does a
check to see if libatomic is needed with some compile/link tests with e.g.
__atomic_exchange_1, but the tests don't provide a prototype for
__atomic_exchange_1 so with clang-16 the test fails, breaking the build.
Let's simplify things by linking to libatomic unconditionally if it is found
and seems to work. If actually unneeded, it might be dropped via --as-needed.
This seems to work with gcc and clang.
declare_dependency() is used instead of cc.find_library(), because the latter
picks up a symlink in gcc private directory (e.g.
/usr/lib/gcc/x86_64-redhat-linux/12/libatomic.so), and we don't want that.
Fixes#25057.
The lists of directives for fuzzer tests are maintained manually in the
repo. There is a tools/check-directives.sh script that runs during test
phase and reports stale directive lists.
Let's rework the script into a generator so that these directive files
are created on-the-flight and needn't be updated whenever a unit file
directives change. The scripts is rewritten in Python to get rid of gawk
dependency and each generated file is a separate meson target so that
incremental builds refresh what is just necessary (and parallelize
(negligible)).
Note: test/fuzz/fuzz-unit-file/directives-all.slice is kept since there
is not automated way to generate it (it is not covered by the check
script neither).
--convert writes the journal files read by journalctl to the given
location. The location should be specified as a full journal file
path (e.g. /a/b/c/converted.journal). The directory specifies where
the converted journal files will be stored. The filename specifies
the naming convention the converted journal files will follow.
- new symbols are available from libbpf 0.6.0 so could be used with
libbpf.so.0, but we're sure the old symbols will be there and this
simplifies code
- detection at runtime should always work, regardless of whether systemd
has been compiled with older or newer libbpf and runs with older or newer
libbpf
We already depend on the skeleton APIs introduced in libbpf 0.7 so
let's bump our minimum version to reflect that.
We don't enforce bpf compilation on mkosi anymore since not all
distros have sufficiently up-to-date libbpf available.
Fixes compile error with -Dopenssl=false.
```
In file included from ../../home/watanabe/git/systemd/src/shared/pkcs11-util.h:12,
from ../../home/watanabe/git/systemd/src/cryptenroll/cryptenroll.c:24:
../../home/watanabe/git/systemd/src/shared/openssl-util.h:56:21: error: conflicting types for ‘X509’; have ‘struct X509’
56 | typedef struct X509 X509;
| ^~~~
In file included from /usr/include/openssl/crypto.h:25,
from /usr/include/openssl/bio.h:20,
from /usr/include/openssl/asn1.h:16,
from /usr/include/openssl/ec.h:17,
from /usr/include/fido.h:10,
from ../../home/watanabe/git/systemd/src/shared/libfido2-util.h:18,
from ../../home/watanabe/git/systemd/src/cryptenroll/cryptenroll-fido2.h:7,
from ../../home/watanabe/git/systemd/src/cryptenroll/cryptenroll.c:6:
/usr/include/openssl/ossl_typ.h:123:24: note: previous declaration of ‘X509’ with type ‘X509’ {aka ‘struct x509_st’}
123 | typedef struct x509_st X509;
| ^~~~
```
Building with GCC 12.2 and binutils 2.39 fails on riscv64 Ubuntu Kinetic
with:
FAILED: systemd-oomd
/usr/bin/ld: systemd-oomd.p/src_oom_oomd-util.c.o:
in function `oomd_cgroup_context_acquire':
build/../src/oom/oomd-util.c:415:
undefined reference to `__atomic_exchange_1'
We have to link with -latomic.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
For now, this simply outputs the PCR hash values expected for a kernel
image, if it's measured like sd-stub would do it.
(Later on, we can extend the tool, to optionally sign these
pre-calculated measurements, in order to implement signed PCR policies
for disk encryption.)
Everywhere else that `conf.get('ENABLE_*')` is used as a boolean key for
something (for example in if statements) it always checks if == 1, but
in this one case it neglects to do so. This is important because
conf.get yields the same int that was stored, but if statements require
booleans.
So does executable's "install" kwarg, at least according to the
documentation. In actuality, it accepts all types without sanity
checking, then uses python "if bool(var)", so you can actually do
`install: 'do not'` and that's treated identical to `true`. This is a
type-checking bug which Meson will eventually fix.
muon fails on the same code, today.
0 UID and GID are special, and should not be acceptable for the settings.
Hence, we can handle 0 as unset.
Strictly speaking, time epoch with 0 is valid, but I guess no one use
0 as a valid value.
The journalctl tool may be needed on cross compilation hosts in order
to run --update-catalog against a target rootfs.
To avoid reliability issues caused by shared linking allow journalctl
to be linked statically.
The idea is that we can peek into /sysroot/etc/fstab and figure out if there's
anything interesting there. We could use a separate binary for this, but we'd
need to duplicate most of the logic that in systemd-fstab-generator. Thus I
think it's nicer to make systemd-fstab-generator work as a multi-call binary.
If called as systemd-sysroot-fstab-check, we look for units that we'd mount and
call daemon-reload and initrd-fs.target/restart, similarly to what we did
before, but in the process itself.
DefaultSmackProcessLabel tells systemd what label to assign to its child
process in case SmackProcessLabel is not set in the service file. By
default, when DefaultSmackProcessLabel is not set child processes inherit
label from systemd.
If DefaultSmackProcessLabel is set to "/" (which is an invalid character
for a SMACK label) the DEFAULT_SMACK_PROCESS_LABEL set during compilation
is ignored and systemd act as if the option was unset.
I opted to tweaking kernel-install to allow overriding config
(with $KERNEL_INSTALL_CONF_ROOT, $KERNEL_INSTALL_PLUGINS). An alternative
would be to build a test environment in test/. We can still do that,
but I think it's nice to have a simple test that is very quick and easy
to debug.
Invocation as installkernel is for #23681.
Before we had the following scheme:
mempool_enabled() would check mempool_use_allowed, and
libsystemd-shared would be linked with a .c file that provides mempool_use_allowed=true,
while other things would linked with a different .c file with mempool_use_allowed=false.
In the new scheme, mempool_enabled() itself is a weak symbol. If it's
not found, we assume false. So it only needs to be provided for libsystemd-shared,
where it can return false or true.
test-set-disable-mempool is libshared, so it gets the symbol. But then we
actually disable the mempool via envvar. mempool_enable() is called to check
its return value directly.
I think developers are particularly unlikely to find the descriptions
useful, and would benefit from being able to copy&paste unit names.
Let's make this choice automatically.
Profiling tools tend to work better when binaries and libraries
are compiled with frame pointers as without them there's no easy
and fast way to get the current stacktrace.
https://github.com/systemd/systemd/pull/23192 caused breakage in
Arch Linux's build tooling. Let's give users an opt-out aside from
reverting the patch. It's hardly any maintenance work on our side
and gives users an easy way to revert the locale change if needed.
Of course, by default we still pick C.UTF-8 if the option is not
specified.
On Debian, libdir is commonly something like 'lib/x86_64-linux-gnu'.
The result of get_option('libdir') is normalized to a prefix-relative
path by meson, so we can just append it to rootprefixdir.
Fixes https://github.com/systemd/systemd/issues/23648.
Introduce rootpkglibdir for installing libsystemd-{shared,core}.so.
The benefit over using rootlibexecdir is that this path can be
multiarch aware, i.e. this path can be architecture qualified.
This is something we'd like to make use of in Debian/Ubuntu to make
libsystemd-shared co-installable, e.g. for i386 the path would be
/usr/lib/i386-linux-gnu/systemd/libsystemd-shared-*.so and for amd64
/usr/lib/x86_64-linux-gnu/systemd/libsystemd-shared-*.so.
This will allow for example to install and run systemd-boot/i386 on an
amd64 host. It also simplifies/enables cross-building/bootstrapping.
For more infos about Multi-Arch see https://wiki.debian.org/Multiarch.
See also https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=990547
We're already using C.UTF-8 as the default locale for nspawn. Let's
make the same change for the default-locale option instead of deciding
what to use based on the locale used by the host system. Users can
still override the locale using the default-locale option if needed.
If both gnutls and openssl are available, prefer openssl.
We are gradually moving toward supporting openssl only as the
crypto library, and the resolved gnutls backend will be dropped
at some point, so start nudging users toward the openssl one.
Even if we use meson >= 0.55, using path() does not produce any error or
warning if the required version is below 0.55.
Let's convert path() with full_path() when we requires meson >= 0.55.
Setting with number is deprecated:
```
meson.build:1008: DEPRECATION: configuration_data.set10 with number. the `set10` method should only be used with booleans
```
Passing potentially arbitrary data into a shellscript is potentially
very broken if you do not correctly quote it for use. This quoting must
be done as part of the interpretation of the data itself, e.g. python's
shlex.quote; simply formatting it into a string with double quotes is
NOT sufficient.
An alternative is to communicate the data reliably via argv to the shell
process, and allow the shell to internally handle it via `"$1"`, which
is quote-safe and will expand the data from argv as a single tokenized
word.
With an intentional mistake:
../src/login/logind-dbus.c: In function ‘bus_manager_log_shutdown’:
../src/login/logind-dbus.c:1542:39: error: format ‘%s’ expects a matching ‘char *’ argument [-Werror=format=]
1542 | LOG_MESSAGE("%s %s", message),
| ^~~~~~~
No need to involve a trivial shell script for this.
We could call the compiler directly, but test() expects arguments
to be passed separately and cc.cmd_array() can contain arguments
itself. Using env is easier than manually slicing the array because
meson has no builtins for that.
A compile time option is added to select behaviour: by default
UNIT_FILE_PRESET_ENABLE_ONLY is still used, but the intent is to change to
UNIT_FILE_PRESET_FULL at some point in the future. Distros that want to
opt-in can use the config option to change the behaviour.
(The option is just a boolean: it would be possible to make it multi-valued,
and allow full, enable-only, disable-only, none. But so far nobody has asked
for this, and it's better not to complicate things needlessly.)
With the configuration option flipped, instead of only doing enablements,
perform a full preset on first boot. The reason is that although
`/etc/machine-id` might be missing, there may be other files provisioned in
`/etc` (in fact, this use case is mentioned in `log_execution_mode`). Some of
those possible files include enablement symlinks even if presets dictate it
should be disabled.
Such a seemingly contradictory situation occurs in {RHEL,Fedora} CoreOS,
where we ship `/etc` as if `preset-all` were called. However, we want to
allow users to disable default-enabled services via Ignition, which does
this by creating preset dropins before switchroot. (For why we do
`preset-all` at compose time, see:
https://github.com/coreos/fedora-coreos-config/pull/77).
For example, the composed FCOS image has a `enable zincati.service`
preset and an enablement for that in `/etc`, while at boot time when we
switch root, there may be a `disable zincati.service` preset with higher
precedence. In that case, we want systemd to disable the service.
This is essentially a revert of 304b3079a2. It seems like systemd
*used* to do this, but it was changed to try to make the container
workflow a bit faster.
Resolves: https://github.com/coreos/fedora-coreos-tracker/issues/392
Co-authored-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
People (and build systems) sometimes set flags through -Dc_args=… or $CFLAGS.
Let's catch this common case too. meson will set c_args from $CFLAGS, so we
only need to check the former.
Allows to 'meson install --tags systemd-boot --no-rebuild' to install only the EFI
binaries, skipping the rest, for a very quick build:
$ ninja src/boot/efi/linuxx64.efi.stub
[21/21] Generating src/boot/efi/linuxx64.efi.stub with a custom command
$ ninja src/boot/efi/systemd-bootx64.efi
[10/10] Generating src/boot/efi/systemd-bootx64.efi with a custom command
$ DESTDIR=/tmp/foo meson install --tags systemd-boot --no-rebuild
Installing src/boot/efi/systemd-bootx64.efi to /tmp/foo/usr/lib/systemd/boot/efi
Requires Meson 0.60 to be used, prints a warning for unknown keyword
in earlier versions, but there's no failure
https://mesonbuild.com/Installing.html#installation-tags
The compression helpers are used both in journal code and in coredump
code, and there's a good chance we'll use them later for other stuff.
Let's hence move them into src/basic/, to make them a proper internal
API we can use from everywhere where that's desirable. (pstore might be
a candidate, for example)
No real code changes, just some moving around, build system
rearrangements, and stripping of journal-def.h inclusion.
Suggested by Daniele Nicolodi:
https://github.com/systemd/systemd/pull/23160#discussion_r855853716
This is possible only if the macro is never used in #if, but only in C code.
This means that all places that use #if have to be refactored into C, but we
reduce the duplication a bit, and C is nicer to read than preprocessor
conditionals.
Follow-up for da13d2ca07. Instead of having
separate definitions of the bitmask flags, just define DEFAULT_COMPRESSION_FOO=0|1
directly.
(It *should* be possible to do this more simply, but the problem is that
anything that is used in #if cannot refer to C constants or enums. This is the
simplest I could come up with that preserves the property that we don't use #ifdef.)
The return value from compress_blob() is changed to propagate the error instead
of always returning -EOPNOTSUPP. The callers don't care about the specific error
value. compress_blob_*() are changed to return the compression method on success, so
that compress_blob() can be simplified. compress_stream_*() and compress_stream() are
changed in the same way for consistency, even though the callers do not currently use
this information (outside of tests).
Follow-up for cd3c6322db
journal-def.h should be self-contained too, as it represents the journal object ABI.
Duplicate the enums, as they also need to be in config.h for it to be self-contained,
and enums are not available to the preprocessor. Use an assert to ensure they don't
diverge.
Compression and decompression are controlled by the same build flag,
so if one wants to use, say, LZ4 to compress, ZSTD has to be disabled,
which means one loses the ability to read zstd-compressed journals.
Add a default-compression meson option, that allows to select any of
the available compression algorithms as the default.
To make sure we don't miss any _exit() calls let's move the
coverage-related tweaks into a separate header file and include it
explicitly on the compiler command line using -include when a coverage
build is requested.
Follow-up to c6552ad381.
In --help output, change "$0" → "kernel-install". We generally don't include
the full path in --help output, and let's not do this here either.
kernel-install is now in build/ directly, not in the subdirectory.
GIT_VERSION is not available as a config.h variable, because it's rendered
into version.h during builds. Let's rework jinja2 rendering to also
parse version.h. No functional change, the new variable is so far unused.
I guess this will make partial rebuilds a bit slower, but it's useful
to be able to use the full version string.
This is very similar to (and directly based on) the test for --help. I think
it's nice to do this: the test is very quick, but it'll catch cases where we
forgot to hook up the option, or forgot to exit after printing --version, and
it'll also increase our test coverage a bit.
This test has overlap with test-install-root, but it tests things at a
different level, so I think it's useful to add. It immediately shows various
bugs which will be fixed in later patches.
Defaults to /bin/bash, no changes in the default configuration
The fallback shell for non-root users is as-specified,
and the interactive shell for nspawn sessions is started as
exec(default-user-shell, "-" + basename(default-user-shell), ...)
before falling through to bash and sh
mostly to make sure that systemd is buildable without some dependencies
but other than that it should make it easier to build it with MSan without
having to compile all the dependencies with MSan.
ERROR:
Cannot use target systemd as a generator because it is built for the
host machine and no exe wrapper is defined or needs_exe_wrapper is
true. You might want to set `native: true` instead to build it for
the build machine.
The script was probably not used for a very long time. It is currently
passed systemd_boot.so as boot loader, which cannot work. The test
entries it creates are all pointing at non-existant efi/linux binaries,
which means they would not even show up in the menu if the created image
were actually booted. There is also nothing that actually tries to run
the image in the first place.
If we end up creating a proper systemd-boot test suite, it would be
better to start from scratch. In the meantime, mkosi already covers
the bare minimum with a simple bootup test.
Currently, running "meson build" followed by "meson test -C build"
will result in many failed tests due to missing dependencies. This
commit adds the missing dependencies to make sure no tests fail.
The whole point of systemd-stdio-bridge is to be executed on "foreign" systems
where the path might be different, so we use $PATH to find the binary everywhere.
to make it easier to fuzz code that uses external libraries like libelf/libdw.
The dependencies are skipped on OSS-Fuzz because they aren't available
at runtime if they aren't linked statically. This restriction can safely
be lifted when the fuzzers are built locally with all the dependencies
installed. As far as I know there is at least one fuzz target in the systemd
repository that can benefit from this: https://github.com/systemd/systemd/issues/11018
Meson would generate the following compile test:
#define crypt_set_metadata_size meson_disable_define_of_crypt_set_metadata_size
#include <limits.h>
#undef crypt_set_metadata_size
#ifdef __cplusplus
extern "C"
#endif
char crypt_set_metadata_size (void);
#if defined __stub_crypt_set_metadata_size || defined __stub___crypt_set_metadata_size
fail fail fail this function is not going to work
#endif
int main(void) {
return crypt_set_metadata_size ();
}
This works fine when the identifier being queried is an actual function. But
crypt_token_max() is an inline function, so getting the address would fail,
leading to a false negative result. Complation would fail because the function
would be defined twice.
With this patch, the check is changed to include the header:
#include <libcryptsetup.h>
#include <limits.h>
#if defined __stub_crypt_set_metadata_size || defined __stub___crypt_set_metadata_size
fail fail fail this function is not going to work
#endif
int main(void) {
void *a = (void*) &crypt_set_metadata_size;
long long b = (long long) a;
return (int) b;
}
which seems to work correctly.
This should simplify overriding the program locations as the binary
names should now not change if cross compiling.
It's likely any attempts at autodetecting these in cross environments will
be brittle at best so lets just disable it.
We have /usr/lib/systemd/libsystemd-{shared,core}-nnn.so. With this
path the 'nnn' part can be changed to something different. The idea
is that during a package build this will be set to the package version.
This way during in-place upgrades with the same major version both
the new and old libraries can cooexit. This should fix the issue
when systemd programs are called during package upgrades and fail
to exec because the expect different symbols in the library they
are linked to.
This should fix https://bugzilla.redhat.com/show_bug.cgi?id=1906010.
The scheme is very similar to libsystemd-shared.so: instead of building a
static library, we build a shared library from the same objects and link the
two users to it. Both systemd and systemd-analyze consist mostly of the fairly
big code in libcore, so we save a bit on the installation:
(-0g, no strip)
-rwxr-xr-x 5238864 Dec 14 12:52 /var/tmp/inst1/usr/lib/systemd/systemd
-rwxr-xr-x 5399600 Dec 14 12:52 /var/tmp/inst1/usr/bin/systemd-analyze
-rwxr-xr-x 244912 Dec 14 13:17 /var/tmp/inst2/usr/lib/systemd/systemd
-rwxr-xr-x 461224 Dec 14 13:17 /var/tmp/inst2/usr/bin/systemd-analyze
-rwxr-xr-x 5271568 Dec 14 13:17 /var/tmp/inst2/usr/lib/systemd/libsystemd-core-250.so
(-0g, strip)
-rwxr-xr-x 2522080 Dec 14 13:19 /var/tmp/inst1/usr/lib/systemd/systemd
-rwxr-xr-x 2604160 Dec 14 13:19 /var/tmp/inst1/usr/bin/systemd-analyze
-rwxr-xr-x 113304 Dec 14 13:19 /var/tmp/inst2/usr/lib/systemd/systemd
-rwxr-xr-x 207656 Dec 14 13:19 /var/tmp/inst2/usr/bin/systemd-analyze
-rwxr-xr-x 2648520 Dec 14 13:19 /var/tmp/inst2/usr/lib/systemd/libsystemd-core-250.so
So for systemd itself we grow a bit (2522080 → 2648520+113304=2761824), but
overall we save. The most is saved on all the test files that link to libcore,
if they are installed, because there's 15 of them:
$ du -s /var/tmp/inst?
220096 /var/tmp/inst1
122960 /var/tmp/inst2
I also considered making systemd-analyze a symlink to /usr/lib/systemd/systemd
and turning systemd into a multicall binary. We did something like this with
udevd and udevadm. But that solution doesn't fit well in this case.
systemd-analyze has a bunch of functionality that is not used in systemd,
so the systemd binary would need to grow quite a bit. And we're likely to
add new types of verification or introspection features in analyze, and this
baggage would only grow. In addition, there are the test binaries which also
benefit from this.
This patch changes busctl capture to generate pcapng format
instead of the legacy pcap format files. It includes basic
meta-data in the file and still uses microsecond time
resolution. In future, more things can be added such as
high resolution timestams, statistics, etc.
PCAP Next Generation capture file format is what tshark uses
and is in process of being standardized in IETF. It is also
readable with libpcap.
$ capinfos /tmp/new.pcapng
File name: /tmp/new.pcapng
File type: Wireshark/... - pcapng
File encapsulation: D-Bus
File timestamp precision: microseconds (6)
Packet size limit: file hdr: (not set)
Packet size limit: inferred: 4096 bytes
Number of packets: 22
File size: 21kB
Data size: 20kB
Capture duration: 0.005694 seconds
First packet time: 2021-12-11 11:57:42.788374
Last packet time: 2021-12-11 11:57:42.794068
Data byte rate: 3,671kBps
Data bit rate: 29Mbps
Average packet size: 950.27 bytes
Average packet rate: 3,863 packets/s
SHA256: b85ed8b094af60c64aa6d9db4a91404e841736d36b9e662d707db9e4096148f1
RIPEMD160: 81f9bac7ec0ec5cd1d55ede136a5c90413894e3a
SHA1: 8400822ef724b934d6000f5b7604b9e6e91be011
Strict time order: True
Capture oper-sys: Linux 5.14.0-0.bpo.2-amd64
Capture application: systemd 250 (250-rc2-33-gdc79ae2+)
Number of interfaces in file: 1
Interface #0 info:
Encapsulation = D-Bus (146 - dbus)
Capture length = 4096
Time precision = microseconds (6)
Time ticks per second = 1000000
Number of stat entries = 0
Number of packets = 22
Previously, when -Ddns-over-tls=false, libopenssl was missing in the
dependency of resolved.
Also, this drops libgpg_error when it is not necessary.
Replaces #21878.
This adds /etc/locale.conf to the set of configuration files
populated by tmpfiles.d factory /etc handling.
In particular, the build-time locale configuration in systemd is
now wired to a /usr factory file, and installed to the system.
On boot, if other locale customization tools did not write
/etc/locale.conf on the system, the factory default file gets
copied to /etc by systemd-tmpfiles.
This is done in order to avoid skews between different system
components when no locale settings are configured. At that point,
systemd can safely act as the fallback owner of /etc/locale.conf.
afl-clang and hufzz-clang try to instrument the code and the
underlying compilers don't like it. It should probably be
fixed in both afl and honggfuzz eventually but until then
let's just use "raw" clang to build bpf-skeletons.
It's a follow-up to https://github.com/systemd/systemd/pull/21607
341890de86 made "bootctl install" create
ESP\MID, in preparation of cf73f65089 that
followed it and created 00-entry-directory.install to make ESP\MID\KVER
if ESP\MID existed ‒ this meant that "bootctl install" followed by
"kernel-install $(uname -r) /boot/vml*$(uname -r) /boot/ini*$(uname -r)"
actually installed the kernel correctly.
Later, 31e57550b5 reverted the first
commit, meaning, that now running those two commands first installs
sd-boot, but then does nothing. Everything appears to work right,
nothing errors out, but no changes are actually done. To the untrained
eye (all of them), even running with -v appears to work:
all the hooks are run, as is depmod, but, again, nothing happens.
This is horrible. Nothing in either manpage suggests what to do
(nor should it, really), but the user is left with a bootloader that
appears fully funxional, since nothing suggests a failure in the output,
but with an unbootable machine, /no way to boot it/, even if they drop
to an EFI shell, since the boot bundle isn't present on the ESP,
and no real recourse even if they boot into a recovery system,
apart from installing like GRUB or whatever.
00- is purely instrumentation for 90-,
and separating one from the other has led to downstream dissatisfaxion
(indeed, the last mentioned commit cited cited exactly that as the
reversion reason), while creating $ENTRY_DIR_ABS is only required
for bootloaders using the BLS, and shouldn't itself toggle anything.
To that end, introduce an /{e,l}/k/install.conf file that allows
overriding the detected layout, and detect it as "bls" if
$BOOT_ROOT/$MACHINE_ID ($ENTRY_DIR_ABS/..) exists, otherwise "other" ‒
if a user wishes to select a different bootloader,
like GRUB, they (or, indeed, the postinst script) can specify
layout=grub. This disables 90- and $ENTRY_DIR_ABS manipulation.
The way that the cryptsetup plugins were built was unnecessarilly complicated.
We would build three static libraries that would then be linked into dynamic
libraries. No need to do this.
While at it, let's use a convenience library to avoid compiling the shared code
more than once.
We want the output .so files to be located in the main build directory,
like with all consumable build artifacts, so we need to maintain the split
between src/cryptsetup/cryptsetup-token/meson.build and the main meson.build
file.
AFAICT, the build artifacts are the same: exported and undefined symbols are
identical. There is a tiny difference in size, but I think it might be caused
by a different build directory name.
Use a 'convenience library' to do the compilation once and then link the
objects into all the files that need it. Those files are small, so this probably
doesn't matter too much for speed, but has the advantage that we don't get the
same error four times if something goes wrong.
The library is conditionalized in the same way importd itself, because we
cannot build it without the deps.
Build option "link-boot-shared" to build a statically linked bootctl and
systemd-bless-boot by using
-Dlink-boot-shared=false
on systems with full systemd stack except bootctl and systemd-bless-boot,
such as CentOS/RHEL 9.
This is a soft disable. Passing `dbus-interfaces-dir` build option
will with path or 'yes' enable exports again even when cross
compiling. (maybe your environment will allow to execute
cross compiled binaries)
Currently, all the logic related to writing journal files lives in
journal-file.c which is part of libsystemd (sd-journal). Because it's
part of libsystemd, we can't depend on any code from src/shared.
To allow using code from src/shared when writing journal files, let's
gradually move the write related logic from journal-file.c to
journald-file.c in src/journal. This directory is not part of libsystemd
and as such can use code from src/shared.
We can safely remove any journal write related logic from libsystemd as
it's not used by any public APIs in libsystemd.
This commit introduces the new file along with the JournaldFile struct
which wraps an instance of JournalFile. The goal is to gradually move
more functions from journal-file.c and fields from JournalFile to
journald-file.c and JournaldFile respectively.
This commit also modifies all call sites that write journal files to
use JournaldFile instead of JournalFile. All sd-journal tests that
write journal files are moved to src/journal so they can make use of
journald-file.c.
Because the deferred closes logic is only used by journald, we move it
out of journal-file.c as well. In journal_file_open(), we would wait for
any remaining deferred closes for the file we're about to open to complete
before continuing if the file was not newly created. In journald_file_open(),
we call this logic unconditionally since it stands that if a file is newly
created, it can't have any outstanding deferred closes.
No changes in behavior are introduced aside from the earlier execution
of waiting for any deferred closes to complete when opening a new journal
file.
In 9cf75222f2 the conf.get() statements for `bpf-framework` and
`valgrind` were dropped, which causes the respective features to always
show as disabled (since they don't follow the "standard" naming scheme
with HAVE_/ENABLE_ prefixes).
It could work, but it doesn't make much sense. If we already have openssl as
the cryptolib that provides the necessary support, let's not bring in another
library. Disallowing this simplifies things and reduces our support matrix.
This allows resolved and importd to be built without libgcrypt.
Note that we now say either 'cryptographic library' or 'cryptolib'.
Co-authored-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
This is heavily based on Kevin Kuehler's work, but the logic is also
significantly changed: instead of a straighforward port to openssl, both
versions of the code are kept, and at compile time we pick one or the other.
The code is purposefully kept "dumb" — the idea is that the libgcrypt codepaths
are only temporary and will be removed after everybody upgrades to openssl 3.
Thus, a separate abstraction layer is not introduced. Instead, very simple
ifdefs are used to select one or the other. If we added an abstraction layer,
we'd have to remove it again afterwards, and it don't think it makes sense to
do that for a temporary solution.
Co-authored-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
# Conflicts:
# meson.build
meson-0.59.4-1.fc35.noarch says:
WARNING: You should add the boolean check kwarg to the run_command call.
It currently defaults to false,
but it will default to true in future releases of meson.
See also: https://github.com/mesonbuild/meson/issues/9300
When working on systemd, it's often useful to be able to comment out
a function to see how a build behaves without it. Currently, when doing
this with a static function that's only used once, the build fails because
the function then becomes unused. As such, Let's downgrade the unused
function error to a warning in local builds.
After reading https://simonbyrne.github.io/notes/fastmath/ I think we
should drop -ffast-math. The JSON code actually looks for NaN, so the
fact it becomes unreliable kinda sucks.
Moreover, we don't do any number crunching. We use floating point fields
only sporadical for trivial math. Hence the optimization is entirely
unnecessary.
Moving all of the gnu-efi detection into src/boot/efi/meson.build makes
more sense than having it partially split.
And thanks to subdir_done() we can simplify the code a lot.
Fixes: #21258
Getting the variable directly from pkg-config (without
adding the sysroot prefix) is prone to host contamination
when building in sysroots as the compiler starts looking for the
headers on the host in addition to the sysroot.
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
This adds support for dm integrity targets and an associated
/etc/integritytab file which is required as the dm integrity device
super block doesn't include all of the required metadata to bring up
the device correctly. See integritytab man page for details.
glibc 2.30 (Aug 2019) added a wrapper for getdents64(). For older
versions let's define our own.
(This syscall exists since Linux 2.4, hence should be safe to use for
us)
In upstream, we have a linearly-growing list of net-naming-scheme defines;
we add a new one for every release where we make user-visible changes to the
naming scheme.
But the general idea was that downstream distributions could define their
own combinations (or even just their own names for existing combinations),
so provide stability for their users. So far this required patching of the
netif-naming-scheme.c and .h files to add the new lines.
With this patch, patching is not required:
$ meson configure build \
-Dextra-net-naming-schemes=gargoyle=v238+npar_ari+allow_rerenames,gargoyle2=gargoyle+nspawn_long_hash \
-Ddefault-net-naming-scheme=gargoyle2
or even
$ meson configure build \
-Dextra-net-naming-schemes=gargoyle=v238+npar_ari+allow_rerenames,gargoyle2=gargoyle+nspawn_long_hash,latest=v249 \
-Ddefault-net-naming-scheme=gargoyle2
The syntax is a comma-separated list of NAME=name+name+…
This syntax is a bit scary, but any typos result in compilation errors,
so I think it should be OK in practice.
With this approach, we don't allow users to define arbitrary combinations:
what is allowed is still defined at compilation time, so it's up to the
distribution maintainers to provide reasonable combinations. In this regard,
the only difference from status quo is that it's much easier to do (and harder
to do incorrectly, for example by forgetting to add a name to one of the
maps).
We used 'combo' type for the scheme list. For a while we forgot to add
new names, and recently aa0a23ec86 added v241, v243, v245, and v247.
I want to allow defining new values during configuration, which means
that we can't use meson to verify the list of options. So any value is
allowed, but then two tests are added: one that will fail compilation if some
invalid name is given (other than "latest"), and one that converts
DEFAULT_NET_NAMING_SCHEME to a NamingScheme pointer.
Before:
```
Compiling C object src/libsystemd-network/libsystemd-network.a.p/dhcp6-option.c.o
../src/libsystemd-network/dhcp6-option.c: In function ‘dhcp6_option_parse_ia’:
../src/libsystemd-network/dhcp6-option.c:633:70: warning: passing argument 3 of ‘dhcp6_option_parse’ makes pointer from integer without a cast [-Wint-conversion]
633 | r = dhcp6_option_parse(option_data, option_data_len, offset, &subopt, &subdata_len, &subdata);
| ^~~~~~
| |
| size_t {aka long unsigned int}
../src/libsystemd-network/dhcp6-option.c:358:25: note: expected ‘size_t *’ {aka ‘long unsigned int *’} but argument is of type ‘size_t’ {aka ‘long unsigned int’}
358 | size_t *offset,
| ~~~~~~~~^~~~~~
```
After:
```
../src/libsystemd-network/dhcp6-option.c: In function ‘dhcp6_option_parse_ia’:
../src/libsystemd-network/dhcp6-option.c:633:70: error: passing argument 3 of ‘dhcp6_option_parse’ makes pointer from integer without a cast [-Werror=int-conversion]
633 | r = dhcp6_option_parse(option_data, option_data_len, offset, &subopt, &subdata_len, &subdata);
| ^~~~~~
| |
| size_t {aka long unsigned int}
../src/libsystemd-network/dhcp6-option.c:358:25: note: expected ‘size_t *’ {aka ‘long unsigned int *’} but argument is of type ‘size_t’ {aka ‘long unsigned int’}
358 | size_t *offset,
| ~~~~~~~~^~~~~~
cc1: some warnings being treated as errors
```
Compilation would fail because we could have HAVE_SMACK_RUN_LABEL without
HAVE_SMACK. This doesn't make much sense, so let's just make -Dsmack=false
completely disable smack.
Also, the logic in smack-setup.c seems dubious: '#ifdef SMACK_RUN_LABEL'
would evaluate to true even if -Dsmack-run-label='' is used. I think
this was introduced in the conversion to meson:
8b197c3a8a added
AC_ARG_WITH(smack-run-label,
AS_HELP_STRING([--with-smack-run-label=STRING],
[run systemd --system with a specific SMACK label]),
[AC_DEFINE_UNQUOTED(SMACK_RUN_LABEL, ["$withval"], [Run with a smack label])],
[])
i.e. it really was undefined if not specified. And it was same
still in 72cdb3e783 when configure.ac
was dropped.
So let's use the single conditional HAVE_SMACK_RUN_LABEL everywhere.
Units are copied out via sendmsg datafd from images, but that means
the SELinux labels get lost in transit. Extract them and copy them over.
Given recvmsg cannot use multiple IOV transparently when the sizes are
variable, use a '\0' as a separator between the filename and the label.
upstream meson stopped allowing combining boolean with the plus
operator, and now requires using the logical and operator
reference:
43302d3296Fixes: #20632
Add support for systemd-pkcs11 based LUKS2 device activation
via libcryptsetup plugin. This make the feature (pkcs11 sealed
LUKS2 keyslot passphrase) usable from both systemd utilities
and cryptsetup cli.
The feature is configured via -Dlibcryptsetup-plugins combo
with default value set to 'auto'. It get's enabled automatically
when cryptsetup 2.4.0 or later is installed in build system.
Add support for systemd-fido2 based LUKS2 device activation
via libcryptsetup plugin. This make the feature (fido2 sealed
LUKS2 keyslot passphrase) usable from both systemd utilities
and cryptsetup cli.
The feature is configured via -Dlibcryptsetup-plugins combo
with default value set to 'auto'. It get's enabled automatically
when cryptsetup 2.4.0 or later is installed in build system.
The output is similar to our hand-crafted status message, but it's nice to use
the built-in functionality. After all, it was amended during development to
support our use case.
This undoes part of 4c890ad3cc: the
implementations of update-dbus-docs and update-man-rules are moved back to
man/meson.build, and alias_target() is used to keep the visible target names
unchanged.
The rules for man pages are reworked so that it's possible to invoke the
targets even if xstlproc is not available. After all, xsltproc is only needed
for the final formatted output, and not other processing.
Ubuntu Bionic 18.04 has 0.45, so it was below the previously required
minimum version already. Focal 20.04 has 0.53.2. Let's require that
and use various features that are available.
Otherwise the build sometimes fails in a racy way:
```
[274/1850] Compiling C object src/cryptsetup/cryptsetup-tokens/libcryptsetup-token-systemd-tpm2_static.a.p/cryptsetup-token-systemd-tpm2.c.o
FAILED: src/cryptsetup/cryptsetup-tokens/libcryptsetup-token-systemd-tpm2_static.a.p/cryptsetup-token-systemd-tpm2.c.o
cc -Isrc/cryptsetup/cryptsetup-tokens/libcryptsetup-token-systemd-tpm2_static.a.p (...) -c ../build/src/cryptsetup/cryptsetup-tokens/cryptsetup-token-systemd-tpm2.c
../build/src/cryptsetup/cryptsetup-tokens/cryptsetup-token-systemd-tpm2.c:12:10: fatal error: version.h: No such file or directory
12 | #include "version.h"
| ^~~~~~~~~~~
compilation terminated.
```
Follow-up to d1ae38d85a.
Add support for systemd-tpm2 based LUKS2 device activation
via libcryptsetup plugin. This make the feature (tpm2 sealed
LUKS2 keyslot passphrase) usable from both systemd utilities
and cryptsetup cli.
The feature is configured via -Dlibcryptsetup-plugins combo
with default value set to 'auto'. It get's enabled automatically
when cryptsetup 2.4.0 or later is installed in build system.
This closes an important gap: so far we would reexecute the system manager and
restart system services that were configured to do so, but we wouldn't do the
same for user managers or user services.
The scheme used for user managers is very similar to the system one, except
that there can be multiple user managers running, so we query the system
manager to get a list of them, and then tell each one to do the equivalent
operations: daemon-reload, disable --now, set-property Markers=+needs-restart,
reload-or-restart --marked.
The total time that can be spend on this is bounded: we execute the commands in
parallel over user managers and units, and additionally set SYSTEMD_BUS_TIMEOUT
to a lower value (15 s by default). User managers should not have too many
units running, and they should be able to do all those operations very
quickly (<< 1s). The final restart operation may take longer, but it's done
asynchronously, so we only wait for the queuing to happen.
The advantage of doing this synchronously is that we can wait for each step to
happen, and for example daemon-reloads can finish before we execute the service
restarts, etc. We can also order various steps wrt. to the phases in the rpm
transaction.
When this was initially proposed, we discussed a more relaxed scheme with bus
property notifications. Such an approach would be more complex because a bunch
of infrastructure would have to be added to system manager to propagate
appropriate notifications to the user managers, and then the user managers
would have to wait for them. Instead, now there is no new code in the managers,
all new functionality is contained in src/rpm/. The ability to call 'systemctl
--user user@' makes this approach very easy. Also, it would be very hard to
order the user manager steps and the rpm transaction steps.
Note: 'systemctl --user disable' is only called for a user managers that are
running. I don't see a nice way around this, and it shouldn't matter too much:
we'll just leave a dangling symlink in the case where the user enabled the
service manually.
A follow-up for https://bugzilla.redhat.com/show_bug.cgi?id=1792468 and
fa97d2fcf6.
Instead of embedding the commands to invoke directly in the macros,
let's use a helper script as indirection. This has a couple of advantages:
- the macro language is awkward, we need to suffix most commands by "|| :"
and "\", which is easy to get wrong. In the new scheme, the macro becomes
a single simple command.
- in the script we can use normal syntax highlighting, shellcheck, etc.
- it's also easier to test the invoked commands by invoking the helper
manually.
- most importantly, the logic is contained in the helper, i.e. we can
update systemd rpm and everything uses the new helper. Before, we would
have to rebuild all packages to update the macro definition.
This raises the question whether it makes sense to use the lua scriptlets when
the real work is done in a bash script. I think it's OK: we still have the
efficient lua scripts that do the short scripts, and we use a single shared
implementation in bash to do the more complex stuff.
The meson version is raised to 0.47 because that's needed for install_mode.
We were planning to raise the required version anyway…
This moves the /var/log/README content out of /var and into the
docs location, replacing the previous file with a symlink
created through a tmpfiles.d entry.
We disabled it in f73fb7b742 in response to an
apparent gcc bug. It seems that depending on the combination of optimization
options, gcc still ignores (void). But this seems to work fine with clang, so
let's re-enable the warning conditionally.
So, as it turns out AF_ALG is turned off in a lot of kernels/container
environments, including our CI. Hence, if we link against OpenSSL
anyway, let's just use that client side. It's also faster.
One of those days we should drop the khash code, and ust use OpenSSL,
once the licensing issues are resolved.
The general idea with users and groups created through sysusers is that an
appropriate number is picked when the allocation is made. The number that is
selected will be different on each system based on the order of creation of
users, installed packages, etc. Since system users and groups are not shared
between installations, this generally is not an issue. But it becomes a problem
for initrd: some file systems are shared between the initrd and the host (/run
and /dev are probably the only ones that matter). If the allocations are
different in the host and the initrd, and files survive switch-root, they will
have wrong ownership.
This makes the gids build-time-configurable for all groups and users where
state may survive the switch from initrd to the host.
In particular, all "hardware access" groups are like this: files in /dev will
be owned by them. Eventually the new udev would change ownership, but there
would be a momemnt where the files were owned by the wrong group. The
allocations are "soft-static" in the language of Fedora packaging guidelines:
the uid/gid will be used if possible, but we'll fall back to a different
one. TTY_GID is the exception, because the number is used directly.
Similarly, the possibility to configure "soft-static" uids is added for daemons
which may usefully run in the initramfs: systemd-network (lease information and
interface state is serialized to /run), systemd-resolve (stub files and
interface state), systemd-timesync (/run/systemd/timesync).
Journal files are owned by the group systemd-journal, and acls are granted
for wheel and adm.
systemd-oom and systemd-coredump are excluded from this patch: I assume that
oomd is not useful in the initrd, and coredump leaves no state (it only creates
a pipe in /run?).
The defaults are not changed: if nothing is configured, dynamic allocation will
be used. I looked at a Debian system, and the numbers are all different than
on Fedora.
For Fedora, see the list of uids and gids at https://pagure.io/setup/blob/master/f/uidgid.
In particular, systemd-network and systemd-resolve got soft-static numbers to
make it easy to transition from a non-host-specific initrd to a host system
already a few years back (https://bugzilla.redhat.com/show_bug.cgi?id=1102002).
I also requested static allocations for sgx, input, render in
https://pagure.io/packaging-committee/issue/1078,
https://pagure.io/setup/pull-request/27.
Debugging udev issues especially during the early boot is fairly
difficult. Currently, you need to enable (at least) debug logging and
start monitoring uevents, try to reproduce the issue and then analyze
and correlate two (usually) huge log files. This is not ideal.
This patch aims to provide much more focused debugging tool,
tracepoints. More often then not we tend to have at least the basic idea
about the issue we are trying to debug further, e.g. we know it is
storage related. Hence all of the debug data generated for network
devices is useless, adds clutter to the log files and generally
slows things down.
Using this set of tracepoints you can start asking very specific
questions related to event processing for given device or subsystem.
Tracepoints can be used with various tracing tools but I will provide
examples using bpftrace.
Another important aspect to consider is that using tracepoints you can
debug production systems. There is no need to install test packages with
added logging, no debuginfo packages, etc...
Example usage (you might be asking such questions during the debug session),
Q: How can I list all tracepoints?
A: bpftrace -l 'usdt:/usr/lib/systemd/systemd-udevd:udev:*'
Q: What are the arguments for each tracepoint?
A: Look at the code and search for use of DEVICE_TRACE_POINT macro.
Q: How many times we have executed external binary?
A: bpftrace -e 'usdt:/usr/lib/systemd/systemd-udevd:udev:spawn_exec { @cnt = count(); }'
Q: What binaries where executed while handling events for "dm-0" device?
A bpftrace -e 'usdt:/usr/lib/systemd/systemd-udevd:udev:spawn_exec / str(arg1) == "dm-0"/ { @cmds[str(arg4)] = count(); }'
Thanks to Thomas Weißschuh <thomas@t-8ch.de> for reviewing this patch
and contributions that allowed us to drop the dependency on dtrace tool
and made the resulting code much more concise.
In commit d895e10a a test was introduced to validate that prefix is a
child of rootprefix. However, it only works when rootprefix is "/".
Since the test is ignored when rootprefix is equal to prefix, this is
only noticed if specifying both -Drootprefix= and -Dprefix=, e.g.:
$ meson foo -Drootprefix=/foo -Dprefix=/foo/bar
meson.build:111:8: ERROR: Problem encountered: Prefix is not below
root prefix (now rootprefix=/foo prefix=/foo/bar)
On Debian, bpftool is installed in /usr/sbin, which is not in $PATH for
non-root users by default, so finding it fails.
Add a secondary, hard-coded '/usr/sbin/bpftool' after 'bpftool' so that
meson can find it.
https://packages.debian.org/sid/amd64/bpftool/filelist
This ensures that the fuzz test code is also built by default.
It also increases the test coverage a bit. Compiling the tests
*with* sanitizers is painfully slow, so this is not enabled. But
just compiling them sauté is hardly noticable. Running the tests
increases the test count and runtime:
622 tests, 26 s
to
922 tests, 35 s
I think this is acceptable.
This doesn't matter too much, but makes things a bit more consistent.
A minor advantage is that the file is not a configuration file for meson
anymore, so:
a) It is not built unless pulled in by another target. Since
we don't usually build man pages by default, this saves a tiny
amount of work.
b) When the .in file is updated, meson does not reconfigure everything,
but just rebuilds the dependent targets.
Now that the conversion is finished, time for benchmarking:
a full build with default settings (and -Dstandalonebinaries=true), yields
before this pull request: 1687 targets, 148.13s user 35.17s system 317% cpu 57.697 total
with the full pull request: 1714 targets, 143.07s user 27.87s system 314% cpu 54.369 total
The difference doesn't seem significant. Partial rebuilds might be faster as
mentioned before.
We had two big 'configuration_data' objects in meson config. (There are in fact
more. On is added in this series, and there's one for efi… But those others
have a handful variables only for specific purposes and don't matter). The two
sets are 'conf' and 'substs', and were inherited from the original autotools
system. In the past there was even a third set ('m4_defines'), but @yuwata
removed it in 348b44372f. And those two/three
systems had very similar data, but with different variable names, because of
historical reasons. They also used subtly different quoting (.set()
vs. .set10() vs. .set_quoted()), which was required because the templating
engines were not flexible enough. This meants we had more work when changing
things, and we needed to search for different variable names, etc.
With a more flexible templating engine we can do with just one
configuration_data object.
The naming of variables is very inconsistent. I tried to use more
modern style naming (UNDERSCORED_TITLE_CASE), but I didn't change existing
names too much. Only SYSTEM_DATA_UNIT_PATH is renamed to SYSTEM_DATA_UNIT_DIR
to match SYSTEM_CONFIG_UNIT_DIR.
I want to stop using 'substs'. But in this case, configure_file() is nicer
than custom_target(), because it causes meson to immediately generate the
helpers after configuration, so it's possible to do
'meson build && build/man/man ...', without building anything first.
We only substitute one variable here, so let's use a custom configuration_data()
object.
m4 was hugely popular in the past, because autotools, automake, flex, bison and
many other things used it. But nowadays it much less popular, and might not even
be installed in the buildroot. (m4 is small, so it doesn't make a big difference.)
(FWIW, Fedora dropped make from the buildroot now,
https://fedoraproject.org/wiki/Changes/Remove_make_from_BuildRoot. I think it's
reasonable to assume that m4 will be dropped at some point too.)
The main reason to drop m4 is that the syntax is not very nice, and we should
minimize the number of different syntaxes that we use. We still have two
(configure_file() with @FOO@ and jinja2 templates with {{foo}} and the
pythonesque conditional expressions), but at least we don't need m4 (with
m4_dnl and `quotes').
HAVE_SMACK_RUN_LABEL was dropped back in 348b44372f,
so one line in etc.conf was not rendered as expected ;(
Checking if names are defined is paying for itself!
We don't need two (and half) templating systems anymore, yay!
I'm keeping the changes minimal, to make the diff manageable. Some enhancements
due to a better templating system might be possible in the future.
For handling of '## ' — see the next commit.
m4 was nice in '85, but the syntax feels a bit dated. Since we use python for
meson, let's use a popular python templating engine to replace some m4 usage.
A little nicety is that typos are caught:
FAILED: sysusers.d/systemd-remote.conf
/usr/bin/meson --internal exe --capture sysusers.d/systemd-remote.conf -- /home/zbyszek/src/systemd/tools/meson-render-jinja2.py config.h ../sysusers.d/systemd-remote.conf.j2
Traceback (most recent call last):
File "/home/zbyszek/src/systemd/tools/meson-render-jinja2.py", line 28, in <module>
print(render(sys.argv[2], defines))
File "/home/zbyszek/src/systemd/tools/meson-render-jinja2.py", line 24, in render
return template.render(defines)
File "/usr/lib/python3.9/site-packages/jinja2/environment.py", line 1090, in render
self.environment.handle_exception()
File "/usr/lib/python3.9/site-packages/jinja2/environment.py", line 832, in handle_exception
reraise(*rewrite_traceback_stack(source=source))
File "/usr/lib/python3.9/site-packages/jinja2/_compat.py", line 28, in reraise
raise value.with_traceback(tb)
File "<template>", line 8, in top-level template code
jinja2.exceptions.UndefinedError: 'HAVE_MICROHTTP' is undefined
This checking mirrors what 349cc4a507 did for C defines.
Old meson fails with:
Element not a string: [<Holder: <ExternalProgram 'sh' -> ['/bin/sh']>>, '-c', 'test -n "$DESTDIR" || /bin/journalctl --update-catalog']
I'm doing it as a revert so that it's easy to undo the revert when we require
newer meson. The effect is not so bad, maybe a dozen or so lines about finding
'sh'.
Meson 0.58 has gotten quite bad with emitting a message every time
a quoted command is used:
Program /home/zbyszek/src/systemd-work/tools/meson-make-symlink.sh found: YES (/home/zbyszek/src/systemd-work/tools/meson-make-symlink.sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program xsltproc found: YES (/usr/bin/xsltproc)
Configuring custom-entities.ent using configuration
Message: Skipping bootctl.1 because ENABLE_EFI is false
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Message: Skipping journal-remote.conf.5 because HAVE_MICROHTTPD is false
Message: Skipping journal-upload.conf.5 because HAVE_MICROHTTPD is false
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Message: Skipping loader.conf.5 because ENABLE_EFI is false
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
...
Let's suffer one message only for each command. Hopefully we can silence
even this when https://github.com/mesonbuild/meson/issues/8642 is
resolved.
Today this is v248 with 938bdfc0fa, which,
if you don't know about the github webflow key fails to configure with
meson.build:724:8: ERROR: String "gpg: Signature made Tue 30 Mar 2021 22:59:02 CEST\ngpg: using RSA key 4AEE18F83AFDEB23\ngpg: Can't check signature: No public key\n1617137942\n" cannot be converted to int
or, if you do, with
meson.build:724:8: ERROR: String 'gpg: Signature made Tue 30 Mar 2021 22:59:02 CEST\ngpg: using RSA key 4AEE18F83AFDEB23\ngpg: Good signature from "GitHub (web-flow commit signing) <noreply@github.com>" [unknown]\ngpg: WARNING: This key is not certified with a trusted signature!\ngpg: There is no indication that the signature belongs to the owner.\nPrimary key fingerprint: 5DE3 E050 9C47 EA3C F04A 42D3 4AEE 18F8 3AFD EB23\n1617137942\n' cannot be converted to int
New glibc deprecated mallocinfo(), even newer glibc added mallocinfo2()
as replacement. Use it, if it exists.
Follow-up for 4b6f74f5a0 and related
commits.
https://github.com/systemd/systemd/pull/19316 failed with:
[1065/1670] Linking target systemd-hwdb
--- command ---
14:28:29 /root/src/test/hwdb-test.sh
--- stdout ---
./systemd-hwdb does not exist, please build first
I'm not sure what is going on here… In principle meson says that tests may be
called from any directory, but in practice is was always the build directory.
So far we were relying on systemd-hwdb being present in '.', and this worked.
Either way, it's nicer to pass the exact path, so let's do that.
* Add `bpf-framework` feature gate with 'auto', 'true' and 'false' choices
* Add libbpf [0] dependency
* Search for clang llvm-strip and bpftool binaries in compile time to
generate bpf skeleton.
For libbpf [0], make 0.2.0 [1] the minimum required version.
If libbpf is satisfied, set HAVE_LIBBPF config option to 1.
If `bpf-framework` feature gate is set to 'auto', means that whether
bpf feature is enabled or now is defined by the presence of all of
libbpf, clang, llvm and bpftool in build
environment.
With 'auto' all dependencies are optional.
If the gate is set to `true`, make all of the libbpf, clang and llvm
dependencies mandatory.
If it's set to `false`, set `BPF_FRAMEWORK` to false and make libbpf
dependency optional.
libbpf dependency is dynamic followed by the common pattern in systemd.
meson, bpf: add build rule for socket_bind program
Try to make this more manageable by reording:
- dependencies / inputs
(with subcategory of compression libraries)
- major components / outputs
- optional features / conditionals that don't fit into the two above categories
The division isn't well defined, because libraries often correspond one-to-one
to feature, but not always.
Let's assert if we ever happen to pass 0 to one of the log functions.
With the preceding commit to return -EIO from log_*(), passing 0 wouldn't
affect the return value any more, but it is still most likely an error.
The unit test code is an exception: we fairly often pass the return value
to print it, before checking what it is. So let's assert that we're not
passing 0 in non-test code. As with the previous check for %m, this is only
done in developer mode. We are depending on external code setting
errno correctly for us, which might not always be true, and which we can't
test, so we shouldn't assert, but just handle this gracefully.
I did a bunch of greps to try to figure out if there are any places where
we're passing 0 on purpose, and couldn't find any.
The one place that failed in tests is adjusted.
About "zerook" in the name: I wanted the suffix to be unambiguous. It's a
single "word" because each of the words in log_full_errno is also meaningful,
and having one term use two words would be confusing.
Using a enum is all nice and generic, but at this point it seems unlikely that
we'll add further build modes. But having an enum means that we need to include
the header file with the enumeration whenerever the conditional is used. I want
to use the conditional in log.h, which makes it hard to avoid circular imports.
We intentionally do not inline initializations with definitions for
a bunch of _cleanup_ variables in tests, to ensure valgrind is triggered.
This triggers a lot of maybe-uninitialized false positives when -O2 and
-flto are used. Suppress them.
The warning was disabled in 8794164fed to avoid
false positives. But it is useful in finding errors, even if it sometimes
results in untrue warnings (c.f. 77fac974fe, da46a1bc3c).
After #19168, #19169, and #19175, there are no warnings with
-Dbuildtype=debug-optimized/-O2 and gcc-11.0.1-0.3.fc34.x86_64. Warnings
are reenabled for -O[23]
-O0 is good for development, and -O2 is the default optimization level for
Fedora package builds. -Os, -O3, -O1, and -Og still generate some warnings. In
fact, with -Os the number of warnings seems completely hopeless. Dozens and
dozens.
With 8f20232fcb systemd-localed supports
generating locales when required. This fails if the locale directory is
read-only, so make it writable.
Closes#19138
sd-boot has a copy of a subset of codes from libbasic. This makes
sd-boot share the code with libbasic, and dedup the code.
Note, startswith_no_case() is dropped from sd-boot, as
- it is not used,
- the previous implementation is not correct,
- gnu-efi does not have StrniCmp() or so.
There is no technical reason to support systems with split-usr, except for
backwards compatibility. Even though systemd itself makes an effort to support
this, many other tools aren't as careful. Despite those efforts, we
(collectively) get it wrong often, because doing it "wrong" on systems with
merged-usr has no consequences. Since almost all developers are on such
systems, any issues are only discovered late. Supporting this split-usr mode
makes both code and documentation more complicated. The split is purely
artificial and has no justification except to allow old installation to not
update. Mechanisms to update existing systems are available though: Fedora
did that in https://fedoraproject.org/wiki/Features/UsrMove, Debian has
the usrmerge package.
The next version of Debian will only support systems with split-usr=false,
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978636#178:
The Technical Committee resolves that Debian 'bookworm' should
support only the merged-usr root filesystem layout, dropping support
for the non-merged-usr layout.
Let's start warning if split-usr mode is used, in preparation to removing the
split in one of the future releases.
The target is update-syscall-tables, so let's call the script
update-syscall-tables.sh to reduce the cognitive overhead when
trying to find the right file.
The script is renamed to match.
Now all targets are named uniformly in a tab-completion-friendly fashion, with
the exception of systemd-update-po which is generated by the i18n module
automatically:
$ ninja -C build -t targets | grep update
systemd-update-po: phony
update-syscall-tables: phony
update-syscall-header: phony
update-hwdb: phony
update-hwdb-autosuspend: phony
update-dbus-docs: CUSTOM_COMMAND
update-man-rules: CUSTOM_COMMAND
Very old versions of meson did not include the subdirectory name in the
target name, so we started adding various "top-level" custom targets in
subdirectories. This was nice because the main meson.build file wasn't
as cluttered. But then meson started including the subdir name in the
target name. So let's move the definition to the root so we can have all
targets named uniformly.
Before this commit, udevd is built with LOG_REALM=LOG_REALM_UDEV.
However, log level specified by e.g. environment variable or kernel
command line option are also passed to LOG_REALM_SYSTEMD. So, the
maximum log level for the two realms are always equivalent, and it is
not necessary to specify the build option. Hence drop it.
