1
0
mirror of https://github.com/systemd/systemd.git synced 2025-02-04 21:47:31 +03:00

64319 Commits

Author SHA1 Message Date
Zbigniew Jędrzejewski-Szmek
46886f130d test_ukify: add test for combining config and cmdline 2023-05-05 18:43:20 +02:00
Zbigniew Jędrzejewski-Szmek
248be6ef37 man: describe all the changes to ukify
As in mkosi(1), let's describe the config file and commandline options
together. This is nice for us, because we don't need to duplicate descriptions
and we're less likely to forget to update one place or the other. This is also
nice for users, because they can easily figure out what can be configured
where.

The options are now ordered by config file section.

--summary was not described before.

More examples are added.
2023-05-05 18:43:20 +02:00
Zbigniew Jędrzejewski-Szmek
bac18826e9 ci: install pytest-flakes
Some web searches say that it's packaged for those distros and not the others…

v2:
- drop arch. https://aur.archlinux.org/packages/python-pytest-flakes exists,
  but installation fails in CI.
2023-05-05 18:42:37 +02:00
Zbigniew Jędrzejewski-Szmek
55be961f48 test_ukify: rework how --flakes argument is appended
The usual approach is to put 'addopts = --flakes' in setup.cfg. Unfortunately
this fails badly when pytest-flakes is not installed:
  ERROR: usage: test_ukify.py [options] [file_or_dir] [file_or_dir] [...]
  test_ukify.py: error: unrecognized arguments: --flakes

pytest-flakes is not packaged everywhere, and this test is not very important,
so let's just do it only if pytest-flakes is available. We now detect if
pytest-flakes is available and only add '--flakes' conditionally. This
unfortunately means that when invoked via 'pytest' or directly as
'src/ukify/test/test_ukify.py', '--flakes' will not be appended automatically.
But I don't see a nice way to achieve previous automatic behaviour.

(I first considered making 'setup.cfg' templated. But then it is created
in the build directory, but we would need it in the source directory for
pytest to load it automatically. So to load the file, we'd need to give an
argument to pytest anyway, so we don't gain anything with this more complex
approach.)
2023-05-05 18:42:37 +02:00
Zbigniew Jędrzejewski-Szmek
041f536f9a test_ukify: propagate failure
Oops. This explains why the tests were "passing" in CI even
though a direct pytest invocation would fail.
2023-05-05 18:42:37 +02:00
Zbigniew Jędrzejewski-Szmek
a758f95c33 ukify: appease mypy
Note to self: PEP 585 introduced using collection types as types,
and is available since 3.9. PEP 604 allows writing unions with "|",
but is only available since 3.10, so not yet here because we maintain
compat with 3.9.
2023-05-05 18:42:37 +02:00
Zbigniew Jędrzejewski-Szmek
a4b329e6aa TODO: remove two entries
0ccfd3564b2532a4da6526a9e030362c4a142b77 implemented one of the items, and this
pull requests handles the other one.
2023-05-05 18:42:37 +02:00
Zbigniew Jędrzejewski-Szmek
3f80c1393e test/60-ukify: override stub location in tests
Without this, build would fail if the stub is not available in /usr/lib/.
2023-05-05 18:42:37 +02:00
Zbigniew Jędrzejewski-Szmek
f9a6cb0e13 test-kernel-install: test 60-ukify.install and 90-uki-copy.install
We install a kernel with layout=uki and uki_generator=ukify, and test
that a UKI gets installed in the expected place. The two plugins cooperate,
so it's easiest to test them together.
2023-05-05 18:42:37 +02:00
Zbigniew Jędrzejewski-Szmek
ca1abaa5c4 60-ukify: kernel-install plugin that calls ukify to create a UKI
60-ukify.install calls ukify with a config file, so singing and policies and
splash will be done through the ukify config file, without 60-ukify.install
knowing anything directly.

In meson.py, the variable for loaderentry.install.in is used just once, let's
drop it. (I guess this approach was copied from kernel_install_in, which is
used in another file.)

The general idea is based on cvlc12's #27119, but now in Python instead of
bash.
2023-05-05 18:42:37 +02:00
Zbigniew Jędrzejewski-Szmek
47a6df4da0 test_ukify: add tests for the new functionality 2023-05-05 18:42:37 +02:00
Zbigniew Jędrzejewski-Szmek
7081db294c ukify: PeError → PEError
We don't lowercase acronyms in systemd usually.
Remove unnused f'' prefix to avoid a pylint warning.
2023-05-05 18:42:37 +02:00
Zbigniew Jędrzejewski-Szmek
5143a47a81 ukify: rework option parsing to support a config file
In some ways this is similar to mkosi: we have a argparse.ArgumentParser()
with a bunch of options, and a configparser.ConfigParser() with an
overlapping set of options. Many options are settable in both places, but
not all. In mkosi, we define this in three places (a dataclass, and a
function for argparse, and a function for configparser). Here, we have one
huge list of ConfigItem instances. Each instance specifies the full metadata
for both parsers. Argparse generates a --help string for all the options,
and we also append a config file sample to --help based on the ConfigItem
data:

$ python src/ukify/ukify.py --help|tail -n 25
config file:
  [UKI]
  Linux = LINUX
  Initrd = INITRD…
  Cmdline = TEXT|@PATH
  OSRelease = TEXT|@PATH
  DeviceTree = PATH
  Splash = BMP
  PCRPKey = KEY
  Uname = VERSION
  EFIArch = ia32|x64|arm|aa64|riscv64
  Stub = STUB
  PCRBanks = BANK…
  SigningEngine = ENGINE
  SecureBootPrivateKey = SB_KEY
  SecureBootCertificate = SB_CERT
  SignKernel = SIGN_KERNEL

  [PCRSignature:NAME]
  PCRPrivateKey = PATH
  PCRPublicKey = PATH
  Phases = PHASE-PATH…

While writing this I needed to check the argument parsing, so I added
a --summary switch. It just pretty-prints the resulting option dictionary:

$ python src/ukify/ukify.py /efi//3a9d668b4db749398a4a5e78a03bffa5/6.2.11-300.fc38.x86_64/linux /efi//3a9d668b4db749398a4a5e78a03bffa5/6.2.11-300.fc38.x86_64/initrd --pcr-private-key=PRIV.key --pcr-public-key=PUB.key --config=man/ukify-example.conf --summary
Host arch 'x86_64', EFI arch 'x64'
{'_groups': [0, 'initrd', 'system'],
 'cmdline': 'A1 B2 C3',
 'config': 'man/ukify-example.conf',
 'devicetree': None,
 'efi_arch': 'x64',
 'initrd': [PosixPath('initrd1'),
            PosixPath('initrd2'),
            PosixPath('initrd3'),
            PosixPath('/efi/3a9d668b4db749398a4a5e78a03bffa5/6.2.11-300.fc38.x86_64/initrd')],
 'linux': PosixPath('/efi/3a9d668b4db749398a4a5e78a03bffa5/6.2.11-300.fc38.x86_64/linux'),
 'measure': None,
 'os_release': PosixPath('/etc/os-release'),
 'output': 'linux.efi',
 'pcr_banks': ['sha1', 'sha384'],
 'pcr_private_keys': [PosixPath('PRIV.key'),
                      PosixPath('pcr-private-initrd-key.pem'),
                      PosixPath('pcr-private-system-key.pem')],
 'pcr_public_keys': [PosixPath('PUB.key'),
                     PosixPath('pcr-public-initrd-key.pem'),
                     PosixPath('pcr-public-system-key.pem')],
 'pcrpkey': None,
 'phase_path_groups': [None,
                       ['enter-initrd'],
                       ['enter-initrd:leave-initrd',
                        'enter-initrd:leave-initrd:sysinit',
                        'enter-initrd:leave-initrd:sysinit:ready']],
 'sb_cert': PosixPath('mkosi.secure-boot.crt'),
 'sb_key': PosixPath('mkosi.secure-boot.key'),
 'sections': [],
 'sign_kernel': None,
 'signing_engine': None,
 'splash': None,
 'stub': PosixPath('/usr/lib/systemd/boot/efi/linuxx64.efi.stub'),
 'summary': True,
 'tools': None,
 'uname': None}

With --summary, existence of input paths is not checked. I think we'll
want to show them, instead of throwing an error, but in red, similarly to
'bootctl list'.

This also fixes tests which were failing with e.g.
E       FileNotFoundError: [Errno 2] No such file or directory: '/ARG1'
=========================== short test summary info ============================
FAILED ../src/ukify/test/test_ukify.py::test_parse_args_minimal - FileNotFoun...
FAILED ../src/ukify/test/test_ukify.py::test_parse_args_many - FileNotFoundEr...
FAILED ../src/ukify/test/test_ukify.py::test_parse_sections - FileNotFoundErr...
=================== 3 failed, 10 passed, 3 skipped in 1.51s ====================
2023-05-05 18:42:03 +02:00
Zbigniew Jędrzejewski-Szmek
3f7e77fae1 test_ukify: fix two failing tests
Fixup for 22ad038ac6e4fe5e4a68555f0e70bd0a16fb5616 and
3fc5eed47091363247012454df458e1a3303bf12. It seems that the tests are
not executed properly in CI. Nevertheless, test-ukify appears in logs:

rpm-build:fedora-rawhide-x86_64:
 409/1191 systemd / test-ukify    OK   0.16s

This is strange.
2023-05-03 12:29:41 +02:00
Zbigniew Jędrzejewski-Szmek
cb3b451e11 test_ukify: fix loop iteration
We'd try to access 'linux' or 'initrd' after failing to set it.
2023-05-03 12:29:41 +02:00
Zbigniew Jędrzejewski-Szmek
b62ee354dd meson: allow building .standalone on demand
We can always build the standalone version whenever we build the normal version
(the dependencies are the same). In most builds standalone binaries would be
disabled. But it is occasionally useful to have them for testing, so move the
conditional to install:, so the binaries can be build by giving the explicit
target name.

The default of 'build_by_default' for executable() is sadly true (since meson
0.38.0), so need to specify build_by_default: too.

Also add systemd-shutdown.standalone to public_programs for additional testing.
2023-05-03 12:29:41 +02:00
Zbigniew Jędrzejewski-Szmek
9a01fe3906 meson: avoid building executables that won't be installed
When executable() or custom_target() has install: that is conditional as is
false (i.e. not install:true), it won't be built by default. (build_by_default:
defaults to install:). But if that program is added to public_programs, it will
be build by default because it is pulled in by the test, effectively defeating
the disablement.

While at it, make 'ukify' follow the same pattern as 'kernel-install'.
They will be used later together.
2023-05-03 12:29:41 +02:00
Zbigniew Jędrzejewski-Szmek
d9c8f075af ukify: add missing header
This file is installed, so it should have the long header.
2023-05-03 12:29:41 +02:00
Zbigniew Jędrzejewski-Szmek
50f4add445 ukify: use UPPERCASE for parameter names
We generally nowadays use UPPERCASE for parameters in variuos help text.
Let's be consistent here too, and also drop duplicated 'usage:':
$ ukify -h
usage: ukify [options…] LINUX INITRD…
       ukify -h | --help

Build and sign Unified Kernel Images

positional arguments:
  LINUX                 vmlinuz file [.linux section]
  INITRD…               initrd files [.initrd section]
...
2023-05-03 12:29:41 +02:00
Zbigniew Jędrzejewski-Szmek
d3533162d1 90-loaderentry: make sure that variables are set
We unconditionally use the variables later on, so let's make sure
that they were passed as expected.
2023-05-03 12:29:40 +02:00
Yu Watanabe
406004a6c3
Merge pull request #27499 from yuwata/sd-journal-fix-loop
sd-journal: check .next_entry_array_offset earlier
2023-05-03 13:30:47 +09:00
Yu Watanabe
de6eb806ff Revert "sd-journal: introduce simple loop detection for entry array objects"
This reverts commit a8fbcc0e3c033a43e511550052cace6b0dcf3df7.

The commit is not necessary, as the invalid entry array object is
filtered earlier by the previous commit.
2023-05-03 06:41:25 +09:00
Yu Watanabe
b5335da7a5 sd-journal: check .next_entry_array_offset earlier
Then, if it is invalid, refuse to use the entry array object.

Follow-up for a8fbcc0e3c033a43e511550052cace6b0dcf3df7.
Fixes #27489.
2023-05-03 06:41:21 +09:00
Jan Janssen
ec232e4abd boot: Use correct memory type for allocations
We were using the wrong memory type when allocating pool memory. This
does not seem to cause a problem on x86, but the kernel will fail to
boot at least on ARM in QEMU.

This is caused by mixing different allocation types which ended up
breaking the kernel or EDK2 during boot services exit. Commit
2f3c3b0bee5534f2338439f04b0aa517479f8b76 appears to fix this boot
failure because it was replacing the gnu-efi xpool_print with xasprintf
thereby unifying the allocation type.
But this same issue can also happen without this fix somehow when the
random-seed logic is in use.

Fixes: #27371
2023-05-02 22:38:15 +01:00
msizanoen1
845824acdd core: check for SERVICE_RELOAD_NOTIFY in manager_dbus_is_running
This ensures that systemd won't erronously disconnect from the system
bus in case a bus recheck is triggered immediately after the bus service
emits `RELOADING=1`.

This fixes an issue where systemd-logind sometimes randomly stops
receiving `UnitRemoved` after a system update.

This also handles SERVICE_RELOAD_SIGNAL just in case somebody ever
creates a D-Bus broker implementation that uses `Type=notify-reload`.
2023-05-03 00:14:35 +09:00
Lennart Poettering
5789c60973 journal: handle EADDRNOTAVAIL in two more cases gracefully
Follow-up for #27488.
2023-05-03 00:13:40 +09:00
Yu Watanabe
c19b827802
Merge pull request #27493 from poettering/generate-sym-test-tweaks
fixes/additions for generate-sym-test
2023-05-03 00:12:48 +09:00
Mike Yuan
5e15fec23d
Merge pull request #27492 from poettering/base-filesystem-000
base-filesystem: create /proc, /sys, /dev mount points as 555
2023-05-02 20:53:35 +08:00
Luca Boccassi
b8fba0cded generators: skip private tmpfs if /tmp does not exist
When spawning generators within a sandbox we want a private /tmp, but it
might not exist, and on some systems we might be unable to create it
because users want a BTRFS subvolume instead.

Fixes https://github.com/systemd/systemd/issues/27436
2023-05-02 12:28:09 +02:00
Daan De Meyer
a3b076f641 tree-wide: Handle EADDRNOTAVAIL as journal corruption
Journal corruption is not only indicated by EBADMSG but also by
EADDRNOTAVAIL so treat that as corruption in a few more cases.
2023-05-02 12:17:41 +02:00
Lennart Poettering
2a10774caa mount-setup: use size_t when iterating through array indexes 2023-05-02 12:06:03 +02:00
Lennart Poettering
7243cefec1 base-filesystem: create /proc, /sys, /dev mount points as 0555
These inodes are going to be overmounted anyway, hence let's create them
with access mode 555, so that they are as close to being immutable as
regular UNIX access modes allow them to be. In other words: this takes
the "w" mode away for root. This of course usually has little effect --
unless CAP_DAC_OVERRIDE is dropped. But at the very least it makes the
point clear that inodes should be considered immutable.

(I intended to make this 0000 originally, but that doesn't work, as many
tools – including our own – have fallback paths that when they see
ENOENT in /proc/ they can handle this gracefully. But changing the mode
to 000 would turn this to EACCES - something they usually have no
fallback path for)
2023-05-02 12:06:03 +02:00
Lennart Poettering
6d3c20e1ff generate-sym-test: only look in .c and .h files
Otherwise it might find backup files and such, which triggered all kinds
of false positives (at least on my devel machine).
2023-05-02 11:30:51 +02:00
Lennart Poettering
4f65fbe990 generate-sym-test: search for missing symbols
This slightly extends the symbol file test and checks which symbols are
listed in one list but missing in the other. This is tremendously useful
to quickly determine which symbols wheren't exposed properly but should
have been.

(This is is implemented in pure C, no systemd helpers, to ensure we see
libsystemd.so API as any other tool would.)
2023-05-02 11:30:46 +02:00
Klaus
ab13274a19
docs: add correct pacman command (#27486)
The `pacman` command in order to install packages on Arch in the documentation is invalid.
This PR fixes the command.
2023-05-02 17:18:10 +08:00
Yu Watanabe
e94756c566 test: replace sleep with timeout
If the test environment is too slow, then sleeping 2 seconds may not be
sufficient.
2023-05-02 11:09:18 +02:00
Yu Watanabe
ecf82f74b6
Merge pull request #27455 from yuwata/test-lib-sym
test: check all public functions are listed in .sym file
2023-05-02 04:45:30 +09:00
Luca Boccassi
67713e37e6
Merge pull request #27442 from DaanDeMeyer/presets
mkosi: Switch to use mkosi presets with prebuilt initrds
2023-05-01 20:10:58 +01:00
Yu Watanabe
8e2449a5c9 test-network: add workaround for bug in iproute2 v6.2.0
Closes #27473.
2023-05-01 20:02:15 +02:00
Jan Janssen
669f16acf7 stub: Relocate kernels below 4G for EFI handover
Old kernels can fail to boot when they are located above the 4G
boundary even if they claim to support it.

Fixes: #27472
2023-05-01 19:08:12 +02:00
Jan Janssen
6fd3ee6916 boot: Fix EFI_SIZE_TO_PAGES macro 2023-05-01 19:08:12 +02:00
Daan De Meyer
d052cc8893 mkosi: Switch to use mkosi presets with prebuilt initrds
Instead of building the initrds for the mkosi images with dracut,
let's switch to using mkosi presets to build the initrd with mkosi
as well.

This commit splits up our single image build into three separate
mkosi presets:

1. The "base" preset. This image contains systemd and all its runtime
dependencies. The sole purpose of this image is to serve as a base image
for the initrd and the final image. It's also responsible for building
systemd from source with the build script. The results are installed into
the base image. Note that we install the systemd and udev packages into this
image as well to prevent package managers from overriding the systemd we built
from source with the distro packaged systemd if it's pulled in as a dependency
by another package from the initrd or final profiles.
2. The "initrd" preset. This image provides the initrd. It's trivial and does
nothing more than packaging the base image up as a zstd compressed initramfs and
adds /init and /etc/initrd-release symlinks to the image.
3. The "final" preset. This image builds on top of the base image and adds
a kernel and extra packages that are useful for testing and debugging.

We also split out the optional kernel build into a separate set of config files
that are only included if a kernel to build is actually provided.

Note that this commit doesn't really change anything about how mkosi is used.
The commands remain the same, except that mkosi will now build all the presets
in order. "mkosi summary" will show the summary of all the presets. "mkosi qemu,
boot, shell" will always boot the final preset. With "-f", all presets will be
built and the final one is booted. "-i" makes a cache of each preset.

The only thing to keep in mind is that specifying config via the mkosi CLI will
apply to each of the presets. e.g. any extra packages added with "-p" will be
installed in both the initrd and the final image. To apply local configuration
to a single preset, create a file 00-local.conf in
mkosi.presets/<profile>/mkosi.conf.d and put all the preset specific configuration
in there.
2023-05-01 15:39:50 +02:00
Daan De Meyer
d9081eb0ae meson: Search for find program
find is required by a few of the unit tests so let's make sure its
available when configuring the build.
2023-05-01 14:58:08 +02:00
Daan De Meyer
4d56d084d8 meson: Search for diff program
diff is required by a few of the unit tests so let's make sure its
available when configuring the build.
2023-05-01 14:58:08 +02:00
Yu Watanabe
5c519a56f5
Merge pull request #27458 from mrc0mmand/test-corrupted-journals
test: test journalctl with corrupted journals
2023-05-01 19:14:22 +09:00
Frantisek Sumsal
0f69a4e9ae test: test journalctl with corrupted journals
Last month I monkey-patched journald to produce a small (64K) but valid
journal and used that as an input to four AFL fuzzers. After a month it
generated quite a nice corpora (4738 test cases) and after filtering
and minimizing it I was left with 619 unique journals with various
levels of corruption that probe the journal code.

It seems to detect past issues like systemd#26567, etc.
2023-05-01 10:14:12 +02:00
Frantisek Sumsal
de965f4893 test: clean up the test script a bit 2023-05-01 15:47:04 +09:00
Yu Watanabe
a8fbcc0e3c sd-journal: introduce simple loop detection for entry array objects
If .next_entry_array_offset points to one of the previous entry or the
self entry, then the loop for entry array objects may run infinitely.
Let's assume that the offsets of each entry array object are in
increasing order, and check that in loop.

Fixes #27470.
2023-05-01 15:46:59 +09:00
Yu Watanabe
4c1b6e699f test: also test all _public_ functions are listed in .sym files
Co-authored-by: Frantisek Sumsal <frantisek@sumsal.cz>
2023-05-01 15:40:18 +09:00
Yu Watanabe
e758735d33 sd-journal: tighten variable scope 2023-05-01 14:25:38 +09:00