1
0
mirror of https://github.com/systemd/systemd.git synced 2024-12-22 17:35:35 +03:00
Commit Graph

275 Commits

Author SHA1 Message Date
Daan De Meyer
6a88774c96 mkosi: Use env where appropriate 2024-06-03 12:52:48 +02:00
Daan De Meyer
c397b5c701 mkosi: drop Debian /tmp hack
/tmp is now a tmpfs in debian testing so let's drop our workaround.
2024-06-03 12:43:20 +02:00
Daan De Meyer
e9ef4d6c7f mkosi: Add missing [Content] section 2024-06-03 12:29:23 +02:00
Daan De Meyer
51018ed1c2 mkosi: Make system dependencies conditional on format
If the none format is specified, we don't use the extra trees at all
so let's make the dependencies on the subimages conditional on Format=
not being set to "none".

This speeds up builds when using Format=none.
2024-06-03 11:51:08 +02:00
Daan De Meyer
f9dd837104 mkosi: Filter out sdubby and grubby from fedora rpm dependencies
When we query dependencies with rpmspec, we get the following:

"""
bash-5.2# mkosi-chroot rpmspec --with upstream --query --requires --define "_topdir /var/tmp" --define "_sourcedir pkg/fedora" "pkg/fedora/systemd.spec" | sort --unique
(grubby > 8.40-72 if grubby)
(sdubby > 1.0-3 if sdubby)
(systemd-rpm-macros = 256~rc3-1.fc41 if rpm-build)
(util-linux-core or util-linux)
/bin/sh
coreutils
dbus >= 1.9.18
firewalld-filesystem
grep
kbd
kmod >= 18-4
libidn2.so.0()(64bit)
libidn2.so.0(IDN2_0.0.0)(64bit)
libkmod.so.2()(64bit)
libkmod.so.2(LIBKMOD_5)(64bit)
openssl-libs
python3dist(cryptography)
python3dist(pefile)
python3dist(psutil)
python3dist(zstd)
systemd = 256~rc3-1.fc41
systemd(x86-64) = 256~rc3-1.fc41
systemd-boot
systemd-libs(x86-64) = 256~rc3-1.fc41
systemd-networkd = 256~rc3-1.fc41
systemd-pam(x86-64) = 256~rc3-1.fc41
systemd-udev = 256~rc3-1.fc41
"""

Trying to install with dnf5 correctly filters out grubby and sdubby
as they are conditional dependencies and shouldn't be installed. However,
dnf doesn't do the right thing and tries to install both grubby and sdubby,
and since they conflict this causes the build to fail.

Let's filter out sdubby and grubby explicitly to work around the bug in dnf
as it's unlikely to get fixed since all development effort is now focused on
dnf5.
2024-05-31 23:46:58 +02:00
Daan De Meyer
f4a0dd0ef6
Merge pull request #33105 from DaanDeMeyer/sanitizers
Sanitizers
2024-05-31 19:44:05 +02:00
Daan De Meyer
8919f86f57 mkosi: Sanitizer improvements
- Let's set the environment on the kernel command line so it applies
to initrd and main system.
- Let's add the necessary wrappers that are also added in test-functions.
Unlike test-functions we don't use gcc/clang to get the library path as
that requires installing gcc/clang in the initrd.
- Let's drop the hack to get journald writing to the console and have
it write to kmsg instead. We'll get the output either way.
- Stop removing libstdc++ and sanitizer libraries from Arch Linux
initrds and other images as it's required by the sanitizer libraries.
- Add a workaround for specifying extra meson options for opensuse
- Add a leak sanitizer suppression file as a workaround for a false
positive leak in verify_selinuxmnt() in libselinux. We do a soname match
because the stacktrace can't be properly symbolized on Debian.
2024-05-31 17:26:13 +02:00
Zbigniew Jędrzejewski-Szmek
00d359e988 Revert "mkosi: Disable add-determinism on Fedora"
This reverts commit e74a994ebf.

The description in that commit was confused, add-determinism was not broken
and was not causing any problems, except a log message.
2024-05-31 13:34:58 +02:00
Daan De Meyer
1baea86287 mkosi: Override rpm's _fixperms script to be a noop
Since rpm 4.20 it tries to fix the permissions of all source files
in the sources. This takes forever and fails in our case with a
permission error. Since we don't want rpm touching our source files,
override the script to be a noop.
2024-05-30 22:19:00 +02:00
Daan De Meyer
e74a994ebf mkosi: Disable add-determinism on Fedora
This was added for reproducible builds which we don't really care
about when building rpms in mkosi, so disable it from running as it's
broken and breaks our rpm builds.
2024-05-30 22:17:29 +02:00
Daan De Meyer
3b046883ad mkosi: Skip debuginfo workaround on newer rpm
The corresponding bug was fixed in rpm 4.20 (of which the alpha is in
rawhide as rpm 4.19.91) so skip the workaround when we detect a newer
rpm version.
2024-05-30 22:16:13 +02:00
Daan De Meyer
8c335463d9 mkosi: Don't install wireguard-tools on Debian
Package was removed from testing. This can be reverted next week once
it's added back to testing.
2024-05-30 20:54:06 +02:00
Daan De Meyer
ba592dc715 mkosi: Replace submodules with our own thing
Unfortunately, git submodules break in all sorts of ways:

- Various github workflows (dependabot, github pages) try to do a shallow
clone of git submodules which does not work at all when the git repository
is hosted on pagure (https://pagure.io/pagure/issue/5453,
https://github.com/dependabot/dependabot-core/issues/9391).
- If the git forge hosting the git repository uses SHA256, then it breaks our
usage of it as a submodule as SHA256 repositories cannot be used as submodules
in SHA1 repositories (src.opensuse.org moved to SHA256 which broke our usage of
opensuse's systemd spec as a submodule).
- git submodules completely break usage of git worktrees.
- ...

Let's avoid all these issues by just doing our own home grown implementation of
git submodules. We lose the automatic dependabot updates this way but since dependabot
fails to run more often that not with submodules we don't really lose anything.
2024-05-30 19:31:32 +02:00
Daan De Meyer
074ac66e88 Revert "mkosi: Sanitizer improvements"
This reverts commit aef13ad029.
2024-05-30 14:50:21 +02:00
Daan De Meyer
aef13ad029 mkosi: Sanitizer improvements
- Let's set the environment on the kernel command line so it applies
to initrd and main system.
- Let's add the necessary wrappers that are also added in test-functions.
Unlike test-functions we don't use gcc/clang to get the library path as
that requires installing gcc/clang in the initrd.
- Let's drop the hack to get journald writing to the console and have
it write to kmsg instead. We'll get the output either way.
- Stop removing libstdc++ and sanitizer libraries from Arch Linux
initrds and other images as it's required by the sanitizer libraries.
- Add a workaround for specifying extra meson options for opensuse
- Add a leak sanitizer suppression file as a workaround for a false
positive leak in verify_selinuxmnt() in libselinux. We do a soname match
because the stacktrace can't be properly symbolized on Debian.
2024-05-30 12:47:45 +02:00
Daan De Meyer
fe2a793b2d mkosi: Stop installing dbus-broker on OpenSUSE
dbus-broker and dbus-daemon have not been made interchangable on
OpenSUSE so we currently end up with dbus-broker used for the system
bus and dbus-daemon for the session bus. Let's stick to dbus-daemon
on OpenSUSE until they switch to dbus-broker.
2024-05-30 09:59:30 +02:00
Daan De Meyer
9b112737fa mkosi: Install bpftrace 2024-05-30 09:59:30 +02:00
Daan De Meyer
556659adfe mkosi: Switch back to cloud/KVM kernel packages
Now that we don't need megasas2 anymore, let's switch back to the
KVM/cloud kernel images to save on image size, build times and boot
times.
2024-05-29 15:24:03 +02:00
Daan De Meyer
14aea4a9f5 mkosi: Disable iscsi service and socket
We'll always start these on demand in integration tests that need
them. No need to start them by default.
2024-05-29 14:10:50 +02:00
Daan De Meyer
3c367df7c6 mkosi: Switch from btrfs to ext4
Mounting multiple btrfs filesystems with the same fsid only works
properly from kernel 6.7 onwards. Let's switch to ext4 for now which
does support this.
2024-05-29 14:10:50 +02:00
Daan De Meyer
6448993a4b mkosi: Set EXTRA_CFLAGS on opensuse
The opensuse spec doesn't unconditionally set FORTIFY_SOURCE=2 anymore
so let's drop our workaround.
2024-05-29 11:26:06 +02:00
Luca Boccassi
386dc7d0fd mkosi: explicitly install g++ in opensuse
Not pulled in by default at build time
2024-05-21 19:07:39 +01:00
Yu Watanabe
90fe8cf209 test: install more kernel modules for Fedora image
Otherwise, several tests for networkd are skipped.

Follow-up for #32666.
2024-05-20 18:38:45 +09:00
Daan De Meyer
e182c7b4db test: Enable TEST-69-SHUTDOWN for mkosi
In mkosi, we run the test inside the VM instead of outside. To simplify
the implementation we drop the reboot part and only verify that we can
schedule and cancel shutdowns and that the wall messages are sent as
expected.
2024-05-16 18:21:32 +02:00
Daan De Meyer
4ac46561e9 test: Enable TEST-24-CRYPTSETUP for mkosi
Encrypted /var is skipped because meson's limitations make per test
images not really feasible and we can't encrypt /var by default because
it slows down the image build too much.

Co-authored-by: Richard Maw <richard.maw@codethink.co.uk>
2024-05-16 14:46:37 +02:00
Daan De Meyer
b13b7978e7 test: Enable TEST-08-INITRD on mkosi 2024-05-15 10:09:53 +02:00
Daan De Meyer
985ea31173
Merge pull request #32666 from DaanDeMeyer/mkosi-network
test: Add TEST-85-NETWORK to run systemd-networkd-tests.py
2024-05-15 07:30:04 +02:00
Daan De Meyer
9e262ef92e test: Enable TEST-54-CREDS on mkosi 2024-05-14 23:19:28 +02:00
Daan De Meyer
16585dbaf4 mkosi: Disable journald rate-limiting 2024-05-14 22:58:28 +02:00
Daan De Meyer
5692f87160 test: Run TEST-64-UDEV-STORAGE with mkosi
We add a configure script for each subtest to add the required
qemu arguments.

Co-authored-by: Richard Maw <richard.maw@codethink.co.uk>
2024-05-14 12:43:28 +02:00
Daan De Meyer
eb4c962aa4 test: Ignore configure scripts in minimal images
These are always aimed at the system image so ignore any specified
configure scripts in the minimal images.
2024-05-14 12:43:28 +02:00
Daan De Meyer
f6af2976aa TEST-06-SELINUX: Simplify auto-relabeling
Let's ship a .autorelabel file so we can get rid of
firstboot-autorelabel.service.
2024-05-14 12:43:28 +02:00
Daan De Meyer
7a321b5a21 test: Rename testsuite-XX units to match test name
Having these named differently than the test itself mostly creates
unecessary confusion and makes writing logic against the tests harder
so let's rename the testsuite-xx units and scripts to just use the
test name itself.
2024-05-14 12:43:28 +02:00
Daan De Meyer
8f59b036de mkosi: Install dhcp-server on OpenSUSE
Required for systemd-networkd-tests.py.
2024-05-06 20:21:32 +02:00
Daan De Meyer
2c5dc1df1d mkosi: Install python3-psutil
Required to run systemd-networkd-tests.py. Already a dependency of
systemd-tests on Fedora so we don't add it there as it'll be pulled
in via the prepare script.
2024-05-06 18:15:54 +02:00
Daan De Meyer
15526060cc mkosi: Install radvd
Required for networkd integration tests.
2024-05-06 18:15:54 +02:00
Daan De Meyer
fdf3820e54 mkosi: Configure coredump to store coredumps in the journal.
Makes it easier to debug coredumps that happen in CI as we can retrieve
them via the journal.
2024-05-06 14:44:43 +02:00
Daan De Meyer
71b7156bd3 mkosi: Add missing SPDX line 2024-05-06 14:44:27 +02:00
Daan De Meyer
21e9282bc2 mkosi: Only look for services in /usr and /etc 2024-05-06 14:40:29 +02:00
Daan De Meyer
bda54ae03f mkosi: Skip build if NO_BUILD is set
This allows running the integration tests against the distro's packaged
version of systemd.
2024-05-06 14:40:26 +02:00
Daan De Meyer
fc4bac8162 mkosi: Add dependency to system image on minimal-base
The output is included as an extra tree so it should be a dependency,
even if it is pulled in transitively via minimal-0 and minimal-1
already.
2024-05-06 10:56:45 +02:00
Daan De Meyer
13686c0e58 mkosi: Install dbus policy required by TEST-23-UNIT-FILE
Co-authored-by: Richard Maw <richard.maw@codethink.co.uk>
2024-05-06 10:56:45 +02:00
Daan De Meyer
6817910e6f mkosi: Remove /etc/default/keyboard if it exists
Required to make TEST-73-LOCALE pass on Ubuntu.
2024-05-06 10:56:45 +02:00
Daan De Meyer
7cf0efd8ec mkosi: Make sure tmp.mount is not messed with on Debian/Ubuntu
We want /tmp to be a tmpfs so let's hack the debian packaging to
make sure that's the case until the debian packaging is fixed to
make /tmp a tmpfs for UPSTREAM=1 builds.
2024-05-06 10:56:45 +02:00
Daan De Meyer
5cb66ff5a1 mkosi: Install stress
Required for TEST-55-OOMD. Not available on opensuse.
2024-05-06 10:56:45 +02:00
Daan De Meyer
3c3e21fb64 mkosi: Install veritysetup
Only missing on CentOS/Fedora/OpenSUSE as in Arch/Debian/Ubuntu it's
part of the cryptsetup package which we already install.

Required for TEST-58-REPART.
2024-05-06 10:56:45 +02:00
Daan De Meyer
a82d897c3d mkosi: Install dig
Required for TEST-75-RESOLVED
2024-05-06 10:56:45 +02:00
Daan De Meyer
2d93e534a3 mkosi: Install knot
Required for TEST-75-RESOLVED
2024-05-06 10:56:45 +02:00
Daan De Meyer
efbd22e476 mkosi: Use /etc/nsswitch.conf from repo in mkosi image 2024-05-06 10:56:45 +02:00
Daan De Meyer
f09947661c mkosi: Move sbsigntools to CentOS/Fedora shared configuration
sbsigntools was added to EPEL 9.
2024-05-06 10:56:45 +02:00
Daan De Meyer
9bf888faf1 mkosi: Drop glibc-langpack-en from Fedora specific packages
It's already added in the CentOS/Fedora shared configuration.
2024-05-06 10:56:45 +02:00
Daan De Meyer
89255c31a4 mkosi: Install dfuzzer on CentOS/Fedora images
Required for TEST-21-DFUZZER.
2024-05-06 10:56:45 +02:00
Richard Maw
e26efe09f3 mkosi: Disable selinux labelling and install policy in initramfs
It is necessary to install the selinux policy in the initramfs
so that userland is entered with the correct label.

SELinuxRelabel defaults to auto, which will skip if the relabelling
command is not installed and will treat failure to relabel as non-fatal.

We can't force it on because root privileges are required if the labels
don't exist on the host system and we would like to be able to
cross-build from other distributions.

Since we are already committed to relabelling on first boot
there is no value in even trying to label.
2024-05-03 12:57:11 +01:00
Richard Maw
a56820d16a mkosi: Fix particle profile
The sbsign and ukify part was moved into a finalize script
rather than a postinst.chroot script so was using host tools
instead of just-built.
2024-05-03 13:49:09 +02:00
Daan De Meyer
f991c3871b mkosi: Fix %__check_files eval in opensuse build script
Make sure the correct buildroot gets expanded instead of the default
opensuse build root location.

We also remove debug files from the unpackaged files list as they're
handled separately by rpm.
2024-04-30 18:17:11 +02:00
Daan De Meyer
3c9a807ac8 mkosi: Install more locales on CentOS/Fedora
Required for TEST-73-LOCALE.
2024-04-30 17:01:41 +02:00
Daan De Meyer
0d73e224e2 mkosi: Install tpm2-tools in system image
Required for TEST-70-TPM2.
2024-04-30 17:01:41 +02:00
Daan De Meyer
695ebbaf7f mkosi: Override /sbin/init in minimal-base
Required for TEST-13-NSPAWN.
2024-04-30 17:01:41 +02:00
Richard Maw
334e707329 mkosi: Add curl package for journal tests 2024-04-30 17:01:41 +02:00
Daan De Meyer
989d46df8b mkosi: Install locales on debian/ubuntu
Required for TEST-73-LOCALE.
2024-04-30 17:01:41 +02:00
Daan De Meyer
a531931cd4 mkosi: Simply remove all the debian patches instead of mounting over them
The build sources are ephemeral anyways, so we can just remove the patches
instead of hiding them.
2024-04-30 17:01:41 +02:00
Daan De Meyer
e839cb1cf8 mkosi: Use symlinks instead of bind mounts for Arch
With bind mounts, the directories we bind mount to get recorded as
the meson source and build directories. This means meson will complain
if we later try to run meson install -C /work/build in the virtual
machine or container. If we use symlinks, the directories we symlink to
will be recorded as the meson source and build directories, which means
meson install -C /work/build will work when executed after booting the
VM or container.

I tried to do the same for debian as well but the debian package tooling
changes directory into the build directory and then does meson setup ..
which is completely broken when switching to a symlink.
2024-04-30 17:01:41 +02:00
Daan De Meyer
48e0399bee mkosi: Install minimal-base to /usr/share/testsuite-13-container-template
Required for TEST-13-NSPAWN.
2024-04-30 17:01:41 +02:00
Daan De Meyer
a8bc9400a8 mkosi: Remove more files from Arch minimal image 2024-04-30 17:01:41 +02:00
Daan De Meyer
c89244842b test: Don't persist journal in mkosi image if we're not debugging tests
If we're not debugging tests, there's no point in persisting the journal,
so let's use the volatile journal storage mode in that case to avoid doing
unnecessary work.

We don't disable journal storage alltogether since various tests check
that stuff is written to the journal.
2024-04-30 15:20:55 +02:00
Daan De Meyer
742b77e1d2 mkosi: Make sure we create an image without /var/log/journal
This allows journald to create the directory itself with the right
chattr settings on first boot.
2024-04-30 11:41:02 +02:00
Daan De Meyer
744fe45b8f mkosi: Simplify rpm build scripts 2024-04-30 11:39:36 +02:00
Daan De Meyer
e7c8507977 mkosi: Introduce particle profile
Unfortunately the current mkosi partitioning setup is a bit too
avant-garde for the integration tests. Both in that distributions
aren't ready for it yet (some more than others), and that software
which we depend on in the integration tests isn't ready for it yet
(e.g. libselinux does not read its configuration from /usr).

Let's switch back to a more boring partioning setup by default but
keep the fancy stuff around as a mkosi profile. This means that it
can still be used for manually testing stuff by running
"mkosi --profile particle -f qemu".
2024-04-30 10:46:18 +02:00
Daan De Meyer
b2173f50dc mkosi: Ignore version from versioned dependencies in .SRCINFO
Dependencies in .SRCINFO can be versioned. Let's make sure we ignore
any specified versions when grepping it for dependencies. Also update
the arch submodule to the latest to make sure the change works.
2024-04-28 16:20:58 +02:00
Richard Maw
6e40124efb mkosi: use generic ubuntu kernel for udev and nvme tests
TEST-17-UDEV expects to find scsi_debug and TEST-84-STORAGETM expects to
find nvmet-tcp.

This isn't ideal as it adds firmware, microcode and other drivers to the
initramfs, but there's no linux-modules-extra virtual package
to just include the extra modules.
2024-04-26 15:55:42 +01:00
Richard Maw
9750972455 mkosi: Use default opensuse kernel for tests
The kvmsmall kernel lacks important modules like the dummy network
device which TEST-17-UDEV requires very early on.
2024-04-26 15:55:42 +01:00
Daan De Meyer
e392c868d2 mkosi: Set CleanPackageMetadata=yes for minimal-base
We don't want to install any more packages in derived images so let's
not store repository metadata in the image to make it smaller.
2024-04-26 13:46:45 +02:00
Daan De Meyer
c77dad706b mkosi: Build minimal images and enable related integration tests
This commit adds definitions to build the minimal_0 and minimal_1
images with mkosi and includes them into the system image. We also
move the building of the various app-xxx and similar images that are
extremely minimal into the tests itself by moving the related logic
from install_verity_minimal() into a new function
install_extension_images() in util.sh. Because the mkosi /usr is
read-only, we now place the extension images in /tmp instead of
/usr/share.

Co-authored-by: Richard Maw <richard.maw@codethink.co.uk>
Co-authored-by: sam-leonard-ct <sam.leonard@codethink.co.uk>
2024-04-25 21:06:30 +02:00
Daan De Meyer
d74ebb175a mkosi: Make sure systemd-resolved is started on boot 2024-04-25 17:06:42 +02:00
Daan De Meyer
a721b93aaa mkosi: Add squashfs-tools
Required for various integration tests.
2024-04-25 17:06:00 +02:00
Daan De Meyer
2c139de254 mkosi: Add attr
Required for messing around with xattrs in integration tests.
2024-04-24 12:09:12 +02:00
Daan De Meyer
ab7253e1ec mkosi: Add nvme-cli
The nvme command is needed for TEST-84-STORAGETM.
2024-04-24 11:02:48 +02:00
Richard Maw
fab270d73e mkosi: Add psmisc to opensuse for killall 2024-04-24 11:01:45 +02:00
Richard Maw
f744ccd5fe mkosi: Add psmisc to debian-ubuntu for killall 2024-04-24 11:01:45 +02:00
Richard Maw
bad25450b8 mkosi: Add psmisc to arch for killall 2024-04-24 11:01:45 +02:00
Daan De Meyer
622678edd7
Merge pull request #32352 from DaanDeMeyer/test
test: Various mkosi integration test improvements
2024-04-23 11:27:23 +02:00
Daan De Meyer
422124072d mkosi: Run integration tests in CI
We do the image build and run the tests in a btrfs loopback so we
can make use of btrfs subvolumes and COW to keep the disk space
requirements to a minimum and speed up the ephemeral copies we make
of the image to run the tests.

We also switch to building debug packages and publishing the built
packages as artifacts.
2024-04-23 10:33:08 +02:00
Daan De Meyer
89b44cbb4d mkosi: Deal with ubuntu's special debug packages
Building debug packages on ubuntu requires the "debug" option to be
specified explicitly. Debug packages on Ubuntu have the .ddeb extension,
so let's make sure we handle that by copying the .ddeb packages in the
build script as well.
2024-04-23 10:25:02 +02:00
Daan De Meyer
388dcc785b mkosi: Use -fdebug-prefix-map= instead of -ffile-prefix-map=
-ffile-prefix-map= implies -fmacro-prefix-map= which is incompatible
with our definition of PROJECT_FILE.

See https://github.com/systemd/systemd/issues/32417.

-fdebug-prefix-map= only affects debuginfo without affecting macros.
2024-04-23 10:25:02 +02:00
Luca Boccassi
41fb4dc334 mkosi: mask isc-dhcp-server
isc-dhcp-server does not ship units, only sysv scripts, so the mkosi
presets that disable it have no effect. The generated unit is started on
each boot and fails, causing delays and noise.

Mask it so that the generated unit is overridden. It is installed only
to bring in binaries used by the networkd tests anyway.
2024-04-23 07:04:27 +02:00
Daan De Meyer
e888d1cae8 mkosi: Drop workaround to re-enable serial console line wrapping
Since we do it in reset_terminal_fd() now, there's no need to carry
this workaround anymore.
2024-04-22 15:28:26 +02:00
Daan De Meyer
8308fcfec3 mkosi: Explicitly disable fortify for debian/ubuntu
We do the same for the other distributions, so let's do so for
debian/ubuntu as well.
2024-04-22 10:49:32 +02:00
Daan De Meyer
4ccc28b8a8 mkosi: Use awk to avoid dpkg-buildpackage warning
Let's strip unnecessary whitespace to avoid a dpkg-buildpackage
warning.
2024-04-22 10:48:52 +02:00
Daan De Meyer
4de750ce90 mkosi: Add environment variable to set the optimization level 2024-04-22 10:29:21 +02:00
Daan De Meyer
fa15d444be mkosi: Install various system users/groups in opensuse images
These don't get pulled in automatically and there doesn't seem to
be a "system-users" package so install the ones we need for the
integration tests manually.
2024-04-22 09:09:32 +02:00
Daan De Meyer
9fd52adacf mkosi: Backport Fedora basic users/groups sysusers files to CentOS
CentOS does not ship these sysusers dropins which set up basic system
users and groups. Until we can move to CentOS Stream 10, let's add the
dropins ourselves to make sure the base system users/groups are available
on CentOS.
2024-04-22 09:09:32 +02:00
Daan De Meyer
245387721b mkosi: Install procps-ng in all images
Required for TEST-01-BASIC.
2024-04-22 09:09:32 +02:00
Daan De Meyer
4edc85a6c9 mkosi: Make sure systemd-libs is updated on Arch
The Arch Linux PKGBUILD does not (yet) have versioned dependencies
between the systemd packages, causing systemd-libs to not get updated
to 256-devel if systemd 256-devel is installed. Let's explicitly install
the newer version of systemd-libs as well for now until this problem is
fixed.
2024-04-22 09:09:32 +02:00
Daan De Meyer
27e7b633bb mkosi: Make sure ping is installed
Required for test-bpf-firewall.
2024-04-22 09:09:32 +02:00
Daan De Meyer
865f67701c mkosi: Disable rpm package notes
This introduces dependencies on various environment variables set
by rpmbuild which will not be set when rerunning meson inside the
VM/container so let's disable package notes to avoid these dependencies,
as the package notes aren't terribly useful in this scenario anyway.
2024-04-22 09:09:32 +02:00
Daan De Meyer
1531413efa mkosi: Make sure serial console line wrapping is re-enabled 2024-04-22 09:09:30 +02:00
Daan De Meyer
a2574ebf4d mkosi: Fix FORTIFY_SOURCE (again)
CentOS/Fedora use annobin which will complain if FORTIFY_SOURCE=0
is used so we disable those checks to avoid the warnings.

We also make sure that when we query the compilation flags so we can
add more, we set _fortify_level=0 and undefine _lto_flags so that we
don't get those flags in the result.
2024-04-19 10:17:09 +02:00
Richard Maw
945b722f13 test: Add mkosi-based integration test runner
The first two tests are included to ensure parallel test execution is
demonstrable.
2024-04-18 16:26:38 +01:00
Daan De Meyer
6c07705213 mkosi: undefine FORTIFY_SOURCE instead of setting it zero
Newer gcc complains if FORTIFY_SOURCE=0 is set so just undefine it
instead.
2024-04-18 14:35:07 +02:00
Daan De Meyer
104c64fae1 mkosi: Disable bash debugging in Arch build script 2024-04-18 14:21:31 +02:00
Daan De Meyer
a432014b01 mkosi: Install debug packages when WITH_DEBUG=1 is enabled
When we're building debuginfo packages, the original binaries and
libraries are stripped so make sure we install the debuginfo
packages to make sure debugging in the container/VM still works.
2024-04-18 14:01:50 +02:00
Daan De Meyer
35c1ea4d69 mkosi: Setup --ffile-prefix-map= for opensuse as well
This doesn't actually work because the opensuse spec doesn't allow
adding extra build flags, but I'm working on fixing that, so let's
already set things up for later.
2024-04-18 14:01:44 +02:00
Daan De Meyer
15bf8bfb69 mkosi: Undefine FORTIFY_SOURCE before setting it again
Otherwise we get warnings from gcc.
2024-04-18 14:01:06 +02:00
Daan De Meyer
d2c0acaf70 mkosi: Install more packages
Let's install everything we can to get more coverage and make sure
all build outputs are available in mkosi containers or VMs.
2024-04-18 13:32:43 +02:00
Daan De Meyer
6911a9ea7f mkosi: Drop systemd-repart from package lists
This is just a Provides for systemd-udev.
2024-04-18 13:27:05 +02:00
Daan De Meyer
22fa7cbccc mkosi: Drop workarounds
With the latest Github Actions image release none of these should
be required anymore (20240414.1.1).
2024-04-17 17:36:36 +02:00
Daan De Meyer
241d792c92 mkosi: Switch to O0 and disable FORTIFY_SOURCE
-Og still causes a lot of "<optimized out>" in GDB so let's use -O0
instead and disable FORTIFY_SOURCE as it doesn't work without
optimizations enabled.
2024-04-17 17:36:33 +02:00
Daan De Meyer
42ca6c2773 mkosi: Disable DWARF debug info optimization
We care more about fast builds in this case so disable the DWARF
debug optimization that's done by default.
2024-04-17 16:11:35 +02:00
Daan De Meyer
7f1d7fed80 mkosi: Override build_cflags instead of _distro_extra_cflags
The latter is not supported on CentOS Stream 9.
2024-04-17 15:38:24 +02:00
Daan De Meyer
8e728c14ec mkosi: Set up -ffile-prefix-map= correctly when building debuginfo packages
This makes sure that the debuginfo files contain source files pointing
to the source files shipped by the debugsource package.

Normally this should be done automatically by rpm invoking debugedit
but for some unknown reason debugedit refuses to rewrite the source
files in our binaries.

Given that debugedit is completely undebugable (does not generate any
logs at all, and its source code is ridiculously obtuse), let's set
-ffile-prefix-map= when building instead which achieves the same
effect.
2024-04-17 15:21:01 +02:00
Daan De Meyer
4980ae0fbc mkosi: Add support for building debug packages
This allows building debug packages by setting WITH_DEBUG=1. This
slows down the build a lot so it's opt in. We don't yet install
these but can do so in a future commit.
2024-04-16 15:25:37 +02:00
Daan De Meyer
f1884281a0 mkosi: Add --noclean when running rpmbuild
The entire build environment is ephemeral anyway so everything is
cleaned regardless. By specifying --noclean, we make debugging
easier as the rpm build root can be inspected when using --debug-shell.
2024-04-16 15:25:37 +02:00
Daan De Meyer
2400b3a590 mkosi: Install which in image
gdb-headless in Fedora is missing a dependency on which:
https://bugzilla.redhat.com/show_bug.cgi?id=2275274

This happens all the time so let's make sure which is always installed
in the image.
2024-04-16 15:25:37 +02:00
Daan De Meyer
9d98617c1a Update debugging with vscode section
- We have ssh-generator now, so need for mkosi's Ssh= option anymore.
- By enabling RuntimeBuildSources= by default, we don't need the gdb
  config file in the image anymore, since the build and source
  directories will be mounted at the expected locations.
2024-04-16 15:25:34 +02:00
Daan De Meyer
9c5216bb9d mkosi: Use new volatile package settings
mkosi just learned to do natively what we currently do with environment
variables and a postinst script, so let's update to the latest version
and start using the new settings instead.
2024-04-14 19:53:43 +02:00
Daan De Meyer
035bdb1472 mkosi; Fix permissions of mkosi.postinst script
Must be executable.
2024-04-09 16:20:58 +01:00
Daan De Meyer
07d52c7a46 mkosi: Drop out-of-date comment 2024-04-09 11:57:38 +02:00
Daan De Meyer
1a2374a3f6 mkosi: Unify initrd post-install scripts
Just like we already have $SYSTEMD_PACKAGES for systemd packages to
re-install in the main image, let's add $INITRD_PACKAGES for all
systemd packages to re-install in the initrd.
2024-04-09 11:57:16 +02:00
Daan De Meyer
e4c8e9dcbd mkosi: Install openSUSE-release instead of distribution-release
distribution-release is a virtual package that is by default satisfied
by the openSUSE MicroOS-release package. Let's make sure we pull in the
generic openSUSE-release package instead by installing
patterns-base-minimal_base which has a Suggests dependency on
openSUSE-release which makes sure it takes priority over the MicroOS one.
2024-04-09 11:57:06 +02:00
Daan De Meyer
d6f206b944 mkosi: Make scripts more generic
We might want to run the build scripts outside of mkosi as well at
some point, e.g. to build an rpm after booting the image, so let's
make them more generic by using /usr/lib/os-release to figure out
which pkg specs we should use instead of $PKG_SUBDIR. To make ubuntu
use the debian pkg spec, we add a symlink pkg/ubuntu which points to
debian/ in the same directory.
2024-04-09 11:57:05 +02:00
Daan De Meyer
9c611095ca mkosi: Update pkg/arch to latest and install systemd-tests
The arch PKGBUILD now packages the tests when building for upstream
so let's make sure we install the new package.
2024-04-09 11:47:37 +02:00
Daan De Meyer
da38f93bd6 mkosi: Fix environment variable in arch prepare script 2024-04-08 11:36:40 +02:00
Daan De Meyer
8440ea8f21 Install build dependencies into final image
This allows us to build and install after booting without having to
build a new image. Together with
https://github.com/systemd/mkosi/pull/2601 and after enabling
RuntimeBuildSources=yes, after booting, "meson install -C /work/build"
can be used to do an incremental build and install. This won't build
proper packages, but will be invaluable for having a quick compile,
edit, test cycle without having to rebuild the image all the time.
2024-04-08 11:36:40 +02:00
Daan De Meyer
84affd46d5 mkosi: Install dnf5 in Fedora image 2024-04-07 19:09:11 +02:00
Lennart Poettering
1d98716ef7 libkmod: turn into dlopen() dependency
As it turns out libkmod has quite a bunch of deps, including various
compressing libs and similar. By turning this into a dlopen()
dependency, we can make our depchain during install time quite a bit
smaller. In particular as inside of containers kmod doesn't help anyway
as CAP_SYS_MODULE is not available anyway.

While we are at it, also share the code that sets up logging/kmod
context.

After:

$ lddtree ./build/systemd
systemd => ./build/systemd (interpreter => /lib64/ld-linux-x86-64.so.2)
    libsystemd-core-255.so => ./build/src/core/libsystemd-core-255.so
        libaudit.so.1 => /lib64/libaudit.so.1
            libcap-ng.so.0 => /lib64/libcap-ng.so.0
                ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
        libm.so.6 => /lib64/libm.so.6
        libmount.so.1 => /lib64/libmount.so.1
            libblkid.so.1 => /lib64/libblkid.so.1
        libseccomp.so.2 => /lib64/libseccomp.so.2
        libselinux.so.1 => /lib64/libselinux.so.1
            libpcre2-8.so.0 => /lib64/libpcre2-8.so.0
    libsystemd-shared-255.so => /home/lennart/projects/systemd/build/src/shared/libsystemd-shared-255.so
        libacl.so.1 => /lib64/libacl.so.1
            libattr.so.1 => /lib64/libattr.so.1
        libcap.so.2 => /lib64/libcap.so.2
        libcrypt.so.2 => /lib64/libcrypt.so.2
        libgcrypt.so.20 => /lib64/libgcrypt.so.20
            libgpg-error.so.0 => /lib64/libgpg-error.so.0
        liblz4.so.1 => /lib64/liblz4.so.1
        libcrypto.so.3 => /lib64/libcrypto.so.3
            libz.so.1 => /lib64/libz.so.1
        libpam.so.0 => /lib64/libpam.so.0
            libeconf.so.0 => /lib64/libeconf.so.0
        liblzma.so.5 => /lib64/liblzma.so.5
        libzstd.so.1 => /lib64/libzstd.so.1
    libc.so.6 => /lib64/libc.so.6

Before:

$ lddtree ./build/systemd
systemd => ./build/systemd (interpreter => /lib64/ld-linux-x86-64.so.2)
    libsystemd-core-255.so => ./build/src/core/libsystemd-core-255.so
        libaudit.so.1 => /lib64/libaudit.so.1
            libcap-ng.so.0 => /lib64/libcap-ng.so.0
                ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
        libkmod.so.2 => /lib64/libkmod.so.2
            libzstd.so.1 => /lib64/libzstd.so.1
            liblzma.so.5 => /lib64/liblzma.so.5
            libz.so.1 => /lib64/libz.so.1
            libcrypto.so.3 => /lib64/libcrypto.so.3
            libgcc_s.so.1 => /lib64/libgcc_s.so.1
        libm.so.6 => /lib64/libm.so.6
        libmount.so.1 => /lib64/libmount.so.1
            libblkid.so.1 => /lib64/libblkid.so.1
        libseccomp.so.2 => /lib64/libseccomp.so.2
        libselinux.so.1 => /lib64/libselinux.so.1
            libpcre2-8.so.0 => /lib64/libpcre2-8.so.0
    libsystemd-shared-255.so => /home/lennart/projects/systemd/build/src/shared/libsystemd-shared-255.so
        libacl.so.1 => /lib64/libacl.so.1
            libattr.so.1 => /lib64/libattr.so.1
        libcap.so.2 => /lib64/libcap.so.2
        libcrypt.so.2 => /lib64/libcrypt.so.2
        libgcrypt.so.20 => /lib64/libgcrypt.so.20
            libgpg-error.so.0 => /lib64/libgpg-error.so.0
        liblz4.so.1 => /lib64/liblz4.so.1
        libpam.so.0 => /lib64/libpam.so.0
            libeconf.so.0 => /lib64/libeconf.so.0
    libc.so.6 => /lib64/libc.so.6
2024-04-04 18:16:45 +02:00
Daan De Meyer
50ad7cf934 mkosi: More package updates
- Install individual asan libraries instead of gcc
- Drop duplicate qrencode package from arch config
- Install dbus-user-session which provides default-dbus-session-bus
- Explicitly install dbus-broker on Arch Linux
2024-04-03 21:05:03 +02:00
Daan De Meyer
56e6bf9e04 mkosi: Install selinux tools in main image instead of initramfs
Also install setools-console and policycoreutils instead of setools
which pulls in the kitchen sink. Also install selinux-policy-targeted
to make sure the right policy is installed.
2024-04-03 20:09:04 +02:00
Daan De Meyer
5ed030a7ac mkosi: Install git-core where possible
This avoids pulling in the perl kitchen sink.
2024-04-03 20:09:04 +02:00
Daan De Meyer
42e2ec23f4 mkosi: Use '-' instead of '.' to separate upstream version and debian revision
The debian revision starts after the '-' character, so make sure the
timestamp we append is treated as the revision instead of being a part
of the upstream version.
2024-04-02 12:31:34 +02:00
Daan De Meyer
e5cd051724 mkosi: Switch to linux-virtual on Ubuntu
This now finally has support for credentials and erofs so let's switch
to this much smaller kernel package that doesn't pull in linux-firmware.
2024-03-29 15:58:13 +01:00
Daan De Meyer
5524d2838c mkosi: Copy built packages to output directory
This allows the built packages to be inspected if needed, or installed
on the host system for anyone who's daring enough to do so.
2024-03-27 16:14:17 +01:00
Daan De Meyer
0a6bf709dd mkosi: Specify --without docs if needed for fedora/centos 2024-03-27 13:00:11 +01:00
Daan De Meyer
6327988d65 mkosi: Merge base and system images
The only reason to have these split up is to be able to build extension
images that use the base image as a base tree and install extra packages.
Until we have such a use case, let's merge the base and system images to
simplify things a bit.

We keep the mkosi.images/ directory to not cause too many conflicts with
the integration tests PR.
2024-03-27 12:20:50 +01:00
Richard Maw
23a617fd8a mkosi: Add selinux support to CentOS and Fedora initrds
The base initrd is also used by the system image
so selinux should be there even if not enabled by the base image.
2024-03-25 15:51:16 +00:00
Richard Maw
10ac38ddd3 mkosi: Remove testuser from base/mkosi.postinst.chroot
The testuser user is only needed for integration tests,
which are used in the system user and this config
can be provided as drop-ins instead of inline in postinst scripts.
2024-03-25 15:51:16 +00:00
Daan De Meyer
b21d56e82c mkosi: Use new environment variables for Arch
These were changed to be prefixed with _systemd_ so let's account
for that.
2024-03-25 12:06:50 +00:00
Daan De Meyer
d0327823d8 mkosi: Switch to Arch Linux packaging sources main branch
https://gitlab.archlinux.org/archlinux/packaging/packages/systemd/-/merge_requests/8
was merged so let's switch branches to the main branch.
2024-03-22 16:51:37 +01:00
Daan De Meyer
97b191c49b mkosi: Disable debug package generation on Arch Linux
This is extremely slow since the latest pacman release, and since
we don't strip binaries, not really needed either.
2024-03-22 16:49:10 +01:00
Daan De Meyer
2d0b8e9fb6 mkosi: Make sure man and man-db are installed everywhere 2024-03-22 15:12:20 +01:00
Daan De Meyer
a1eeb4d753 mkosi: Install systemd-experimental in OpenSUSE initrd
systemd-repart is part of systemd-experimental and we need systemd-repart
in the initrd so let's install systemd-experimental in the initrd.
2024-03-18 16:20:15 +01:00
Daan De Meyer
bd9bec99d0 mkosi: Install python3-pefile in OpenSUSE image
ukify is part of systemd-experimental on OpenSUSE and not its own
package. Because the OpenSUSE systemd maintainers do not want to
introduce a python dependency for systemd-experimental, we have to
install python3-pefile manually to make sure ukify works properly.
2024-03-18 16:20:00 +01:00
Daan De Meyer
d1c29b517a mkosi: Allow booting without secure boot
Don't fail if SecureBoot is not enabled. Instead, only execute
the secure boot related checks if secure boot is actually enabled.
2024-03-13 23:42:34 +01:00
Daan De Meyer
37bd860c22 mkosi: Introduce packaging sources as submodules
By always cloning the latest branch commit, we can't bisect properly
using mkosi as when bisecting wildly different packaging sources will
be used compared to when the commit was merged. By using submodules, we
track individual commits which means when bisecting the same packaging
sources will be used.

We use git submodules as dependabot has support for automatically making
PRs to update git submodules. This commit also includes the necessary
dependabot configuration to enable this.

We make ubuntu/debian use the same submodule instead of adding the debian
packaging sources twice by introducing a new $PKG_SUBDIR environment variable
and using it instead of $DISTRIBUTION.
2024-03-13 12:07:49 +01:00
Daan De Meyer
d008a9cfad mkosi: Remove some leftover shell debugging 2024-03-13 12:07:09 +01:00
Daan De Meyer
4cc17467a1 mkosi: Use same pkg/ subdirectory for debian and ubuntu
Instead of cloning the same repository twice, let's make sure we
use the same directory for both debian/ubuntu.
2024-03-13 12:07:09 +01:00
Yu Watanabe
1d87a00a95 mkosi: fix typo
Follow-up for 4d0f1451b5.
2024-03-10 14:09:41 +09:00
Yu Watanabe
17be4d7b98 mkosi: fix typo
Follow-up for 4d0f1451b5.
2024-03-08 18:17:37 +01:00
Daan De Meyer
4d0f1451b5 Build distribution packages in mkosi
Instead of running meson install and hoping for the best, let's build
distribution packages from the downstream packaging specs. This gets
us the following:

- Vastly simplified mkosi scripts since we don't need a separate initrd
  image anymore but can just reuse the default mkosi initrd.
- Almost everything can move to the base image as its not the basis
  anymore for the initrd and as such we don't need to care about the
  size anymore.
- The systemd packages that get pulled in as dependencies of other
  packages get properly uninstalled and replaced with our packages that
  we built instead of just installing on top of an existing systemd
  installation with no guarantee that everything from that previous
  installation was removed.
- Much better testing coverage as what we're testing is much closer
  to what will actually be deployed in distributions.
- Immediate feedback if something we change breaks distribution packaging
- We get integration with the distribution for free as we'll automatically
  use the proper directories and such instead of having to hack this
  into a mkosi build script.
- ...
2024-03-07 10:47:19 +01:00
Daan De Meyer
bb478caa86 mkosi: Use specifier to refer to the output directory
Otherwise the output directory cannot be configured by users.
2024-03-05 19:42:18 +01:00
Frantisek Sumsal
e86b1a9b0f mkosi: make shellcheck happy 2024-03-05 12:41:30 +01:00