IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
NO_BUILD=1 indicates that we want to test systemd from the local system and not
the one from the local build. Hence there should be no need to call
find-build-dir.sh when NO_BUID=1 especially since it's likely that the script
will fail to find a local build in this case.
This avoids find-build-dir.sh to emit 'Specify build directory with $BUILD_DIR'
message when NO_BUILD=1 and no local build can be found.
This introduces a behavior change though: systemd from the local system will
always be preferred when NO_BUILD=1 even if a local build can be found.
In some cases image names are unpredictable - some orchestrators/deployment
tools like to mangle names to suit their internal formats. In these cases,
the requirement that the extension-release file matches exactly the image
name where it's contained cannot work.
Allow falling back to loading the first regular file which name starts with
'extension-release' located in /usr/lib/extension-release.d/ and tagged with
a user.extension-release.strict extended attribute with a true value, if the
one with the expected name cannot be found.
Skip a harmless error when running the tests on a system with a significantly
older systemd version (ldd tries to resolve the unprefixed RPATH for libsystemd.so.0,
which is in this case older than the already installed libsystemd.so.0 in $initdir).
The issue is triggered by installing test dependencies in install_missing_libraries().
Spotted on CentOS 8.
```
$ ldd /var/tmp/systemd-test.nZO11F/root/lib/systemd/tests/test-sd-device-thread
/var/tmp/systemd-test.nZO11F/root/lib/systemd/tests/test-sd-device-thread: /lib64/libsystemd.so.0: version `LIBSYSTEMD_240' not found (required by /var/tmp/systemd-test.nZO11F/root/lib/systemd/tests/test-sd-device-thread)
linux-vdso64.so.1 (0x00007fffb79d0000)
libclang_rt.asan-powerpc64le.so => /usr/lib64/clang/11.0.0/lib/linux/libclang_rt.asan-powerpc64le.so (0x00007fffb6ef0000)
libsystemd.so.0 => /lib64/libsystemd.so.0 (0x00007fffb6d20000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fffb6cd0000)
libc.so.6 => /lib64/libc.so.6 (0x00007fffb6ab0000)
$ LD_LIBRARY_PATH=/var/tmp/systemd-test.nZO11F/root/lib64/ ldd /var/tmp/systemd-test.nZO11F/root/lib/systemd/tests/test-sd-device-thread
linux-vdso64.so.1 (0x00007fffaba80000)
libclang_rt.asan-powerpc64le.so => /usr/lib64/clang/11.0.0/lib/linux/libclang_rt.asan-powerpc64le.so (0x00007fffaafa0000)
libsystemd.so.0 => /var/tmp/systemd-test.nZO11F/root/lib64/libsystemd.so.0 (0x00007fffaa5f0000)
libpthread.so.0 => /var/tmp/systemd-test.nZO11F/root/lib64/libpthread.so.0 (0x00007fffaa5a0000)
libc.so.6 => /var/tmp/systemd-test.nZO11F/root/lib64/libc.so.6 (0x00007fffaa380000)
```
When `linux-headers` is installed on Arch Linux, it stores the module
source tree in the kernel module directory, which is then picked up by
`find` and we get a lot of harmless but annoying errors:
```
...
modprobe: FATAL: Module Kconfig.iosched not found in directory /lib/modules/5.13.7-arch1-1
modprobe: FATAL: Module Kconfig not found in directory /lib/modules/5.13.7-arch1-1
modprobe: FATAL: Module Kconfig not found in directory /lib/modules/5.13.7-arch1-1
modprobe: FATAL: Module dm-mpath.h not found in directory /lib/modules/5.13.7-arch1-1
modprobe: FATAL: Module dm-bio-prison-v2.h not found in directory /lib/modules/5.13.7-arch1-1
modprobe: FATAL: Module raid0.h not found in directory /lib/modules/5.13.7-arch1-1
...
```
Let's fix this by trying to install only kernel modules (*.ko files with
an optional compression).
Let's unify handling of the boolean values throughout the test-functions
code, since we use 0/1, true/false, and yes/no almost randomly in many
places, so picking the right values during CI configuration can be a real
pain.
Saving the journal for passing tests creates a huge amount of unneeded
data stored for each full test run. Add a env var to allow saving the
journal only for failed tests.
Due to a little misunderstanding the last patch doesn't work as
expected, since test_create_image() is called only for the first image
(usually TEST-01-BASIC), and all subsequent images are then (possibly)
modified with test_append_files().
Follow-up to 179ca4d2b1.
It turns out the "supporting services" were run in _all_ tests if
TEST-01-BASIC was run as the first test (which is usually the case),
since with the original condition in test_create_image() we would skip
the masking and then propagate the change to the default image used by
other tests. This has been causing multiple bogus test timeouts
(especially when the hwdb was being rebuilt in tests with short
timeouts, like TEST-52-HONORFIRSTSHUTDOWN).
Let's "fix" this by making the call to mask_supporting_services()
uncoditional and override the test_create_image() function in
TEST-01-BASIC to avoid the masking in this single case.
The three-argument match() is a GNU AWK extension, thus breaking the
compatibility with mawk (used on Ubuntu/Debian, for example). Let's
replace it with a (hopefully) more portable sed expression to drop the
inadvertently introduced gawk dependency.
Fixes: #19957
This fixes an issue introduced by the commit 954c77c251.
For some reasons, setting default ACL on $TESTDIR makes TEST-29-PORTABLE
fail. Let's drop the default ACL, and set ACL on saved results instead.
Fixes#19519.
"systemd-testsuite" gets in the way when grepping for "testsuite-*.sh".
Also, the name doesn't matter for anything, so let's just use something
very short to save space.
When editing this function in 7bf20e48bd, I couldn't
decide whether to initialize ret at the top and only reset it on success, or
whether to assign a value in each branch. In the end I did neither ;( So if the
test finished without creating any of the result files, we would echo a
message, but return "success".
But there was bigger confusion with /failed: some tests create it empty, some
don't. I think we may want to do away pre-creation of /failed completely, and
assume the test failed unless /testok is found. But I'm leaving that for later
rework. For now let's just make sure we report return success only if /testok
or /skipped is found.
Basically the same scenario as in
a33e2692e1, where `awk` exits as soon
as it finds a match, thus sending SIGPIPE to `ldd` if it's not fast
enough. That, in combination with `set -o pipefail` causes random &
unexpected fails, like:
```
No journal files were found.
-rw-r----- 1 root root 16777216 Apr 30 10:31
/var/tmp/TEST-01-BASIC_sanitizers-nspawn/system.journal
TEST-01-BASIC RUN: Basic systemd setup [OK]
systemd is not linked against the ASan DSO
gcc does this by default, for clang compile with -shared-libasan
make: *** [Makefile:2: clean-again] Error 1
make: Leaving directory '/build/test/TEST-01-BASIC'
```
Specifying the test number manually is tedious and prone to errors (as
recently proven). Since we have all the necessary data to work out the
test number, let's do it automagically.
We have to invoke the tests as superuser, and not being able to read
the journal as the invoking user is annoying. I don't think there are
any security considerations here, since the invoking user can already
put arbitrary code in the Makefile and test scripts which get executed
with root privileges.
The logic to query test state was rather complex. I don't quite grok the point
of ret=$((ret+1))… But afaics, the precise result was always ignored by the
caller anyway.
This code was partially broken, since the firmware directory was
undefined. Also, some of the parts were a dead code, since they relied
on code from the original dracut test suite.
`command -v <bin> | grep ...` can under certain conditions cause the
`command` to exit with SIGPIPE, which in combination with `set -o
pipefail` means that the tests sometimes randomly die during setup.
Let's avoid using pipes in such cases.
This breaks some existing loops which previously ignored if the piped
program exited with EC >0. Rewrite them to mitigate this (and also make
them more robust in some cases).
When trying to calculate the next firing of 'Sun *-*-* 01:00:00', we'd fall
into an infinite loop, because mktime() moves us "backwards":
Before this patch:
tm_within_bounds: good=0 2021-03-29 01:00:00 → 2021-03-29 00:00:00
tm_within_bounds: good=0 2021-03-29 01:00:00 → 2021-03-29 00:00:00
tm_within_bounds: good=0 2021-03-29 01:00:00 → 2021-03-29 00:00:00
...
We rely on mktime() normalizing the time. The man page does not say that it'll
move the time forward, but our algorithm relies on this. So let's catch this
case explicitly.
With this patch:
$ TZ=Europe/Dublin faketime 2021-03-21 build/systemd-analyze calendar --iterations=5 'Sun *-*-* 01:00:00'
Normalized form: Sun *-*-* 01:00:00
Next elapse: Sun 2021-03-21 01:00:00 GMT
(in UTC): Sun 2021-03-21 01:00:00 UTC
From now: 59min left
Iter. #2: Sun 2021-04-04 01:00:00 IST
(in UTC): Sun 2021-04-04 00:00:00 UTC
From now: 1 weeks 6 days left <---- note the 2 week jump here
Iter. #3: Sun 2021-04-11 01:00:00 IST
(in UTC): Sun 2021-04-11 00:00:00 UTC
From now: 2 weeks 6 days left
Iter. #4: Sun 2021-04-18 01:00:00 IST
(in UTC): Sun 2021-04-18 00:00:00 UTC
From now: 3 weeks 6 days left
Iter. #5: Sun 2021-04-25 01:00:00 IST
(in UTC): Sun 2021-04-25 00:00:00 UTC
From now: 1 months 4 days left
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1941335.
otherwise udev complains about the file being world-writable:
systemd-udevd[228]: Configuration file /etc/udev/rules.d/00-set-LD_PRELOAD.rules is marked world-writable. Please remove world writability permission bits. Proceeding anyway.
Fixes: systemd/systemd-centos-ci#354
This test would normally get stuck when trying to mount the verity image
due to:
systemd-udevd[299]: dm-0: '/usr/sbin/dmsetup udevflags 6293812'(err) '==371==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.'
systemd-udevd[299]: dm-0: Process '/usr/sbin/dmsetup udevflags 6293812' failed with exit code 1
...
systemd-udevd[299]: dm-0: '/usr/sbin/dmsetup udevcomplete 6293812'(err) '==372==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.'
systemd-udevd[299]: dm-0: Process '/usr/sbin/dmsetup udevcomplete 6293812' failed with exit code 1.
systemd-udevd[299]: dm-0: Command "/usr/sbin/dmsetup udevcomplete 6293812" returned 1 (error), ignoring.
so let's add a simple udev rule which sets $LD_PRELOAD for the block
subsystem.
Also, install the ASan library along with necessary dependencies into
the verity minimal image, to get rid of the annoying (yet harmless)
errors about missing library from $LD_LIBRARY.
When running integration tests under sanitizers D-Bus fails to
shutdown cleanly, causing unnecessary noise in the logs:
```
dbus-daemon[272]: ==272==LeakSanitizer has encountered a fatal error.
dbus-daemon[272]: ==272==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
dbus-daemon[272]: ==272==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)
```
Since we're not "sanitizing" D-Bus anyway let's disable LSan's at_exit
check for the dbus.service to get rid of this error.
When a subshell is used ('make' or 'make all') the LOOPDEV environment
variable, which is used to store the opened loop device, is lost.
So the cleanup on trap/exit doesn't do anything, and the loop
device used to mount the test image is left around.
Avoid using a subshell to fix the issue.
The source package in the apt cache might be older than the
packaging from salsa.debian.org/systemd-team/systemd so it might not
list all the current binary packages.
This is currently the case for systemd-timesyncd, so TEST-30 fails.
Simply grep the control file rather than using apt-cache when iterating
over the packages contents.
Binaries on the latest Arch Linux use `call` instructions instead of
`callq`, which breaks the ASan detection and eventually the image
building process (due to insufficient space).
There may be situations where a cgroup should be protected from killing
or deprioritized as a candidate. In FB oomd xattrs are used to bias oomd
away from supervisor cgroups and towards worker cgroups in container
tasks. On desktops this can be used to protect important units with
unpredictable resource consumption.
The patch allows systemd-oomd to understand 2 xattrs:
"user.oomd_avoid" and "user.oomd_omit". If systemd-oomd sees these
xattrs set to 1 on a candidate cgroup (i.e. while attempting to kill something)
AND the cgroup is owned by root, it will either deprioritize the cgroup as
a candidate (avoid) or remove it completely as a candidate (omit).
Usage is restricted to root owned cgroups to prevent situations where an
unprivileged user can set their own cgroups lower in the kill priority than
another user's (and prevent them from omitting their units from
systemd-oomd killing).
Add NO_BUILD var to allow testing with no local build, by installing
local systemd files into the image.
This only works for debian-like distros currently, that use the
tools 'apt' and 'dpkg' for package management.
The $BUILD_DIR is only used in test-functions, and doesn't need to
be specified in any other scripts. Additionally, to be able to allow
the integration test suite to be run against locally installed binaries,
instead of built binaries, moving BUILD_DIR logic completely into
test-functions allows later patches to be simpler.
Building custom images for each test takes a lot of time.
Build the default one, and if the test needs incompatible changes
just copy it and extend it instead.
This reverts commit 73484ecff9.
3976f372ae moved libudev.so to be built in the
main directory, so this addition to $LD_LIBRARY_PATH is now obsolete.
After that commit, we build the following shared libraries:
build/libnss_myhostname.so.2
build/libnss_mymachines.so.2
build/libnss_resolve.so.2
build/libnss_systemd.so.2
build/libsystemd.so.0.30.0
build/libudev.so.1.7.0
build/pam_systemd.so
build/pam_systemd_home.so
build/src/boot/efi/stub.so
build/src/boot/efi/systemd_boot.so
build/src/shared/libsystemd-shared-247.so
EFI stubs don't matter, and libsystemd-shared-nnn.so is loaded through rpath,
and is doesn't need to and shouldn't be in $LD_LIBRARY_PATH. In effect, we only
ever need to add the main build directory to the search path.
Not all optional libraries might be available on developers machines,
so log and skip.
Also some pkg-config files are broken (eg: tss2 on Debian Stable) so
skip if the required variables are missing, and improve logs.
By default the test suite prefers qemu, and uses nspawn only if
a test specifically says it doesn't support qemu.
Add a variable to allow flipping the default, and run as many
tests under nspawn as possible.
Allows to split the test run in two parts. Most tests can run under
nspawn which is much faster, and they can be ran in one chunk with
TEST_NO_QEMU=1. The qemu-only tests, which are just a handful, can
be ran in another chunk with TEST_QEMU_ONLY=1.
Allows autopkgtest to be split in two parts.
The image build function greps for ExecStart lines in unit files, but some
of them (eg: systemd-firstboot) do not use a full path.
It then falls back to 'type -P' but that only works if you have the binary
installed. For optional binaries like systemd-firstboot, the installation
can then fail.
Manually check if the binary already exists in /[usr/][s]bin.
Usually on Debian ROOTLIBDIR is /lib/<arch triplet>, which is not the right place.
Use pkg-config since we define it, and then fallback to /usr/lib/systemd/user which is
the canonical location.
On both Debian&friends and Fedora dbus/dbus-broker install the user socket/service
under /usr/lib/systemd/user, not /lib/systemd/systemd/user.
I suspect the original version of the regex was written on a system,
which prints both the QEMU version and the QEMU package version in the
--version output, like Fedora:
$ /bin/qemu-system-x86_64 --version
QEMU emulator version 4.2.1 (qemu-4.2.1-1.fc32)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers
However, Arch Linux prints only the QEMU version:
$ /bin/qemu-system-x86_64 --version
QEMU emulator version 5.2.0
Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers
This causes the awk regex to not match the version string, since there's
no whitespace after it, causing the version check to fail (as well as the
TEST-36-NUMAPOLICY) as well.
Follow-up for 43b49470d1.
Upgrading to qemu 5.2 breaks TEST-36-NUMAPOLICY like:
qemu-system-x86_64: total memory for NUMA nodes (0x0) should
equal RAM size (0x20000000)
Use the new (as in >=2014) form of memdev in test 36:
-object memory-backend-ram,id=mem0,size=512M -numa node,memdev=mem0,nodeid=0
Since some target systems are as old as qemu 1.5.3 (CentOS7) but the new
kind to specify was added in qemu 2.1 this needs to add version parsing and
add the argument only when qemu is >=5.2.
Fixes#17986.
Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
The systemd-user file has been moved from /etc/pam.d into /usr/lib/pam.d,
so test-functions needs to copy it from /usr/lib/pam.d instead.
This will copy it from either location.
When invoking "ldd" to find dependency libraries we already set
$LD_LIBRARY_PATH to point to our own build tree, so that our libraries
are checked, not the host libraries. This is not sufficient howeever, as
libudev is built in a subdir. Add that, too.
We have four legal cases:
1. /usr/lib/os-release exists and /etc/os-release is a symlink to it
2. both exist but /etc/os-release is not a symlink to /usr/lib/os-release
3. only /usr/lib/os-release exists
4. only /etc/os-release exists
The generic setup code in test-functions and create-busybox-image didn't handle
case 3.
The test-specific code in TEST-50 didn't handle 2 (because the general setup
code would only install /etc/os-release in the image and
grep -f /usr/lib/os-release would not work) and 4 (same reason) and would fail
in case 3 in generic setup.
Since the hwdb update from a79be2f807
the systemd-hwdb-update service started timing out under ASan when
compiled with gcc, as we started tripping over the 3 minutes timeout.
This affects only gcc runs, since the current gcc on Arch still suffers
from the detect_stack_use_after_return performance penalty[0]. Until
the fixed gcc is present in the respective repositories, let's bump
the timeout to 4 minutes, as we might not be able to upgrade right
away, due to systemd/systemd#16199.
Before the hwdb update:
[ 7958.292540] systemd[63]: systemd-hwdb-update.service: Executing: /usr/bin/time systemd-hwdb update
[ 7958.304005] systemd[1]: systemd-journald.service: Got notification message from PID 44 (FDSTORE=1)
[ 7958.314434] systemd[1]: systemd-journald.service: Added fd 3 (n/a) to fd store.
[ 8008.520082] systemd[1]: systemd-journald.service: Got notification message from PID 44 (WATCHDOG=1)
[ 8068.520151] systemd[1]: systemd-journald.service: Got notification message from PID 44 (WATCHDOG=1)
[ 8125.682843] time[63]: 84.47user 82.92system 2:47.50elapsed 99%CPU (0avgtext+0avgdata 811512maxresident)k
[ 8125.682843] time[63]: 0inputs+19680outputs (0major+25000853minor)pagefaults 0swaps
After the hwdb update:
[ 6215.491958] systemd[63]: systemd-hwdb-update.service: Executing: /usr/bin/time systemd-hwdb update
[ 6215.503380] systemd[1]: systemd-journald.service: Got notification message from PID 44 (FDSTORE=1)
[ 6215.514172] systemd[1]: systemd-journald.service: Added fd 3 (n/a) to fd store.
[ 6329.392918] systemd[1]: systemd-journald.service: Got notification message from PID 44 (WATCHDOG=1)
[ 6394.920205] time[63]: 89.48user 89.98system 2:59.55elapsed 99%CPU (0avgtext+0avgdata 812764maxresident)k
[ 6394.920205] time[63]: 0inputs+20568outputs (0major+27318354minor)pagefaults 0swaps
[0] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94910
Prompted by systemd/systemd#16111.
* check if /var is a mountpoint - if not, something went wrong. In case
of systemd/systemd#16111 the /failed file was created, because
systemd-cryptsetup failed, but it ended up being empty, making the result
check incorrectly pass
* forward journal messages to console - if we fail to mount /var,
journald won't flush logs to the persistent storage and we end up
empty handed and with no clue what went wrong
For example, without systemd/systemd#16111 and with this patch:
...
[FAILED] Failed to start systemd-cryptsetup@varcrypt.service.
See 'systemctl status systemd-cryptsetup@varcrypt.service' for details.
[DEPEND] Dependency failed for cryptsetup.target.
...
[ 3.882451] systemd-cryptsetup[581]: Key file /etc/varkey is world-readable. This is not a good idea!
[ 3.883946] systemd-cryptsetup[581]: WARNING: Locking directory /run/cryptsetup is missing!
[ 3.884846] systemd-cryptsetup[581]: Failed to load Bitlocker superblock on device /dev/disk/by-uuid/180ba5ef-873b-4018-9968-47c23431f71a: Invalid argument
...
[ 4.099451] sh[606]: + mountpoint /var
[ 4.100025] sh[603]: + systemctl poweroff --no-block
[ 4.101636] systemd[1]: Finished systemd-user-sessions.service.
[ 4.102598] sh[608]: /var is not a mountpoint
[FAILED] Failed to start testsuite-02.service.
Let's create new images public by default and then symlink/copy them
into the respective private directories afterwards, not the other way
around. This should fix a nasty race condition in parallel runs where
one tests attempts to copy the backing public image at the same moment
another test is already modifying it.
Support running tests in parallel by switching to copying of the
base image instead of symlinking it..
This still requires some setup steps, like running `make setup` on tests
which have unique $IMAGE_NAME beforehand (and sequentially), otherwise
they'll all try to create the same base image when started in parallel,
leading to nasty issues. However, as running the integration tests in
parallel is such an unusual use case it should be good enough, for now.
As Debian/Ubuntu use /lib/systemd instead of /usr/lib/systemd,
add systemd-journal-remote to the list of programs that test-functions
detects the correct path to, and replace its direct usage with
$SYSTEMD_JOURNAL_REMOTE
Also use $JOURNALCTL instead of journalctl.
Also minor correction in install_plymouth() to look in /lib/... as
well as /usr/lib/... and /etc/...
Remove the artifact files indicating test result (testok, failed, and
skipped) just before running the test so we always get the latest and
most relevant result instead of incorrectly consuming previous results.
Discovered in https://github.com/systemd/systemd/pull/15378#issuecomment-616801873
This doesn't really matter, since in non-/usr-merged systems plymouth
needs to be in /bin and on merged ones it doesn't matter, but it is
still prettier to insert the right path, and avoid /bin on merged
systems, since it's just a compat symlink.
Replaces: #15351
