1
0
mirror of https://github.com/systemd/systemd.git synced 2024-11-01 09:21:26 +03:00
Commit Graph

260 Commits

Author SHA1 Message Date
Zbigniew Jędrzejewski-Szmek
129cb6e249 shared/calendarspec: when mktime() moves us backwards, jump forward
When trying to calculate the next firing of 'Sun *-*-* 01:00:00', we'd fall
into an infinite loop, because mktime() moves us "backwards":

Before this patch:
tm_within_bounds: good=0 2021-03-29 01:00:00 → 2021-03-29 00:00:00
tm_within_bounds: good=0 2021-03-29 01:00:00 → 2021-03-29 00:00:00
tm_within_bounds: good=0 2021-03-29 01:00:00 → 2021-03-29 00:00:00
...

We rely on mktime() normalizing the time. The man page does not say that it'll
move the time forward, but our algorithm relies on this. So let's catch this
case explicitly.

With this patch:
$ TZ=Europe/Dublin faketime 2021-03-21 build/systemd-analyze calendar --iterations=5 'Sun *-*-* 01:00:00'
Normalized form: Sun *-*-* 01:00:00
    Next elapse: Sun 2021-03-21 01:00:00 GMT
       (in UTC): Sun 2021-03-21 01:00:00 UTC
       From now: 59min left
       Iter. #2: Sun 2021-04-04 01:00:00 IST
       (in UTC): Sun 2021-04-04 00:00:00 UTC
       From now: 1 weeks 6 days left           <---- note the 2 week jump here
       Iter. #3: Sun 2021-04-11 01:00:00 IST
       (in UTC): Sun 2021-04-11 00:00:00 UTC
       From now: 2 weeks 6 days left
       Iter. #4: Sun 2021-04-18 01:00:00 IST
       (in UTC): Sun 2021-04-18 00:00:00 UTC
       From now: 3 weeks 6 days left
       Iter. #5: Sun 2021-04-25 01:00:00 IST
       (in UTC): Sun 2021-04-25 00:00:00 UTC
       From now: 1 months 4 days left

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1941335.
2021-03-23 00:35:02 +01:00
Frantisek Sumsal
5acfe54e98 test: fix permissions of the ASan udev workaround
otherwise udev complains about the file being world-writable:
systemd-udevd[228]: Configuration file /etc/udev/rules.d/00-set-LD_PRELOAD.rules is marked world-writable. Please remove world writability permission bits. Proceeding anyway.

Fixes: systemd/systemd-centos-ci#354
2021-03-11 00:28:13 +09:00
Frantisek Sumsal
9f6235e1b4 test: fix TEST-50-DISSECT under sanitizers
This test would normally get stuck when trying to mount the verity image
due to:

systemd-udevd[299]: dm-0: '/usr/sbin/dmsetup udevflags 6293812'(err) '==371==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.'
systemd-udevd[299]: dm-0: Process '/usr/sbin/dmsetup udevflags 6293812' failed with exit code 1
...
systemd-udevd[299]: dm-0: '/usr/sbin/dmsetup udevcomplete 6293812'(err) '==372==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.'
systemd-udevd[299]: dm-0: Process '/usr/sbin/dmsetup udevcomplete 6293812' failed with exit code 1.
systemd-udevd[299]: dm-0: Command "/usr/sbin/dmsetup udevcomplete 6293812" returned 1 (error), ignoring.

so let's add a simple udev rule which sets $LD_PRELOAD for the block
subsystem.

Also, install the ASan library along with necessary dependencies into
the verity minimal image, to get rid of the annoying (yet harmless)
errors about missing library from $LD_LIBRARY.
2021-03-06 22:44:00 +01:00
Frantisek Sumsal
648fd18924 test: tidy up the ASan-related stuff 2021-03-06 22:43:58 +01:00
Frantisek Sumsal
3b5fc147d6 test: disable at_exit LSan check for dbus.service
When running integration tests under sanitizers D-Bus fails to
shutdown cleanly, causing unnecessary noise in the logs:

```
dbus-daemon[272]: ==272==LeakSanitizer has encountered a fatal error.
dbus-daemon[272]: ==272==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
dbus-daemon[272]: ==272==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)
```

Since we're not "sanitizing" D-Bus anyway let's disable LSan's at_exit
check for the dbus.service to get rid of this error.
2021-03-06 11:27:04 +09:00
Luca Boccassi
93f597013a Add ExtensionImages directive to form overlays
Add support for overlaying images for services on top of their
root fs, using a read-only overlay.
2021-02-23 15:34:46 +00:00
Luca Boccassi
0761da386a test: avoid leaking open loop devices
When a subshell is used ('make' or 'make all') the LOOPDEV environment
variable, which is used to store the opened loop device, is lost.
So the cleanup on trap/exit doesn't do anything, and the loop
device used to mount the test image is left around.

Avoid using a subshell to fix the issue.
2021-02-17 18:55:05 +01:00
Luca Boccassi
7eba0c50f5 test: install binaries from local d/control file
The source package in the apt cache might be older than the
packaging from salsa.debian.org/systemd-team/systemd so it might not
list all the current binary packages.
This is currently the case for systemd-timesyncd, so TEST-30 fails.

Simply grep the control file rather than using apt-cache when iterating
over the packages contents.
2021-02-15 10:17:39 +01:00
Zbigniew Jędrzejewski-Szmek
b3c57df0f5
Merge pull request #18401 from anitazha/oomdxattr
oomd: implement avoid/omit support for cgroups
2021-02-13 10:00:31 +01:00
Luca Boccassi
8b535f3aac test: setup var/tmp in the test image as well 2021-02-10 18:54:12 +00:00
Luca Boccassi
e763342091 test: add empty /etc/resolv.conf in test image
Portable services have a BindPath pointing to it, so it needs to
be available in the image
2021-02-10 18:54:12 +00:00
Luca Boccassi
9785c44dd8 test-50-dissect: move minimal image setup to common setup function
So that it can be re-used for other tests by simply setting
TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to
replicate the setup commands.
2021-02-10 18:54:12 +00:00
Frantisek Sumsal
2de908aead test: count call instructions as well
Binaries on the latest Arch Linux use `call` instructions instead of
`callq`, which breaks the ASan detection and eventually the image
building process (due to insufficient space).
2021-02-10 20:20:31 +03:00
Anita Zhang
59331b8e29 oom: implement avoid/omit xattr support
There may be situations where a cgroup should be protected from killing
or deprioritized as a candidate. In FB oomd xattrs are used to bias oomd
away from supervisor cgroups and towards worker cgroups in container
tasks. On desktops this can be used to protect important units with
unpredictable resource consumption.

The patch allows systemd-oomd to understand 2 xattrs:
"user.oomd_avoid" and "user.oomd_omit". If systemd-oomd sees these
xattrs set to 1 on a candidate cgroup (i.e. while attempting to kill something)
AND the cgroup is owned by root, it will either deprioritize the cgroup as
a candidate (avoid) or remove it completely as a candidate (omit).

Usage is restricted to root owned cgroups to prevent situations where an
unprivileged user can set their own cgroups lower in the kill priority than
another user's (and prevent them from omitting their units from
systemd-oomd killing).
2021-02-09 02:27:40 -08:00
Luca Boccassi
21be71eefb test: fix 'make clean' not removing shared image 2021-01-29 09:56:45 +00:00
Dan Streetman
8fa038085e test/test-functions: allow installing systemd files from local system
Add NO_BUILD var to allow testing with no local build, by installing
local systemd files into the image.

This only works for debian-like distros currently, that use the
tools 'apt' and 'dpkg' for package management.
2021-01-26 17:09:48 -05:00
Dan Streetman
12d31e4ea5 test: find $BUILD_DIR in test-functions, remove from other scripts
The $BUILD_DIR is only used in test-functions, and doesn't need to
be specified in any other scripts. Additionally, to be able to allow
the integration test suite to be run against locally installed binaries,
instead of built binaries, moving BUILD_DIR logic completely into
test-functions allows later patches to be simpler.
2021-01-26 17:09:48 -05:00
Dan Streetman
42f3b48c97 test/test-functions: add variables for several dir locations 2021-01-26 17:09:39 -05:00
Dan Streetman
1918406900 test/test-functions: move var assignment
This makes no code change, only moves a small block of vars higher in the
file. This makes the next commit a bit easier to read.
2021-01-26 16:05:49 -05:00
Luca Boccassi
d9e606e89f tests: build the image once and then copy/extend it
Building custom images for each test takes a lot of time.
Build the default one, and if the test needs incompatible changes
just copy it and extend it instead.
2021-01-24 13:07:39 +00:00
Zbigniew Jędrzejewski-Szmek
6eab0c6dba Revert "test-functions: make sure we test our own libudev instead of the host libudev"
This reverts commit 73484ecff9.

3976f372ae moved libudev.so to be built in the
main directory, so this addition to $LD_LIBRARY_PATH is now obsolete.

After that commit, we build the following shared libraries:

build/libnss_myhostname.so.2
build/libnss_mymachines.so.2
build/libnss_resolve.so.2
build/libnss_systemd.so.2
build/libsystemd.so.0.30.0
build/libudev.so.1.7.0
build/pam_systemd.so
build/pam_systemd_home.so
build/src/boot/efi/stub.so
build/src/boot/efi/systemd_boot.so
build/src/shared/libsystemd-shared-247.so

EFI stubs don't matter, and libsystemd-shared-nnn.so is loaded through rpath,
and is doesn't need to and shouldn't be in $LD_LIBRARY_PATH. In effect, we only
ever need to add the main build directory to the search path.
2021-01-20 14:04:52 +09:00
Lennart Poettering
b5e0c17654 test: improve a log message while building test images 2021-01-19 13:41:42 +01:00
Luca Boccassi
fa7a3cd00e test: run strace with -f and copy log out 2021-01-18 17:24:05 +00:00
Luca Boccassi
a9d34376e6 test: skip missing optional libraries in image install
Not all optional libraries might be available on developers machines,
so log and skip.
Also some pkg-config files are broken (eg: tss2 on Debian Stable) so
skip if the required variables are missing, and improve logs.
2021-01-18 17:24:05 +00:00
Luca Boccassi
eb3785f367 tests: add TEST_PREFER_NSPAWN variable to run as many as possible under nspawn
By default the test suite prefers qemu, and uses nspawn only if
a test specifically says it doesn't support qemu.
Add a variable to allow flipping the default, and run as many
tests under nspawn as possible.
2021-01-01 21:33:16 +00:00
Luca Boccassi
51d56d3be0 tests: add TEST_QEMU_ONLY variable to run only tests where qemu is mandatory
Allows to split the test run in two parts. Most tests can run under
nspawn which is much faster, and they can be ran in one chunk with
TEST_NO_QEMU=1. The qemu-only tests, which are just a handful, can
be ran in another chunk with TEST_QEMU_ONLY=1.

Allows autopkgtest to be split in two parts.
2021-01-01 21:33:16 +00:00
Luca Boccassi
aeac20fc69 test: shortcut skip if both TEST_NO_QEMU and TEST_NO_NSPAWN are set
Allows to run all tests in bulk with TEST_NO_QEMU, skipping those where it is
mandatory, without wasting time building the image.
2021-01-01 21:33:16 +00:00
Luca Boccassi
0515e4c17d test: check for binariers in [usr/][s]bin
The image build function greps for ExecStart lines in unit files, but some
of them (eg: systemd-firstboot) do not use a full path.
It then falls back to 'type -P' but that only works if you have the binary
installed. For optional binaries like systemd-firstboot, the installation
can then fail.

Manually check if the binary already exists in /[usr/][s]bin.
2021-01-01 21:33:16 +00:00
Luca Boccassi
53a1c94480 test: use pkg-config to get user unit dir when installing dbus user socket
Usually on Debian ROOTLIBDIR is /lib/<arch triplet>, which is not the right place.
Use pkg-config since we define it, and then fallback to /usr/lib/systemd/user which is
the canonical location.
On both Debian&friends and Fedora dbus/dbus-broker install the user socket/service
under /usr/lib/systemd/user, not /lib/systemd/systemd/user.
2021-01-01 21:33:16 +00:00
Lennart Poettering
a60d5b2f38 test: add tpm2 and fido2 libs to dlopen test 2020-12-17 20:02:24 +01:00
Frantisek Sumsal
f26d091460 test: drop the trailing whitespace from the QEMU version check
I suspect the original version of the regex was written on a system,
which prints both the QEMU version and the QEMU package version in the
--version output, like Fedora:

$ /bin/qemu-system-x86_64 --version
QEMU emulator version 4.2.1 (qemu-4.2.1-1.fc32)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

However, Arch Linux prints only the QEMU version:
$ /bin/qemu-system-x86_64 --version
QEMU emulator version 5.2.0
Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers

This causes the awk regex to not match the version string, since there's
no whitespace after it, causing the version check to fail (as well as the
TEST-36-NUMAPOLICY) as well.

Follow-up for 43b49470d1.
2020-12-16 16:38:26 +01:00
Christian Ehrhardt
43b49470d1 test: use modern qemu numa arguments
Upgrading to qemu 5.2 breaks TEST-36-NUMAPOLICY like:
  qemu-system-x86_64: total memory for NUMA nodes (0x0) should
  equal RAM size (0x20000000)

Use the new (as in >=2014) form of memdev in test 36:
 -object memory-backend-ram,id=mem0,size=512M -numa node,memdev=mem0,nodeid=0

Since some target systems are as old as qemu 1.5.3 (CentOS7) but the new
kind to specify was added in qemu 2.1 this needs to add version parsing and
add the argument only when qemu is >=5.2.

Fixes #17986.

Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
2020-12-16 05:16:41 +09:00
Luca Boccassi
b7fca1b059 test: pull in weak deps into tests
(Pulled from @bluca's comment here:
https://github.com/systemd/systemd/pull/17884#issuecomment-740005624 and
turned into a commit by @poettering)
2020-12-09 13:34:49 +01:00
Dan Streetman
3965279c67 test/test-functions: copy /usr/lib/pam.d into $initdir
The systemd-user file has been moved from /etc/pam.d into /usr/lib/pam.d,
so test-functions needs to copy it from /usr/lib/pam.d instead.

This will copy it from either location.
2020-11-23 17:17:19 -05:00
Anita Zhang
e4ff80404a test: add basic memory pressure extended test for oomd
Simple test to generate a lot of pressure in one unit and assert that
systemd-oomd kills the right one.
2020-10-22 17:04:22 -07:00
Lennart Poettering
73484ecff9 test-functions: make sure we test our own libudev instead of the host libudev
When invoking "ldd" to find dependency libraries we already set
$LD_LIBRARY_PATH to point to our own build tree, so that our libraries
are checked, not the host libraries. This is not sufficient howeever, as
libudev is built in a subdir. Add that, too.
2020-09-01 17:40:12 +02:00
Zbigniew Jędrzejewski-Szmek
9f927e46ac test/test-functions: do not execute strip_binaries twice
It is called from setup_basic_environment(), which also calls install_symtemd()
a bit earlier, so in effect it would be called twice.
2020-08-27 13:10:30 +02:00
Zbigniew Jędrzejewski-Szmek
ae6c598791 tests/TEST-50: support the case when /etc/os-release is present
We have four legal cases:
1. /usr/lib/os-release exists and /etc/os-release is a symlink to it
2. both exist but /etc/os-release is not a symlink to /usr/lib/os-release
3. only /usr/lib/os-release exists
4. only /etc/os-release exists

The generic setup code in test-functions and create-busybox-image didn't handle
case 3.

The test-specific code in TEST-50 didn't handle 2 (because the general setup
code would only install /etc/os-release in the image and
grep -f /usr/lib/os-release would not work) and 4 (same reason) and would fail
in case 3 in generic setup.
2020-08-27 13:08:14 +02:00
Elisei Roca
2aa5a13aa9 test: adapt test-functions for SUSE 2020-07-22 10:42:42 +02:00
Frantisek Sumsal
8f843190ff test: bump the timeout for systemd-hwdb-update.service under ASan
Since the hwdb update from a79be2f807
the systemd-hwdb-update service started timing out under ASan when
compiled with gcc, as we started tripping over the 3 minutes timeout.

This affects only gcc runs, since the current gcc on Arch still suffers
from the detect_stack_use_after_return performance penalty[0]. Until
the fixed gcc is present in the respective repositories, let's bump
the timeout to 4 minutes, as we might not be able to upgrade right
away, due to systemd/systemd#16199.

Before the hwdb update:
[ 7958.292540] systemd[63]: systemd-hwdb-update.service: Executing: /usr/bin/time systemd-hwdb update
[ 7958.304005] systemd[1]: systemd-journald.service: Got notification message from PID 44 (FDSTORE=1)
[ 7958.314434] systemd[1]: systemd-journald.service: Added fd 3 (n/a) to fd store.
[ 8008.520082] systemd[1]: systemd-journald.service: Got notification message from PID 44 (WATCHDOG=1)
[ 8068.520151] systemd[1]: systemd-journald.service: Got notification message from PID 44 (WATCHDOG=1)
[ 8125.682843] time[63]: 84.47user 82.92system 2:47.50elapsed 99%CPU (0avgtext+0avgdata 811512maxresident)k
[ 8125.682843] time[63]: 0inputs+19680outputs (0major+25000853minor)pagefaults 0swaps

After the hwdb update:
[ 6215.491958] systemd[63]: systemd-hwdb-update.service: Executing: /usr/bin/time systemd-hwdb update
[ 6215.503380] systemd[1]: systemd-journald.service: Got notification message from PID 44 (FDSTORE=1)
[ 6215.514172] systemd[1]: systemd-journald.service: Added fd 3 (n/a) to fd store.
[ 6329.392918] systemd[1]: systemd-journald.service: Got notification message from PID 44 (WATCHDOG=1)
[ 6394.920205] time[63]: 89.48user 89.98system 2:59.55elapsed 99%CPU (0avgtext+0avgdata 812764maxresident)k
[ 6394.920205] time[63]: 0inputs+20568outputs (0major+27318354minor)pagefaults 0swaps

[0] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94910
2020-06-29 16:07:11 +02:00
Elisei Roca
9e1732924d
test-functions: read /usr/etc/nsswitch.conf if /etc/nsswitch.conf does not exist (#16195)
See this for more info why and since when this change is needed: https://build.opensuse.org/request/show/807179.
2020-06-23 07:42:15 +02:00
Frantisek Sumsal
e47add9edc test: make TEST-02-CRYPTSETUP a bit more robust
Prompted by systemd/systemd#16111.

* check if /var is a mountpoint - if not, something went wrong. In case
  of systemd/systemd#16111 the /failed file was created, because
  systemd-cryptsetup failed, but it ended up being empty, making the result
  check incorrectly pass
* forward journal messages to console - if we fail to mount /var,
  journald won't flush logs to the persistent storage and we end up
  empty handed and with no clue what went wrong

For example, without systemd/systemd#16111 and with this patch:
...
[FAILED] Failed to start systemd-cryptsetup@varcrypt.service.
See 'systemctl status systemd-cryptsetup@varcrypt.service' for details.
[DEPEND] Dependency failed for cryptsetup.target.
...
[    3.882451] systemd-cryptsetup[581]: Key file /etc/varkey is world-readable. This is not a good idea!
[    3.883946] systemd-cryptsetup[581]: WARNING: Locking directory /run/cryptsetup is missing!
[    3.884846] systemd-cryptsetup[581]: Failed to load Bitlocker superblock on device /dev/disk/by-uuid/180ba5ef-873b-4018-9968-47c23431f71a: Invalid argument
...
[    4.099451] sh[606]: + mountpoint /var
[    4.100025] sh[603]: + systemctl poweroff --no-block
[    4.101636] systemd[1]: Finished systemd-user-sessions.service.
[    4.102598] sh[608]: /var is not a mountpoint
[FAILED] Failed to start testsuite-02.service.
2020-06-10 17:42:25 +02:00
Jay Burger
eed44b715d fix_test_function_timeout 2020-05-30 12:01:29 +02:00
Frantisek Sumsal
2991fa41e4 test: create public images by default
Let's create new images public by default and then symlink/copy them
into the respective private directories afterwards, not the other way
around. This should fix a nasty race condition in parallel runs where
one tests attempts to copy the backing public image at the same moment
another test is already modifying it.
2020-05-27 08:41:02 +02:00
Frantisek Sumsal
7a57256c73 test: introduce TEST_PARALLELIZE to support running tests in parallel
Support running tests in parallel by switching to copying of the
base image instead of symlinking it..

This still requires some setup steps, like running `make setup` on tests
which have unique $IMAGE_NAME beforehand (and sequentially), otherwise
they'll all try to create the same base image when started in parallel,
leading to nasty issues. However, as running the integration tests in
parallel is such an unusual use case it should be good enough, for now.
2020-05-17 10:48:16 +02:00
Frantisek Sumsal
e89450921f test: fix public/private image detection and prefer the latter one 2020-05-17 10:46:34 +02:00
Zbigniew Jędrzejewski-Szmek
b76ef59756
Merge pull request #13512 from msekletar/freezer
core: introduce support for cgroup freezer
2020-05-01 07:52:29 +02:00
Dan Streetman
cad6727906 test: find path for systemd-journal-remote
As Debian/Ubuntu use /lib/systemd instead of /usr/lib/systemd,
add systemd-journal-remote to the list of programs that test-functions
detects the correct path to, and replace its direct usage with
$SYSTEMD_JOURNAL_REMOTE

Also use $JOURNALCTL instead of journalctl.

Also minor correction in install_plymouth() to look in /lib/... as
well as /usr/lib/... and /etc/...
2020-04-30 22:33:47 +02:00
Michal Sekletár
d446ae89c0 test: add test for cgroup v2 freezer support 2020-04-30 19:02:55 +02:00
Frantisek Sumsal
80b44b38b5 test: correctly clean up test artifacts
Remove the artifact files indicating test result (testok, failed, and
skipped) just before running the test so we always get the latest and
most relevant result instead of incorrectly consuming previous results.

Discovered in https://github.com/systemd/systemd/pull/15378#issuecomment-616801873
2020-04-22 08:57:38 +02:00