IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Let's drop stuff from the current $BUILD_DIR from the final coverage
report, as it's all generated files (mostly gperf) which we don't
really care about and it makes the Coveralls report confusing, since it
reports "source not available" for all such files.
When running on Debian/Ubuntu, I get a minute delay or so on every boot
because the local initramfs tries to resume from hibernation. This is
not really useful here, so always skip it
This gets rid of the all-but-one remaining uses of perl. I tested the new code
on my machine, so I'm fairly confident that it works as expected.
install_iscsi() has one majestic perl invocation, but we can't get rid of it
easily: it extends the code of tgt-admin to print some list of files. Obviously
this only works because tgt-admin is written in perl, and perl will be installed
if tgt-admin is installed. install_iscsi() is used in TEST-64-UDEV-STORAGE
conditionally if tgtadm is installed, so this can stay as is.
Stripping the binaries in the test images makes potential stack straces
quite useless, so let's drop the stripping stuff to make test fails a bit
more developer friendly.
Related: https://github.com/systemd/systemd-centos-ci/pull/616
Let's make the dropin, to make the build dir writable for gcov, a bit
more generic, so it can be used by all units starting with prefix test-.
This should help with a bunch of recent reports about missing coverage I
got, as well as with existing test units using DynamicUser=true.
This might feel a bit like a magic trick from behind the curtains, but I
want to touch the actual tests as little as possible, since it makes them
unnecessarily messy (see the various workarounds for sanitizers), and
the coverage reports are generated only in a specific CI job anyway.
systemd-repart needs to find mkfs.ext4 for the test.
This is located in the directory /usr/sbin on openSUSE Tumbleweed.
But since the variable ALWAYS_SET_PATH in /etc/login.defs is set to yes,
su re-initializes the $PATH variable and removes /usr/sbin.
Hence, mkfs.ext4 is not found and the test fails.
Using setpriv instead of su fixes this issue and is more appropriate to
do the switch user task from root.
[zjs: move setpriv to $BASICTOOLS and force-push to retrigger CI]
This is useful to identify log messages with metadata from the images
they run on. Look for ID/VERSION_ID/IMAGE_ID/IMAGE_VERSION/BUILD_ID,
with a SYSEXT_ prefix if we are looking at an extension, and append via
LogExtraFields= as respectively PORTABLE_NAME_AND_VERSION= in case of a
single image. In case of extensions, append as PORTABLE_ROOT_NAME_AND_VERSION=
for the base and one PORTABLE_EXTENSION_AND_VERSION= for each extension.
Example with a base and two extensions, with the unit coming from the
first extension:
[Service]
RootImage=/home/bluca/git/systemd/base.raw
Environment=PORTABLE=app0.raw
BindReadOnlyPaths=/etc/os-release:/run/host/os-release
LogExtraFields=PORTABLE=app0.raw
Environment=PORTABLE_ROOT=base.raw
LogExtraFields=PORTABLE_ROOT=base.raw
LogExtraFields=PORTABLE_ROOT_NAME_AND_VERSION=debian_10
ExtensionImages=/home/bluca/git/systemd/app0.raw
LogExtraFields=PORTABLE_EXTENSION=app0.raw
LogExtraFields=PORTABLE_EXTENSION_NAME_AND_VERSION=app_0
ExtensionImages=/home/bluca/git/systemd/app1.raw
LogExtraFields=PORTABLE_EXTENSION=app1.raw
LogExtraFields=PORTABLE_EXTENSION_NAME_AND_VERSION=app_1
When testing the binaries from the host, make sure to not store the state data
below /usr but use a dedicated directory in /var/tmp/ instead.
The working directories of the tests, initially located in /var/tmp, are also
moved in a dedicated directory /var/tmp/systemd-tests.
I noticed that our coverage reports miss some files completely - this
happens when the test doesn't touch the code in them at all, so the
generated coverage data (and resulting reports) have no information
about them. Let's fix this by doing an initial zero coverage capture
that contains a zeroed counter for every instrumented line in every
file, so when we later merge it with a capture from the test, it shows up
with a missing coverage instead of not showing at all.
/usr/lib/systemd/tests may contain more than the unit tests. For example on
SUSE we also install the integration tests there.
Putting the unit tests in a dedicated directory named 'unit-tests' makes the
layout cleaner.
Note that `run-unit-tests.py` has not been moved so we don't need to adjust
(Fedora) packaging and users also don't need to descend into the subdirectory.
Let's attempt to reduce the amount of flakes further when the AWS region
we run in is under heavy load and the hypervisor stars stealing our CPU
time.
Follow-up to e0cbb73911 and c78d18215b.
(The one case that is left unchanged is '< <(subcommand)'.)
This way, the style with no gap was already dominant. This way, the reader
immediately knows that ' < ' is a comparison operator and ' << ' is a shift.
In a few cases, replace custom EOF replacement by just EOF. There is no point
in using someting like "_EOL" unless "EOF" appears in the text.
On Arch both delv and dig pull in libnss_resolve:
```
$ grep resolve /etc/nsswitch.conf
hosts: mymachines resolve [!UNAVAIL=return] files myhostname dns
```
Since c78d18215b D-Bus services now have 60s to start, but the client
side (sd-bus) still waits only for 25s before giving up:
```
[ 226.196380] testsuite-71.sh[556]: + assert_in 'Static hostname: H' ''
[ 226.332965] testsuite-71.sh[576]: + set +ex
[ 226.332965] testsuite-71.sh[576]: FAIL: 'Static hostname: H' not found in:
[ 228.910782] sh[577]: + systemctl poweroff --no-block
[ 232.255584] hostnamectl[565]: Failed to query system properties: Connection timed out
[ 236.827514] systemd[1]: end.service: Consumed 2.131s CPU time.
[ 237.476969] dbus-daemon[566]: [system] Successfully activated service 'org.freedesktop.hostname1'
[ 237.516308] systemd[1]: system-modprobe.slice: Consumed 1.533s CPU time.
[ 237.794635] systemd[1]: testsuite-71.service: Main process exited, code=exited, status=1/FAILURE
[ 237.818469] systemd[1]: testsuite-71.service: Failed with result 'exit-code'.
[ 237.931415] systemd[1]: Failed to start testsuite-71.service.
[ 238.000833] systemd[1]: testsuite-71.service: Consumed 5.651s CPU time.
[ 238.181030] systemd[1]: Reached target testsuite.target.
```
Let's override the timeout in sd-bus as well to mitigate this.
Follow-up to c78d18215b.
- Use inst_recursive() and image_install() helpers where appropriate
- Update comments to explain why we need to install the test data manually in
$initdir
- Install manual/ in $initdir as TEST-35-LOGIN relies on
manual/test-session-properties
mkfs.vfat doesn't support specifying a root directory to bootstrap
the filesystem from (see https://github.com/dosfstools/dosfstools/issues/183).
Instead, we can use the mcopy tool from the mtools package to copy
files into the vfat filesystem after creating it without needing to
mount the vfat filesystem.
The glibc stuff on ppc64le C8S is a little bit wild, as there are two
versions:
```
$ ldconfig -p | grep libc.so
libc.so.6 (libc6,64bit, hwcap: "power9", OS ABI: Linux 3.10.0) => /lib64/glibc-hwcaps/power9/libc-2.28.so
libc.so.6 (libc6,64bit, OS ABI: Linux 3.10.0) => /lib64/libc.so.6
```
and with `/etc/ld.so.cache` present all binaries use the first one:
```
$ ldd /bin/cat
linux-vdso64.so.1 (0x00007fffa8070000)
libc.so.6 => /lib64/glibc-hwcaps/power9/libc-2.28.so (0x00007fffa7e20000)
/lib64/ld64.so.2 (0x00007fffa8090000)
```
However, without the cache the binaries will fall back to `/lib64/libc.so.6`
which breaks tests that use the minimal verity images (like TEST-29),
because we install only the first version (that's shown by `ldd` at
the time the images are created):
```
[ 91.595343] testsuite-29.sh[747]: + portablectl --profile=trusted attach --now --runtime /usr/share/minimal_0.raw minimal-app0
Starting systemd-portabled.service...
[ OK ] Started systemd-portabled.service.
Starting minimal-app0-foo.service...
Starting minimal-app0.service...
[ 104.432217] cat[858]: cat: error while loading shared libraries: libc.so.6: cannot open shared object file: No such file or directory
[ 104.435080] cat[857]: cat: error while loading shared libraries: libc.so.6: cannot open shared object file: No such file or directory
[FAILED] Failed to start minimal-app0.service.
See 'systemctl status minimal-app0.service' for details.
```
```
$ chroot /var/tmp/systemd-test.nMHPfc/minimal/
/bin/bash: error while loading shared libraries: libc.so.6: cannot open shared object file: No such file or directory
```
With the ldconfig's cache it seems to work as expected:
```
$ chroot /var/tmp/systemd-test.gVtYLg/minimal
bash-4.4# cat --version
cat (GNU coreutils) 8.30
...
```
The iscsi-init.service calls `sh` which might, in certain circumstances,
pull in instrumented systemd NSS modules causing `sh` to fail. Let's mitigate
this by pulling in an env file crafted by `create_asan_wrapper()` that
(among others) pre-loads ASan's DSO.
Since open-iscsi 2.1.2 [0] the initiator name should be generated via
a one-time service instead of distro package's post-install scripts.
However, some distros still use this approach even after this patch,
so prefer the already existing initiatorname.iscsi file if it exists.
[0] f37d5b653f
In most cases we refernced the concept as "initrd". Let's convert most
remaining uses of "initramfs" to "initrd" too, to stay internally
consistent.
This leaves "initramfs" only where it's relevant to explain historical
concepts or where "initramfs" is part of the API (i.e. in
/run/initramfs).
Follow-up for: b66a6e1a58
On certain systems the `install_libnss()` function might end up with an
empty list of libraries to install, which triggers an assertion in
`image_install()`:
```
I: Install libnss
..//test-functions: line 2721: 1: parameter null or not set
make: *** [Makefile:4: setup] Error 1
```
E.g.:
```
# LD_DEBUG=files getent passwd 2>&1 >/dev/null | sed -n '/calling init: .*libnss_/ {s!^.* /!/!; p}'
/lib64/libnss_sss.so.2
/lib64/libnss_systemd.so.2
# dnf -y remove sssd-client systemd-libs
# LD_DEBUG=files getent passwd 2>&1 >/dev/null | sed -n '/calling init: .*libnss_/ {s!^.* /!/!; p}'
<no output>
```
Let's handle this case gracefully.
Until now using the INTERACTIVE_DEBUG=yes stuff together with sanitizers
was almost impossible, since the console kept eating up our inputs or
not responding at all. After a painful day of debugging I noticed that
if we use a shell script in the initrd -> root transition, we might end up
with a plymouthd still running, which kept screwing with the tty.
E.g. with initrd -> wrapper -> systemd transition, where the `wrapper`
is a simple script:
```
exec -- /usr/lib/systemd/systemd "$@"
```
we'd end up with a stray plymouthd process after the bootup:
```
1 0 440 2 20 0 0 0 worker I ? 0:00 [kworker/5:2-ata_sff]
1 0 453 2 20 0 0 0 worker I ? 0:00 [kworker/9:2-rcu_gp]
5 0 456 1 20 0 7252 1960 do_epo S ? 0:00 @usr/sbin/plymouthd --mode=boot --pid-file=/run/plymouth/pid --attach-to-session
```
After killing it, the tty works finally as expected.
Set both timeouts to some reasonable values instead of just `infinity`
to provide some form of a safe-net in case the test goes haywire and the
environment didn't set the timeouts itself (like our CIs do).
to avoid scope clashing in case some previous test crashed and/or didn't
clean up properly. Currently all test machines are called `root`, since
the name is automagically derived from the container path (in this case
`/var/tmp/systemd-test.XXXXXX/root`).
E.g. (from Ubuntu CI):
```
[23:10:12] --x-- Running TEST-71-HOSTNAME --x--
make: Entering directory '/tmp/autopkgtest.5LjnBV/build.0mE/systemd/test/TEST-71-HOSTNAME'
+ make -C TEST-71-HOSTNAME setup run
TEST-71-HOSTNAME SETUP: test hostnamed
Reusing existing cached image /tmp/autopkgtest.5LjnBV/build.0mE/systemd/test/TEST-71-HOSTNAME/../default.img → /tmp/autopkgtest.5LjnBV/build.0mE/systemd/test/default.img
'/var/tmp/systemd-test.1yy2SS/default.img' -> '/tmp/autopkgtest.5LjnBV/build.0mE/systemd/test/default.img'
I: Masking supporting services
'/var/tmp/systemd-test.1yy2SS/root/etc/systemd/system/systemd-hwdb-update.service' -> '/dev/null'
'/var/tmp/systemd-test.1yy2SS/root/etc/systemd/system/systemd-journal-catalog-update.service' -> '/dev/null'
'/var/tmp/systemd-test.1yy2SS/root/etc/systemd/system/systemd-networkd.service' -> '/dev/null'
'/var/tmp/systemd-test.1yy2SS/root/etc/systemd/system/systemd-networkd.socket' -> '/dev/null'
'/var/tmp/systemd-test.1yy2SS/root/etc/systemd/system/systemd-resolved.service' -> '/dev/null'
TEST-71-HOSTNAME RUN: test hostnamed
+ env --unset=UNIFIED_CGROUP_HIERARCHY --unset=SYSTEMD_NSPAWN_UNIFIED_HIERARCHY timeout --foreground 1200 /bin/systemd-nspawn --register=no --kill-signal=SIGKILL --directory=/var/tmp/systemd-test.1yy2SS/root --setenv=SYSTEMD_UNIT_PATH=/usr/lib/systemd/tests/testdata/testsuite-71.units:/usr/lib/systemd/tests/testdata/units: /lib/systemd/systemd systemd.unit=testsuite.target systemd.wants=testsuite-71.service systemd.wants=end.service
Spawning container root on /var/tmp/systemd-test.1yy2SS/root.
Press ^] three times within 1s to kill container.
Failed to allocate scope: Unit root.scope already exists.
E: nspawn failed with exit code 1
```
Some tests (like TEST-02) set a multiline string to $KERNEL_APPEND
(which is a valid thing to do), unfortunately we'd use only the first
line of it and throw the rest away, e.g:
```
$ printf "%s" "$x"
hello
this is a multiline
kernel command line
$ read -ra out <<< "$x"
$ printf "%s" "${out[@]}"
hello
```
Let's use readarray/mapfile instead to avoid this:
```
$ readarray out <<< "$x"
$ printf "%s" "${out[@]}"
hello
this is a multiline
kernel command line
```
When an extension image has binaries they should match the host
architecture. Currently there is no way to specify this requirement.
Introduce an ARCHITECTURE field in the extension's release file that
may be set to prevent loading on the wrong host architecture.
Since this new field is introduced late, we don't want to make
specifying it mandatory as it would break existing sysext images.
See https://github.com/systemd/systemd/issues/24061
A sysext image that merely contains static binaries has no dependency
on the host distribution and should be able to be used anywhere.
Support the special '_any' value for the ID field in the extension to
opt-out of ID and VERSION_ID/SYSEXT_LEVEL matching.
See https://github.com/systemd/systemd/issues/24061
Since the library is dlopen()ed by libpthread and required during
pthread_exit()/pthread_cancel(), let's install it explicitly if available to
avoid unexpected fails in tests. This also consolidates all related
workarounds for this library across the test scripts.
Bash allows only one handler per signal, so let's overcome this
limitation by having one dedicated EXIT signal which runs all registered
handlers from all over the place.
so we can run TEST-24 under sanitizers as well.
Also, when at it, use the 'named-fields' sfdisk format to make the code
a bit more descriptive without needing a manual.
- use test_append_files() to install additional commands
- drop use of expect
- include assert.sh and use assertions at several places
- use timeout command at several places
- always use logind-test-user
- etc
since we delete the guest journals as part of the save_journal() step in
check_result_common(), making journal inaccessible from the custom check
hooks.
Let's detect & wrap binaries which are linked against systemd DSOs and
we're running under ASan, since otherwise running such binaries ends
with:
```
==633==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.
```
Since we unset $LD_PRELOAD in the testsuite-* units (due to another
issue), let's store the path to the ASan DSO in another env variable, so
we can easily access it in the testsuite scripts when needed.
If rngd is included in the host initrd, QEMU guests need at least one source of
entropy otherwise rngd will refuse to start. Hence this patch enables the
virtio RNG device in QEMU guests (exposed as a HW RNG device available at
/dev/hwrng).
As a safety measure, the patch limits the data sent to the guest to 1KB per
second in order to not let the guest starve the host entropy.
We used both "qemu" and "QEMU", let's use the lower-case version everywhere
since it's also the name of the binary and the version that people are
most familiar with.
The stuff under test/ is not only for the integeration tests, but also
for various other test-related stuff, so adjust the docs a bit.
Since c6552ad we now try to collect coverage even in situations where
it's basically impossible (like in test-mount-util where the whole / is
mounted as read-only). As dealing with this is not worth the trouble,
let's ignore the missing coverage errors thrown by gcov in such cases.
Add tests for enrolling and unlocking. Various cases are tested:
- Default PCR 7 policy w/o PIN, good and bad cases (wrong PCR)
- PCR 7 + PIN policy, good and bad cases (wrong PCR, wrong PIN)
- Non-default PCR 0+7 policy w/o PIN, good and bad cases (wrong PCR 0)
v2: rename test, fix tss2 library installation, fix CI failures
v3: fix ppc64, load module
IIUC, pipefail doesn't matter for a sequence of commands joined with &&, and we
don't have any pipes. And such a failing expression also does not trigger an
exit, so the set +e/set -e were noops.
It should make it easier to figure out what exactly services do there.
For example, with SYSTEMD_LOG_LEVEL=debug userdbd (v249) prints
```
varlink-5: New incoming message: {"method":"io.systemd.UserDatabase.GetUserRecord","parameters":{}}
```
before it crashes and systemd-resolved prints
```
varlink-21: New incoming message: {"method":"io.systemd.Resolve.ResolveAddress","parameters":{"address":[127,0,0,1],"flags":0,"ifindex":1000000,"family":0}}
```
and those messages are helpful (especially when scripts causing them
aren't clever enough to keep track of random stuff they send to systemd
:-))
otherwise units using `DynamicUser=yes` won't be able to write the
coverage stats (currently affecting TEST-20-MAINPIDGAMES).
`DynamicUser=yes` implies `ProtectSystem=strict` and
`ProtectHome=read-only` and can't be overridden hence we need to
utilize `ReadWritePaths=` to work around that.
If the test logs contain lines like:
```
...systemd-resolved[735885]: profiling:/systemd-meson-build/src/shared/libsystemd-shared-250.a.p/base-filesystem.c.gcda:Cannot open
```
it means we're possibly missing some coverage since gcov can't write the stats,
usually due to the sandbox being too restrictive (e.g. ProtectSystem=yes,
ProtectHome=yes) or the $BUILD_DIR being inaccessible to non-root users.
If we're built with `-Dportable=false`, the portable profiles won't get
installed into the image. Since we need only the profile files and
nothing else, let's copy them into the image explicitly in such case.
A bind mount is added directly from private on the host to the actual
destination directory, no need for the symlinks (which cannot be created
as the bind mount happens first and creates the target as an actual directory)
Fixes https://github.com/systemd/systemd/issues/22264
Wraps nspawn to be able to use pexpect. The test logs in on the console
and runs screen. In one screen window it types in shutdown commands and
checks whether a wall message was sent to the other.
19:50:59 F: Missing a shared library required by /var/tmp/systemd-test.NIPT2q/root/lib/systemd/libsystemd-core-250.so.
19:50:59 F: Run "ldd /var/tmp/systemd-test.NIPT2q/root/lib/systemd/libsystemd-core-250.so" to find out what it is.
19:50:59 F: libsystemd-shared-250.so => not found
19:50:59 F: Cannot create a test image.
Writing a byte to test10.socket is actually the root cause of issue #19154:
depending on the timing, it's possible that PID1 closes the socket before socat
(or nc, it doesn't matter which tool is actually used) tries to write that one
byte to the socket. In this case writing to the socket returns EPIPE, which
causes socat to exit(1) and subsequently make the test fail.
Since we're only interested in connecting to the socket and triggering the rate
limit of the socket, this patch removes the parts that write the single byte to
the socket, which should remove the race for good.
Since it shouldn't matter whether the test uses socat or nc, let's switch back
to nc and hence remove the sole user of socat. The exit status of nc is however
ignored because some versions might choke when the socket is closed
unexpectedly.
In order to avoid inflating the dependency list for the core
library, use dlopen when inspecting elfs, since it's only
used in two non-core executables.
so we can run TEST-46 under sanitizers once again.
`systemd-homed` runs fsck on home directories, which reports a memory
leak we're not interested in. Let's introduce an LSan suppression file
to get around this. Since the patterns in the suppression file are
matched using basic substring match[0], they're a bit cumbersome, but
should get the work one.
[0] https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer#suppressions
Example leaks (as reported by TEST-46):
```
systemd-homed[1333]: =================================================================
systemd-homed[1333]: ==1333==ERROR: LeakSanitizer: detected memory leaks
systemd-homed[1333]: Direct leak of 24 byte(s) in 1 object(s) allocated from:
systemd-homed[1333]: #0 0x7f0c8facccd1 in calloc (/usr/lib/clang/12.0.1/lib/linux/libclang_rt.asan-x86_64.so+0xf4cd1)
systemd-homed[1333]: #1 0x558d9494ff67 (/usr/bin/fsck+0x3f67)
systemd-homed[1333]: Direct leak of 6 byte(s) in 1 object(s) allocated from:
systemd-homed[1333]: #0 0x7f0c8fa906c1 in strdup (/usr/lib/clang/12.0.1/lib/linux/libclang_rt.asan-x86_64.so+0xb86c1)
systemd-homed[1333]: #1 0x558d949518fd (/usr/bin/fsck+0x58fd)
systemd-homed[1333]: SUMMARY: AddressSanitizer: 30 byte(s) leaked in 2 allocation(s).
systemd-homed[1337]: ==1337==WARNING: Symbolizer was blocked from starting itself!
systemd-homed[1337]: =================================================================
systemd-homed[1337]: ==1337==ERROR: LeakSanitizer: detected memory leaks
systemd-homed[1337]: Direct leak of 67584 byte(s) in 1 object(s) allocated from:
systemd-homed[1337]: #0 0x7f01edb84b19 (/usr/lib/clang/12.0.1/lib/linux/libclang_rt.asan-x86_64.so+0xf4b19)
systemd-homed[1337]: #1 0x7f01e8326829 (/usr/bin/../lib/libLLVM-12.so+0xb46829)
systemd-homed[1337]: SUMMARY: AddressSanitizer: 67584 byte(s) leaked in 1 allocation(s).
```
With the suppression file:
```
systemd-homed[1339]: -----------------------------------------------------
systemd-homed[1339]: Suppressions used:
systemd-homed[1339]: count bytes template
systemd-homed[1339]: 2 30 /bin/fsck$
systemd-homed[1339]: -----------------------------------------------------
systemd-homed[1343]: ==1343==WARNING: Symbolizer was blocked from starting itself!
systemd-homed[1343]: -----------------------------------------------------
systemd-homed[1343]: Suppressions used:
systemd-homed[1343]: count bytes template
systemd-homed[1343]: 1 67584 /lib/libLLVM
systemd-homed[1343]: -----------------------------------------------------
```
