1
0
mirror of https://github.com/systemd/systemd.git synced 2024-12-22 17:35:35 +03:00
Commit Graph

371 Commits

Author SHA1 Message Date
Zbigniew Jędrzejewski-Szmek
35f6e5c327 tools/update-distro-hash: add a helper script to sync submodule 2024-06-06 10:02:23 +02:00
Daan De Meyer
ba592dc715 mkosi: Replace submodules with our own thing
Unfortunately, git submodules break in all sorts of ways:

- Various github workflows (dependabot, github pages) try to do a shallow
clone of git submodules which does not work at all when the git repository
is hosted on pagure (https://pagure.io/pagure/issue/5453,
https://github.com/dependabot/dependabot-core/issues/9391).
- If the git forge hosting the git repository uses SHA256, then it breaks our
usage of it as a submodule as SHA256 repositories cannot be used as submodules
in SHA1 repositories (src.opensuse.org moved to SHA256 which broke our usage of
opensuse's systemd spec as a submodule).
- git submodules completely break usage of git worktrees.
- ...

Let's avoid all these issues by just doing our own home grown implementation of
git submodules. We lose the automatic dependabot updates this way but since dependabot
fails to run more often that not with submodules we don't really lose anything.
2024-05-30 19:31:32 +02:00
Daan De Meyer
f779fd1f8f meson: Set up git submodule update on post checkout as well 2024-05-05 14:49:21 +02:00
Luca Boccassi
6dfaa0edaf
Merge pull request #32204 from DaanDeMeyer/post-rewrite
git: Add post-rewrite hook that invokes git submodule update
2024-04-10 22:52:45 +01:00
Daan De Meyer
f1e9e8041c git: Add post-rewrite hook that invokes git submodule update
git rebase does not support a --recurse-submodules switch to automatically
check out the submodules at their registered commits during or after a rebase.

Instead, let's use the post-rewrite git hook to do this ourselves.
2024-04-10 14:39:49 +02:00
Antonio Alvarez Feijoo
2ad44c22fe
tools/check-version-history: skip test if lxml is not installed
```
================================= 1070/1408 ==================================
test:         systemd:dist / check-version-history
start time:   07:55:05
duration:     0.03s
result:       exit status 1
...
----------------------------------- stderr -----------------------------------
Traceback (most recent call last):
  File "/mnt/work/systemd/upstream-fork/main/tools/check-version-history.py", line 6, in <module>
    import lxml.etree as tree
ModuleNotFoundError: No module named 'lxml'
==============================================================================
```

Follow-up for 3691e7fce7
2024-04-10 11:10:02 +02:00
Yu Watanabe
dcc6256593 git-contrib: use 'git shortlog' command
Also, this drops Weblate (again) and dependabot from the contributers list.

Moreover, this makes the contributers sorted by git command, rather
than sort command. Then, the authors are sorted by their first name, e.g.
- before
Xiaotian Wu, Yuri Chornoivan, Yu Watanabe, Zbigniew Jędrzejewski-Szmek,
- after
Xiaotian Wu, Yu Watanabe, Yuri Chornoivan, Zbigniew Jędrzejewski-Szmek,

Suggested-by: Matteo Croce <teknoraver@meta.com>
2024-04-03 10:07:54 +09:00
Antonio Alvarez Feijoo
f0896ca557 efi: skip check-alignment-* tests if pefile is not installed
Otherwise, when building with ukify disabled, no one else requires pefile, and
if it is not installed, those tests fail:

```
================================== 12/1212 ===================================
test:         systemd:efi / check-alignment-systemd-bootx64.efi
start time:   14:21:54
duration:     0.08s
result:       exit status 1
command:      /mnt/work/systemd/upstream-fork/main/tools/check-efi-alignment.py /mnt/work/systemd/upstream-fork/main/build/src/boot/efi/systemd-bootx64.efi
----------------------------------- stderr -----------------------------------
Traceback (most recent call last):
  File "/mnt/work/systemd/upstream-fork/main/tools/check-efi-alignment.py", line 7, in <module>
    import pefile
ModuleNotFoundError: No module named 'pefile'
==============================================================================

================================== 13/1212 ===================================
test:         systemd:efi / check-alignment-linuxx64.efi.stub
start time:   14:21:54
duration:     0.08s
result:       exit status 1
command:      /mnt/work/systemd/upstream-fork/main/tools/check-efi-alignment.py /mnt/work/systemd/upstream-fork/main/build/src/boot/efi/linuxx64.efi.stub
----------------------------------- stderr -----------------------------------
Traceback (most recent call last):
  File "/mnt/work/systemd/upstream-fork/main/tools/check-efi-alignment.py", line 7, in <module>
    import pefile
ModuleNotFoundError: No module named 'pefile'
==============================================================================

================================== 14/1212 ===================================
test:         systemd:efi / check-alignment-addonx64.efi.stub
start time:   14:21:54
duration:     0.07s
result:       exit status 1
command:      /mnt/work/systemd/upstream-fork/main/tools/check-efi-alignment.py /mnt/work/systemd/upstream-fork/main/build/src/boot/efi/addonx64.efi.stub
----------------------------------- stderr -----------------------------------
Traceback (most recent call last):
  File "/mnt/work/systemd/upstream-fork/main/tools/check-efi-alignment.py", line 7, in <module>
    import pefile
ModuleNotFoundError: No module named 'pefile'
==============================================================================
```

Follow-up for 7ff3b88396
2024-03-27 02:32:42 +09:00
Zbigniew Jędrzejewski-Szmek
da867fa3c3
Merge pull request #31907 from mrc0mmand/efi-shenanigans
efi: check if all sections of our EFI binaries are properly aligned
2024-03-23 12:04:14 +01:00
Zbigniew Jędrzejewski-Szmek
9a50330476
Merge pull request #31779 from keszybz/elf2efi-clang-18
Make elf2efi work with clang-18
2024-03-23 12:02:34 +01:00
Zbigniew Jędrzejewski-Szmek
125f06dd1c tools/elf2efi: elif→if to make pylint happy 2024-03-22 15:44:17 +01:00
Zbigniew Jędrzejewski-Szmek
6d03e5523c tools/elf2efi: skip empty .got section and its .relro_padding
Resolves https://github.com/systemd/systemd/issues/31637.

lld-18 does the section setup differently than older versions. There is a bunch
of ordering chagnes, but it also inserts the following:

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
...
  9 .got          00000000  00000000000283c0  00000000000283c0  000283c0  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 10 .relro_padding 00000c40  00000000000283c0  00000000000283c0  000283c0  2**0
                  ALLOC
 11 .data         00000024  00000000000293c0  00000000000293c0  000283c0  2**4
                  CONTENTS, ALLOC, LOAD, DATA
...

This causes a problem for us, because we try to map the .got to .rodata,
and the subsequent .data to .data, and round down the VMA to the nearest
page, which causes the PE sections to overlap.

https://github.com/llvm/llvm-project/pull/66042 adds .relro_padding to make
sure that the RELRO segment is properly write protected and allocated. For our
binaries, the .got section is empty, so we can skip it safely, and the
.relro_padding section is not useful once .got has been dropped.

We don't expect .got sections, but they are apparently inserted on i386 and
aarch64 builds. Emit a warning until we figure out why they are there.
2024-03-22 15:42:57 +01:00
Frantisek Sumsal
7ff3b88396 efi: check if all sections of our EFI binaries are properly aligned 2024-03-22 14:02:03 +01:00
Daan De Meyer
e33d43b059 docs: Add one more git submodule setting to configure 2024-03-21 23:02:52 +00:00
Daan De Meyer
f680c4c95e dbus-exporter: Set explicit mode on output directory
Otherwise the created directory might have the sticky bit or the setgid
bit set as these are inherited from the parent directory.
2024-03-22 07:54:25 +09:00
Daan De Meyer
6233a76932 meson: Rename add-git-hook.sh to git-setup.sh and configure git in it
Let's automatically apply the recommended git config with meson.
2024-03-15 17:27:49 +01:00
Zbigniew Jędrzejewski-Szmek
8a753717f8 tools/elf2efi: rework exception messages
RuntimeError is documented as "Unspecified run-time error". It doesn't make
much sense for Python. (It originated in Java, where exceptions that can be
thrown by a function are declared in the function signature. All code calling
such a function must either explicitly catch all possible exception types, or
allow them to propagate by listing them in its own exception type list. This is
nice in theory, but in practice very annoying. Especially during development,
when the list of possible exception types is not finalized, we would end up
adding and removing exceptions to functions signatures all the time. Also for
code which is designed to call functions recursively, we would soon end up with
all functions declaring all possible exception types… To avoid this, people
would quite often do fake handling with a block that either prints and ignores
an exception, or has just a comment like "fix me later", or even nothing. This
often lead to people forgetting to adjust this later on and production code
containing such constructs. An escape hatch was opened with RuntimeException and
its subclasses, which do not need to be pre-declared. Various memory-related
exceptions were added as subclasses of RuntimeException. But later on, people
starting using this to not to have to declare all exception types everywhere.)

In Python, exceptions do no have to be pre-declared, and for code which just
encounters a failure, we should raise a specific exception type. The catch-all
class for unexpected input is ValueError.

For https://github.com/systemd/systemd/issues/31637:
BadSectionError: Section '.data' @0x28000 overlaps previous section @0x28000+0x300=@0x28300

Also, exception strings should not contain trailing periods, because they are
often embedded in sentences.
2024-03-14 11:08:41 +01:00
Zbigniew Jędrzejewski-Szmek
642f991b70 tools/elf2efi: split out function to create parser
main() is supposed to be lean and mean.
2024-03-14 10:32:17 +01:00
Zbigniew Jędrzejewski-Szmek
a0797b4ad7 tools/elf2efi: align columns in tables, unify formatting
For tables which represent binary data structures, readability is greatly
enhanced if the part which shows field size and type is aligned. This follows
the usual style for tables in the rest of the systemd codebase.

Also, use the same style for functions: if the function signature is too long
to fit in one line, put each parameter on a separate line.

Also, for comprehension expressions, if they are split, use the usual Python
style.

Also, drop format annotations, since the code isn't automatically formatted
anymore, and automatic formatting is neither feasible nor a goal for the
systemd codebase.
2024-03-14 10:32:17 +01:00
Daan De Meyer
5c25f9dbc5 meson: Remove version_h dependency from jinja2_cmdline
version_h includes GIT_VERSION which only makes sense for C files
which aren't preprocessed by jinja2 so remove the argument.

The end result of this change is that the man pages are not recompiled
anymore every time GIT_VERSION changes.
2024-03-06 15:16:52 +01:00
Zbigniew Jędrzejewski-Szmek
0ace2560ea
Merge pull request #31352 from DaanDeMeyer/versioning
Rework meson-vcs-tag.sh
2024-02-23 11:07:56 +01:00
Daan De Meyer
3f6ce3d4f0 meson: Decouple the version tag from the vcs tag
Let's split off a new vcs-tag option from version-tag that configures whether
the current commit should be appended to the version tag. Doing this saves
us from having to fiddle around with generating git versions in packaging
specs and instead let's meson do it for us, even if we pass in a custom
version tag.

With this approach there's no more need for tools/meson-vcs-tag.sh so
we remove it.
2024-02-21 14:01:25 +01:00
Frantisek Sumsal
5083e42765 test: verify our own units (where applicable)
This is inspired by one of our internal tests that does pretty much the
same thing. However, it is slightly more convoluted than I'd like it to
be, since I really don't want to duplicate the list of our units in
another place, so we need to, somehow, pass the list from the meson file
to the test script. I originally envisioned this to be a part of the
unit test suite, but this doesn't work for unit files with absolute
paths to binaries, as we'd have to install the build first (maybe using
a chroot would work?).

It doesn't check man pages (since they might not be installed on the
test machine) and also skip recursive dependencies (as that would trip
over issues in files that are not under our direct control), but it
should still cover typos and such.

There are currently two units for which the check had to be disabled -
syslog.socket, as the corresponding syslog.service might not be
installed, and rc-local.service as that's a compat API and the necessary
/etc/rc.d/rc.local file may not (and most likely won't be) present.
2024-02-20 15:45:50 +01:00
Daan De Meyer
dbb9a6ad69 meson-vcs-tag: Say version format specification
That's the official name so let's use it.
2024-02-14 19:58:19 +01:00
Daan De Meyer
ea2a57bee3 meson: Start adding devel and rc suffixes to the project version
Let's make sure that versions generated by meson-vcs-tag.sh always
sort higher than official and stable releases. We achieve this by
immediately updating the meson version in meson.build after a new
release. To make sure this version always sorts lower than future
rcs, we suffix it with "~devel" which will sort lower than "~rcX".

The new release workflow is to update the version in meson.build
for each rc and the official release and to also update the version
number after a new release to the next development version.

The full version is exposed as PROJECT_VERSION_FULL and used where
it makes sense over PROJECT_VERSION.

We also switch to reading the version from a meson.version file in
the repo instead of hardcoding it in meson.build. This makes it
easier to access both inside and outside of the project.

The meson-vcs-tag.sh script is rewritten to query the version from
meson.version instead of passing it in via the command line. This
makes it easier to use outside of systemd since users don't have to
query the version themselves first.
2024-02-14 15:36:34 +01:00
Yu Watanabe
dcd484e545 tools: fix typo
Follow-up for 6d55e3a364.
2024-02-14 03:58:35 +09:00
Daan De Meyer
6d55e3a364 Use tilde for rc tag versioning
tilde sorts lower in the version comparison spec:
https://uapi-group.org/specifications/specs/version_format_specification/

➜  systemd git:(strip) systemd-analyze compare-versions 249\~rc1 249
249\~rc1 < 249
➜  systemd git:(strip) systemd-analyze compare-versions 249-rc1 249
249-rc1 > 249

Also update tools/meson-vcs-tag.sh to use carets instead of hyphens
for the git part of the version as carets are allowed to be part of
a version by pacman while hyphens are not and both sort higher than
a version without the git part.
2024-02-13 18:14:36 +01:00
Zbigniew Jędrzejewski-Szmek
9c8a0ca69e tmpfiles: split out verbs in help
One of the three must always be specified, but they buried in a long list of
options in the output of --help. Make them more visible to draw the eye.

Also, drop "marked" from the description. It's supposed to mean "configured",
but it's a strange way to say that, and also it's generally obvious that the
program does what its configuration tells it to, and it's not going to remove
all files found on the system.
2024-02-09 17:57:41 +01:00
Frantisek Sumsal
43b238f1c1 man: suffix signals with ()
Since signals can take arguments, let's suffix them with () as we
already do with functions. To make sure we remain consistent, make the
`update-dbus-docs.py` script check & fix any occurrences where this is
not the case.

Resolves: #31002
2024-01-23 16:27:50 +01:00
Frantisek Sumsal
25cb4c1d53 update-man-rules: skip over standard-conf.xml
bc6fdcbf5d switched its doctype to refentry, so the script started
picking it up and complaining that it's missing required stuff. Since
this file is only included from other man pages, let's skip it when
putting together a list of valid targets.

Resolves: #30715
Follow-up for: bc6fdcbf5d
2024-01-04 15:02:28 +01:00
David Tardon
c101b65619 man: make ID valid
The id attribute is of type ID, defined at
https://www.w3.org/TR/1998/REC-xml-19980210#id . It may contain only
selected non-alphanumeric characters; '@' is not among them.
2023-12-25 10:40:40 +01:00
Mike Yuan
ced3e6bc0e elf2efi: remove outdated comment mentioning linker script
Follow-up for 142f0c61a3
2023-12-10 12:50:14 +00:00
Yu Watanabe
67bc3c1a8c tools/meson-vcs-tag: the third argument is optional
Follow-up for 1a71ac07ad.
2023-12-04 10:29:52 +00:00
Yu Watanabe
b3b33fa3aa contrib: include co-authors to the contributor list of NEWS
Closes #30261.
2023-11-30 09:57:49 +00:00
Zbigniew Jędrzejewski-Szmek
94113d5f19 hwdb: rename .html=>.csv
The old suffix is now confusing.
2023-11-15 15:21:26 +01:00
Frantisek Sumsal
a3d3bf559c fuzz: don't panic without a C++ compiler
meson's `cpp_args` option is defined only if it detects a C++ compiler,
otherwise we get an error:

../test/fuzz/meson.build:56:28: ERROR: Tried to access unknown option 'cpp_args'.
2023-11-15 12:07:42 +00:00
Frantisek Sumsal
17ee59c9c9 fuzz: pass -Dc_args=/-Dcpp_args= to fuzzer targets
Prompted by #29972, because right now it's practically impossible to pass
-fno-sanitize=function to the fuzzer targets without some extensive
sed'ing.

This splits both c_args and cpp_args to separate arguments for
tools/meson-build.sh, because the other way would be to use `eval`, so
the space-separated but quoted strings passed to these options are not
split where they shouldn't, and I'd rather avoid using `eval` if
possible.

Also, this switches the positional arguments we pass to `meson setup`,
as they were in incorrect order (docs say it should be buildir followed
by sourcedir); meson is apparently clever enough to figure this out and
switch the arguments around if necessary, so it didn't complain.
2023-11-10 17:22:08 +00:00
Luca Boccassi
4561c679bc tools: syscall tables moved to a subdirectory 2023-11-01 14:07:54 +00:00
Luca Boccassi
73b2d48558 Update hwdb autosuspend rules 2023-11-01 14:07:54 +00:00
Zbigniew Jędrzejewski-Szmek
725e646854 meson: generate proper version tag when git fails on permission errors
When building with mkosi I would get the following:

    [1/477] Generating version.h with a custom command
    fatal: detected dubious ownership in repository at '/work/src'
    To add an exception for this directory, call:

            git config --global --add safe.directory /work/src

and then the tag would be generated as 'v254-'. This is obviously some problem
with the setup, but we should handle this gracefully. Let's fall back to 'v254'
instead.

In the case where we have a repo but no tags, use --dirty=^ too, as in the case
with tags.

I tested four cases:
- normal checkout
- checkout with .git removed
- checkout with .git chowned to root
- checkout wiht all tags removed
2023-10-19 17:52:44 +02:00
Abderrahim Kitouni
e8868e8354 doc-sync: add support for uploading the documentation for main
It will refuse running on any other branch than main or stable branches.

Also update the release instructions to run it on the stable branch.
2023-10-10 17:50:04 +01:00
Abderrahim Kitouni
00fc4a3945 doc-sync: automatically detect whether we're updating the latest version
also update the release instructions to push release candidates to -stable
2023-10-09 18:37:41 +01:00
Abderrahim Kitouni
75481bebc1 doc-sync: add man/ to the passed directory
This mirrors the behaviour before multiple version support
2023-10-09 18:03:57 +01:00
Abderrahim Kitouni
3c1f396f69 man: support multiple versions of the documentation on the website
This changes the doc-sync meson target from a simple rsync command to a
script that:

* puts the documentation in a subdirectory according to the version
* injects a bit of javascript to add a drop-down to switch between versions
* updates an index.json file with the newly uploaded version
* keeps the latest/ directory up to date with the latest version
* supports a --no-latest switch to be used when uploading older versions
2023-10-09 11:16:20 +01:00
Abderrahim Kitouni
3691e7fce7 man: add checks for missing version information
This adds a new script tools/check-version-history.py and a corresponding
test when building in developer mode. It checks manpages (except dbus
documentation which is handled by update-dbus-docs) for missing version
history information.

It also adds ignore lists based on version 183 (the version that our version
annotations go back to). These can be augmented if we want to ignore other
elements if it doesn't make sense for them to have version annotations.
2023-10-01 11:54:29 +01:00
Jan Janssen
823bf39a49 elf2efi: Add GNU_RELRO support 2023-09-29 17:05:11 +02:00
Jan Janssen
898e9edc46 elf2efi: Add --copy-sections option
This makes the special PE sections available again in our output EFI
images.

Since the compiler provides no way to mark a section as not allocated,
we use GNU assembler syntax to emit the sections instead. This ensures
the section data isn't emitted twice as load segments will only contain
allocating input sections.
2023-09-29 16:56:30 +02:00
Jan Janssen
7d6fd7f099 elf2efi: Add next_section_address helper 2023-09-29 16:56:30 +02:00
Jan Janssen
5713c50d84 elf2efi: Check ELF image base if possible 2023-09-29 16:56:30 +02:00
Jan Janssen
142f0c61a3 elf2efi: Rework ELF section conversion
The main reason we need to apply a whole lot of logic to the section
conversion logic is because PE sections have to be aligned to the page
size (although, currently not even EDK2 enforces this). The process of
achieving this with a linker script is fraught with errors, they are a
pain to set up correctly and suck in general. They are also not
supported by mold, which requires us to forcibly use bfd, which also
means that linker feature detection is easily at odds as meson has a
differnt idea of what linker is in use.

Instead of forcing a manual ELF segment layout with a linker script we
just let the linker do its thing. We then simply copy/concatenate the
sections while observing proper page boundaries.
Note that we could just copy the ELF load *segments* directly and
achieve the same result. Doing this manually allows us to strip sections
we don't need at runtime like the dynamic linking information (the
elf2efi conversion is effectively the dynamic loader).

Important sections like .sbat that we emit directly from code will
currently *not* be exposed as individual PE sections as they are
contained within the ELF segments. A future commit will fix this.
2023-09-29 16:56:30 +02:00