1
0
mirror of https://github.com/systemd/systemd.git synced 2024-12-28 11:21:59 +03:00
Commit Graph

219 Commits

Author SHA1 Message Date
Zbigniew Jędrzejewski-Szmek
19c79550b4 tools/analyze-dump-sort: a helper to compare two 'systemd-analyze dump' outputs
Lines in the dumps are ordered by some pseudo-random hashmap entry order, which
makes it hard to diff two outputs. This sort the entries alphabetically, and
also sorts items within the entries, and supresses timestamps and other fields
which always vary.

We could sort the output inside of systemd itself, but it'd make things more
complex, and we probably don't need output to be sorted in most cases. It also
wouldn't be enough, because timestamps and such would still need to be ignored
to do a nice diff. So I think doing the sorting and suppression in a python
helper is a better approach.
2021-06-04 12:09:58 +02:00
Luca Boccassi
c5fd89adcd core: make libbpf a dlopen() dependency 2021-05-25 12:59:26 +01:00
Zbigniew Jędrzejewski-Szmek
44ff8df777 Drop dependency on m4
m4 was hugely popular in the past, because autotools, automake, flex, bison and
many other things used it. But nowadays it much less popular, and might not even
be installed in the buildroot. (m4 is small, so it doesn't make a big difference.)

(FWIW, Fedora dropped make from the buildroot now,
https://fedoraproject.org/wiki/Changes/Remove_make_from_BuildRoot. I think it's
reasonable to assume that m4 will be dropped at some point too.)

The main reason to drop m4 is that the syntax is not very nice, and we should
minimize the number of different syntaxes that we use. We still have two
(configure_file() with @FOO@ and jinja2 templates with {{foo}} and the
pythonesque conditional expressions), but at least we don't need m4 (with
m4_dnl and `quotes').
2021-05-19 10:24:43 +09:00
Zbigniew Jędrzejewski-Szmek
6b1aac3ccf meson: replace some m4 templates with jinja2
m4 was nice in '85, but the syntax feels a bit dated. Since we use python for
meson, let's use a popular python templating engine to replace some m4 usage.

A little nicety is that typos are caught:

FAILED: sysusers.d/systemd-remote.conf
/usr/bin/meson --internal exe --capture sysusers.d/systemd-remote.conf -- /home/zbyszek/src/systemd/tools/meson-render-jinja2.py config.h ../sysusers.d/systemd-remote.conf.j2
Traceback (most recent call last):
  File "/home/zbyszek/src/systemd/tools/meson-render-jinja2.py", line 28, in <module>
    print(render(sys.argv[2], defines))
  File "/home/zbyszek/src/systemd/tools/meson-render-jinja2.py", line 24, in render
    return template.render(defines)
  File "/usr/lib/python3.9/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/lib/python3.9/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/lib/python3.9/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 8, in top-level template code
jinja2.exceptions.UndefinedError: 'HAVE_MICROHTTP' is undefined

This checking mirrors what 349cc4a507 did for C defines.
2021-05-19 10:24:43 +09:00
Jörg Thalheim
f96bc66901 fix bash shebangs 2021-05-17 22:27:41 +01:00
Zbigniew Jędrzejewski-Szmek
b8f1045fe7 Revert "tools/make-directive-index: parallelize"
This reverts commit a2031de849.

The patch itself seems OK, but it exposes a bug in lxml or libxml2-2.9.12 which
was just released. This is being resolved in
https://gitlab.gnome.org/GNOME/libxml2/-/issues/255, but it might be while. So
let's revert this for now to unbreak our CI.

Fixes #19601.
2021-05-14 23:16:24 +01:00
Zbigniew Jędrzejewski-Szmek
a2031de849 tools/make-directive-index: parallelize
I occasionally do 'build/man/man systemd.directives' when working on man pages,
and it's annoying slow. By paralellizing the parsing of xml, we can make it a
bit faster.

This is still rather innefficient. Only the parsing part is serialized, xml is
still produced serially at the end, which is hard to avoid.

$ ninja -C build man/systemd.directives.xml
before:
8.20s user 0.21s system 99% cpu 8.460 total
8.33s user 0.18s system 98% cpu 8.619 total
8.72s user 0.19s system 98% cpu 9.019 total

after:
13.99s user 0.73s system 345% cpu 4.262 total
14.15s user 0.35s system 348% cpu 4.161 total
14.33s user 0.35s system 339% cpu 4.321 total

I.e. it uses almost twice as much cpu, but cuts the wallclock time down (on a
2-core/4-thread cpu) to about half too, which is an overall win if you're just
trying to render the man page.

The change from list and .append() to set and .add() is something that could
have been done before too, but it's noticable now. It cuts down on the
serialization/deserialization time (about .2s).
2021-05-13 11:53:01 +02:00
Julia Kartseva
cf4f9a57f2 bpf: add build script for bpf programs
Add a build script to compile bpf source code. A program in restricted
C is compiled into an object file. Object file is converted to BPF
skeleton [0] header file.
If build with custom meson build rule, the target header will reside in
build/ directory (not in source tree), e.g the path for socket_bind:
`build/src/core/bpf/socket_bind/socket-bind.skel.h`

Script runs the phases:
* clang to generate *.o from restricted C
* llvm-strip to remove useless DWARF info
* bpf skeleton generation with bpftool
These phases are logged to stderr for debug purposes.

To include BTF debug information, -g option is passed to clang.

[0] https://lwn.net/Articles/806911/
2021-04-26 16:07:41 -07:00
Frantisek Sumsal
3b6fd3c1de tools: shellcheck-ify most of the tool scripts 2021-04-20 20:11:13 +02:00
Frantisek Sumsal
34fde9f898 test: check if the unit file fuzzer corpora is up to date
This follows a similar pattern we already have in place for
networkd-related directives.
2021-04-05 14:41:05 +01:00
Zbigniew Jędrzejewski-Szmek
b242d2dec9 git-contrib: use non-breaking spaces in names
Some people have initials or abbreviated parts in the name and looks strange
when a line break occurs in the middle. Let's keep each name in one line.
2021-03-30 13:17:59 +02:00
Lennart Poettering
1ab4edae12 tools: exclude Weblate bot from "ninja git-contrib"
Bots are not people, no reason to say thanks to them and list them as
authors or contributors.
2021-03-17 16:53:14 +01:00
Zbigniew Jędrzejewski-Szmek
a3fbf67180 hwdb: update for v248
Seems to be additions and corrections as usual.
Includes an update of the chromiumos autosuspend rules.
2021-03-08 14:27:48 +01:00
Zbigniew Jędrzejewski-Szmek
64eb60b8c5 update-dbus-docs: use color in summary 2021-02-06 11:41:42 +01:00
Zbigniew Jędrzejewski-Szmek
7bd5b8614d update-dbus-docs: say "MODIFIED" not "OUTDATED"
When executed in test mode, "OUTDATED" is appropriate. But when executed
to actually update the text, after the tool executes, those pages are the
opposite, not outdated.
2021-02-06 11:41:42 +01:00
Evgeny Vereshchagin
77591e9732 oss-fuzz: show meson logs
It should help to make it more clear what causes issues like
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=30140
and https://github.com/google/oss-fuzz/pull/5084
2021-02-01 22:51:43 +09:00
Anita Zhang
934d0d023e tools: make update-dbus-docs compatible with Python 3.6
668b3a42fe allowed update-dbus-docs.py to start
running on Cent OS 8 (instead of skipping). But subprocess.check_output()'s
text argument didn't exist until Python 3.7 and C8 is still running
Python 3.6. Use universal_newlines instead for backwards compatibility.
2021-02-01 11:21:10 +00:00
Luca Boccassi
668b3a42fe tools: make update-dbus-docs compatible with Python 3.7
Debian Stable uses Python 3.7, but there are a couple of 3.8 features used
in the script. Add fallbacks.
2021-01-29 09:53:34 +00:00
Zbigniew Jędrzejewski-Szmek
d3821a339e tools: rename helper to match target name
The target is update-syscall-tables, so let's call the script
update-syscall-tables.sh to reduce the cognitive overhead when
trying to find the right file.
2021-01-28 09:55:35 +01:00
Zbigniew Jędrzejewski-Szmek
1f6f8cc803 Use .txt as the extension of arch syscall lists
This makes it easier to filter those files and tells editors that they should
be treated as plain text.
2021-01-28 09:55:35 +01:00
Zbigniew Jędrzejewski-Szmek
984b529684 Use .txt as the extension for syscall list file
Upstream uses .text, but this is rather unusual. Let's use .txt as the usual
suffix for text files. This tells various editors and such that the file should
be treated as plain text. I also want to a script to summarize license status,
and having an easy-to-recognize suffix makes this easier.
2021-01-28 09:55:35 +01:00
Zbigniew Jędrzejewski-Szmek
9ee03516df tree-wide: add spdx header on all scripts and helpers
Even though many of those scripts are very simple, it is easier to include
the header than to try to say whether each of those files is trivial enough
not to require one.
2021-01-28 09:55:35 +01:00
Zbigniew Jędrzejewski-Szmek
816f31d018 meson: rename target to update-hwdb-autosuspend
The script is renamed to match.

Now all targets are named uniformly in a tab-completion-friendly fashion, with
the exception of systemd-update-po which is generated by the i18n module
automatically:

$ ninja -C build -t targets | grep update
systemd-update-po: phony
update-syscall-tables: phony
update-syscall-header: phony
update-hwdb: phony
update-hwdb-autosuspend: phony
update-dbus-docs: CUSTOM_COMMAND
update-man-rules: CUSTOM_COMMAND
2021-01-27 09:24:30 +01:00
Zbigniew Jędrzejewski-Szmek
4095cff07e meson: rename target to update-hwdb
The goal is to have all "update-*" targets named uniformly so that
tab-completion works. The script is renamed to match.
2021-01-27 09:22:15 +01:00
Zbigniew Jędrzejewski-Szmek
e3c368f63c meson: rename target to update-man-rules
Same justification as for update-dbus-docs.
2021-01-27 09:10:25 +01:00
Zbigniew Jędrzejewski-Szmek
4c890ad3cc meson: rename target to update-dbus-docs
Very old versions of meson did not include the subdirectory name in the
target name, so we started adding various "top-level" custom targets in
subdirectories. This was nice because the main meson.build file wasn't
as cluttered. But then meson started including the subdir name in the
target name. So let's move the definition to the root so we can have all
targets named uniformly.
2021-01-27 08:46:42 +01:00
Benjamin Berg
b25e76eada hwdb: Pull autosuspend rules from upstream libfprint
libfprint includes a list of known fingerprint readers that can be
autosuspended. Upstream libfprint generates this file from the USB IDs
registered to drivers and a list of well-known readers that are
currently unsupported.

Closes: #17663
2021-01-20 22:14:23 +01:00
Benjamin Berg
84171ba179 tools: Pass source directory to autosuspend-update.sh
This makes sense so that we can extend the script to also update other
files.
2021-01-19 15:05:27 +01:00
Yu Watanabe
a0e150b2f4 meson: add missing license header 2021-01-19 07:06:32 +09:00
Zbigniew Jędrzejewski-Szmek
2b5a1402f6
Merge pull request #18263 from keszybz/syscalls-auto
Generate missing syscalls headers programatically
2021-01-16 17:21:34 +01:00
Daan De Meyer
064b8e2c99 meson: Use configure_file when version-tag is specified
vcs_tag() is slow. When the version-tag meson option is set,
we can use configure_file() directly to speed up incremental
builds.

Before (with version-tag set to v247):

```
‣ Running build script...
[1/418] Generating version.h with a custom command

real    0m0.521s
user    0m0.229s
sys     0m0.067s
```

After (with version-tag set to v247):

```
‣ Running build script...
ninja: no work to do.

real    0m0.094s
user    0m0.048s
sys     0m0.022s
```
2021-01-15 19:34:44 +01:00
Zbigniew Jędrzejewski-Szmek
7975857079 tree-wide: use curl --fail
curl will save the 404 response page (or another error) if the page
download fails, which we never want. Let it error out instead.
2021-01-15 18:35:02 +01:00
Zbigniew Jędrzejewski-Szmek
9a6da617db meson: download full syscall tables from hrw/syscalls-table
The target is renamed to 'update-syscall-tables'. (Other targets
with similar names will be added later.)
2021-01-15 18:35:02 +01:00
Frantisek Sumsal
86660d160b ci: move the Coverity job to GitHub Actions 2021-01-11 15:50:25 +01:00
Daan De Meyer
09422f9a28 meson: Respect MESON_INSTALL_QUIET
MESON_INSTALL_QUIET is set when --quiet is passed to meson install.
Make sure we check the variable in our custom install scripts and
don't output anything if it is set.
2020-12-06 22:11:11 +00:00
Zbigniew Jędrzejewski-Szmek
fe7576e345 hwdb: update chromiumos autosuspend rules 2020-11-26 13:54:37 +01:00
Yu Watanabe
45752a2495 tools: drop unnecessary "else" after for loop 2020-11-20 19:47:11 +09:00
Lennart Poettering
53eedd5011 git-contrib: exclude -rc tags 2020-11-10 14:12:27 +01:00
Yu Watanabe
db9ecf0501 license: LGPL-2.1+ -> LGPL-2.1-or-later 2020-11-09 13:23:58 +09:00
Zbigniew Jędrzejewski-Szmek
3a7771c9c4 Partially revert "hwdb: add trailing ":*" everywhere"
This reverts commit c0443b97b7.

I got various cases wrong:
"usb:v04F3p2B7Cd5912dc00dsc00dp00ic03isc00ip00in00"
"usb:v0627p0001:QEMU USB Tablet"
"input:b0003v0627p0001e0001-e0,1,2,4,k110,111,112,r0,1,8,B,am4,lsfw"

OTOH:
-evdev:name:ETPS/2 Elantech Touchpad:dmi:*svnASUSTeKComputerInc.:pnN53SV:*
+evdev:name:ETPS/2 Elantech Touchpad:dmi:*svnASUSTeKComputerInc.:pnN53SV*
is OK. Other parts follow after 'pn'.

-mouse:*:name:*Trackball*:*
-mouse:*:name:*trackball*:*
-mouse:*:name:*TrackBall*:*
+mouse:*:name:*Trackball*:
+mouse:*:name:*trackball*:
+mouse:*:name:*TrackBall*:
... and anything else with :name should be OK too, because our imports always
include ":" at the end:
IMPORT{builtin}="hwdb 'joystick:$env{ID_BUS}:v$attr{id/vendor}p$attr{id/product}:name:$attr{name}:'"
Including '*' at the end makes the pattern work even if we decide to add
something to the match string later.

Fixes #17499.
2020-11-03 14:17:53 +01:00
Zbigniew Jędrzejewski-Szmek
c0443b97b7 hwdb: add trailing ":*" everywhere
No functional change is intended.
The general pattern of changes:

-usb:v04F3p2B7C*
+usb:v04F3p2B7C:*
This is mostly a clarification, to make the part that makes the usb vXXXXpYYYY
part visually separated. It would only make a difference if we added further
keys with a different number of digits, which is unlikely.

-usb:v0627p0001:*QEMU USB Keyboard*
-usb:v0627p0001:*QEMU USB Mouse*
-usb:v0627p0001:*QEMU USB Tablet*
+usb:v0627p0001:*QEMU USB Keyboard*:*
+usb:v0627p0001:*QEMU USB Mouse*:*
+usb:v0627p0001:*QEMU USB Tablet*:*
Again, only a clarification. We know that ":" will appear somewhere later in
the match key, so anything that matches "…Keyboard*" will also match "…Keyboard*:*".

-evdev:name:ETPS/2 Elantech Touchpad:dmi:*svnASUSTeKComputerInc.:pnN53SV*
+evdev:name:ETPS/2 Elantech Touchpad:dmi:*svnASUSTeKComputerInc.:pnN53SV:*
This makes the match narrower. Previously we would match product "N53SV"
and "N53SV2", "N53SV3", and others. Here we are saying that the ':pn' part must
match exactly. Most of the changes in this patch match this pattern. I made a few
judgement calls and used "pn…*:*" when I wasn't sure if the full pn is included:
-evdev:name:Dell WMI hotkeys:dmi:bvn*:bvr*:bd*:svnDell*:pnPrecision*
+evdev:name:Dell WMI hotkeys:dmi:bvn*:bvr*:bd*:svnDell*:pnPrecision*:*

-evdev:name:Cypress APA Trackpad ?cyapa?:dmi:*:svnHewlett-Packard*:pnFalco*:
+evdev:name:Cypress APA Trackpad ?cyapa?:dmi:*:svnHewlett-Packard*:pnFalco*:*
This more like the "QEMU" example above, since all dmi strings end in ":", so
anything which matches the old version will also match the new version.

-evdev:atkbd:dmi:bvn*:bvr*:bd*:svnGateway*:pnA0A1*:pvr*
+evdev:atkbd:dmi:bvn*:bvr*:bd*:svnGateway*:pnA0A1*:*

I replaced trailing ":pvr*" by ":*". This makes no functional difference because
we expect "pvr" to always appear in the dmi string. This makes patterns shorter.

-evdev:atkbd:dmi:bvn*:bvr*:bd*:svnAcer*:pn*
+evdev:atkbd:dmi:bvn*:bvr*:bd*:svnAcer*:pn*:*

OTOH, ":pn*" is kept. This is because almost patterns include ":pn*", and if we
skip it, we should make it clear that this is on purpose, that we really want to
match any product name.

The python script to generate autosuspend rules is updated to use ":*" too.

Inspired by https://github.com/systemd/systemd/pull/17281#discussion_r501489750.
2020-10-15 18:01:32 +02:00
Zbigniew Jędrzejewski-Szmek
cc17fa8511 hwdb: update chromiumos autosuspend rules 2020-10-15 13:29:01 +02:00
Zbigniew Jędrzejewski-Szmek
198fda4f48 update-dbus-docs: skip test if python is too old 2020-09-20 13:15:44 +02:00
Zbigniew Jędrzejewski-Szmek
8aaf611b9a update-dbus-docs: skip test lxml is not available 2020-09-18 18:51:42 +02:00
Zbigniew Jędrzejewski-Szmek
c91e3116f4 update-dbus-docs: add hint 2020-08-27 21:30:23 +02:00
Zbigniew Jędrzejewski-Szmek
04aa6fa8cd update-dbus-docs: omit verbose output when in --test mode
It makes the ninja output listing very long for no good purpose.
2020-08-27 21:30:23 +02:00
Zbigniew Jędrzejewski-Szmek
1b584f38a8 update-dbus-docs: add test mode 2020-08-27 20:22:52 +02:00
Zbigniew Jędrzejewski-Szmek
0f5cea0212 update-dbus-docs: use argparse 2020-08-27 20:22:52 +02:00
Zbigniew Jędrzejewski-Szmek
af4c7dc269 update-dbus-docs: print statistics at the end
Right now:
org.freedesktop.LogControl1.xml: 3/3
org.freedesktop.home1.xml:       44/44
org.freedesktop.hostname1.xml:   21/21
org.freedesktop.import1.xml:     17/19
org.freedesktop.locale1.xml:     10/10
org.freedesktop.login1.xml:      172/172
org.freedesktop.machine1.xml:    49/65
org.freedesktop.resolve1.xml:    25/61
org.freedesktop.systemd1.xml:    214/1468
org.freedesktop.timedate1.xml:   12/12
total:                           567/1875

:(
2020-08-27 20:22:40 +02:00
Zbigniew Jędrzejewski-Szmek
47354b440e meson: add syscall-names-update target
The calls to find_program("tools/*") are moved earlier so they can be used
in libshared/ (and it doesn't make sense to split them).
2020-08-19 15:30:24 +02:00