1
0
mirror of https://github.com/systemd/systemd.git synced 2024-10-30 06:25:37 +03:00
Commit Graph

32 Commits

Author SHA1 Message Date
Frantisek Sumsal
bef8d18b3f oss-fuzz: include generated corpora in the final zip file
Since the files with generated directives are now automatically
generated during build, they're now under the respective build directory
which the current oss-fuzz CI script didn't account for.

Follow-up to: #24958
Resolves: #25859
2023-01-07 17:44:23 +01:00
Evgeny Vereshchagin
b22f5ed5fb oss-fuzz: support i386
Closes https://github.com/systemd/systemd/issues/23532
2022-05-29 09:32:00 +00:00
Evgeny Vereshchagin
ebd4541efe oss-fuzz: turn off fuzz-introspector
fuzz-introspector passes -fuse-ld=gold and -flto using CFLAGS/LDFLAGS and due to
https://github.com/mesonbuild/meson/issues/6377#issuecomment-575977919 and
https://github.com/mesonbuild/meson/issues/6377 it doesn't mix well with meson.
It's possible to build systemd with duct tape there using something like
https://github.com/google/oss-fuzz/pull/7583#issuecomment-1104011067 but
apparently even with gold and lto some parts of systemd are missing from
reports (presumably due to https://github.com/google/oss-fuzz/issues/7598).
Let's just fail here for now to make it clear that fuzz-introspector isn't supported.
2022-04-22 09:08:47 +09:00
Evgeny Vereshchagin
69aa4982bc ci: merge seed corpora with public OSS-Fuzz corpora on CFLite
to let it cover as much code as possible. It should help to catch
most regressions in a few minutes.
2022-01-30 10:17:25 +00:00
Evgeny Vereshchagin
81f84a2c0b ci: use CFLite to test forks (including systemd-stable)
It's like CIFuzz but unlike CIFuzz it's compatible with forks and
it should make it possible to run the fuzzers to make sure that
patches backported to them are backported correctly without introducing
new bugs and regressions.
2022-01-29 18:37:17 +00:00
Evgeny Vereshchagin
e0ec0450e9 tests: fuzz etc_hosts_parse
That's just a follow-up to https://github.com/systemd/systemd/pull/22179
2022-01-20 15:53:48 +09:00
Evgeny Vereshchagin
85ed912479 oss-fuzz: drop line-tables-only
It was copy-pasted directly from OSS-Fuzz where it makes sense to
kind of strip binaries to get nice backtraces but when the fuzzers
are built and run locally with gdb it would be nice to have a little
bit more than that.

It was initially discovered in elfutils where I put the same flags
and was surprised when I couldn't run the fuzzer comfortably step
by step, which led to the same change there: https://github.com/google/oss-fuzz/pull/7092
:-)
2022-01-05 12:04:27 +01:00
Evgeny Vereshchagin
c84059f16e oss-fuzz: turn on the alignment check 2021-12-29 10:39:06 +00:00
Evgeny Vereshchagin
4b65fc8725 tests: add fuzz-bcd 2021-12-29 10:39:06 +00:00
Zbigniew Jędrzejewski-Szmek
4287c85589 meson: create new libsystemd-core.so private shared library
The scheme is very similar to libsystemd-shared.so: instead of building a
static library, we build a shared library from the same objects and link the
two users to it. Both systemd and systemd-analyze consist mostly of the fairly
big code in libcore, so we save a bit on the installation:

(-0g, no strip)
-rwxr-xr-x 5238864 Dec 14 12:52 /var/tmp/inst1/usr/lib/systemd/systemd
-rwxr-xr-x 5399600 Dec 14 12:52 /var/tmp/inst1/usr/bin/systemd-analyze
-rwxr-xr-x  244912 Dec 14 13:17 /var/tmp/inst2/usr/lib/systemd/systemd
-rwxr-xr-x  461224 Dec 14 13:17 /var/tmp/inst2/usr/bin/systemd-analyze
-rwxr-xr-x 5271568 Dec 14 13:17 /var/tmp/inst2/usr/lib/systemd/libsystemd-core-250.so

(-0g, strip)
-rwxr-xr-x 2522080 Dec 14 13:19 /var/tmp/inst1/usr/lib/systemd/systemd
-rwxr-xr-x 2604160 Dec 14 13:19 /var/tmp/inst1/usr/bin/systemd-analyze
-rwxr-xr-x  113304 Dec 14 13:19 /var/tmp/inst2/usr/lib/systemd/systemd
-rwxr-xr-x  207656 Dec 14 13:19 /var/tmp/inst2/usr/bin/systemd-analyze
-rwxr-xr-x 2648520 Dec 14 13:19 /var/tmp/inst2/usr/lib/systemd/libsystemd-core-250.so

So for systemd itself we grow a bit (2522080 → 2648520+113304=2761824), but
overall we save. The most is saved on all the test files that link to libcore,
if they are installed, because there's 15 of them:

$ du -s /var/tmp/inst?
220096	/var/tmp/inst1
122960	/var/tmp/inst2

I also considered making systemd-analyze a symlink to /usr/lib/systemd/systemd
and turning systemd into a multicall binary. We did something like this with
udevd and udevadm. But that solution doesn't fit well in this case.
systemd-analyze has a bunch of functionality that is not used in systemd,
so the systemd binary would need to grow quite a bit. And we're likely to
add new types of verification or introspection features in analyze, and this
baggage would only grow. In addition, there are the test binaries which also
benefit from this.
2021-12-25 15:18:47 +09:00
Evgeny Vereshchagin
4997d1b965 ci: pin python dependencies and let Dependabot keep track of them 2021-12-07 09:08:26 +00:00
Evgeny Vereshchagin
2fd1beb3e2 oss-fuzz: move apt-gets and pips to the systemd repository
to be able to control our dependencies right here without
sending PRs like https://github.com/google/oss-fuzz/pull/5199 and
https://github.com/google/oss-fuzz/pull/5601.

It should also allow us to pin meson to let Dependabot keep track of
it and jump from one version to another without breaking anything
2021-11-19 08:52:28 +00:00
Frantisek Sumsal
f7e0d22d76 tools: shellcheck-ify tool scripts 2021-09-30 12:27:06 +02:00
Evgeny Vereshchagin
77591e9732 oss-fuzz: show meson logs
It should help to make it more clear what causes issues like
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=30140
and https://github.com/google/oss-fuzz/pull/5084
2021-02-01 22:51:43 +09:00
Yu Watanabe
db9ecf0501 license: LGPL-2.1+ -> LGPL-2.1-or-later 2020-11-09 13:23:58 +09:00
Evgeny Vereshchagin
1f034000a5 oss-fuzz: turn on the pointer-overflow check
It's off by default on OSS-Fuzz but it should be safe to turn it on
manually: https://github.com/google/oss-fuzz/issues/659#issuecomment-631897889

Just a follow-up to https://github.com/systemd/systemd/pull/15865.
2020-05-21 08:52:18 +02:00
Jörg Thalheim
ff12a7954c treewide: more portable bash shebangs
As in 2a5fcfae02
and in 3e67e5c992
using /usr/bin/env allows bash to be looked up in PATH
rather than being hard-coded.

As with the previous changes the same arguments apply
- distributions have scripts to rewrite shebangs on installation and
  they know what locations to rely on.
- For tests/compilation we should rather rely on the user to have setup
  there PATH correctly.

In particular this makes testing from git easier on NixOS where do not provide
/bin/bash to improve compose-ability.
2020-03-05 17:27:07 +01:00
Frantisek Sumsal
a3e42c468f test: unpin meson from v0.52.1
The compatibility issue in meson v0.53 has been fixed in v0.53.1, which
is already available through pip, so let's remove the pin for meson
introduced before.

Reverts: 514793658c
2020-01-25 23:14:35 +03:00
Frantisek Sumsal
514793658c test: pin meson to 0.52.1 for fuzzit/fuzzbuzz
Latest meson doesn't work with older python 3.5, which is present on
Ubuntu 16.04. Let's pin in to the latest working version (0.52.1) until
we properly bump all necessary Ubuntu images to 18.04.

See: https://github.com/mesonbuild/meson/issues/6427
2020-01-08 13:56:30 +03:00
Evgeny Vereshchagin
2dd9b65778 oss-fuzz.sh: stop downloading the skia seed corpus
When the fuzz target was integrated, it was added as a stopgap
to get fuzz-json up and running. It served its purpose and can
safely be removed to prevent tools/oss-fuzz.sh from failing with
```
+wget -O /home/travis/build/systemd/systemd/out/fuzz-json_seed_corpus.zip https://storage.googleapis.com/skia-fuzzer/oss-fuzz/skjson_seed_corpus.zip
--2019-09-10 22:40:44--  https://storage.googleapis.com/skia-fuzzer/oss-fuzz/skjson_seed_corpus.zip
Resolving storage.googleapis.com (storage.googleapis.com)... 74.125.70.128, 2607:f8b0:4001:c05::80
Connecting to storage.googleapis.com (storage.googleapis.com)|74.125.70.128|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2019-09-10 22:40:44 ERROR 403: Forbidden.
```

Ideally we should put our seed corpus somewhere and download it from there
but I haven't got round to it.
2019-09-11 03:24:29 +03:00
Evgeny Vereshchagin
9c5c4677fa fuzzers: use -fsanitizer=fuzzer if clang supports it
Closes https://github.com/systemd/systemd/issues/10645
2019-05-14 13:42:17 +02:00
Zbigniew Jędrzejewski-Szmek
cc5549ca12 scripts: use 4 space indentation
We had all kinds of indentation: 2 sp, 3 sp, 4 sp, 8 sp, and mixed.
4 sp was the most common, in particular the majority of scripts under test/
used that. Let's standarize on 4 sp, because many commandlines are long and
there's a lot of nesting, and with 8sp indentation less stuff fits. 4 sp
also seems to be the default indentation, so this will make it less likely
that people will mess up if they don't load the editor config. (I think people
often use vi, and vi has no support to load project-wide configuration
automatically. We distribute a .vimrc file, but it is not loaded by default,
and even the instructions in it seem to discourage its use for security
reasons.)

Also remove the few vim config lines that were left. We should either have them
on all files, or none.

Also remove some strange stuff like '#!/bin/env bash', yikes.
2019-04-12 08:30:31 +02:00
Evgeny Vereshchagin
3ff074bd85 oss-fuzz.sh: copy dictionaries along with "options" files
We currently don't have any upstream but it doesn't mean that
it should be impossible to experiment with local ones :-)
2018-11-06 19:55:11 +01:00
Evgeny Vereshchagin
6315d12bba tests: add a fuzzer for the json parser and dumper 2018-10-10 10:13:30 +02:00
Zbigniew Jędrzejewski-Szmek
93b575b266 fuzz: rename "fuzz-corpus" directory to just "fuzz"
Also, all corpus subdirectories are named exactly the same as the fuzzer they
are for. This makes the paths a bit longer, but easier.
2018-10-02 09:41:25 +02:00
Evgeny Vereshchagin
7cd98cc495 oss-fuzz.sh: just install the shared library
The workaround is no longer necessary, because the scripts
checking fuzzers have stopped going down to the subdirectories
of $OUT and started to look for the string "LLVMFuzzerTestOneInput"
to tell fuzzers and random binaries apart. Some more details can be
found at https://github.com/google/oss-fuzz/issues/1566.
2018-08-16 06:45:49 +00:00
Zbigniew Jędrzejewski-Szmek
d9b02e1697 tree-wide: drop copyright headers from frequent contributors
Fixes #9320.

for p in Shapovalov Chevalier Rozhkov Sievers Mack Herrmann Schmidt Rudenberg Sahani Landden Andersen Watanabe; do
  git grep -e 'Copyright.*'$p -l|xargs perl -i -0pe 's|/([*][*])?[*]\s+([*#]\s+)?Copyright[^\n]*'$p'[^\n]*\s*[*]([*][*])?/\n*|\n|gms; s|\s+([*#]\s+)?Copyright[^\n]*'$p'[^\n]*\n*|\n|gms'
done
2018-06-20 11:58:53 +02:00
Lennart Poettering
96b2fb93c5 tree-wide: beautify remaining copyright statements
Let's unify an beautify our remaining copyright statements, with a
unicode ©. This means our copyright statements are now always formatted
the same way. Yay.
2018-06-14 10:20:21 +02:00
Zbigniew Jędrzejewski-Szmek
3c31f3a575 tools/oss-fuzz: install private shared library non-executable (#8927)
Apparently oss-fuzz's "bad build check" is confused by the library.
Let's make it non-executable, so the checker ignores it.

Should fix https://github.com/google/oss-fuzz/issues/1330.
2018-05-08 18:00:19 +03:00
Zbigniew Jędrzejewski-Szmek
11a1589223 tree-wide: drop license boilerplate
Files which are installed as-is (any .service and other unit files, .conf
files, .policy files, etc), are left as is. My assumption is that SPDX
identifiers are not yet that well known, so it's better to retain the
extended header to avoid any doubt.

I also kept any copyright lines. We can probably remove them, but it'd nice to
obtain explicit acks from all involved authors before doing that.
2018-04-06 18:58:55 +02:00
Zbigniew Jędrzejewski-Szmek
36cd9913ce tools/oss-fuzz: add clang library dir using -L
I have no idea why clang doesn't do this on its own, and why clang
makes it so hard to query this path (-dumpversion returns something
unrelated...).

I know this is an ugly hack, but this is a very specialized script,
so it should be OK to make it a bit hacky.

Tested to work on Fedora (27) and Debian (unstable).

Fixes #8428.
2018-03-12 15:59:25 +01:00
Zbigniew Jędrzejewski-Szmek
9037a0e022 Rename scripts/oss-fuzz.sh to tools/oss-fuzz.sh 2018-03-12 15:40:37 +01:00