1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-10 01:18:15 +03:00
Commit Graph

35 Commits

Author SHA1 Message Date
Joseph Sutton
97566f1b83 lib/fuzzing: Fix code spelling
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2023-10-25 22:23:37 +00:00
Andreas Schneider
1bfa2c2938 lib:fuzzing: Fix code spelling
Best reviewed with: `git show --word-diff`.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2023-04-03 03:56:35 +00:00
Andreas Schneider
233a0cd6de lib:fuzzing: Fix shellcheck errors in build_samba.sh
lib/fuzzing/oss-fuzz/build_samba.sh:24:27: error: Double quote array
expansions to avoid re-splitting elements. [SC2068]

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Pavel Filipenský <pfilipensky@samba.org>
2022-08-17 10:08:35 +00:00
Andreas Schneider
1e88064170 lib:fuzzing: Reformat shell scripts
shfmt -f lib/fuzzing/ | xargs shfmt -w -p -i 0 -fn

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2022-02-24 09:15:34 +00:00
Andreas Schneider
fc69206f8b lib:fuzzing: Fix quoting of --fuzz-target-ldflags
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

Autobuild-User(master): Uri Simchoni <uri@samba.org>
Autobuild-Date(master): Mon Oct  4 11:36:06 UTC 2021 on sn-devel-184
2021-10-04 11:36:06 +00:00
Uri Simchoni
4f300d672a fuzzing/oss-fuzz: strip RUNPATH from dependencies
Strip all RUNPATH headers from all dependency shared objects that
we copy to the fuzzing target, as those libraries aren't placed
in their original place.

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2021-09-09 00:53:54 +00:00
Uri Simchoni
f94b1d3b31 fuzzing/oss-fuzz: fix samba build script for Ubuntu 20.04
Add a linker flag to generate fuzzer binaries with an RPATH
header instead of RUNPATH.

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2021-09-09 00:53:54 +00:00
Uri Simchoni
541f9ee5ab fuzzing/oss-fuzz: fix RPATH comments for post-Ubuntu-16.04 era
Remove what appears to be a copy+paste error in one place, and
explain that RPATH/RUNPATH is set by the linker, not by chrpath
utility.

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2021-09-09 00:53:54 +00:00
Uri Simchoni
2fe8d3eeac fuzzing/oss-fuzz: fix image build recipe for Ubuntu 20.04
Update the build_image.sh script to install Ubuntu 20.04 packages
instead of Ubuntu 16.04 on the oss-fuzz container - this will
allow the oss-fuzz container to be based on Ubuntu 20.04.

REF: https://github.com/google/oss-fuzz/issues/6301#issuecomment-911705365

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2021-09-09 00:53:54 +00:00
Samuel Cabrero
49a0f6170b oss-fuzz: Update build script to be compatible with rpm distros
The /etc/default/locale file does not exists in the rpm family distros
so the do_build.sh script failed with:

./lib/fuzzing/oss-fuzz/do_build.sh: line 31: /etc/default/locale: No
such file or directory

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: David Mulder <dmulder@samba.org>

Autobuild-User(master): Samuel Cabrero <scabrero@samba.org>
Autobuild-Date(master): Tue Apr  6 15:54:54 UTC 2021 on sn-devel-184
2021-04-06 15:54:54 +00:00
Andrew Bartlett
8f66ce0a3d oss-fuzz: Add very verbose explaination for RPATH vs RUNPATH
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Fri Oct 23 00:33:57 UTC 2020 on sn-devel-184
2020-10-23 00:33:57 +00:00
Andrew Bartlett
a57702db1d oss-fuzz: Always run the check, even on the oss-fuzz platform
It is much harder to determine why we get messages like
    Step #6: Error occured while running fuzz_reg_parse:
    Step #6: /workspace/out/coverage/fuzz_reg_parse: error while loading shared libraries: libavahi-common.so.3: cannot open shared object file: No such file or directory
instead this detects the failure to use RPATH (which is
strictly required instead of the modern RUNPATH)
otherwise.

We do this by creating a new build_samba.sh after renaming
build_samba.sh to do_build.sh because this is what oss-fuzz
runs, meaning we don't need to coordinate a MR there as well.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2020-10-22 23:08:31 +00:00
Andrew Bartlett
b5f8073431 oss-fuzz: update comment to reference RPATH for the static-ish binaries
We strictly require RPATH, so fix the comment to avoid mentioning
the modern RUNPATH which is almost but not entirely similar.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2020-10-22 23:08:31 +00:00
Andrew Bartlett
c03a265030 oss-fuzz: standardise on RPATH for the static-ish binaries
This includes a revert of commit e60df21499.

We strictly require RPATH, not the modern RUNPATH for the behaviour
we need in oss-fuzz, which is that not just the first line of dependencies
but the full set of libraries used by the program are looked for in the
'$ORIGIN/lib' directory.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Oct 22 14:10:04 UTC 2020 on sn-devel-184
2020-10-22 14:10:04 +00:00
Andrew Bartlett
048725080b fuzzing: Improve robustness and documentation of the ldd-base library copy
This tries to make progress towards understanding why we sometime see errors like
Step #6: Error occured while running fuzz_reg_parse:
Step #6: /workspace/out/coverage/fuzz_reg_parse: error while loading shared libraries: libavahi-common.so.3: cannot open shared object file: No such file or directory

in the previously failing coverage builds.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2020-10-22 12:47:37 +00:00
Andrew Bartlett
d031391bed fuzzing: Fix the oss-fuzz coverage build
It was long thought that the issue here was that no seed corpus was
provided, but actually the issue is that to obtain coverage output
just as we already know for gcc gcov, you must provide fuzzing flags
to both the compile and link phase.

Thankfully clang as a linker does not mind the strange non-linker options
from $COVERAGE_FLAGS.

REF: https://stackoverflow.com/questions/56112019/clang-does-not-generate-profraw-file-when-linking-manually
REF: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=19495#c48

Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Oct 21 23:07:37 UTC 2020 on sn-devel-184
2020-10-21 23:07:37 +00:00
Douglas Bagnall
9dfeb81d08 fuzz/oss-fuzz/build_samba: fetch fuzz seeds
There is a git repository at
https://gitlab.com/samba-team/samba-fuzz-seeds that contains the
seeds. When the master branch of that repository is updated, a CI job
runs that creates a zip file of all the seeds as an artifact. That zip
file is downloaded and unpacked by oss_fuzz/build_samba. The contents
of that zip are further zips that contain the seeds for each fuzzing
binary; these are placed next to the binaries in the manner that
oss-fuzz expects.

That is, beside 'fuzz_foo', we put 'fuzz_foo_seed_corpus.zip' which
contains a pile of fuzz_foo seeds.

There may be times when a new fuzz target does not have a seed corpus,
and times when a removed fuzz target leaves behind a seed corpus.
This is OK, so we don't insist on an exact match between the target
names and the zip names, only that there is some overlap.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Oct 21 03:47:35 UTC 2020 on sn-devel-184
2020-10-21 03:47:35 +00:00
Douglas Bagnall
6d388da765 fuzz/oss-fuzz/build-samba: note the calling site
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-10-21 02:28:38 +00:00
Andrew Bartlett
e60df21499 oss-fuzz: standardise on RUNPATH for the static-ish binaries
We use ld.bfd for the coverage builds, rather than the faster ld.gold.

We run the oss-fuzz autobuild target on Ubuntu 16.04 to more closely
mirror the environment provided by the Google oss-fuzz build
container.

On Ubuntu 16.04, when linking with ld.bfd built binaries get a RPATH,
but builds in Ubuntu 18.04 and those using ld.gold get a RUNPATH.

Just convert them all to RUNPATH to make the check_build.sh test (run
by the oss-fuzz autobuild target) easier.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2020-09-11 03:43:40 +00:00
Andrew Bartlett
830c020645 oss-fuzz: Ensure a UTF8 locale is set for the samba build
This ensures that LANG=en_US.UTF8 is set, which
Samba's build system needs to operate in UTF8 mode.

The change to use flex to generate code meant that this
difference between GitLab CI and oss-fuzz was exposed.

REF: https://github.com/google/oss-fuzz/pull/4366

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Aug 26 03:20:46 UTC 2020 on sn-devel-184
2020-08-26 03:20:45 +00:00
Andrew Bartlett
49f58b2b09 oss-fuzz: Try harder to ensure we always fail fast
During a previous attempt to fix the LANG= issue I changed
the script invocation to be via a shell, so the set -x et al
ensures these are always in place and we fail fast
rather than failures only being detected by lack of output.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2020-08-26 01:57:33 +00:00
Douglas Bagnall
326bc84c0d oss-fuzz: use uninstrumented dynamic python
We can't link to the instrumented statically built Python, so instead
we use the system Python in the docker image.

REF: https://github.com/google/oss-fuzz/issues/4223
REF: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=22618
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14451

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-08-03 02:51:35 +00:00
Andrew Bartlett
bbc4ebbcaf lib/fuzzer: Allow coverage build for oss-fuzz
This still does not seem to be enough but it is one step towards a working
coverage build.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-12-18 06:39:26 +00:00
Andrew Bartlett
0be0c044b6 autobuild: extend autobuild with samba-fuzz job to build the fuzzers in AFL mode using oss-fuzz scripts
This helps ensure the build_samba.sh file keeps working and the fuzzers build
(because they are excluded from the main build).

This is not in the default autobuild because it uses too much
space on sn-devel (4GB).

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>
2019-12-11 02:55:32 +00:00
Andrew Bartlett
8b06cabc7d bootstrap: Add chrpath as a required package
This is used to test build.sh, part of the oss-fuzz integration, and so also that we
correctly build our fuzzers.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Dec 10 09:15:43 UTC 2019 on sn-devel-184
2019-12-10 09:15:43 +00:00
Andrew Bartlett
4c8388fb19 lib/fuzzing Truncate the original files after RUNPATH manipulation in build.sh
This saves space on the rackspace runners in particular.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-12-10 07:50:29 +00:00
Andrew Bartlett
5e5d18c5b1 lib/fuzzing Add comments to explain RUNPATH manipulation in build.sh
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-12-10 07:50:29 +00:00
Andrew Bartlett
5bb9ecdf15 lib/fuzzing: Support an oss-fuzz build with either address or undefined behaviour sanitizers
Add handler for $SANITIZER in build.sh

This allows a build with the undefined behaviour sanitizer.

Otherwise we fail the oss-fuzz CI because the UBSan build links with ASan.

Once this in in then https://github.com/google/oss-fuzz/pull/3094
can be merged to oss-fuzz.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-12-10 07:50:29 +00:00
Andrew Bartlett
f79caf3b6b lib/fuzzing: Remove oss-fuzz build.sh stub from the Samba repo
We need to ship the stub build.sh in the oss-fuzz repo, not ours.
This is because otherwise the travis CI checks skip the build
(it thinks we are not set up yet, or have been disabled).

See https://github.com/google/oss-fuzz/pull/3094 for the PR
creating a similar file there.  This is very similar to how
janus-gateway operates, so this is an accepted pattern.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-12-10 07:50:29 +00:00
Andrew Bartlett
8382fa6408 oss-fuzz: Align build.sh sh parameters with pattern from the oss-fuzz project
We should run build_samba.sh with -eux to ensure we exit on failure,
refuse to use an unset varible and print the commands we are running.

(The suggested build.sh on the oss-fuzz side uses -eu).

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-12-10 07:50:29 +00:00
Andrew Bartlett
cc128c7885 lib/fuzzing/oss-fuzz: copy required libraries to the build target
This is an alternative to static linking as we do not have static source
libraries for all the things we depend on.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
2019-11-20 23:19:36 +00:00
Andrew Bartlett
4946811eb6 lib/fuzzing/oss-fuzz: Install chrpath as we use it in the build.sh script to set -rpath
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
2019-11-20 23:19:35 +00:00
Douglas Bagnall
fbb2377d51 lib/fuzzing/oss-fuzz: Add build_image.sh using Samba's bootstrap tools
Google's oss-fuzz environment is Ubuntu 16.04 based so we can
just use the maintained bootstrap system rather than a manual
package list here that will get out of date.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
Pair-programmed-by: Andrew Bartlett <abartlet@samba.org>
2019-11-20 23:19:35 +00:00
Andrew Bartlett
ec4f6f8fd3 lib/fuzzing/oss-fuzz: add stub build.sh that will not change often
This makes local development of build_samba.sh easier as it will remain in the source tree.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
2019-11-20 23:19:35 +00:00
Andrew Bartlett
f57c0238e9 lib/fuzzing/oss-fuzz: Add build_samba.sh for oss-fuzz
We work hard to put the primary logic for oss-fuzz here, and
where possible into waf, so that only a tiny stub needs to
be maintained in the Google oss-fuzz repo.

This will be called by build.sh (not copied directly because
it is too easy to forget to copy in an updated version when
doing development in the docker image).

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Uri Simchoni <uri@samba.org>
2019-11-20 23:19:35 +00:00