1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-22 13:34:15 +03:00
Commit Graph

556 Commits

Author SHA1 Message Date
Stefan Metzmacher
3189807278 script/autobuild.py: skip lcov step for samba-fips
This doesn't really work and only generates an empty samba-fips.info
file.

Someone familiar with gcov/lcov should look at this and fix it.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2021-04-13 08:23:35 +00:00
Stefan Metzmacher
63853b823c script/autobuild.py: split samba-ad-dc-backup into samba-ad-back{1,2}
This will make it possible to run them in parallel (hopefully on shared
gitlab runners).

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2021-04-13 08:23:35 +00:00
Stefan Metzmacher
c2a725b224 script/autobuild.py: move ad_dc_backup to samba-ad-dc-6
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2021-04-13 08:23:35 +00:00
Stefan Metzmacher
307edf8202 script/autobuild.py: split out samba-{nt4,h5l,no-opath}-build
These will be used to move the build stages from private gitlab runner
jobs to shared runners.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2021-04-13 08:23:35 +00:00
Stefan Metzmacher
37619d399b script/autobuild.py: split out "samba-{def,mit}-build"
This means we avoid a lot of cpu usage for the build.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2021-04-13 08:23:35 +00:00
Stefan Metzmacher
95849d3d6a script/autobuild.py: add support for dependencies
This will make it possible to split build and test stages
in the next steps.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2021-04-13 08:23:35 +00:00
Stefan Metzmacher
404cd173e9 script/autobuild.py: store the directory for the running builder in self.builder_dir
For now it keeps being the same as self.test_source_dir, but that will
change soon.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2021-04-13 08:23:35 +00:00
Stefan Metzmacher
e315ce40c2 script/autobuild.py: defer cp and git clone
This will make further reconstruction easier.

Use 'git show -w' to see the minimal diff.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2021-04-13 08:23:35 +00:00
Stefan Metzmacher
ba6f6a3cce script/autobuild.py: pass --with-selftest-prefix via make instead of configure
We do start for almost all jobs, just samba-ctdb keeps passing it via
configure in order to have a regression test for it.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2021-04-13 08:23:35 +00:00
Stefan Metzmacher
2933c02718 script/autobuild.py: split out a CLEAN_SOURCE_TREE_CMD
This works for all cases even if the builder runs in a subdirectory.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2021-04-13 08:23:35 +00:00
Stefan Metzmacher
0e7a7440f7 script/autobuild.py: change the task definitions into an dictionary
The will make it easier to add more meta data properties to task
definitions.

Use 'git show -w' to see the minimal diff.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2021-04-13 08:23:35 +00:00
Douglas Bagnall
d26b700258 autobuild: fuzz: correctly spell AFL build option
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
2021-03-16 17:09:32 +00:00
Stefan Metzmacher
9883ac4593 script/autobuild.py: let cleanup() ignore errors from rmdir_force() by default
It's not useful to generate a python backtrace from within the cleanup code.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Jan 27 18:17:17 UTC 2021 on sn-devel-184
2021-01-27 18:17:17 +00:00
Stefan Metzmacher
7a5df2deaa script/autobuild.py: split out a rmdir_force() helper function
That also tries to re-add write permissions before removing.
In future we'll have jobs changing there directory to read-only.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2021-01-27 17:07:09 +00:00
Karolin Seeger
715b208b51 script/release.sh: always select the GPG key by it's ID
Signed-off-by: Karolin Seeger <kseeger@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
2021-01-21 13:57:30 +01:00
Karolin Seeger
2f6cea063d script/release.sh: Use new GPG key.
Signed-off-by: Karolin Seeger <kseeger@samba.org>
2021-01-04 11:45:30 +00:00
Stefan Metzmacher
3e61d1ff31 autobuild.py: use --enable-clangdb for the "samba-ctdb" task
The key is that we only enable it for just one task.

I plan to restructure the autobuild tasks, but 'samba-ctdb'
will stay the way it works currently.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-12-16 12:50:37 +00:00
Ralph Boehme
436903afe2 CI: add samba-no-opath
Add a job that builds with O_PATH undefined.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-12-16 09:08:30 +00:00
SATOH Fumiyasu
6ac16232de autobuild: Encode text/plain into base64 to wrap long-lines
MIMEText(text, 'plain', 'utf-8') encodes the text into
base64 and adds 'Content-Transfer-Encoding: base64' header.

Signed-off-by: SATOH Fumiyasu <fumiyas@osstech.co.jp>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Nov 16 22:43:35 UTC 2020 on sn-devel-184
2020-11-16 22:43:34 +00:00
Andrew Bartlett
a57702db1d oss-fuzz: Always run the check, even on the oss-fuzz platform
It is much harder to determine why we get messages like
    Step #6: Error occured while running fuzz_reg_parse:
    Step #6: /workspace/out/coverage/fuzz_reg_parse: error while loading shared libraries: libavahi-common.so.3: cannot open shared object file: No such file or directory
instead this detects the failure to use RPATH (which is
strictly required instead of the modern RUNPATH)
otherwise.

We do this by creating a new build_samba.sh after renaming
build_samba.sh to do_build.sh because this is what oss-fuzz
runs, meaning we don't need to coordinate a MR there as well.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2020-10-22 23:08:31 +00:00
Andrew Bartlett
c03a265030 oss-fuzz: standardise on RPATH for the static-ish binaries
This includes a revert of commit e60df21499.

We strictly require RPATH, not the modern RUNPATH for the behaviour
we need in oss-fuzz, which is that not just the first line of dependencies
but the full set of libraries used by the program are looked for in the
'$ORIGIN/lib' directory.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Oct 22 14:10:04 UTC 2020 on sn-devel-184
2020-10-22 14:10:04 +00:00
Volker Lendecke
570c425dba test: Get the clusteredmember environment out of its smb1 corner
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Oct 16 18:30:18 UTC 2020 on sn-devel-184
2020-10-16 18:30:18 +00:00
Andrew Bartlett
88663eb949 autobuild: Merge no-modules test with the library --disable-python build
This avoids another full compile cycle.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2020-10-01 01:18:39 +00:00
Andrew Bartlett
584f5106cd autobuild: Remove more "make install" steps
Running a "make install" involves a full re-link which takes quite some time
we really only need to test this in a couple of basic combinations, so remove
from a few more targets.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2020-10-01 01:18:39 +00:00
Andrew Bartlett
5419774b23 autobuild: Remove the os.getpid() from the autobuild directory
This might help our CI runners get a ccache hit by keeping the path constant.

Otherwise, we only get a good ccache hit rate if the docker container gives
us the same pid each time.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2020-10-01 01:18:39 +00:00
Andrew Bartlett
3e48825507 build: Remove Python2 support from the build
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14488

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2020-10-01 01:18:38 +00:00
Andrew Bartlett
8557a52937 autobuild.py: Combine samba-static and samba-nopython
We expect these will complete in under an hour and reduce the number of
parallel jobs.  Hopefully there will be some ccache hits between these
as well.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2020-10-01 01:18:38 +00:00
Andrew Bartlett
4dbe8d1131 python: Remove remaining references to third_party python libs
For now at least we do not have any in third_party.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: David Mulder <dmulder@suse.com>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Aug 21 00:12:52 UTC 2020 on sn-devel-184
2020-08-21 00:12:51 +00:00
Andrew Bartlett
0208d5f64b Add docs build to CI
We did not check we could actually build the HTML of the
Samba Developers guide and HTML of the manpages previously.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Jun 10 07:11:59 UTC 2020 on sn-devel-184
2020-06-10 07:11:59 +00:00
Andreas Schneider
10b195fe12 gitlab-ci: Create a single samba-fips runner
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-05-24 23:55:36 +00:00
Andreas Schneider
242b6cf2c4 autobuild: Use sane random sleep values for samba-admem-mit
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
2020-05-18 14:43:41 +00:00
Andreas Schneider
4fe51b27e0 autobuild: Add ad_member_fips target
Reviewed-by: Alexander Bokovoy <ab@samba.org>
2020-05-18 14:43:40 +00:00
Andrew Bartlett
4e08ea2aa3 selftest: Avoid running the slowest of the "none" tests in samba-o3
This job is already quite long and these tests are unlikely
to vary between hosts or under the -O3 compile

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2020-05-15 06:05:30 +00:00
Andreas Schneider
9918abd732 gitlab-ci: Add runner for fips compliance testing
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Apr  8 14:45:18 UTC 2020 on sn-devel-184
2020-04-08 14:45:18 +00:00
Noel Power
a11c58c5f9 autobuild: Add ad_dc_default_smb1 to autobuild jobs
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-04-03 15:08:27 +00:00
Noel Power
ffe856ce32 autobuild: Add nt4_dc_smb1_done to autobuild jobs
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-04-03 15:08:27 +00:00
Noel Power
814a6d43f4 autobuild: Add fileserver_smb1_done to autbuild jobs
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-04-03 15:08:27 +00:00
Noel Power
bbce98c264 autobuild: Add ad_dc_smb1_done to autobuild jobs
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-04-03 15:08:27 +00:00
Noel Power
e468b03fc4 autobuild: Add ad_dc_default_smb1 to autobuild jobs
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-04-03 15:08:27 +00:00
Noel Power
82e4dd5025 autobuild: Add new nt4_dc_smb1 env to jobs
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-04-03 15:08:26 +00:00
Noel Power
087eb9d744 script: Add ad_dc_smb1 to autobuild jobs
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-04-03 15:08:26 +00:00
Noel Power
1954972cfd script: Add testenv fileserver_smb1 to autobuild jobs
Adjust the existing autobuild jobs that currently include/exclude
test env fileserver to also use fileserver_smb1.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
2020-04-03 15:08:26 +00:00
Ralph Boehme
a45e637025 selftest: allow SMB1 in clusteredmember env
This is a relatively new test environment that only uses very few tests
yet. Skip the env name rename dance for this env, but someone has to convert the
tests later on.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2020-04-03 15:08:25 +00:00
Stefan Metzmacher
9b1e96197e script/autobuild.py: allow write_system_info commands to fail
These commands are just there as hints to debug possible problems.
In order to support autobuild.py on non-linux platforms we should
just ignore errors here.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Apr  2 07:36:07 UTC 2020 on sn-devel-184
2020-04-02 07:36:07 +00:00
Stefan Metzmacher
0312a10e09 script/autobuild.py: use more portable 'cp -R -a -l'
On platforms like FreeBSD 12 cp doesn't accept the long options,
using the one letter options works there and keeps working on Linux
as well.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-04-02 06:01:39 +00:00
Stefan Metzmacher
f9374d0a4e script/autobuild.py: add support git worktree
.git is not always a directory, with 'git worktree' it's a file.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2020-04-02 06:01:38 +00:00
Andrew Bartlett
9bdc5a6742 autobuild: Merge samba-simpleserver into samba-nt4
This saves CI resources (20mins build time) at the expense of making
the samba-nt4 job take 45min (5 min longer).

The new maximum job time is 1:15 by samba-o3 on the different hosts.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Mar 23 14:45:59 UTC 2020 on sn-devel-184
2020-03-23 14:45:59 +00:00
Andrew Bartlett
7a44bd1c42 autobuild: Merge the samba-ktest-heimdal and samba-fileserver jobs
This avoids a full compile of Samba just to test Kerberos with a system Heimdal
while still providing an environment to test other fileserver features.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-03-23 13:02:32 +00:00
Andrew Bartlett
607c9ab307 build: Require --without-ad-dc for --without-ads
Building an AD DC while setting --without-ads makes no sense and just wastes compile time on our build hosts.

To allow samba-nt4 to build --without-ad-dc we set rpc.spoolss.notify
(which is built on the NTVFS fileserver for the callbacks) to run in
the ad_member environment rather than nt4_dc and ad_dc.

This is also just more realistic in any case.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-03-23 13:02:31 +00:00
Andreas Schneider
54f26cfcf2 autobuild: Run the none env in the samba-o3 build
This includes tests which should make sure that certain code is not
optimized away, like memset_s().

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Mar  9 23:42:26 UTC 2020 on sn-devel-184
2020-03-09 23:42:26 +00:00
Volker Lendecke
8441471d5d autobuild: samba-ctdb does not need an AD DC
Speed up the build a bit

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Feb 19 11:28:56 UTC 2020 on sn-devel-184
2020-02-19 11:28:56 +00:00
Volker Lendecke
cfb8e4de8f autobuild: Split a long line
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2020-02-19 09:38:40 +00:00
Volker Lendecke
ebf67b6da7 autobuild: Run a simple test for clustered samba
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2020-02-19 09:38:40 +00:00
Volker Lendecke
602f27d6a2 autobuild: Avoid clusteredmember in normal environments
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2020-02-19 09:38:40 +00:00
Andrew Bartlett
cf4984947b build: Do not check if system perl modules should be bundled
We do not ship any perl modules in third_party at this time, so
this check is pointless and breaks the build for --bundled-libraries=ALL.

As reported by aaptel on https://gitlab.com/samba-team/samba/-/merge_requests/1104#note_281050331

This changes our autobuild script to cover this case in the
samba-static job.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2020-02-04 05:13:39 +00:00
Karolin Seeger
f699df32cd script/release.sh: Don't use quotations any longer.
Signed-off-by: Karolin Seeger <kseeger@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(master): Mon Feb  3 12:45:39 UTC 2020 on sn-devel-184
2020-02-03 12:45:39 +00:00
Stefan Metzmacher
648f94d203 script/release.sh: make it possible to run from a git worktree
.git is a regular file in that case.

Also check that script/release.sh is present as a relative path
to ensure we're called from the expected location.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
2020-01-21 19:29:38 +01:00
Andrew Bartlett
0be0c044b6 autobuild: extend autobuild with samba-fuzz job to build the fuzzers in AFL mode using oss-fuzz scripts
This helps ensure the build_samba.sh file keeps working and the fuzzers build
(because they are excluded from the main build).

This is not in the default autobuild because it uses too much
space on sn-devel (4GB).

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>
2019-12-11 02:55:32 +00:00
Andrew Bartlett
d349d344f8 autobuild.py: Avoid listing jobs twice
We use the tasks table instead, to avoid the issue shown in the previous commit.

Now we just have to keep .gitlab-ci.yml and the tasks table in sync.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>
2019-12-11 02:55:32 +00:00
Andrew Bartlett
df38d51104 autobuild.py: Add missing samba-simpleserver job
This was missed when the job was split out in f0e8dd1a08.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>
2019-12-11 02:55:32 +00:00
Ralph Boehme
24905ba82d docs-xml: remove explicit "constant"
The constant mark applied to types "string" and "ustring". The previous patches
in this patchset already markes all string options as either constant or
substituted, but it's still possible to add options or change existing ones to
be neither constant nor substituted.

In order to enforce strings to be either constant or substitued, remove the
explicit constant marker. Instead, any option that is not marked as substituted
is implicitly made constant.

This patch doesn't change behaviour and all generated files are the same before
and after this change.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-11-27 10:25:37 +00:00
Stefan Metzmacher
ef1f0e9ffe param: add FN_{GLOBAL,LOCAL}_SUBSTITUTED_STRING support
Pair-Programmed-With: Ralph Boehme <slow@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
2019-11-27 10:25:32 +00:00
Christof Schmitt
7965a204d6 autobuild: Add test for build with system heimdal library
The configure option --with-system-heimdalkrb5 requires --without-ad-dc.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14179

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-11-13 17:49:38 +00:00
Andreas Schneider
831796b2d7 gitlab-ci: Run 'samba' tests also with MIT Kerberos
This runs the tests of the ktest environment.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Nov  4 14:19:17 UTC 2019 on sn-devel-184
2019-11-04 14:19:17 +00:00
Ralph Boehme
6a9041ba43 autobuild: rename samba-ad-member-* to samba-admem-*
Needed to come around MAX pathlen limit for UNIX domain socket triggered by
the new samba-ad-member-mitkrb5 target added in samba-ad-member-mitkrb5.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Oct 22 14:53:17 UTC 2019 on sn-devel-184
2019-10-22 14:53:17 +00:00
Uri Simchoni
649d15bb96 autobuild: harden samba-xc test suite
Add more checks which directly test the behavior of
--cross-answers and --cross-execute.

Previous test tested things in a round-about way, checking
that running in all three modes (native, cross-execute,
cross-answers) yields the same result. It was vulnerable
to a degradation in which cross-compilation modes didn't
work at all and were running native tests, which is
what happened with the upgrade of waf.

The added tests check the following:
- That cross-excute with cross-answers sets the cross-answers file
- That the content of cross-answers file actually affects the build
  configuration
- That a missing line in cross-answers fails the build

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13846

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Uri Simchoni <uri@samba.org>
Autobuild-Date(master): Sun Oct 20 13:29:58 UTC 2019 on sn-devel-184
2019-10-20 13:29:58 +00:00
Andreas Schneider
46068d5f28 gitlab-ci: Run several AD tests with MIT KDC
This will avoid introducing regressions in either client or server code.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-10-12 16:18:39 +00:00
Ralph Boehme
f0e8dd1a08 selftest: split fileserver testenv into simpleserver and fileserver
The simpleserver testenv continues to be built with minimal
dependencies. fileserver otoh will be built with bells and whistles including
JSON which I need for using the env as target for Spotlight tests.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
2019-10-09 14:35:29 +00:00
Stefan Metzmacher
0df28b3d14 script/autobuild.py: remove explicit --picky-developer"
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2019-08-06 06:32:28 +00:00
Gary Lockyer
6d6fe49938 traffic replay: Store the instance id in the replay context
Store the traffic runner instance id in the replay context.  Will be
used in subsequent commits.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-24 02:24:27 +00:00
Douglas Bagnall
96148436b7 script/compare_cc_results: print usage on too few args
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-05 01:05:20 +00:00
Douglas Bagnall
5331f913ac script/bisect-test: fix for py3
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-02 04:21:36 +00:00
Joe Guo
8de56ddc92 script/autobuild.py: make code more pythonic
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:17 +00:00
Joe Guo
fb12a25261 script/autobuild.py: avoid nested try except block
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:17 +00:00
Joe Guo
ae90e5263b script/autobuild.py: rm redundant empty str in options
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:16 +00:00
Joe Guo
04a6967bc3 script/autobuild.py: define cwd for builder and avoid chdir
We can pass cwd in Popen, no need to chdir for each cmd.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:16 +00:00
Joe Guo
6fc3eb8926 script/autobuild.py: improve run_cmd with check_output
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:16 +00:00
Joe Guo
0daeb6046b script/autobuild.py: add helper functions to simplify make test cmd
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:16 +00:00
Joe Guo
0383747dcc script/autobuild.py: replace random-sleep.sh with python function
The random-sleep.sh script is tricky, and there is no need to rely on another
script just for random sleep.

Using the python function, the random number generation will happen when
load autobuild.py other than execute cmd, but shouldn't affect the
result.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:16 +00:00
Joe Guo
1b70d43ed1 script/autobuild.py: rm unused mime type text/plain in tasks
The mime type `text/plain` is repeated everywhere but not used any more.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:16 +00:00
Joe Guo
41d93a6da5 script/autobuild.py: fix tasks indent
Indent tasks at same level, make it easier to copy/move lines arround.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:16 +00:00
Volker Lendecke
7c9eb428d5 autobuild: Fix autobuild for python3
"keys()" in python3 returns an unmodifiable view object. Two lines
down we might want to modify it, which python3 does not allow.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2019-06-18 05:34:07 +00:00
Joe Guo
1b48d8a9ce script/autobuild.py: define LCOV_CMD and run it after each make test
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 08:19:17 +00:00
Joe Guo
c5ae200b56 script/autobuild.py: add ENABLE_COVERAGE placeholder in configure cmd option
The value will be "--enable-coverage" or "".

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 08:19:17 +00:00
Joe Guo
0cdd8bd6e1 script/autobuild.py: replace more placeholders in cmds for coverage
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 08:19:17 +00:00
Joe Guo
87ea55e436 script/autobuild.py: add --enable-coverage option
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 08:19:17 +00:00
Joe Guo
332f19c7f2 script/autobuild.py: mv optionparse to top
so the rest of the code can use the option values directly.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 08:19:17 +00:00
Joe Guo
9ea81c0e37 script/autobuild.py: mv find_git_root and gitroot to top
so the reset of the code can use gitroot directly.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 08:19:16 +00:00
Joe Guo
af0e0c544b script/autobuild.py: rename sdir to test_source_dir
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 08:19:16 +00:00
Douglas Bagnall
bd53819b28 script/attr_count_read: load and correlate all data
This changes script/attr_count_read to take the samba private directory
as an argument and load all the databases at once, printing them as
one big table. It isn't extremely clear what it all means, but it
*tries* to tell you.

With --plot, it will attempt to load matplotlib and plot the number of
requested attributes against the number returned, with colour
of each point indicating its relative frequency. It is a scatterplot
that wants to be a heatmap.

With --no-casefold, you can get an extra confusing table where,
for instance, something repeatedly asks for "attributeId" which is not
accounted for, while in a completely different row an unrequested
"attributeID" is found many times over.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May  1 06:46:36 UTC 2019 on sn-devel-184
2019-05-01 06:46:36 +00:00
Douglas Bagnall
60620273db dsdb/modules: a module to count attribute searches and results
The dsdb module stack can turn a simple search request into a
complicated tree of sub-queries that include attributes not originally
asked for and excluding those that were. The corresponding replies
might contain unrequested attributes or (for good reasons, according
to some module) hide requested ones. The entire stack is there to
meddle and that is what is does. Except *this* module. It just counts.
To understand dsdb performance it helps to have some idea what
requests and replies are flying too and fro. This module, when
inserted anywhere in the stack, counts the requests and replies
passing through and the attributes they contain. This data is stored
in on-disk tdbs in the private/debug directory.

The module is not loaded by default. To load it you need to patch the
source4/dsdb/samdb/ldb_modules/samba_dsdb.c and put "count_attrs"
somewhere in the module lists in the samba_dsdb_init() function. For
example, to examine the traffic between repl_meta_data and
group_audit_log, you would do something like this around line 316:

          "subtree_delete",
          "repl_meta_data",
  +       "count_attrs",
          "group_audit_log",
          "encrypted_secrets",

and recompile. Samba will then write to a number of tdb files in the
debug directory as requests and replies pass through. A simple script
is included to read these files. Doing this:

./script/attr_count_read st/ad_dc/private/debug/debug/attr_counts_not_found.tdb

will print a table showing how often various attritbutes were
requested but not found (from the point of view of the module).

A more sophisticated version of the script is coming in the next
commit, but this one is included first because in its simplicity it
documents the storage format reasonably well. The tdb keys are
attribute names, and the values are uint32_t in machine native order.

When the module is included in the stack there will be a very small
decrease in performance.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-01 05:32:25 +00:00
Joe Guo
2ee72cc615 traffic: load dns query from file and write stats to file
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May  1 01:10:42 UTC 2019 on sn-devel-184
2019-05-01 01:10:42 +00:00
Tim Beale
e387cf9288 traffic_replay: Avoid Exception if no packet rate is specified
traffic_replay would throw an exception if you didn't specify some sort
of packet rate. We can avoid this by using --scale-traffic=1.0 as the
default if nothing else was specified.

 script/traffic_replay model.txt $SERVER.$REALM --duration=10
   --fixed-password=blahblah12# -U$USERNAME%$PASSWORD
INFO 2019-04-10 01:03:01,809 pid:47755 script/traffic_replay #280: Using
the specified model file to generate conversations
Traceback (most recent call last):
  File "script/traffic_replay", line 438, in <module>
    main()
  File "script/traffic_replay", line 293, in main
    opts.conversation_persistence)
  File "bin/python/samba/emulate/traffic.py", line 1295, in
generate_conversation_sequences
    target_packets = int(packet_rate * duration)
TypeError: unsupported operand type(s) for *: 'NoneType' and 'float'

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2019-04-30 23:18:28 +00:00
Tim Beale
641d74cb26 traffic_replay: Assign users to groups by default
The traffic_replay script has a myriad of options, but by default when
it creates user accounts it does not assign these users to any groups
(you have to specify extra options to do that). This isn't really a fair
test of samba performance, because it's unlikely that real world setups
will have users that are in no groups (other than the default ones).

This patch changes the default behaviour so that it will assign the new
users to groups automatically, if no other group options were
specified.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2019-04-30 23:18:28 +00:00
Andreas Schneider
f1bf02c78a autobuild: Build also Samba AD with MIT Kerberos
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2019-04-25 10:47:16 +00:00
Lutz Justen
92f30f91e4 waf: build: Respect --disable-python for third_party modules
Skips installation of samba/third_party stuff into the python directory if
--disable-python is set.

Added test after install that confirms no python modules installed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13905

Signed-off-by: Lutz Justen <ljusten@google.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Apr 24 07:32:31 UTC 2019 on sn-devel-184
2019-04-24 07:32:31 +00:00
Garming Sam
bfd762b53a selftest: rename schemaupgrade_dc (+pair) to schema_dc
This is needed because the name of the autobuild job and
the name of the selftest env end up in the socket path
for ncalrpc sockets.

The challenge is that (for example)
/memdisk/autobuild/fl/b2424063/samba-schemaupgrade/bin/ab/schemaupgrade_pair_dc/ncalrpc/np/protected_storage
does not fit in a struct sockaddr_un.

Signed-off-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Apr 12 05:41:36 UTC 2019 on sn-devel-144
2019-04-12 05:41:36 +00:00
Aaron Haslett
59ee3c864c selftest: split schemaupgrade testenv out
Schemaupgrade tests are particularly resource intensive and are causing
runners to hit their memory and CPU limits, so we need to split them
out.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2019-04-11 04:17:11 +00:00
Douglas Bagnall
dad98d0334 autobuild: attempt authenticated email if environment suggests it
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-05 04:41:25 +00:00
Stefan Metzmacher
be6cf83c01 autobuild: try to distribute the tasks a bit more
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2019-02-27 10:16:11 +00:00