1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-22 13:34:15 +03:00
Commit Graph

460 Commits

Author SHA1 Message Date
Noel Power
0e35d74193 script: Fix failing build_xc job
build_xc job uses compare_cc_results.py to compare cache
files, the cache files are stringified hash maps, the results
in python 3.4 don't compare well due to inconsistent order of
dict key/value pairs when the cache files are created. While
comparing the file contents works fine in python3.6 it fails
with python3.4. This patch detects problematic dict lines and
rewrites the value for comparison

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-12-10 10:38:25 +01:00
Noel Power
9bdeb95f2c autobuild: Remove temporary purepy3 tasks
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-12-10 10:38:24 +01:00
Noel Power
61adb75b9f CI: Add new (TEMP) pure python3 autobuild jobs for samba-build & samba-nt4
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-12-10 10:38:24 +01:00
Noel Power
b920d80eca script: PY3 port traffic_learner
Use python3 compatable print

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-12-10 10:38:23 +01:00
Noel Power
fd87c89c64 script: Add new (temporary) pure python3 ad-dc test
Ideally we want all the tests to run under python3 by default (no
special task for this) and then convert the existing '-py3' tasks
to run the python tests with python3.
However at the moment the convertion process is not ready to do this,
for a while we need to run separate autobuild tasks for this.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-12-10 10:38:22 +01:00
Tim Beale
5517d60653 traffic_replay: Add a max-members option to cap group size
traffic_replay tries to distribute the users among the groups in a
realistic manner - some groups will have almost all users in them.
However, this becomes a problem when testing a really large database,
e.g. we may want 100K users, but no more than 5K users in each group.

This patch adds a max-member option so we can limit how big the groups
actually get.

If we detect that a group exceeds the max-members, we reset the group's
probability (of getting selected) to zero, and then recalculate the
cumulative distribution. The means that the group should no longer get
selected by generate_random_membership(). (Note we can't completely
remove the group from the list because that changes the
list-index-to-group-ID mapping).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Dec  4 12:22:50 CET 2018 on sn-devel-144
2018-12-04 12:22:50 +01:00
Andreas Schneider
a9477eb3bd autobuild: Add _FORTIFY_SOURCE=2 to the -O3 build
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-11-28 23:19:21 +01:00
Noel Power
2ca3b3ebd8 script: Add new (temporary) pure python3 ad-dc-2 test
Ideally we want all the tests to run under python3 by default (no
special task for this) and then convert the existing '-py3' tasks
to run the python tests with python3.
However at the moment the convertion process is not ready to do this,
for a while we need to run separate autobuild tasks for this.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-14 05:07:17 +01:00
Tim Beale
a0b5f4b7b0 traffic_replay: Rework machine accounts to remove redundant code
generate_users_and_groups() now generates the machine acounts as well as
the user accounts, so it seems there's no need to also have
generate_traffic_accounts(), which does the same job.

Instead, we can just pass through the number of machine acounts to
generate_users_and_groups() and delete the other function.

Also updated generate_users_and_groups() so that machine_accounts is
no longer optional (we want to create machine accounts in all cases).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07 17:55:09 +01:00
Tim Beale
51917fc07f traffic_replay: Move 'traffic account' flag up a level
We create machine accounts for 2 different purposes:
1). For traffic generation, i.e. testing realistic network packets.
2). For generating a realistic large DB.

Unfortunately, we want to use different userAccountControl flags for
the 2 different cases. Commit 3338a3e257 changed the flags used
for case #2, but this breaks case #1.

The problem is generate_users_and_groups() is called in both cases,
so we want the 'traffic account' flag passed into that function.
This ensures that the machine accounts get created with the appropriate
userAccountControl flags for the particular case you want to test.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07 17:55:08 +01:00
Tim Beale
85b6d88989 traffic_replay: Move machine account creation
I was assuming that generate_users_and_groups() only gets called in the
--generate-users-only case. However, it also gets called in the default
traffic replay case.

This patch reworks the code so that the number of machine accounts to
create gets passed in, and the 'create 25% more computers than users'
assumption only applies to the --generate-users-only case.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-07 17:55:08 +01:00
Douglas Bagnall
5e79630822 autobuild: remove unused variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:15 +01:00
Douglas Bagnall
9055602e63 traffic_replay: Exception has no .message
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:13 +01:00
Douglas Bagnall
2013cd2889 script/show_test_time: attempt py3 compat
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:10 +01:00
Tim Beale
4a8a0ab32b traffic_replay: logger was ignoring smb.conf log-level
We were trying to access the debug-level (in python C bindings) before
the smb.conf had been loaded and actually set the debug-level. So it
would default to zero, regardless of what was in the smb.conf.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-10-31 00:30:16 +01:00
Tim Beale
b40daca6d4 traffic_replay: Generate users faster by writing to local DB
We can create user accounts much faster if the LDB connection talks
directly to the local sam.ldb file rather than going via LDAP. This
patch allows the 'host' argument to the tool to be a .ldb file (e.g.
"/usr/local/samba/private/sam.ldb") instead of a server name/IP.

In most cases, the traffic_replay tool wants to run on a remote device
(because the point of it is to send traffic to the DC). However, the
--generate-users-only is one case where the tool can be run locally,
directly on the test DC. (The traffic_replay user generation is handy
for standalone testing, because it also handles assigning group
memberships to the generated user accounts).

Note that you also need to use '--option="ldb:nosync = true"' to get
the improvement in performance.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-10-31 00:30:16 +01:00
Douglas Bagnall
cd5ac17654 script/show_test_time: approach python 3 compatibility
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
2018-10-25 21:45:58 +02:00
Douglas Bagnall
5773290a6f script/generate_param.py: remove unused imports
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
2018-10-25 21:45:54 +02:00
Noel Power
d357141546 CI/Autobuild: Remove samba-none-env-py3 test
We now run a purepython3 none-env test, later when the whole
build is running under python3 we will resurrect this job
but as (samba-none-env-py2) for python2

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Oct 23 09:10:22 CEST 2018 on sn-devel-144
2018-10-23 09:10:22 +02:00
Noel Power
f8b5dd9544 script: Add new (temporary) autobuild task for none-env tests.
Ideally we want all the tests to run under python3 by default (no
special task for this) and then convert the existing '-py3' tasks
to run the python tests with python3.
However at the moment the convertion process is not ready to do this,
for a while we need to run separate autobuild tasks for this.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-10-23 05:50:29 +02:00
Joe Guo
4260fb87ed script/traffic_replay: apply new logger to replace print
Use logger to replace print

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-10-10 06:16:21 +02:00
Joe Guo
57594c8772 script/traffic_replay: get debug level via api
The -d option will set samba global debug level automatically.
We should not parse and use the passed in value.

Use samba.get_debug_level instead.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-10-10 06:16:21 +02:00
Joe Guo
33ce1fa23e script/traffic_replay: print packets data to stderr
This is debug info, should print to stderr.
Otherwise it will flood stdout.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-10-10 06:16:21 +02:00
Mathieu Parent
7dd388a1f9 third_party: Add pam_set_items.so from pam_wrapper
Pair-Programmed-With: Andreas Schneider <asn@samba.org>
Signed-off-by: Mathieu Parent <math.parent@gmail.com>
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
2018-10-02 14:12:13 +02:00
Noel Power
de1e6d7856 script: add new autobuild task for building pure python3
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-28 01:01:23 +02:00
Philipp Gesang
c98f997192 turn --with-json-audit into global --with-json
Fold the build option --with-json-audit into the toplevel wscript
to reflect the fact that JSON support is no longer local to the
audit subsystem.

Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-26 20:41:07 +02:00
Alexander Bokovoy
8f022a0be8 script/autobuild: Fix formatting in send_email
Commit cb40e2bbc8 introduced a print
statement with a broken formatting. Reported by pylint.

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-05 06:37:28 +02:00
Alexander Bokovoy
67ed1ea6c4 script/autobuild: re-use CACHE_SUFFIX from waflib
Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-05 06:37:28 +02:00
Thomas Nagy
247d12c4fa build:wafsamba: Ignore cfg_file absolute paths differences
Due to build variants, cfg_file paths are written as absolute paths.

Signed-off-by: Thomas Nagy <tnagy@waf.io>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-05 06:37:22 +02:00
Thomas Nagy
5967335387 script/autobuild: use --out instead of -b when calling configure
Signed-off-by: Thomas Nagy <tnagy@waf.io>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-05 06:37:21 +02:00
Thomas Nagy
9aa8f2badd build:wafsamba: detail where we are processing the autobuild
Signed-off-by: Thomas Nagy <tnagy@waf.io>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-05 06:37:21 +02:00
Andrew Bartlett
825d67fd35 autobuild: Avoid subshell for tail -f invocation
This should mean one less process in the process tree, and less places to hold
FDs open.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13591

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Aug 29 08:20:55 CEST 2018 on sn-devel-144
2018-08-29 08:20:55 +02:00
Andrew Bartlett
47c14ef64e autobuild: use close_fds=True to avoid *.stderr and *.stdout inheriting into every process
This closes fds other than 0, 1, 2.

This ensures only the correct *.stderr and *.stdout is attached, via
the stdout/stderr parameter to Popen(), but not every other FD
currently open in python at the time Popen is called.

For the tail invocation and other calls to Popen(), because fds 0, 1,
2 are still attached, these function as before.

Per https://docs.python.org/2.6/library/subprocess.html:

"If close_fds is true, all file descriptors except 0, 1 and
2 will be closed before the child process is executed. (Unix only)."

And regarding the passed in parameters:

"stdin, stdout and stderr specify the executed programs’ standard
input,
standard output and standard error file handles, respectively.  "
...

"With None (the default), no redirection will occur;
the child’s file handles will be inherited from the parent. "

(The unwanted inherited files would be on a random high FD, where the
program wouldn't know what to do with them, but counting towards the
process FD limit).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13591

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-08-29 05:28:17 +02:00
Andrew Bartlett
6b11643617 autobuild: Move backup/restore environments into to samba-ad-dc-2 autobuild job
This tries to to split up the tasks more evenly and may help with the python3
work by isolating them from the long samba job.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-08-29 05:28:16 +02:00
Andrew Bartlett
f9e494a6f2 autobuild: Move ad_dc_no_nss into to samba-ad-dc-2 autobuild job
This tries to to split up the tasks more evenly and may help with the python3 tests
against this environment if started from a more isolated job.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-08-29 05:28:16 +02:00
Andrew Bartlett
38784f2b37 autobuild: Reduce duplication of task list in autobuild
The defaulttasks or builddirs are often updated out of sync, which causes confusion until
it is resolved.

We simply choose "." as the builddir for the tasks that
are not in the default set.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-08-29 05:28:16 +02:00
Andrew Bartlett
0da15fa764 autobuild: Implement a split python2 and python2 build pattern
The default tasks will run the tests without --extra-python specified and
the new -py3 tasks will run the python3 tests only.

This will reduce the complexity of the build combinations

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-08-29 05:28:16 +02:00
Andrew Bartlett
9a3b64707f autobuild: Fix -py3 support to look in tasks (the table with the tests) not tasknames (the requested list)
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-08-27 08:53:20 +02:00
Andrew Bartlett
05389bcbf8 autobuild: Allow automatic handling of autobuild tasks ending in -py3
This will allow splitting up of the python2 and python3 tests without
duplication of this already complex file.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Aug 25 11:53:15 CEST 2018 on sn-devel-144
2018-08-25 11:53:15 +02:00
Douglas Bagnall
2ccb0b0384 PEP8: improve spacing around colons
These dropped out of Joe's patches during rebase and review.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-08-24 07:49:31 +02:00
Joe Guo
aa243d1a7f PEP8: fix W291: trailing whitespace
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:31 +02:00
Joe Guo
cabb299749 PEP8: fix E703: statement ends with a semicolon
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:30 +02:00
Joe Guo
95c36d825c PEP8: fix E401: multiple imports on one line
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:30 +02:00
Joe Guo
115f2a71b8 PEP8: fix E305: expected 2 blank lines after class or function definition, found 1
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:30 +02:00
Joe Guo
fd6b2086cb PEP8: fix E303: too many blank lines (2)
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:30 +02:00
Joe Guo
211c9a5f85 PEP8: fix E302: expected 2 blank lines, found 1
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:29 +02:00
Joe Guo
c809a86023 PEP8: fix E261: at least two spaces before inline comment
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:29 +02:00
Joe Guo
1ccc36b401 PEP8: fix E251: unexpected spaces around keyword / parameter equals
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:29 +02:00
Joe Guo
12d3fbe15c PEP8: fix E231: missing whitespace after ','
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:28 +02:00
Joe Guo
87bbc2df97 PEP8: fix E226: missing whitespace around arithmetic operator
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:28 +02:00
Joe Guo
32266d2d48 PEP8: fix E225: missing whitespace around operator
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:28 +02:00
Joe Guo
fb5ea356dd PEP8: fix E222: multiple spaces after operator
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:27 +02:00
Joe Guo
ba0827b560 PEP8: fix E203: whitespace before ':'
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:27 +02:00
Joe Guo
562411bd95 PEP8: fix E202: whitespace before ')'
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:27 +02:00
Joe Guo
a9551edaee PEP8: fix E201: whitespace after '('
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:27 +02:00
Joe Guo
5d532543ab PEP8: fix E128: continuation line under-indented for visual indent
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:27 +02:00
Joe Guo
484ce0634b PEP8: fix E123: closing bracket does not match indentation of opening bracket's line
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:26 +02:00
Joe Guo
297faf3252 PEP8: fix E122: continuation line missing indentation or outdented
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:26 +02:00
Joe Guo
b43408b383 PEP8: fix E121: continuation line under-indented for hanging indent
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:26 +02:00
Joe Guo
4fc08d8f14 PEP8: fix E111: indentation is not a multiple of four
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:25 +02:00
Joe Guo
ceed07fe89 traffic-replay: add extra check
Make sure --average-groups-per-user is not more than --number-of-users

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-15 07:08:25 +02:00
Andrew Bartlett
2ba5fb2031 autobuild: Test with and without building bundled popt
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Jul 28 03:39:48 CEST 2018 on sn-devel-144
2018-07-28 03:39:48 +02:00
Ralph Boehme
010bbe536e autobuild: add some basic tests for the all static build
This makes sure each module is at least loaded once
and registers itself as a module.

It means that the skel_opaque and skel_transparent vfs examples
are loaded.

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
2018-07-25 00:23:12 +02:00
Andrew Bartlett
9a7e9e527f autobuild: Fix random-sleep.sh invocation in autobuild.py
The scripts were not running with the correct path and this causes sn-devel to hit
a very high load as many of the compile jobs start at once.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Jul  5 06:51:26 CEST 2018 on sn-devel-144
2018-07-05 06:51:26 +02:00
Andrew Bartlett
f33749fc84 autobuild: Build samba-fileserver --without-json-audit
This build target is already --without-ad-dc and is the one we need to ensure is
compatible with a host without the Jansson JSON library.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Jun 26 02:03:30 CEST 2018 on sn-devel-144
2018-06-26 02:03:30 +02:00
Andrew Bartlett
e9b638c43f autobuild: cover the Gentoo case with python disabled all down the stack
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Fri May 25 13:07:47 CEST 2018 on sn-devel-144
2018-05-25 13:07:47 +02:00
Andrew Bartlett
0e11fabbfa autobuild: Include information of disk free in system-info.txt
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu May 24 00:42:48 CEST 2018 on sn-devel-144
2018-05-24 00:42:48 +02:00
Andrew Bartlett
754a840946 autobuild: build ldb --without-ldb-lmdb
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 23 05:11:13 CEST 2018 on sn-devel-144
2018-05-23 05:11:13 +02:00
Tim Beale
c0fd6cd386 script: Add 'random-seed' option to traffic_replay
When using a traffic-model file to generate traffic, there is some
randomness in the actual packets that get generated. This means it's
hard to use the tool to detect an increase/decrease in Samba
performance - we don't know whether a decrease in packets sent is due
to a regression in the Samba codebase, or just due to the tool sending
different types of packets (i.e. ones that take longer to process).

This patch adds an option to seed the python random number generator.
This means that exactly the same traffic can be generated across
multiple test runs.

(Previously we were using the '--traffic-summary' option to avoid this
problem - we can generate a summary-file based on the model, and then
use the same summary file across multiple runs. However, this proved
impractical when you want to run multiple combinations of scale/rate
parameters, e.g. 21 x 8 different permutations just fills up disk space
with summary-files.)

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: William Brown <william@blackhats.net.au>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 16 13:53:26 CEST 2018 on sn-devel-144
2018-05-16 13:53:26 +02:00
Joe Guo
7181af9f94 traffic_replay: fetch domain from creds other than opts
For traffic_replay script, when user provides `--workgroup` or `-W` option,
it will be set on the creds option group, not the default opts one.

The previous code will not work properly when smb.conf file is missing.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2018-05-12 02:09:29 +02:00
Joe Guo
4e37336632 traffic_replay: fix typo in message string
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2018-05-12 02:09:28 +02:00
Joe Guo
5ed1682905 traffic_replay: set gensec features to encrypt credentials
While running traffic_replay script against windows dc, it will fail
with a `LDAP_UNWILLING_TO_PERFORM` error for adding user.

Windows requires the credentials to be encrypted before sending.
`set_gensec_features` will fix it.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2018-05-12 02:09:28 +02:00
Douglas Bagnall
cb40e2bbc8 autobuild: do not try to send email to no recipient
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-05-05 04:32:42 +02:00
Douglas Bagnall
4eeb43d06c autobuild: add compiler version to results tarball
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-05-05 04:32:42 +02:00
Andrew Bartlett
4365741fe5 autobuild: Extend build combinations tested to include --without-ldap
This bumps --without-ads to the samba-nt4 job so that option alone is still covered.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-05-03 08:17:44 +02:00
Ralph Boehme
5757d25932 script/git-hooks: add check-trailing-whitespace
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Apr 26 03:54:39 CEST 2018 on sn-devel-144
2018-04-26 03:54:38 +02:00
Ralph Boehme
8649e21665 Add a wrapper script as git pre-commit hook
When developer mode is enabled, the wrapper script
"script/git-hooks/pre-commit-hook" gets installed as

  .git/hooks/pre-commit

and calls "script/git-hooks/pre-commit-script".

This way we can later modify the "script/git-hooks/pre-commit-script"
without the need to ever change the installed commit hook itself.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Jeremy Allison <jra at samba.org>
2018-04-26 01:09:27 +02:00
Joe Guo
ce63db26a1 traffic_relay: bulk port print to modern py3 style
Change print to function and avoid the ugly `print >>sys.stderr`.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Fri Apr 13 10:36:32 CEST 2018 on sn-devel-144
2018-04-13 10:36:32 +02:00
Andrew Bartlett
97d0203151 autobuild: Run all envs that depend on ad_dc in the ad_dc job
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-03-27 23:03:14 +02:00
Andrew Bartlett
56191dcfb7 autobuild: Split up the build further with samba-ad-dc-2
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-03-27 23:03:14 +02:00
Andrew Bartlett
d6ac8beacf autobuild: Try and test different configure options for new environments
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-03-27 23:03:14 +02:00
Andrew Bartlett
e4a969ccc5 autobuild: Remove fileserver tests from the main build
Again, this is to allow these to run in the 50min timelimit
of travis-ci and so gain test coverage.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-03-27 23:03:13 +02:00
Andrew Bartlett
d47fb2232b autobuild: Run all "ad_dc" environment tests in samba-ad-dc
This allows us not to run ad_dc tests in the main build, making the
autobuild process faster.  The ad_dc tests run in less than 50mins
on travis-ci, which allows this part of the tests to be run.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-03-27 23:03:13 +02:00
Andrew Bartlett
7544f35741 autobuild: Run nt4_dc and nt4_member tests in parallel
These do not interact with the main AD DC environments, so can run in parallel

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-03-27 23:03:13 +02:00
Andrew Bartlett
da72c4018e autobuild: Move "none" environment to samba-none-env
This takes this part of the test out of the main, slow samba task
but also keeps it away from samba-o3 which is up against the 50min
budget on travis-ci.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Mar 23 12:00:40 CET 2018 on sn-devel-144
2018-03-23 12:00:40 +01:00
Noel Power
2a64f77d4f script: convert print func to be py2/py3 compatible
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-03-23 07:28:24 +01:00
Andrew Bartlett
d950556755 autobuild: Move defaulttasks to one-per-line
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2018-03-22 08:00:25 +01:00
Douglas Bagnall
5fbb471443 scripts/ python: convert 'except X, e' to 'except X as e'
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-02-15 00:18:30 +01:00
Douglas Bagnall
16e173ad2e selftest and autrobuild: convert 'except X, e' to 'except X as e'
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-02-15 00:18:30 +01:00
Jamie McClymont
2245a4bf9a autobuild: fix quoting of --restrict-tests
Currently, passing multiple tests causes those other than the first to be
passed to make, causing failures.

Signed-off-by: Jamie McClymont <jamiemcclymont@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Fri Jan  5 02:51:09 CET 2018 on sn-devel-144
2018-01-05 02:51:09 +01:00
Gary Lockyer
beeec1ff7c tests: replace traffic_summary test with python blackbox test
Replace the shell subunit test for script/traffic_summary.pl with a
python black box test.

This involves moving the test files to more standard locations.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Aug 17 07:59:38 CEST 2017 on sn-devel-144
2017-08-17 07:59:38 +02:00
Gary Lockyer
7057abcfcd scripts: Scripts to replay and generate samba traffic
Scripts to generate representative network traffic and replay this to a
samba instance.  For load testing, performance profiling and capacity
planning.

traffic_learner  process a file generated by traffic_summary and
                 generate a model that can be used by traffic_replay to
                 generate samba network traffic.

traffic_replay   Replay a summary file generated by traffic_summary, or
                 use a model created by traffic_learner to generate
                 network traffic.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
Pair-Programmed-With: Tim Beale <timbeale@catalyst.net.nz>
2017-08-17 04:06:06 +02:00
Douglas Bagnall
e12dbc7307 traffic_summary: avoid uninitialised variable warning
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-08-17 04:06:06 +02:00
Andrew Bartlett
32de1f6aa4 autobuild: Use new selftest.pl feature to run only some environments
This is cleaner than test filtering with regular expressions

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-06-30 02:12:22 +02:00
Andrew Bartlett
9f7ed6b82c autobuild: Do not require cmocka to be installed for samba-libs to build
cmocka does not have an install target, but is a depencency of ldb for the tests

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Apr 19 19:43:15 CEST 2017 on sn-devel-144
2017-04-19 19:43:15 +02:00
Andreas Schneider
a46566ea5e lib: Add pam_wrapper 1.0.3
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-04-07 10:32:13 +02:00
Stefan Metzmacher
a26ba9d806 script/compare_cc_results.py: ignore all LIB*_WRAPPER_SO_PATH values
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-07 10:32:12 +02:00
Martin Schwenke
05b5af4ae5 autobuild: Stop waf uninstall from removing test_tmpdir
Most of the autobuild tasks run "make distcheck", which does a
recursive "waf configure make install uninstall".  "waf uninstall"
(via BuildContext.install() in Build.py) removes empty directories all
the way up the directory tree.  This means that it removes
test_tmpdir, if it is empty, and any empty directories above it.

While this is arguably a waf bug, the simplest solution is to make
test_tmpdir non-empty so it don't get removed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12703

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Mar 21 10:37:08 CET 2017 on sn-devel-144
2017-03-21 10:37:08 +01:00
Andrew Bartlett
4455547204 autobuild: Add nopython environment to test --disable-python builds (but without tests)
This ensures we keep this option building as we extend our use of python.

The rule is that new features and changes to existing features that
require python are most welcome, they just need to be disabled for the
minimalistic targets we still ecourage Samba on, that typically just
want smbd

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:13 +01:00
Douglas Bagnall
132cbad9ce scripts/traffic_summary: documentation typo
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-10 07:31:10 +01:00
Gary Lockyer
b12562fac0 script: Add test script for traffic_summary.pl
Add the test script for traffic_summary.pl, test data in previous
commit.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Mar  1 09:01:07 CET 2017 on sn-devel-144
2017-03-01 09:01:07 +01:00
Gary Lockyer
2b62cafeac script: Add script to provide an anonymous summary from tshark
The tshark command needs to output a PDML XML stream, which this command will
read. The summary is intended not to expose private or customer data while
allowing a good view on the range and frequency of the network traffic.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-01 05:09:21 +01:00
Gary Lockyer
0dc54a42e7 script: Add test data for traffic_summary.pl
This network capture summary tool will be added in the next commit

This sample is taken from make test under SOCKET_WRAPPER_PCAP_FILE

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-03-01 05:09:21 +01:00
Stefan Metzmacher
cad23629ac script/autobuild.py: ignore missing test_tmpdir
It is still unknown what removes it...

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-02-21 20:17:28 +01:00
Stefan Metzmacher
278c921263 script/autobuild.py: try to make TMPDIR handling more verbose
This hopefully gives some hints regarding flakey tests where
the tmpdir is not available.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-01-12 15:35:14 +01:00
Stefan Metzmacher
96277a9f82 script/autobuild.py: add a do_print() wrapper function that flushes after each message
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-01-12 15:35:14 +01:00
Stefan Metzmacher
5a8d7a5446 script/autobuild.py: export PYTHONUNBUFFERED=1
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-01-12 15:35:13 +01:00
Stefan Metzmacher
f9e1887477 script/autobuild.py: cleanup the task subdirs when they're done.
This hopefully reduces the used space on the memdisk.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-01-12 15:35:13 +01:00
Stefan Metzmacher
14d65fbc77 script/autobuild.py: use --enable-developer and --picky-developer for the ctdb build
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2017-01-12 15:35:13 +01:00
Stefan Metzmacher
7870c645b7 script/release.sh: fix off by 1 error in announce.${tagname}.mail.txt creation
Pair-Programmed-With: Karolin Seeger <kseeger@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Karolin Seeger <kseeger@samba.org>

Autobuild-User(master): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(master): Thu Jan 12 15:34:25 CET 2017 on sn-devel-144
2017-01-12 15:34:25 +01:00
Stefan Metzmacher
e6a5e6a01a script/autobuild.py: try make test TESTS=samba3.*ktest for samba-systemkrb5
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-01-10 13:54:17 +01:00
Stefan Metzmacher
60a8e70fe1 script/autobuild.py: create tmpdir for each try and export it as TMPDIR
This way the compiler and other tools hopefully don't use /tmp
anymore.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2016-12-17 19:16:14 +01:00
Stefan Metzmacher
49de2f3a79 script/autobuild.py: cleanup testbase/prefix before each retry
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2016-12-17 19:16:14 +01:00
Stefan Metzmacher
b8c63307bd script/autobuild.py: remove pointless mkdir/rmdir commands
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2016-12-17 19:16:14 +01:00
Stefan Metzmacher
6f0ffd767c script/autobuild.py: don't add subdirs of testbase to cleanup_list
We already have testbase in there.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2016-12-17 19:16:14 +01:00
Günther Deschner
9e98ac05c2 autobuild: add system-mitkrb5 build environment.
We finally need to make sure the build with MIT Kerberos does not break all the
time.

Guenther

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Guenther Deschner <gd@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
2016-09-29 18:30:18 +02:00
Stefan Metzmacher
e0ef054fa9 script/release.sh: use 8 byte gpg key ids
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Sep  2 22:05:33 CEST 2016 on sn-devel-144
2016-09-02 22:05:32 +02:00
Douglas Bagnall
af7948495f autobuild: disable abi check on O3 build
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2016-08-31 07:09:26 +02:00
Douglas Bagnall
db85692e4c autobuild --restrict-tests to test a restricted set
This allows autobuild to be used as a test framework for running
particular Samba tests in a cloud environment.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2016-08-31 07:09:26 +02:00
Douglas Bagnall
25efdf330b autobuild: remove unused argument
The "tasklist" parameter is the same as the global "tasks" variable,
but only the latter is used.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2016-08-31 07:09:26 +02:00
Stefan Metzmacher
98d289d2e5 script/autobuild.py: include the branch name in the output
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Thu Aug 11 08:38:47 CEST 2016 on sn-devel-144
2016-08-11 08:38:47 +02:00
Stefan Metzmacher
4370870957 script/autobuild.py: check for AUTOBUILD_SKIP_SAMBA_O3 environment variable
We need to skip the samba-o3 target on older systems like sn-devel-104.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12108

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2016-08-04 18:26:06 +02:00
Michael Adam
b32df94527 autobuild: Don't compare socket wrapper so_path for xc check
This uses the build-directory which, hence is not the same.
Achieve this by adding the path itself and the whole
defines dictionary to the exclude list.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-07-11 12:19:13 +02:00
Michael Adam
2317800488 autobuild: run the samba-o3 target by default
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2016-05-13 10:16:17 +02:00
Michael Adam
cc9d380271 autobuild: add a target samba-o3 that is built with -O3
Only run quicktest against the ad_dc env.
This currently just takes some 6 odd minutes.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2016-05-13 10:16:17 +02:00
Garming Sam
2fe2e662db autobuild: fix typo in autobuild success subject line
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Thu May  5 04:15:16 CEST 2016 on sn-devel-144
2016-05-05 04:15:16 +02:00
Garming Sam
3751ffbbe7 autobuild: Return the last 50 log lines
This means that you don't have to deal with tars for quickly determining
the cause of a failure.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2016-05-03 08:10:10 +02:00
Stefan Metzmacher
0234e3ac93 Revert "autobuild: Return the last 50 log lines"
This reverts commit aab2f39bb7.

This breaks if log_base is an https url...

Traceback (most recent call last):
  File "script/autobuild.py", line 744, in <module>
    elapsed_time, log_base=options.log_base)
  File "script/autobuild.py", line 608, in email_failure
    f = open("%s/%s.stdout" % (log_base, failed_tag), 'r')
IOError: [Errno 2] No such file or directory:
'https://git.samba.org/uri/samba-autobuild/ctdb.stdout'

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Mar 23 18:59:17 CET 2016 on sn-devel-144
2016-03-23 18:59:17 +01:00
Garming Sam
aab2f39bb7 autobuild: Return the last 50 log lines
This means that you don't have to deal with tars for quickly determining
the cause of a failure.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Mar 22 11:39:38 CET 2016 on sn-devel-144
2016-03-22 11:39:38 +01:00
Karolin Seeger
19fdc7fd57 script/release.sh: generate announce.${tagname}.patch.txt in announcement_samba_rc()
This was only done in announcement_samba_stable() before.

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Karolin Seeger <kseeger@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(master): Mon Feb 15 17:41:49 CET 2016 on sn-devel-144
2016-02-15 17:41:49 +01:00
Karolin Seeger
d87077afb4 script/release.sh: improve error messages if the tag verification fails
This makes it more obvious if a gpg key is expired.

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Karolin Seeger <kseeger@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
2016-02-15 14:30:13 +01:00
Stefan Metzmacher
6968ee4b3a script/autobuild.py: use --extra-python=/usr/bin/python3 by default
export AUTOBUILD_NO_EXTRA_PYTHON=1 can overwrite this if required.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Mon Feb 15 14:28:27 CET 2016 on sn-devel-144
2016-02-15 14:28:27 +01:00
Andrew Bartlett
05c34810a8 autobuild: Use make -j on samba-libs/samba-static build as well
We need this build to be fast when split out as a seperate VM

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2016-01-05 09:10:24 +01:00
Andrew Bartlett
af89d18c14 autobuild: Put the static build in another stanza
This will allow it to run on a different VM in travis-ci and so keep
us below the 50min limit.

Because we do not rebuild the libs (and real world static builds are
likely to have internal libs), this changes us to build them
internally.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2016-01-05 09:10:24 +01:00
Andrew Bartlett
cce16123fe autobuild: Use cp --recursive --link --archive
This works on waf based builds as source files are not modified during the build

TODO: In order to make sure build doesn't influence each other,
we need to add something like:

         try:
             if options.rebase is not None:
                 rebase_tree(options.rebase, rebase_branch=options.branch)
+            run_cmd("find -type f | xargs chmod -w", show=True, dir=test_master)
         except Exception:

But that means we need to change the way 'make distcheck' works for the
standalone libraries.

For now this will help to reduce the (mem)disk usage of an autobuild a lot.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2016-01-05 09:10:24 +01:00
Andrew Bartlett
9a91fce2de autobuild: Give a clearer failure message
This helps when autobuild.py is used in --tail mode and
where there is neither e-mail nor access to the logs.tar.gz

Working back to find where the error happened is typically
quite difficult, as many failures are actually due to the
cleanup.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2016-01-05 09:10:24 +01:00
Stefan Metzmacher
db04752f3e script/autobuild.py: exclude !pytalloc-util and !pyldb-util from bundling in samba-libs
We need to make sure we're really not bundling any of these.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11458

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jelmer Vernooij <jelmer@samba.org>
2015-10-31 22:03:14 +01:00
Andrew Bartlett
3e6af7109e autobuild: Confirm we can build without --enable-developer
We also confirm that such builds do not contain the NTVFS file server

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2015-10-23 22:27:30 +02:00
Stefan Metzmacher
f99d0b9174 script/release.sh: make it possible to create stable .x releases (x >= 1)
This version was used to create samba-4.3.1.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Karolin Seeger <kseeger@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Oct 21 14:27:53 CEST 2015 on sn-devel-104
2015-10-21 14:27:53 +02:00
Douglas Bagnall
800437c1b1 autobuild: add some system information to the autobuild tarball
When running multiple autobuilds on VMs with various parameters, you
can easily get confused about which was which, and the tarball doesn't
help much. This adds an extra file with information about the system.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2015-10-21 11:27:09 +02:00
Stefan Metzmacher
19a0a7f33c script/release.sh: make it possible to create stable .0 releases
This version was used to create samba-4.3.0.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Karolin Seeger <kseeger@samba.org>
2015-10-20 11:51:16 +02:00
Stefan Metzmacher
ba4c9bd08c script/autobuild.py: make sure --nonshared-binary=smbtorture,smbd/smbd keeps working
- It's very useful to have a static smbtorture binary that can be copied arround.
- It's sometimes also useful to have a static smbd binary in order avoid
  runtime overhead via do_lookup_x() (in ld*.so), note that

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Thu Aug 20 19:10:19 CEST 2015 on sn-devel-104
2015-08-20 19:10:18 +02:00
Stefan Metzmacher
86fa1d9059 script/autobuild.py: test some --with-{static,shared}-modules combinations
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2015-08-20 16:06:21 +02:00
Stefan Metzmacher
3b1aa7a665 script/autobuild.py: use -Wmissing-prototypes and --picky-developer for samba-libs*
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2015-08-20 16:06:21 +02:00
Stefan Metzmacher
303032559d script/autobuild.py: split out a samba_libs_configure variable
The avoids too long lines.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2015-08-20 16:06:21 +02:00
Stefan Metzmacher
54cbecbe30 script/librelease.sh: this is replaced by script/release.sh now
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Aug  6 03:49:40 CEST 2015 on sn-devel-104
2015-08-06 03:49:40 +02:00
Stefan Metzmacher
71128e0a3c script/release.sh: This is a new script to do releases
This will replace script/librelease.sh and is more flexible
and powerful.

  Usage: release.sh <PRODUCT> <COMMAND>

  PRODUCT: ldb, talloc, tevent, tdb, samba-rc
  COMMAND: fullrelease, create, push, upload, announce

GNUPGHOME=/path/to/private/gpg script/librelease.sh tdb
becomes
GNUPGHOME=/path/to/private/gpg script/release.sh tdb fullrelease

GNUPGHOME=/path/to/private/gpg script/librelease.sh samba
becomes
GNUPGHOME=/path/to/private/gpg script/release.sh samba-rc create
or later
GNUPGHOME=/path/to/private/gpg script/release.sh samba-stable create

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2015-08-06 00:43:24 +02:00
Michael Adam
5deeba69d4 generate_param: add a means to generate param_table_gen.c from the docs
This is triggered by the new argument PARAMTABLE.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2015-07-31 01:55:32 +02:00
Michael Adam
223aad4f91 generate_param: teach missing types cmdlist, bytes, octal, ustring.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2015-07-31 01:55:32 +02:00
Michael Adam
ca861f2b73 generate_param: more uniform formatting of various type dicts
This allows for easier extension later on (patch context-wise).

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2015-07-31 01:55:32 +02:00