1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-23 17:34:34 +03:00
Commit Graph

460 Commits

Author SHA1 Message Date
Andrew Bartlett
9bdc5a6742 autobuild: Merge samba-simpleserver into samba-nt4
This saves CI resources (20mins build time) at the expense of making
the samba-nt4 job take 45min (5 min longer).

The new maximum job time is 1:15 by samba-o3 on the different hosts.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Mar 23 14:45:59 UTC 2020 on sn-devel-184
2020-03-23 14:45:59 +00:00
Andrew Bartlett
7a44bd1c42 autobuild: Merge the samba-ktest-heimdal and samba-fileserver jobs
This avoids a full compile of Samba just to test Kerberos with a system Heimdal
while still providing an environment to test other fileserver features.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-03-23 13:02:32 +00:00
Andrew Bartlett
607c9ab307 build: Require --without-ad-dc for --without-ads
Building an AD DC while setting --without-ads makes no sense and just wastes compile time on our build hosts.

To allow samba-nt4 to build --without-ad-dc we set rpc.spoolss.notify
(which is built on the NTVFS fileserver for the callbacks) to run in
the ad_member environment rather than nt4_dc and ad_dc.

This is also just more realistic in any case.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2020-03-23 13:02:31 +00:00
Andreas Schneider
54f26cfcf2 autobuild: Run the none env in the samba-o3 build
This includes tests which should make sure that certain code is not
optimized away, like memset_s().

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Mar  9 23:42:26 UTC 2020 on sn-devel-184
2020-03-09 23:42:26 +00:00
Volker Lendecke
8441471d5d autobuild: samba-ctdb does not need an AD DC
Speed up the build a bit

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Feb 19 11:28:56 UTC 2020 on sn-devel-184
2020-02-19 11:28:56 +00:00
Volker Lendecke
cfb8e4de8f autobuild: Split a long line
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2020-02-19 09:38:40 +00:00
Volker Lendecke
ebf67b6da7 autobuild: Run a simple test for clustered samba
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2020-02-19 09:38:40 +00:00
Volker Lendecke
602f27d6a2 autobuild: Avoid clusteredmember in normal environments
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2020-02-19 09:38:40 +00:00
Andrew Bartlett
cf4984947b build: Do not check if system perl modules should be bundled
We do not ship any perl modules in third_party at this time, so
this check is pointless and breaks the build for --bundled-libraries=ALL.

As reported by aaptel on https://gitlab.com/samba-team/samba/-/merge_requests/1104#note_281050331

This changes our autobuild script to cover this case in the
samba-static job.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2020-02-04 05:13:39 +00:00
Karolin Seeger
f699df32cd script/release.sh: Don't use quotations any longer.
Signed-off-by: Karolin Seeger <kseeger@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(master): Mon Feb  3 12:45:39 UTC 2020 on sn-devel-184
2020-02-03 12:45:39 +00:00
Stefan Metzmacher
648f94d203 script/release.sh: make it possible to run from a git worktree
.git is a regular file in that case.

Also check that script/release.sh is present as a relative path
to ensure we're called from the expected location.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
2020-01-21 19:29:38 +01:00
Andrew Bartlett
0be0c044b6 autobuild: extend autobuild with samba-fuzz job to build the fuzzers in AFL mode using oss-fuzz scripts
This helps ensure the build_samba.sh file keeps working and the fuzzers build
(because they are excluded from the main build).

This is not in the default autobuild because it uses too much
space on sn-devel (4GB).

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>
2019-12-11 02:55:32 +00:00
Andrew Bartlett
d349d344f8 autobuild.py: Avoid listing jobs twice
We use the tasks table instead, to avoid the issue shown in the previous commit.

Now we just have to keep .gitlab-ci.yml and the tasks table in sync.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>
2019-12-11 02:55:32 +00:00
Andrew Bartlett
df38d51104 autobuild.py: Add missing samba-simpleserver job
This was missed when the job was split out in f0e8dd1a08.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>
2019-12-11 02:55:32 +00:00
Ralph Boehme
24905ba82d docs-xml: remove explicit "constant"
The constant mark applied to types "string" and "ustring". The previous patches
in this patchset already markes all string options as either constant or
substituted, but it's still possible to add options or change existing ones to
be neither constant nor substituted.

In order to enforce strings to be either constant or substitued, remove the
explicit constant marker. Instead, any option that is not marked as substituted
is implicitly made constant.

This patch doesn't change behaviour and all generated files are the same before
and after this change.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-11-27 10:25:37 +00:00
Stefan Metzmacher
ef1f0e9ffe param: add FN_{GLOBAL,LOCAL}_SUBSTITUTED_STRING support
Pair-Programmed-With: Ralph Boehme <slow@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
2019-11-27 10:25:32 +00:00
Christof Schmitt
7965a204d6 autobuild: Add test for build with system heimdal library
The configure option --with-system-heimdalkrb5 requires --without-ad-dc.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14179

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-11-13 17:49:38 +00:00
Andreas Schneider
831796b2d7 gitlab-ci: Run 'samba' tests also with MIT Kerberos
This runs the tests of the ktest environment.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Nov  4 14:19:17 UTC 2019 on sn-devel-184
2019-11-04 14:19:17 +00:00
Ralph Boehme
6a9041ba43 autobuild: rename samba-ad-member-* to samba-admem-*
Needed to come around MAX pathlen limit for UNIX domain socket triggered by
the new samba-ad-member-mitkrb5 target added in samba-ad-member-mitkrb5.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Oct 22 14:53:17 UTC 2019 on sn-devel-184
2019-10-22 14:53:17 +00:00
Uri Simchoni
649d15bb96 autobuild: harden samba-xc test suite
Add more checks which directly test the behavior of
--cross-answers and --cross-execute.

Previous test tested things in a round-about way, checking
that running in all three modes (native, cross-execute,
cross-answers) yields the same result. It was vulnerable
to a degradation in which cross-compilation modes didn't
work at all and were running native tests, which is
what happened with the upgrade of waf.

The added tests check the following:
- That cross-excute with cross-answers sets the cross-answers file
- That the content of cross-answers file actually affects the build
  configuration
- That a missing line in cross-answers fails the build

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13846

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Uri Simchoni <uri@samba.org>
Autobuild-Date(master): Sun Oct 20 13:29:58 UTC 2019 on sn-devel-184
2019-10-20 13:29:58 +00:00
Andreas Schneider
46068d5f28 gitlab-ci: Run several AD tests with MIT KDC
This will avoid introducing regressions in either client or server code.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-10-12 16:18:39 +00:00
Ralph Boehme
f0e8dd1a08 selftest: split fileserver testenv into simpleserver and fileserver
The simpleserver testenv continues to be built with minimal
dependencies. fileserver otoh will be built with bells and whistles including
JSON which I need for using the env as target for Spotlight tests.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
2019-10-09 14:35:29 +00:00
Stefan Metzmacher
0df28b3d14 script/autobuild.py: remove explicit --picky-developer"
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2019-08-06 06:32:28 +00:00
Gary Lockyer
6d6fe49938 traffic replay: Store the instance id in the replay context
Store the traffic runner instance id in the replay context.  Will be
used in subsequent commits.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-24 02:24:27 +00:00
Douglas Bagnall
96148436b7 script/compare_cc_results: print usage on too few args
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-05 01:05:20 +00:00
Douglas Bagnall
5331f913ac script/bisect-test: fix for py3
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-07-02 04:21:36 +00:00
Joe Guo
8de56ddc92 script/autobuild.py: make code more pythonic
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:17 +00:00
Joe Guo
fb12a25261 script/autobuild.py: avoid nested try except block
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:17 +00:00
Joe Guo
ae90e5263b script/autobuild.py: rm redundant empty str in options
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:16 +00:00
Joe Guo
04a6967bc3 script/autobuild.py: define cwd for builder and avoid chdir
We can pass cwd in Popen, no need to chdir for each cmd.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:16 +00:00
Joe Guo
6fc3eb8926 script/autobuild.py: improve run_cmd with check_output
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:16 +00:00
Joe Guo
0daeb6046b script/autobuild.py: add helper functions to simplify make test cmd
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:16 +00:00
Joe Guo
0383747dcc script/autobuild.py: replace random-sleep.sh with python function
The random-sleep.sh script is tricky, and there is no need to rely on another
script just for random sleep.

Using the python function, the random number generation will happen when
load autobuild.py other than execute cmd, but shouldn't affect the
result.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:16 +00:00
Joe Guo
1b70d43ed1 script/autobuild.py: rm unused mime type text/plain in tasks
The mime type `text/plain` is repeated everywhere but not used any more.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:16 +00:00
Joe Guo
41d93a6da5 script/autobuild.py: fix tasks indent
Indent tasks at same level, make it easier to copy/move lines arround.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-06-20 23:40:16 +00:00
Volker Lendecke
7c9eb428d5 autobuild: Fix autobuild for python3
"keys()" in python3 returns an unmodifiable view object. Two lines
down we might want to modify it, which python3 does not allow.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2019-06-18 05:34:07 +00:00
Joe Guo
1b48d8a9ce script/autobuild.py: define LCOV_CMD and run it after each make test
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 08:19:17 +00:00
Joe Guo
c5ae200b56 script/autobuild.py: add ENABLE_COVERAGE placeholder in configure cmd option
The value will be "--enable-coverage" or "".

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 08:19:17 +00:00
Joe Guo
0cdd8bd6e1 script/autobuild.py: replace more placeholders in cmds for coverage
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 08:19:17 +00:00
Joe Guo
87ea55e436 script/autobuild.py: add --enable-coverage option
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 08:19:17 +00:00
Joe Guo
332f19c7f2 script/autobuild.py: mv optionparse to top
so the rest of the code can use the option values directly.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 08:19:17 +00:00
Joe Guo
9ea81c0e37 script/autobuild.py: mv find_git_root and gitroot to top
so the reset of the code can use gitroot directly.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 08:19:16 +00:00
Joe Guo
af0e0c544b script/autobuild.py: rename sdir to test_source_dir
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 08:19:16 +00:00
Douglas Bagnall
bd53819b28 script/attr_count_read: load and correlate all data
This changes script/attr_count_read to take the samba private directory
as an argument and load all the databases at once, printing them as
one big table. It isn't extremely clear what it all means, but it
*tries* to tell you.

With --plot, it will attempt to load matplotlib and plot the number of
requested attributes against the number returned, with colour
of each point indicating its relative frequency. It is a scatterplot
that wants to be a heatmap.

With --no-casefold, you can get an extra confusing table where,
for instance, something repeatedly asks for "attributeId" which is not
accounted for, while in a completely different row an unrequested
"attributeID" is found many times over.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May  1 06:46:36 UTC 2019 on sn-devel-184
2019-05-01 06:46:36 +00:00
Douglas Bagnall
60620273db dsdb/modules: a module to count attribute searches and results
The dsdb module stack can turn a simple search request into a
complicated tree of sub-queries that include attributes not originally
asked for and excluding those that were. The corresponding replies
might contain unrequested attributes or (for good reasons, according
to some module) hide requested ones. The entire stack is there to
meddle and that is what is does. Except *this* module. It just counts.
To understand dsdb performance it helps to have some idea what
requests and replies are flying too and fro. This module, when
inserted anywhere in the stack, counts the requests and replies
passing through and the attributes they contain. This data is stored
in on-disk tdbs in the private/debug directory.

The module is not loaded by default. To load it you need to patch the
source4/dsdb/samdb/ldb_modules/samba_dsdb.c and put "count_attrs"
somewhere in the module lists in the samba_dsdb_init() function. For
example, to examine the traffic between repl_meta_data and
group_audit_log, you would do something like this around line 316:

          "subtree_delete",
          "repl_meta_data",
  +       "count_attrs",
          "group_audit_log",
          "encrypted_secrets",

and recompile. Samba will then write to a number of tdb files in the
debug directory as requests and replies pass through. A simple script
is included to read these files. Doing this:

./script/attr_count_read st/ad_dc/private/debug/debug/attr_counts_not_found.tdb

will print a table showing how often various attritbutes were
requested but not found (from the point of view of the module).

A more sophisticated version of the script is coming in the next
commit, but this one is included first because in its simplicity it
documents the storage format reasonably well. The tdb keys are
attribute names, and the values are uint32_t in machine native order.

When the module is included in the stack there will be a very small
decrease in performance.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-01 05:32:25 +00:00
Joe Guo
2ee72cc615 traffic: load dns query from file and write stats to file
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May  1 01:10:42 UTC 2019 on sn-devel-184
2019-05-01 01:10:42 +00:00
Tim Beale
e387cf9288 traffic_replay: Avoid Exception if no packet rate is specified
traffic_replay would throw an exception if you didn't specify some sort
of packet rate. We can avoid this by using --scale-traffic=1.0 as the
default if nothing else was specified.

 script/traffic_replay model.txt $SERVER.$REALM --duration=10
   --fixed-password=blahblah12# -U$USERNAME%$PASSWORD
INFO 2019-04-10 01:03:01,809 pid:47755 script/traffic_replay #280: Using
the specified model file to generate conversations
Traceback (most recent call last):
  File "script/traffic_replay", line 438, in <module>
    main()
  File "script/traffic_replay", line 293, in main
    opts.conversation_persistence)
  File "bin/python/samba/emulate/traffic.py", line 1295, in
generate_conversation_sequences
    target_packets = int(packet_rate * duration)
TypeError: unsupported operand type(s) for *: 'NoneType' and 'float'

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2019-04-30 23:18:28 +00:00
Tim Beale
641d74cb26 traffic_replay: Assign users to groups by default
The traffic_replay script has a myriad of options, but by default when
it creates user accounts it does not assign these users to any groups
(you have to specify extra options to do that). This isn't really a fair
test of samba performance, because it's unlikely that real world setups
will have users that are in no groups (other than the default ones).

This patch changes the default behaviour so that it will assign the new
users to groups automatically, if no other group options were
specified.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2019-04-30 23:18:28 +00:00
Andreas Schneider
f1bf02c78a autobuild: Build also Samba AD with MIT Kerberos
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2019-04-25 10:47:16 +00:00
Lutz Justen
92f30f91e4 waf: build: Respect --disable-python for third_party modules
Skips installation of samba/third_party stuff into the python directory if
--disable-python is set.

Added test after install that confirms no python modules installed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13905

Signed-off-by: Lutz Justen <ljusten@google.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Apr 24 07:32:31 UTC 2019 on sn-devel-184
2019-04-24 07:32:31 +00:00