IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Adjust the existing autobuild jobs that currently include/exclude
test env fileserver to also use fileserver_smb1.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
This is a relatively new test environment that only uses very few tests
yet. Skip the env name rename dance for this env, but someone has to convert the
tests later on.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
These commands are just there as hints to debug possible problems.
In order to support autobuild.py on non-linux platforms we should
just ignore errors here.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Apr 2 07:36:07 UTC 2020 on sn-devel-184
On platforms like FreeBSD 12 cp doesn't accept the long options,
using the one letter options works there and keeps working on Linux
as well.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
.git is not always a directory, with 'git worktree' it's a file.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This saves CI resources (20mins build time) at the expense of making
the samba-nt4 job take 45min (5 min longer).
The new maximum job time is 1:15 by samba-o3 on the different hosts.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Mar 23 14:45:59 UTC 2020 on sn-devel-184
This avoids a full compile of Samba just to test Kerberos with a system Heimdal
while still providing an environment to test other fileserver features.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Building an AD DC while setting --without-ads makes no sense and just wastes compile time on our build hosts.
To allow samba-nt4 to build --without-ad-dc we set rpc.spoolss.notify
(which is built on the NTVFS fileserver for the callbacks) to run in
the ad_member environment rather than nt4_dc and ad_dc.
This is also just more realistic in any case.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
This includes tests which should make sure that certain code is not
optimized away, like memset_s().
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Mar 9 23:42:26 UTC 2020 on sn-devel-184
Speed up the build a bit
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Feb 19 11:28:56 UTC 2020 on sn-devel-184
We do not ship any perl modules in third_party at this time, so
this check is pointless and breaks the build for --bundled-libraries=ALL.
As reported by aaptel on https://gitlab.com/samba-team/samba/-/merge_requests/1104#note_281050331
This changes our autobuild script to cover this case in the
samba-static job.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
.git is a regular file in that case.
Also check that script/release.sh is present as a relative path
to ensure we're called from the expected location.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
This helps ensure the build_samba.sh file keeps working and the fuzzers build
(because they are excluded from the main build).
This is not in the default autobuild because it uses too much
space on sn-devel (4GB).
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>
We use the tasks table instead, to avoid the issue shown in the previous commit.
Now we just have to keep .gitlab-ci.yml and the tasks table in sync.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>
This was missed when the job was split out in f0e8dd1a08.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>
The constant mark applied to types "string" and "ustring". The previous patches
in this patchset already markes all string options as either constant or
substituted, but it's still possible to add options or change existing ones to
be neither constant nor substituted.
In order to enforce strings to be either constant or substitued, remove the
explicit constant marker. Instead, any option that is not marked as substituted
is implicitly made constant.
This patch doesn't change behaviour and all generated files are the same before
and after this change.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This runs the tests of the ktest environment.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Nov 4 14:19:17 UTC 2019 on sn-devel-184
Needed to come around MAX pathlen limit for UNIX domain socket triggered by
the new samba-ad-member-mitkrb5 target added in samba-ad-member-mitkrb5.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Oct 22 14:53:17 UTC 2019 on sn-devel-184
Add more checks which directly test the behavior of
--cross-answers and --cross-execute.
Previous test tested things in a round-about way, checking
that running in all three modes (native, cross-execute,
cross-answers) yields the same result. It was vulnerable
to a degradation in which cross-compilation modes didn't
work at all and were running native tests, which is
what happened with the upgrade of waf.
The added tests check the following:
- That cross-excute with cross-answers sets the cross-answers file
- That the content of cross-answers file actually affects the build
configuration
- That a missing line in cross-answers fails the build
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13846
Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Uri Simchoni <uri@samba.org>
Autobuild-Date(master): Sun Oct 20 13:29:58 UTC 2019 on sn-devel-184
This will avoid introducing regressions in either client or server code.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
The simpleserver testenv continues to be built with minimal
dependencies. fileserver otoh will be built with bells and whistles including
JSON which I need for using the env as target for Spotlight tests.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
Store the traffic runner instance id in the replay context. Will be
used in subsequent commits.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We can pass cwd in Popen, no need to chdir for each cmd.
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The random-sleep.sh script is tricky, and there is no need to rely on another
script just for random sleep.
Using the python function, the random number generation will happen when
load autobuild.py other than execute cmd, but shouldn't affect the
result.
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The mime type `text/plain` is repeated everywhere but not used any more.
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Indent tasks at same level, make it easier to copy/move lines arround.
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
"keys()" in python3 returns an unmodifiable view object. Two lines
down we might want to modify it, which python3 does not allow.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
so the rest of the code can use the option values directly.
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This changes script/attr_count_read to take the samba private directory
as an argument and load all the databases at once, printing them as
one big table. It isn't extremely clear what it all means, but it
*tries* to tell you.
With --plot, it will attempt to load matplotlib and plot the number of
requested attributes against the number returned, with colour
of each point indicating its relative frequency. It is a scatterplot
that wants to be a heatmap.
With --no-casefold, you can get an extra confusing table where,
for instance, something repeatedly asks for "attributeId" which is not
accounted for, while in a completely different row an unrequested
"attributeID" is found many times over.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 1 06:46:36 UTC 2019 on sn-devel-184
The dsdb module stack can turn a simple search request into a
complicated tree of sub-queries that include attributes not originally
asked for and excluding those that were. The corresponding replies
might contain unrequested attributes or (for good reasons, according
to some module) hide requested ones. The entire stack is there to
meddle and that is what is does. Except *this* module. It just counts.
To understand dsdb performance it helps to have some idea what
requests and replies are flying too and fro. This module, when
inserted anywhere in the stack, counts the requests and replies
passing through and the attributes they contain. This data is stored
in on-disk tdbs in the private/debug directory.
The module is not loaded by default. To load it you need to patch the
source4/dsdb/samdb/ldb_modules/samba_dsdb.c and put "count_attrs"
somewhere in the module lists in the samba_dsdb_init() function. For
example, to examine the traffic between repl_meta_data and
group_audit_log, you would do something like this around line 316:
"subtree_delete",
"repl_meta_data",
+ "count_attrs",
"group_audit_log",
"encrypted_secrets",
and recompile. Samba will then write to a number of tdb files in the
debug directory as requests and replies pass through. A simple script
is included to read these files. Doing this:
./script/attr_count_read st/ad_dc/private/debug/debug/attr_counts_not_found.tdb
will print a table showing how often various attritbutes were
requested but not found (from the point of view of the module).
A more sophisticated version of the script is coming in the next
commit, but this one is included first because in its simplicity it
documents the storage format reasonably well. The tdb keys are
attribute names, and the values are uint32_t in machine native order.
When the module is included in the stack there will be a very small
decrease in performance.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 1 01:10:42 UTC 2019 on sn-devel-184
traffic_replay would throw an exception if you didn't specify some sort
of packet rate. We can avoid this by using --scale-traffic=1.0 as the
default if nothing else was specified.
script/traffic_replay model.txt $SERVER.$REALM --duration=10
--fixed-password=blahblah12# -U$USERNAME%$PASSWORD
INFO 2019-04-10 01:03:01,809 pid:47755 script/traffic_replay #280: Using
the specified model file to generate conversations
Traceback (most recent call last):
File "script/traffic_replay", line 438, in <module>
main()
File "script/traffic_replay", line 293, in main
opts.conversation_persistence)
File "bin/python/samba/emulate/traffic.py", line 1295, in
generate_conversation_sequences
target_packets = int(packet_rate * duration)
TypeError: unsupported operand type(s) for *: 'NoneType' and 'float'
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
The traffic_replay script has a myriad of options, but by default when
it creates user accounts it does not assign these users to any groups
(you have to specify extra options to do that). This isn't really a fair
test of samba performance, because it's unlikely that real world setups
will have users that are in no groups (other than the default ones).
This patch changes the default behaviour so that it will assign the new
users to groups automatically, if no other group options were
specified.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Skips installation of samba/third_party stuff into the python directory if
--disable-python is set.
Added test after install that confirms no python modules installed.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13905
Signed-off-by: Lutz Justen <ljusten@google.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Apr 24 07:32:31 UTC 2019 on sn-devel-184
This is needed because the name of the autobuild job and
the name of the selftest env end up in the socket path
for ncalrpc sockets.
The challenge is that (for example)
/memdisk/autobuild/fl/b2424063/samba-schemaupgrade/bin/ab/schemaupgrade_pair_dc/ncalrpc/np/protected_storage
does not fit in a struct sockaddr_un.
Signed-off-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Apr 12 05:41:36 UTC 2019 on sn-devel-144
Schemaupgrade tests are particularly resource intensive and are causing
runners to hit their memory and CPU limits, so we need to split them
out.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
This will allow us to run really most tests in an isolated
autobuild/ci task later.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
This will allow us to run really most tests in an isolated
autobuild/ci task later.
This will apply to tests, which may not rely on the ntvfs backend, so
the ad_dc_default alias can point to another environment in future.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
This will allow us to run really slow tests in an isolated
autobuild/ci task later.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Using aliases it will be possible to split the large amount
of tests which use ad_dc_ntvfs into multiple autobuild/ci
tasks/jobs later.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Previously we were only checking samba compiled OK with
--disable-python, not that it actually ran.
The main problem is all the make test framework is based around
subunit/smbtorture, neither of which we seem to build with
disable-python. However, for just a simple sanity-check, we can just
bypass all the subunit-filter work and just call the Perl code directly.
This works OK as long as it's just simple shell script tests that we're
running, as we can check the script's exit code directly.
The main thing that we really want to test is that we can start up the
smbd testenv and connect to it (i.e. a simple smbclient test).
This patch adds a new 'make test-nopython' target. This disables the
subunit filtering, and runs a small test-list that was generated manually.
Note that currently this has the limitation that it doesn't support known
failures or flapping tests. However, just checking that smd starts up OK
is probably OK for now.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Feb 20 02:10:00 CET 2019 on sn-devel-144
This isn't used any more. It was only being set, never referenced.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
For Samba 4.11, the minimum python2 functionality we will support (for
now, at least - we may change our minds) is for the --disable-python
target, i.e. if you're excluding all the python functionality from
samba, then WAF should still support being built with python2.
The use case here is old unix platforms that want to use smbd, but don't
have python3 support.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We no longer build the python2 bindings, only python3. So we can get rid
of this variable now.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Now that we've dropped the {PY3_ONLY} variable, there's no need for
line-breaks in some of the 'TESTS=' values. We can tidy this up a bit.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This variable is no longer needed as all the tests run using python3
now.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
When we switched from python2 being the default to python3, we didn't
update this variable name. It's now handling the python2 case, but it's
a boolean flag named 'py3', which is rather confusing.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samba v4.11 will no longer support python2, so let's drop the autobuild
jobs. This will save some gitlab/sn-devel time and money, as it's less
work for CI to do.
Note that this highlights some previous inconsistencies:
- samba-none-env-py2 was being built for gitlab but not sn-devel.
- samba-nt4-py2 was being built for sn-devel but not gitlab
I've left samba-buildpy2-only for now, which will be addressed in a
subsequent patch.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Many AD tests currently use the "samba" target. Split out a new target
"samba-ad-dc-ntvfs" and have all tests that use the "ad_dc_ntvfs" env
use the new target. This should greatly speed up the runtime for the "samba"
target and avoid swapping.
This reduces the total CI time by ~ 55%, I got an autobuild and a gitlab
pipeline finished in just ~ 100 mins!
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Mon Feb 11 14:10:12 CET 2019 on sn-devel-144
The samba-ad-dc-2 job was reaching its limits with the number of
testenvs and what the resource-limited CI machines can handle.
Samba processes were getting swapped out of memory, causing CI runs
to fail.
This patch splits the backup/restore testenv targets into a separate
autobuild job: samba-ad-dc-backup.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Feb 5 12:23:31 CET 2019 on sn-devel-144
-S, --scale--traffic defaults to 1.0
when we switch to new option -T, both -T and -S are set, which raise an error:
script/traffic_replay #234: --scale-traffic and --packets-per-second are incompatible. Use one or the other.
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>
Autobuild-User(master): Tim Beale <timbeale@samba.org>
Autobuild-Date(master): Wed Jan 23 03:09:41 CET 2019 on sn-devel-144
The old -S/--scale-traffic is relative to the original model, which made
its relationship to true traffic volumes quite opaque
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
traffic_replay had a broken sense of traffic scale. That is fixed, but
in order to compare old and new tests, it helps to be able to
approximate the old behaviour.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The traffic model is generated from a window in time, which makes
conversations appear to start and stop unnaturally at the window
boundaries. When the window is short compared to the traffic replay
time and the true expected conversation length, this has a significant
distorting effect, leading to more conversations than would be
expected to generate a given number of packets.
To offset this slightly we add the --conversation-persistence option
which tries to convert apparent death into a longish wait.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
tracebacks and less nonsense at higher debug levels.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Conversations that haven't finished within some acceptable margin of
on-time can be said to have failed. This is where you specify that
margin.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Sometimes you want to know if any client is crashing for any reason.
In those times use --stop-on-any-error for an early exit.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Use less memory altogether and don't allocated shared mutable before
the fork.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Rather than building all the packets at this point, we stick to the
barest details of the packets (which is all the model gives us
anyway).
The advantage is that will take a lot less memory, which matters
because this process forks into many clients that were sharing and
mutate the conversation list.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
even if someone asked to keep the temporary data, they don't want
to see 5000 empty directries. Non-empty directories will remain.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
add_argument(type=argparse.FileType('w'), ...) will open the file
and leave it empty if the script fails.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
And use it in tests, rather than expecting exact strings.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We could not do this when we replayed traffic summaries (as opposed to
models), but now this script does not do that.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The traffic_replay script has been able to replay a replay log as well
as a model, which was not used in practice and complicated the script.
If we want that feature, we can make a new script for it.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The command output looks like b'foo\nbar' in string-space.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
sn-devel autobuild runing autobuild.py (via git hooks I suppose) but
if run directly (e.g. depending on script shebang) then 'python' aka
python2 will run. This will cause an error when building some targets
because the autobuild script itself sometimes builds paths based
on the version of python executing the script e.g ${PYTHON_PREFIX}.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Now that we are building with python3 by default we need to
convert the old python3 test tasks to python2 (e.g. reverse how
we used do it)
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The resuls of get_python_lib are different between python2 & python3
and this results in autobuild generating the wrong PYTHONPATH with
python3.
python2
=======
print ("%s" % get_python_lib(standard_lib=1, prefix='/my/prefix'))
/my/prefix/lib64/python2.7
python3
print ("%s" % get_python_lib(standard_lib=1, prefix='/my/prefix'))
/my/prefix/lib/python3.6
But with addition of plat_specific param the results are the same
python2
=======
print ("%s" % get_python_lib(plat_specific=1, standard_lib=0, prefix='/my/prefix'))
/my/prefix/lib64/python2.7/site-packages
python3
=======
print ("%s" % get_python_lib(plat_specific=1, standard_lib=0, prefix='/my/prefix'))
/my/prefix/lib64/python3.6/site-packages
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Since autobuild now builds python3 by default we need to change
the previously buildpy3 only job to python2
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
build_xc job uses compare_cc_results.py to compare cache
files, the cache files are stringified hash maps, the results
in python 3.4 don't compare well due to inconsistent order of
dict key/value pairs when the cache files are created. While
comparing the file contents works fine in python3.6 it fails
with python3.4. This patch detects problematic dict lines and
rewrites the value for comparison
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Ideally we want all the tests to run under python3 by default (no
special task for this) and then convert the existing '-py3' tasks
to run the python tests with python3.
However at the moment the convertion process is not ready to do this,
for a while we need to run separate autobuild tasks for this.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
traffic_replay tries to distribute the users among the groups in a
realistic manner - some groups will have almost all users in them.
However, this becomes a problem when testing a really large database,
e.g. we may want 100K users, but no more than 5K users in each group.
This patch adds a max-member option so we can limit how big the groups
actually get.
If we detect that a group exceeds the max-members, we reset the group's
probability (of getting selected) to zero, and then recalculate the
cumulative distribution. The means that the group should no longer get
selected by generate_random_membership(). (Note we can't completely
remove the group from the list because that changes the
list-index-to-group-ID mapping).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Dec 4 12:22:50 CET 2018 on sn-devel-144
Ideally we want all the tests to run under python3 by default (no
special task for this) and then convert the existing '-py3' tasks
to run the python tests with python3.
However at the moment the convertion process is not ready to do this,
for a while we need to run separate autobuild tasks for this.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
generate_users_and_groups() now generates the machine acounts as well as
the user accounts, so it seems there's no need to also have
generate_traffic_accounts(), which does the same job.
Instead, we can just pass through the number of machine acounts to
generate_users_and_groups() and delete the other function.
Also updated generate_users_and_groups() so that machine_accounts is
no longer optional (we want to create machine accounts in all cases).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We create machine accounts for 2 different purposes:
1). For traffic generation, i.e. testing realistic network packets.
2). For generating a realistic large DB.
Unfortunately, we want to use different userAccountControl flags for
the 2 different cases. Commit 3338a3e257 changed the flags used
for case #2, but this breaks case #1.
The problem is generate_users_and_groups() is called in both cases,
so we want the 'traffic account' flag passed into that function.
This ensures that the machine accounts get created with the appropriate
userAccountControl flags for the particular case you want to test.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
I was assuming that generate_users_and_groups() only gets called in the
--generate-users-only case. However, it also gets called in the default
traffic replay case.
This patch reworks the code so that the number of machine accounts to
create gets passed in, and the 'create 25% more computers than users'
assumption only applies to the --generate-users-only case.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We were trying to access the debug-level (in python C bindings) before
the smb.conf had been loaded and actually set the debug-level. So it
would default to zero, regardless of what was in the smb.conf.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
We can create user accounts much faster if the LDB connection talks
directly to the local sam.ldb file rather than going via LDAP. This
patch allows the 'host' argument to the tool to be a .ldb file (e.g.
"/usr/local/samba/private/sam.ldb") instead of a server name/IP.
In most cases, the traffic_replay tool wants to run on a remote device
(because the point of it is to send traffic to the DC). However, the
--generate-users-only is one case where the tool can be run locally,
directly on the test DC. (The traffic_replay user generation is handy
for standalone testing, because it also handles assigning group
memberships to the generated user accounts).
Note that you also need to use '--option="ldb:nosync = true"' to get
the improvement in performance.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
We now run a purepython3 none-env test, later when the whole
build is running under python3 we will resurrect this job
but as (samba-none-env-py2) for python2
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Oct 23 09:10:22 CEST 2018 on sn-devel-144
Ideally we want all the tests to run under python3 by default (no
special task for this) and then convert the existing '-py3' tasks
to run the python tests with python3.
However at the moment the convertion process is not ready to do this,
for a while we need to run separate autobuild tasks for this.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Use logger to replace print
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The -d option will set samba global debug level automatically.
We should not parse and use the passed in value.
Use samba.get_debug_level instead.
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This is debug info, should print to stderr.
Otherwise it will flood stdout.
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Fold the build option --with-json-audit into the toplevel wscript
to reflect the fact that JSON support is no longer local to the
audit subsystem.
Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Commit cb40e2bbc8 introduced a print
statement with a broken formatting. Reported by pylint.
Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Due to build variants, cfg_file paths are written as absolute paths.
Signed-off-by: Thomas Nagy <tnagy@waf.io>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This should mean one less process in the process tree, and less places to hold
FDs open.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13591
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Aug 29 08:20:55 CEST 2018 on sn-devel-144
This closes fds other than 0, 1, 2.
This ensures only the correct *.stderr and *.stdout is attached, via
the stdout/stderr parameter to Popen(), but not every other FD
currently open in python at the time Popen is called.
For the tail invocation and other calls to Popen(), because fds 0, 1,
2 are still attached, these function as before.
Per https://docs.python.org/2.6/library/subprocess.html:
"If close_fds is true, all file descriptors except 0, 1 and
2 will be closed before the child process is executed. (Unix only)."
And regarding the passed in parameters:
"stdin, stdout and stderr specify the executed programs’ standard
input,
standard output and standard error file handles, respectively. "
...
"With None (the default), no redirection will occur;
the child’s file handles will be inherited from the parent. "
(The unwanted inherited files would be on a random high FD, where the
program wouldn't know what to do with them, but counting towards the
process FD limit).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13591
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This tries to to split up the tasks more evenly and may help with the python3
work by isolating them from the long samba job.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This tries to to split up the tasks more evenly and may help with the python3 tests
against this environment if started from a more isolated job.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
The defaulttasks or builddirs are often updated out of sync, which causes confusion until
it is resolved.
We simply choose "." as the builddir for the tasks that
are not in the default set.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
The default tasks will run the tests without --extra-python specified and
the new -py3 tasks will run the python3 tests only.
This will reduce the complexity of the build combinations
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This will allow splitting up of the python2 and python3 tests without
duplication of this already complex file.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Aug 25 11:53:15 CEST 2018 on sn-devel-144
