1
0
mirror of https://github.com/samba-team/samba.git synced 2025-02-03 13:47:25 +03:00

115304 Commits

Author SHA1 Message Date
Douglas Bagnall
b2a9d4c1f6 xml_docs: update traffic script documentation
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Tim Beale <timbeale@samba.org>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Jan  9 03:53:58 CET 2019 on sn-devel-144
2019-01-09 03:53:58 +01:00
Douglas Bagnall
3c10cecac1 traffic_replay: use packets per second as primary scale
The old -S/--scale-traffic is relative to the original model, which made
its relationship to true traffic volumes quite opaque

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:35 +01:00
Douglas Bagnall
c672a92261 traffic: assert json equality in learner test
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:34 +01:00
Douglas Bagnall
daf039afcb traffic tests: use TestCaseInTempdir.mktemp()
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:34 +01:00
Douglas Bagnall
71cb2605e8 python/tests: add TestCaseInTempdir.mktemp()
This gives you a name of a temporary file within the test case's tempdir.
Use it like this:

  with self.mktemp() as filename:
     self.check_run("samba-tool foo --output %s" % filename)
     self.assertStringsEqual(open(filename).read(), expected)

and filename will flick out of existence when the with block ends.

This is based on an idea used in the traffic_runner tests, which will
soon be adapted to use this method.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:34 +01:00
Douglas Bagnall
affaeb944f traffic_replay: --old-scale to mimic the old traffic_replay
traffic_replay had a broken sense of traffic scale. That is fixed, but
in order to compare old and new tests, it helps to be able to
approximate the old behaviour.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:34 +01:00
Douglas Bagnall
524777e681 traffic: add option to reanimate dying conversations
The traffic model is generated from a window in time, which makes
conversations appear to start and stop unnaturally at the window
boundaries. When the window is short compared to the traffic replay
time and the true expected conversation length, this has a significant
distorting effect, leading to more conversations than would be
expected to generate a given number of packets.

To offset this slightly we add the --conversation-persistence option
which tries to convert apparent death into a longish wait.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:34 +01:00
Douglas Bagnall
bda7f35a5e traffic_model: don't report generation errors as parse errors
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:34 +01:00
Douglas Bagnall
9e0effc175 traffic: improve debug messages in traffic_replay
tracebacks and less nonsense at higher debug levels.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:34 +01:00
Douglas Bagnall
b5d493f927 traffic: Conversation.add_short_packet is discerning about packets
If the packets really wouldn't do anything, we might as well not add them.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:34 +01:00
Douglas Bagnall
db51004307 traffic: remove useless branch in stats report
This completes the work of 68c64c634ad7576b2ab5ef838c203320afd90f44,
but differs from that in that it makes no actual change because isatty
was not being called so was always evaluated as true.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:34 +01:00
Douglas Bagnall
d727dcc82a traffic: generate more statistics in replay
Add more "header" values indicating the progress of the run as a
whole.

The new fields are:

Max sleep miss            - the longest sleep() oversleep. Indicates client load.
Maximum lag               - the longest gap between a planned packet
                            time and its actual time.
Start lag                 - the longest gap between intended and actual
                            conversation start.
Planned conversations     - how many conversations we meant to have.
Planned packets           - how many "packets" we thought we were making. Not
                            all "packets" result in actual operations or packets.
Unfinished conversations  - how many conversations had not finished
                            when they were killed.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:34 +01:00
Douglas Bagnall
31c0809e4f traffic: avoid bare except: clauses
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:34 +01:00
Douglas Bagnall
41b0cfce2c traffic: simplify tests for badpassword_frequency
x <= 0 will fail one or both of the other test clauses.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:34 +01:00
Douglas Bagnall
65c02fd68c traffic: assign context domain at start
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:33 +01:00
Douglas Bagnall
17579dd810 traffic: traffic_replay --latency-timeout to control final wait
Conversations that haven't finished within some acceptable margin of
on-time can be said to have failed. This is where you specify that
margin.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:33 +01:00
Douglas Bagnall
b737552ed3 traffic_replay: --stop-on-any-error option to not ignore client trouble
Sometimes you want to know if any client is crashing for any reason.
In those times use --stop-on-any-error for an early exit.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:33 +01:00
Douglas Bagnall
22cba7011c traffic: evoke OS error codes by name, not number
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:33 +01:00
Douglas Bagnall
35bc6ee0a1 traffic: reinitialise RNG in each child
Otherwise they all replay using the same random sequence.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:33 +01:00
Douglas Bagnall
6268effb11 traffic: avoid generating conversations without packets
Some "packets" don't generate any actual traffic. If we have a
conversation consisting only of those, we can avoid forking a client
for it.

This *slightly* increases the load over that which would be generated
otherwise for a given traffic rate, but that's OK.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:33 +01:00
Douglas Bagnall
c4d5bb5952 traffic: rework conversation generation to better use memory
Use less memory altogether and don't allocated shared mutable before
the fork.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:33 +01:00
Douglas Bagnall
7b03e81c61 traffic: generate sparser descriptions of conversations
Rather than building all the packets at this point, we stick to the
barest details of the packets (which is all the model gives us
anyway).

The advantage is that will take a lot less memory, which matters
because this process forks into many clients that were sharing and
mutate the conversation list.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:33 +01:00
Douglas Bagnall
7edf58dc58 traffic: new version of model with packet_rate, version number
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:33 +01:00
Douglas Bagnall
273eb3dffb traffic: fix hash non-determinism when loading JSON
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Tim Beale <timbeale@samba.org>
2019-01-08 23:55:33 +01:00
Douglas Bagnall
ef747953d4 traffic: Conversation gets given id, list of short packets
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:33 +01:00
Douglas Bagnall
a430b11ca2 traffic: rename packet_rate -> replay_speed for accuracy and room
We are soon going to have a self.packet_rate, and replay_speed is more
accurate in this case.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:32 +01:00
Douglas Bagnall
a352060f97 traffic: initialise conversation specific lists after the fork
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:32 +01:00
Douglas Bagnall
e866782a15 traffic: split is_a_real_packet() function out of class
So we can use it to determine whether a packet should be a Packet before
making the leap.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:32 +01:00
Douglas Bagnall
24d09d1df8 traffic: give Packet __slots__ for reduced memory
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:32 +01:00
Douglas Bagnall
4f7ae5d925 traffic: make random_colour_print more deterministic, silent on demand
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:32 +01:00
Douglas Bagnall
b77ee628cc traffic: delete empty temp directories
even if someone asked to keep the temporary data, they don't want
to see 5000 empty directries. Non-empty directories will remain.


Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:32 +01:00
Douglas Bagnall
158f172845 traffic: use namedtuple for ConversationAccounts namespace class
Saving memory, which reduces fork overhead.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:32 +01:00
Douglas Bagnall
9da79b54fa traffic: fix mk_masked_dir doc and comments
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:32 +01:00
Douglas Bagnall
709d2c84cb traffic: Packet.from_line classmethod uses cls var
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:32 +01:00
Douglas Bagnall
d1a1c5d601 traffic learner: avoid truncated output files on error
add_argument(type=argparse.FileType('w'), ...) will open the file
and leave it empty if the script fails.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:32 +01:00
Douglas Bagnall
b275baebd7 traffic_learner: use samba.logger, not print(file=sys.stderr)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:32 +01:00
Douglas Bagnall
3b3b4fa6f9 traffic_learner: return an error code
And use it in tests, rather than expecting exact strings.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:32 +01:00
Douglas Bagnall
f6b91d5b21 traffic: use default value for --duration
We could not do this when we replayed traffic summaries (as opposed to
models), but now this script does not do that.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:31 +01:00
Douglas Bagnall
79e8459817 traffic_replay: drop summary replay
The traffic_replay script has been able to replay a replay log as well
as a model, which was not used in practice and complicated the script.

If we want that feature, we can make a new script for it.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:31 +01:00
Douglas Bagnall
4c46333499 traffic replay: allow --random-seed=0
Zero is a valid seed.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:31 +01:00
Douglas Bagnall
8760168aa8 python/traffic: remove duplicate test file
The other one is python/samba/tests/blackbox/testdata/traffic-sample-very-short.model

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:31 +01:00
Douglas Bagnall
c57f3794e2 python/tests/blackbox: check_exit_code() explains by default
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-01-08 23:55:31 +01:00
Volker Lendecke
ee4a33a4ff idmap: In _wbint_Sids2UnixIDs, pass back what we have
SOME_UNMAPPED does not mean that nothing worthwhile is in here. We
need to pass what we have.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Jan  8 13:15:35 CET 2019 on sn-devel-144
2019-01-08 13:15:35 +01:00
Volker Lendecke
6f2ccb5973 idmap_tdb: If one SID fails to map, try the rest
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-01-08 09:32:10 +01:00
Volker Lendecke
0e2e635205 selftest: Test sids-to-xids with one failing sid
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
2019-01-08 09:32:10 +01:00
Björn Jacke
f23287bcb6 waf: fix tri-state of --with-sendfile-support being auto the default
--with-sendfile-support should be used automatically if it's found.

Configure now also fails reliably for all platforms when sendfile was
explicitly requested but failed to be detected.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=9707

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Jan  8 07:03:49 CET 2019 on sn-devel-144
2019-01-08 07:03:49 +01:00
Björn Jacke
99c030bce6 waf: fix waf errors with sendfail detection on Solaris
Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-01-08 03:40:29 +01:00
Justin Stephenson
a2402f9714 s3: net: Do not set NET_FLAGS_ANONYMOUS with -k
This affects net rpc getsid and net rpc changetrustpw commands.
This avoids an anonymous IPC connection being made when -k is used,
this only affects net rpc getsid and net rpc changetrustpw commands.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13726

Signed-off-by: Justin Stephenson <jstephen@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2019-01-08 03:40:28 +01:00
Volker Lendecke
b06f0912fc libnet4: Use dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-01-08 03:40:28 +01:00
Volker Lendecke
f1e71a2c1e winbind4: Use dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-01-08 03:40:28 +01:00