script: Add 'random-seed' option to traffic_replay

When using a traffic-model file to generate traffic, there is some
randomness in the actual packets that get generated. This means it's
hard to use the tool to detect an increase/decrease in Samba
performance - we don't know whether a decrease in packets sent is due
to a regression in the Samba codebase, or just due to the tool sending
different types of packets (i.e. ones that take longer to process).

This patch adds an option to seed the python random number generator.
This means that exactly the same traffic can be generated across
multiple test runs.

(Previously we were using the '--traffic-summary' option to avoid this
problem - we can generate a summary-file based on the model, and then
use the same summary file across multiple runs. However, this proved
impractical when you want to run multiple combinations of scale/rate
parameters, e.g. 21 x 8 different permutations just fills up disk space
with summary-files.)

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: William Brown <william@blackhats.net.au>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 16 13:53:26 CEST 2018 on sn-devel-144

docs-xml/manpages/traffic_replay.7.xml

@@ -30,6 +30,7 @@
 		<arg choice="opt">-B, --badpassword-frequency &lt;frequency&gt;</arg>
 		<arg choice="opt">--dns-rate &lt;rate&gt;</arg>
 		<arg choice="opt">-t, --timing-data &lt;file&gt;</arg>
+		<arg choice="opt">--random-seed &lt;seed&gt;</arg>
 		<arg choice="opt">-U, --username user</arg>
 		<arg choice="opt">--password &lt;password&gt;</arg>
 		<arg choice="opt">-W --workgroup &lt;workgroup&gt;</arg>
@@ -136,6 +137,16 @@
 	</para></listitem>
 	</varlistentry>
 
+	<varlistentry>
+	<term>random-seed</term>
+	<listitem><para>
+	A number to seed the random number generator with. When traffic is
+	generated from a model-file, use this option to keep the traffic
+	consistent across multiple test runs. This allows you to compare the
+	performance of Samba between different releases.
+	</para></listitem>
+	</varlistentry>
+
 	<varlistentry>
 	<term>Traffic Model Options</term>
 	<listitem><para>
@@ -187,7 +198,7 @@
 		<para>
 		For taking performance measurements over several test runs,
 		it's recommended to use this option and replay the traffic from
-		a traffic-summary file.
+		a traffic-summary file, or to use the --random-seed option.
 		</para></listitem>
 		</varlistentry>
 	</itemizedlist>
@@ -474,7 +485,8 @@
 	<para>The traffic generated is an approximation of what was seen in
 	the network capture. The traffic generation involves some randomness,
 	so running the same command multiple times may result in slightly
-	different traffic being generated.</para>
+	different traffic being generated (although you can avoid this, by
+	specifying the --random-seed option).</para>
 
 	<para>As well as changing how long the model runs for, you can also
 	change how many conversations get generated and how fast the traffic

script/traffic_replay

@@ -22,6 +22,7 @@ import os
 import optparse
 import tempfile
 import shutil
+import random
 
 sys.path.insert(0, "bin/python")
 
@@ -67,6 +68,8 @@ def main():
     parser.add_option('-c', '--clean-up',
                       action="store_true",
                       help='Clean up the generated groups and user accounts')
+    parser.add_option('--random-seed', type='int', default=0,
+                      help='Use to keep randomness consistent across multiple runs')
 
     model_group = optparse.OptionGroup(parser, 'Traffic Model Options',
                                        'These options alter the traffic '
@@ -155,6 +158,9 @@ def main():
                              " for the users created as part of this test"))
         sys.exit(1)
 
+    if opts.random_seed:
+        random.seed(opts.random_seed)
+
     lp = sambaopts.get_loadparm()
     creds = credopts.get_credentials(lp)
     creds.set_gensec_features(creds.get_gensec_features() | gensec.FEATURE_SEAL)