mirror of
https://github.com/samba-team/samba.git
synced 2025-01-11 05:18:09 +03:00
c0fd6cd386
When using a traffic-model file to generate traffic, there is some randomness in the actual packets that get generated. This means it's hard to use the tool to detect an increase/decrease in Samba performance - we don't know whether a decrease in packets sent is due to a regression in the Samba codebase, or just due to the tool sending different types of packets (i.e. ones that take longer to process). This patch adds an option to seed the python random number generator. This means that exactly the same traffic can be generated across multiple test runs. (Previously we were using the '--traffic-summary' option to avoid this problem - we can generate a summary-file based on the model, and then use the same summary file across multiple runs. However, this proved impractical when you want to run multiple combinations of scale/rate parameters, e.g. 21 x 8 different permutations just fills up disk space with summary-files.) Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Alexander Bokovoy <ab@samba.org> Reviewed-by: William Brown <william@blackhats.net.au> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Wed May 16 13:53:26 CEST 2018 on sn-devel-144
579 lines
20 KiB
XML
579 lines
20 KiB
XML
<?xml version="1.0" encoding="iso-8859-1"?>
|
|
<!DOCTYPE refentry PUBLIC "-//Samba-Team//DTD DocBook V4.2-Based Variant V1.0//EN" "http://www.samba.org/samba/DTD/samba-doc">
|
|
<refentry id="traffic_replay.7">
|
|
|
|
<refmeta>
|
|
<refentrytitle>traffic_replay</refentrytitle>
|
|
<manvolnum>7</manvolnum>
|
|
<refmiscinfo class="source">Samba</refmiscinfo>
|
|
<refmiscinfo class="manual">User Commands</refmiscinfo>
|
|
<refmiscinfo class="version">&doc.version;</refmiscinfo>
|
|
</refmeta>
|
|
|
|
|
|
<refnamediv>
|
|
<refname>traffic_replay</refname>
|
|
<refpurpose>Samba traffic generation tool.
|
|
</refpurpose>
|
|
</refnamediv>
|
|
|
|
<refsynopsisdiv>
|
|
<cmdsynopsis>
|
|
<command>traffic_replay</command>
|
|
<arg choice="opt">-F, --fixed-password <test-password></arg>
|
|
<arg choice="opt">-S, --scale-traffic <scale by factor></arg>
|
|
<arg choice="opt">-r, --replay-rate <scale by factor></arg>
|
|
<arg choice="opt">-D, --duration <seconds></arg>
|
|
<arg choice="opt">--traffic-summary <output file></arg>
|
|
<arg choice="opt">-I, --instance-id <id></arg>
|
|
<arg choice="opt">-K, --prefer-kerberos</arg>
|
|
<arg choice="opt">-B, --badpassword-frequency <frequency></arg>
|
|
<arg choice="opt">--dns-rate <rate></arg>
|
|
<arg choice="opt">-t, --timing-data <file></arg>
|
|
<arg choice="opt">--random-seed <seed></arg>
|
|
<arg choice="opt">-U, --username user</arg>
|
|
<arg choice="opt">--password <password></arg>
|
|
<arg choice="opt">-W --workgroup <workgroup></arg>
|
|
<arg choice="opt">--realm <realm></arg>
|
|
<arg choice="opt">-s, --config-file <file></arg>
|
|
<arg choice="opt">-k, --kerberos <kerberos></arg>
|
|
<arg choice="opt">--ipaddress <address></arg>
|
|
<arg choice="opt">-P, --machine-pass</arg>
|
|
<arg choice="opt">--option <option></arg>
|
|
<arg choice="opt">-d, --debuglevel <debug level></arg>
|
|
<arg choice="req">summary-file</arg>
|
|
<arg choice="req">dns-hostname</arg>
|
|
</cmdsynopsis>
|
|
|
|
<cmdsynopsis>
|
|
<command>traffic_replay</command>
|
|
<arg choice="opt">-G, --generate-users-only</arg>
|
|
<arg choice="opt">-F, --fixed-password <test-password></arg>
|
|
<arg choice="opt">-n, --number-of-users <total users></arg>
|
|
<arg choice="opt">--number-of-groups <total groups></arg>
|
|
<arg choice="opt">--average-groups-per-user <average number></arg>
|
|
<arg choice="opt">--group-memberships <total memberships></arg>
|
|
<arg choice="req">dns-hostname</arg>
|
|
</cmdsynopsis>
|
|
|
|
<cmdsynopsis>
|
|
<command>traffic_replay</command>
|
|
<arg choice="req">-c|--clean-up</arg>
|
|
<arg choice="req">dns-hostname</arg>
|
|
</cmdsynopsis>
|
|
|
|
<cmdsynopsis>
|
|
<command>traffic_replay</command>
|
|
<arg choice="opt">-h, --help</arg>
|
|
<arg choice="opt">-V, --version</arg>
|
|
</cmdsynopsis>
|
|
</refsynopsisdiv>
|
|
|
|
<refsect1>
|
|
<title>DESCRIPTION</title>
|
|
<para>This tool is part of the <citerefentry><refentrytitle>samba</refentrytitle>
|
|
<manvolnum>7</manvolnum></citerefentry> suite.</para>
|
|
<para>This tool generates traffic in order to measure the performance
|
|
of a Samba DC, and to test how well Samba will scale as a network
|
|
increases in size. It can simulate multiple different hosts making
|
|
multiple different types of requests to a DC.</para>
|
|
|
|
<para>This tool is intended to run against a dedicated test DC (rather
|
|
than a live DC that is handling real network traffic).</para>
|
|
|
|
<para>Note that a side-effect of running this tool is that user
|
|
accounts will be created on the DC, in order to test various Samba
|
|
operations. As creating accounts can be very time-consuming, these
|
|
users will remain on the DC by default. To remove these accounts, use
|
|
the --clean-up option.
|
|
</para>
|
|
</refsect1>
|
|
|
|
<refsect1>
|
|
<title>OPTIONS</title>
|
|
|
|
<variablelist>
|
|
|
|
<varlistentry>
|
|
<term>-h|--help</term>
|
|
<listitem><para>
|
|
Print a summary of command line options.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>summary-file</term>
|
|
<listitem><para>
|
|
File containing the network traffic to replay. This should either be
|
|
a traffic-summary (generated by <command>traffic_summary.pl</command>)
|
|
or a traffic-model (generated by <command>traffic_learner</command>).
|
|
Based on this file, this tool will generate 'conversations' which
|
|
represent Samba activity between a network host and the DC.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>dns-hostname</term>
|
|
<listitem><para>
|
|
The full DNS hostname of the DC that's being tested. The Samba activity
|
|
in the summary-file will be replicated and directed at this DC. It's
|
|
recommended that you use a dedicated DC for testing and don't try to run
|
|
this tool against a DC that's processing live network traffic.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>-F|--fixed-password <test-password></term>
|
|
<listitem><para>
|
|
Test users are created when this tool is run, so that actual Samba
|
|
activity, such as authorizing users, can be mimicked. This option
|
|
specifies the password that will be used for any test users that are
|
|
created.</para>
|
|
|
|
<para>Note that any users created by this tool will remain on the DC
|
|
until you run the --clean-up option. Therefore, the fixed-password
|
|
option needs to be the same each time the tool is run, otherwise the
|
|
test users won't authenticate correctly.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>random-seed</term>
|
|
<listitem><para>
|
|
A number to seed the random number generator with. When traffic is
|
|
generated from a model-file, use this option to keep the traffic
|
|
consistent across multiple test runs. This allows you to compare the
|
|
performance of Samba between different releases.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Traffic Model Options</term>
|
|
<listitem><para>
|
|
When the summary-file is a traffic-model (produced by
|
|
<command>traffic_learner</command>), use these options to alter the
|
|
traffic that gets generated.</para>
|
|
<itemizedlist>
|
|
<varlistentry>
|
|
<term>-D|--duration <seconds></term>
|
|
<listitem><para>
|
|
Specifies the approximate duration in seconds to generate
|
|
traffic for. The default is 60 seconds.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>-r|--replay-rate <factor></term>
|
|
<listitem><para>
|
|
Replays the traffic faster by this factor. This option won't
|
|
affect the number of conversations (which is based on the
|
|
traffic model), but the rate at which the packets are sent will
|
|
be increased.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>-S|--scale-traffic <factor></term>
|
|
<listitem><para>
|
|
Increases the number of conversations by this factor. This
|
|
option won't affect the rate at which packets get sent (which
|
|
is still based on the traffic model), but it will mean more
|
|
conversations get replayed.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>--traffic-summary <output-file></term>
|
|
<listitem><para>
|
|
Instead of replaying a traffic-model, this option generates a
|
|
traffic-summary file based on what traffic would be sent. Using
|
|
a traffic-model allows you to scale the packet rate and number
|
|
of packets sent. However, using a traffic-model introduces
|
|
some randomness into the traffic generation. So running the
|
|
same traffic_replay command multiple times using a model file
|
|
may result in some differences in the actual traffic sent.
|
|
However, running the same traffic_replay command multiple times
|
|
with a traffic-summary file will always result in the same
|
|
traffic being sent. </para>
|
|
<para>
|
|
For taking performance measurements over several test runs,
|
|
it's recommended to use this option and replay the traffic from
|
|
a traffic-summary file, or to use the --random-seed option.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
</itemizedlist>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>--generate-users-only</term>
|
|
<listitem><para>Add extra user/groups on the DC to increase the DB
|
|
size. By default, this tool automatically creates test users that map
|
|
to the traffic conversations being generated. This option allows extra
|
|
users to be created on top of this. Note that these extra users may
|
|
not actually used for traffic generation - the traffic generation is
|
|
still based on the number of conversations from the model/summary file.
|
|
</para>
|
|
|
|
<para>
|
|
Generating a large number of users can take a long time, so it this
|
|
option allows this to be done only once.</para>
|
|
|
|
<para>Note that the users created will remain on the DC until the
|
|
tool is run with the --clean-up option. This means that it is best to
|
|
only assign group memberships once, i.e. run --clean-up before
|
|
assigning a different allocation of group memberships.</para>
|
|
<itemizedlist>
|
|
|
|
<varlistentry>
|
|
<term>-n|--number-of-users <total-users></term>
|
|
<listitem><para>
|
|
Specifies the total number of test users to create (excluding
|
|
any machine accounts required for the traffic). Note that these
|
|
extra users simply populate the DC's DB - the actual user
|
|
traffic generated is still based on the summary-file.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>--number-of-groups <total-groups></term>
|
|
<listitem><para>
|
|
Creates the specified number of groups, for assigning the test
|
|
users to. Note that users are not automatically assigned to
|
|
groups - use either --average-groups-per-user or
|
|
--group-membership to do this.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>--average-groups-per-user <average-groups></term>
|
|
<listitem><para>
|
|
Randomly assigns the test users to the test groups created.
|
|
The group memberships are distributed so that the overall
|
|
average groups that a user is member of matches this number.
|
|
Some users will belong to more groups and some users will
|
|
belong to fewer groups. This option is incompatible with
|
|
the --group-membership option.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>--group-memberships <total-memberships></term>
|
|
<listitem><para>
|
|
Randomly assigns the test users to the test groups created.
|
|
The group memberships are distributed so that the total
|
|
groups that a user is member of, across all users, matches
|
|
this number. For example, with 100 users and 10 groups,
|
|
--group-memberships=300 would assign a user to 3 groups
|
|
on average. Some users will belong to more groups and some
|
|
users will belong to fewer groups, but the total of all
|
|
member linked attributes would be 300. This option is
|
|
incompatible with the --group-membership option.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
</itemizedlist>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>--clean-up</term>
|
|
<listitem><para>
|
|
Cleans up any users and groups that were created by previously running
|
|
this tool. It is recommended you always clean up after running the tool.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>-I|--instance-id <id></term>
|
|
<listitem><para>
|
|
Use this option to run multiple instances of the tool on the same DC at
|
|
the same time. This adds a prefix to the test users generated to keep
|
|
them separate on the DC.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>-K|--prefer-kerberos</term>
|
|
<listitem><para>
|
|
Use Kerberos to authenticate the test users.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>-B|--badpassword-frequency <frequency></term>
|
|
<listitem><para>
|
|
Use this option to simulate users trying to authenticate with an
|
|
incorrect password.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>--dns-rate <rate></term>
|
|
<listitem><para>
|
|
Increase the rate at which DNS packets get sent.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>-t|--timing-data <file></term>
|
|
<listitem><para>
|
|
This writes extra timing data to the file specified. This is mostly
|
|
used for reporting options, such as generating graphs.
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Samba Common Options</term>
|
|
<listitem>
|
|
<itemizedlist>
|
|
&stdarg.client.debug;
|
|
&stdarg.configfile;
|
|
&stdarg.option;
|
|
<varlistentry>
|
|
<term>--realm=REALM</term>
|
|
<listitem><para>
|
|
Set the realm name
|
|
</para></listitem>
|
|
</varlistentry>
|
|
&stdarg.version;
|
|
</itemizedlist>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Credential Options</term>
|
|
<listitem>
|
|
<itemizedlist>
|
|
<varlistentry>
|
|
<term>--simple-bind-dn=DN</term>
|
|
<listitem><para>
|
|
DN to use for a simple bind
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>--password=PASSWORD</term>
|
|
<listitem><para>
|
|
Password
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>-U USERNAME|--username=USERNAME</term>
|
|
<listitem><para>
|
|
Username
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>-W WORKGROUP|--workgroup=WORKGROUP</term>
|
|
<listitem><para>
|
|
Workgroup
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
&stdarg.kerberos;
|
|
|
|
<varlistentry>
|
|
<term>--ipaddress=IPADDRESS</term>
|
|
<listitem><para>
|
|
IP address of the server
|
|
</para></listitem>
|
|
</varlistentry>
|
|
|
|
&stdarg.machinepass;
|
|
</itemizedlist>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
</variablelist>
|
|
</refsect1>
|
|
|
|
<refsect1>
|
|
<title>OPERATIONS</title>
|
|
|
|
<refsect2>
|
|
<title>Generating a traffic-summary file</title>
|
|
<para>To use this tool, you need either a traffic-summary file or a
|
|
traffic-model file. To generate either of these files, you will need a
|
|
packet capture of actual Samba activity on your network.</para>
|
|
|
|
<para>Use Wireshark to take a packet capture on your network of the
|
|
traffic you want to generate. For example, if you want to simulate lots
|
|
of users logging on, then take a capture at 8:30am when users are
|
|
logging in.</para>
|
|
|
|
<para>Next, you need to convert your packet capture into a traffic
|
|
summary file, using <command>traffic_summary.pl</command>. Basically
|
|
this removes any sensitive information from the capture and summarizes
|
|
what type of packet was sent and when.</para>
|
|
|
|
<para>Refer to the <command>traffic_summary.pl --help</command> help for more
|
|
details, but the basic command will look something like:</para>
|
|
|
|
<para><command>tshark -r capture.pcapng -T pdml |
|
|
traffic_summary.pl > traffic-summary.txt</command></para>
|
|
</refsect2>
|
|
|
|
<refsect2>
|
|
<title>Replaying a traffic-summary file</title>
|
|
<para>Once you have a traffic-summary file, you can use it to generate
|
|
traffic. The traffic_replay tool gets passed the traffic-summary file,
|
|
along with the full DNS hostname of the DC being tested. You also need
|
|
to provide some user credentials, and possibly the Samba realm and
|
|
workgroup (although the realm and workgroup may be determined
|
|
automatically, for example from the /etc/smb.conf file, if one is
|
|
present). E.g.</para>
|
|
|
|
<para><command>traffic_replay traffic-summary.txt
|
|
my-dc.samdom.example.com -UAdmin%password -W samdom
|
|
--realm=samdom.example.com --fixed-password=blahblah123!</command>
|
|
</para>
|
|
|
|
<para>This simply regenerates Samba activity seen in the traffic
|
|
summary. The traffic is grouped into 'conversations' between a host and
|
|
the DC. A user and machine account is created on the DC for each
|
|
conversation, in order to allow logon and other operations to succeed.
|
|
The script generates the same types of packets as those seen in the
|
|
summary.</para>
|
|
|
|
<para>Creating users can be quite a time-consuming process, especially
|
|
if a lot of conversations are being generated. To save time, the test
|
|
users remain on the DC by default. You will need to run the --clean-up
|
|
option to remove them, once you have finished generating traffic.
|
|
Because the same test users are used across multiple runs of the tool,
|
|
a consistent password for these users needs to be used - this is
|
|
specified by the --fixed-password option.
|
|
</para>
|
|
|
|
<para>The benefit of this tool over simply using tcprelay is that the
|
|
traffic generated is independent of any specific network. No setup is
|
|
needed beforehand on the test DC. The traffic no longer contains
|
|
sensitive details, so the traffic summary could be potentially shared
|
|
with other Samba developers.</para>
|
|
|
|
<para>However, replaying a traffic-summary directly is somewhat limited
|
|
in what you can actually do. A more flexible approach is to generate
|
|
the traffic using a model file.</para>
|
|
</refsect2>
|
|
|
|
<refsect2>
|
|
<title>Generating a traffic-model file</title>
|
|
<para>To create a traffic-model file, simply pass the traffic-summary
|
|
file to the <command>traffic_learner</command> script. E.g.</para>
|
|
|
|
<para><command>traffic_learner traffic-summary.txt
|
|
-o traffic-model.txt</command></para>
|
|
|
|
<para>This generates a model of the Samba activity in your network.
|
|
This model-file can now be used to generate traffic.</para>
|
|
</refsect2>
|
|
|
|
<refsect2>
|
|
<title>Replaying the traffic-model file</title>
|
|
<para>Packet generation using a traffic-model file uses the same
|
|
command as a traffic-summary file, e.g.</para>
|
|
|
|
<para><command>traffic_replay traffic-model.txt
|
|
my-dc.samdom.example.com -UAdmin%password</command>
|
|
</para>
|
|
|
|
<para>By default, this will generate 60 seconds worth of traffic based
|
|
on the model. You can specify longer using the --duration parameter.
|
|
</para>
|
|
|
|
<para>The traffic generated is an approximation of what was seen in
|
|
the network capture. The traffic generation involves some randomness,
|
|
so running the same command multiple times may result in slightly
|
|
different traffic being generated (although you can avoid this, by
|
|
specifying the --random-seed option).</para>
|
|
|
|
<para>As well as changing how long the model runs for, you can also
|
|
change how many conversations get generated and how fast the traffic
|
|
gets replayed. To roughly double the number of conversations that get
|
|
replayed, use --scale-traffic=2 or to approximately halve the number
|
|
use --scale-traffic=0.5. To approximately double how quickly the
|
|
conversations get replayed, use --replay-rate=2, or to halve this use
|
|
--replay-rate=0.5</para>
|
|
|
|
<para>For example, to generate approximately 10 times the amount of
|
|
traffic seen over a two-minute period (based on the network capture),
|
|
use:</para>
|
|
|
|
<para><command>traffic_replay traffic-model.txt
|
|
my-dc.samdom.example.com -UAdmin%password --fixed-password=blahblah123!
|
|
--scale-traffic=10 --duration=120</command></para>
|
|
</refsect2>
|
|
|
|
<refsect2>
|
|
<title>Scaling the number of users</title>
|
|
<para>The performance of a Samba DC running a small subset of test
|
|
users will be different to a fully-populated Samba DC running in a
|
|
network. As the number of users increases, the size of the DB
|
|
increases, and a very large DB will perform worse than a smaller DB.
|
|
</para>
|
|
|
|
<para>To increase the size of the Samba DB, this tool can also create
|
|
extra users and groups. These extra users are basically 'filler' for
|
|
the DB. They won't actually be used to generate traffic, but they may
|
|
slow down authentication of the test users.</para>
|
|
|
|
<para>For example, to populate the DB with an extra 5000 users (note
|
|
this will take a while), use the command:</para>
|
|
|
|
<para><command>traffic_replay my-dc.samdom.example.com
|
|
-UAdmin%password --generate-users-only --fixed-password=blahblah123!
|
|
--number-of-users=5000</command></para>
|
|
|
|
<para>You can also create groups and assign users to groups. The users
|
|
can be randomly assigned to groups - this includes any extra users
|
|
created as well as the users that map to conversations. Use either
|
|
--average-groups-per-user or --group-memberships to specify how many
|
|
group memberships should be assigned to the test users.</para>
|
|
|
|
<para>For example, to assign the users in the replayed conversations
|
|
into 10 groups on average, use a command like:</para>
|
|
|
|
<para><command>traffic_replay traffic-model.txt my-dc.samdom.example.com
|
|
-UAdmin%password --fixed-password=blahblah123!
|
|
--generate-users-only --number-of-groups=25 --average-groups-per-user=10
|
|
</command></para>
|
|
|
|
<para>The users created by the test will have names like STGU-0-xyz.
|
|
The groups generated have names like STGG-0-xyz.</para>
|
|
</refsect2>
|
|
</refsect1>
|
|
|
|
|
|
<refsect1>
|
|
<title>VERSION</title>
|
|
|
|
<para>This man page is complete for version &doc.version; of the Samba
|
|
suite.</para>
|
|
</refsect1>
|
|
|
|
<refsect1>
|
|
<title>SEE ALSO</title>
|
|
<para>
|
|
<citerefentry>
|
|
<refentrytitle>traffic_learner</refentrytitle><manvolnum>7</manvolnum>
|
|
</citerefentry>.
|
|
</para>
|
|
</refsect1>
|
|
|
|
<refsect1>
|
|
<title>AUTHOR</title>
|
|
|
|
<para>The original Samba software and related utilities
|
|
were created by Andrew Tridgell. Samba is now developed
|
|
by the Samba Team as an Open Source project similar
|
|
to the way the Linux kernel is developed.</para>
|
|
|
|
<para>The traffic_replay tool was developed by the Samba team at
|
|
Catalyst IT Ltd.</para>
|
|
|
|
<para>The traffic_replay manpage was written by Tim Beale.</para>
|
|
</refsect1>
|
|
|
|
</refentry>
|