linux/tools/testing/selftests
Joe Damato a61a459f58 testing: net-drv: use stats64 for testing
Testing a network device that has large numbers of bytes/packets may
overflow. Using stats64 when comparing fixes this problem.

I tripped on this while iterating on a qstats patch for mlx5. See below
for confirmation without my added code that this is a bug.

Before this patch (with added debugging output):

$ NETIF=eth0 tools/testing/selftests/drivers/net/stats.py
KTAP version 1
1..4
ok 1 stats.check_pause
ok 2 stats.check_fec
rstat: 481708634 qstat: 666201639514 key: tx-bytes
not ok 3 stats.pkt_byte_sum
ok 4 stats.qstat_by_ifindex

Note the huge delta above ^^^ in the rtnl vs qstats.

After this patch:

$ NETIF=eth0 tools/testing/selftests/drivers/net/stats.py
KTAP version 1
1..4
ok 1 stats.check_pause
ok 2 stats.check_fec
ok 3 stats.pkt_byte_sum
ok 4 stats.qstat_by_ifindex

It looks like rtnl_fill_stats in net/core/rtnetlink.c will attempt to
copy the 64bit stats into a 32bit structure which is probably why this
behavior is occurring.

To show this is happening, you can get the underlying stats that the
stats.py test uses like this:

$ ./cli.py --spec ../../../Documentation/netlink/specs/rt_link.yaml \
           --do getlink --json '{"ifi-index": 7}'

And examine the output (heavily snipped to show relevant fields):

 'stats': {
           'multicast': 3739197,
           'rx-bytes': 1201525399,
           'rx-packets': 56807158,
           'tx-bytes': 492404458,
           'tx-packets': 1200285371,

 'stats64': {
             'multicast': 3739197,
             'rx-bytes': 35561263767,
             'rx-packets': 56807158,
             'tx-bytes': 666212335338,
             'tx-packets': 1200285371,

The stats.py test prior to this patch was using the 'stats' structure
above, which matches the failure output on my system.

Comparing side by side, rx-bytes and tx-bytes, and getting ethtool -S
output:

rx-bytes stats:    1201525399
rx-bytes stats64: 35561263767
rx-bytes ethtool: 36203402638

tx-bytes stats:      492404458
tx-bytes stats64: 666212335338
tx-bytes ethtool: 666215360113

Note that the above was taken from a system with an mlx5 NIC, which only
exposes ndo_get_stats64.

Based on the ethtool output and qstat output, it appears that stats.py
should be updated to use the 'stats64' structure for accurate
comparisons when packet/byte counters get very large.

To confirm that this was not related to the qstats code I was iterating
on, I booted a kernel without my driver changes and re-ran the test
which shows the qstats are skipped (as they don't exist for mlx5):

NETIF=eth0 tools/testing/selftests/drivers/net/stats.py
KTAP version 1
1..4
ok 1 stats.check_pause
ok 2 stats.check_fec
ok 3 stats.pkt_byte_sum # SKIP qstats not supported by the device
ok 4 stats.qstat_by_ifindex # SKIP No ifindex supports qstats

But, fetching the stats using the CLI

$ ./cli.py --spec ../../../Documentation/netlink/specs/rt_link.yaml \
           --do getlink --json '{"ifi-index": 7}'

Shows the same issue (heavily snipped for relevant fields only):

 'stats': {
           'multicast': 105489,
           'rx-bytes': 530879526,
           'rx-packets': 751415,
           'tx-bytes': 2510191396,
           'tx-packets': 27700323,
 'stats64': {
             'multicast': 105489,
             'rx-bytes': 530879526,
             'rx-packets': 751415,
             'tx-bytes': 15395093284,
             'tx-packets': 27700323,

Comparing side by side with ethtool -S on the unmodified mlx5 driver:

tx-bytes stats:    2510191396
tx-bytes stats64: 15395093284
tx-bytes ethtool: 17718435810

Fixes: f0e6c86e4b ("testing: net-drv: add a driver test for stats reporting")
Signed-off-by: Joe Damato <jdamato@fastly.com>
Link: https://lore.kernel.org/r/20240520235850.190041-1-jdamato@fastly.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
2024-05-23 10:18:29 +02:00
..
alsa selftests/alsa: make dump_config_tree() as void function 2024-05-08 18:15:42 +02:00
amd-pstate selftests/amd-pstate: Added option to provide perf binary path 2023-10-16 13:06:58 -06:00
arm64 kselftest: arm64: Add a null pointer check 2024-04-28 08:42:58 +01:00
bpf selftests/bpf: Adjust btf_dump test to reflect recent change in file_operations 2024-05-17 01:50:11 +02:00
breakpoints selftests/breakpoints: Fix format specifier in ksft_print_msg in step_after_suspend_test.c 2023-11-30 14:32:34 -07:00
cachestat selftests/cachestat: Fix print_cachestat format 2023-10-13 14:08:10 -06:00
capabilities selftests/capabilities: fix warn_unused_result build warnings 2024-05-08 16:48:25 -06:00
cgroup selftests/cgroup: Drop define _GNU_SOURCE 2024-05-13 07:02:36 -10:00
clone3 selftests/clone3: ksft_exit functions do not return 2024-05-06 13:57:20 -06:00
connector selftests: connector: Fix input argument error paths to skip 2023-07-31 20:11:42 -07:00
core selftests: add F_DUPDFD_QUERY selftests 2024-05-10 08:49:13 +02:00
cpu-hotplug
cpufreq selftests: cpufreq: conform test to TAP 2024-05-06 13:57:20 -06:00
damon selftests: damon: add access_memory to .gitignore 2024-03-04 17:01:16 -08:00
devices kselftest: devices: Add sample board file for XPS 13 9300 2024-01-27 17:38:53 -08:00
dma
dmabuf-heaps selftests/dmabuf-heap: conform test to TAP format output 2024-05-06 13:57:19 -06:00
drivers testing: net-drv: use stats64 for testing 2024-05-23 10:18:29 +02:00
dt linux_kselftest-next-6.9-rc1 2024-03-11 09:25:33 -07:00
efivarfs selftests/efivarfs: create-read: fix a resource leak 2023-10-18 14:33:44 -06:00
exec selftests: exec: Use new ksft_exit_fail_perror() helper 2024-05-06 13:57:19 -06:00
fchmodat2 selftests: link libasan statically for tests with -fsanitize=address 2023-09-19 13:21:32 -07:00
filelock selftests: add OFD lock tests 2023-08-24 10:41:47 -04:00
filesystems selftests: filesystems: add missing stddef header 2024-05-08 16:48:13 -06:00
firmware selftests: firmware: remove duplicate unneeded defines 2023-10-06 17:33:52 -06:00
fpu
ftrace Probes updates for v6.10: 2024-05-17 18:29:30 -07:00
futex selftests: fuxex: Report a unique test name per run of futex_requeue_pi 2024-02-19 15:00:51 -07:00
gpio selftest: gpio: remove obsolete gpio-mockup test 2024-03-05 20:01:08 +01:00
hid selftests/hid: skip tests with HID-BPF if udev-hid-bpf is not installed 2024-05-07 15:39:58 +02:00
ia64
intel_pstate
iommu iommufd: Add config needed for iommufd_fail_nth 2024-04-14 13:52:08 -03:00
ipc selftests/ipc: ksft_exit functions do not return 2024-05-06 13:57:20 -06:00
ir
kcmp
kexec
kmod
kselftest selftests: ktap_helpers: Make it POSIX-compliant 2024-05-06 13:57:20 -06:00
kvm selftests/kvm: remove dead file 2024-05-15 13:40:16 -04:00
landlock selftests/harness: Fix vfork() side effects 2024-05-11 19:18:47 +02:00
lib
livepatch linux_kselftest-next-6.9-rc1 2024-03-11 09:25:33 -07:00
lkdtm selftests/lkdtm: Disable CONFIG_UBSAN_TRAP in test config 2023-09-28 16:39:08 -07:00
locking
lsm lsm: use 32-bit compatible data types in LSM syscalls 2024-03-14 11:31:26 -04:00
media_tests
membarrier selftests: membarrier: ksft_exit_pass() does not return 2024-05-06 13:57:20 -06:00
memfd selftests/memfd: delete unused declarations 2024-02-22 10:24:38 -08:00
memory-hotplug
mincore
mm linux_kselftest-next-6.10-rc1 2024-05-14 11:36:54 -07:00
mount
mount_setattr
move_mount_set_group selftests/move_mount_set_group:Make tests build with old libc 2024-01-22 15:33:37 +01:00
mqueue selftests/mqueue: Set timeout to 180 seconds 2024-02-20 17:28:19 -07:00
nci
net selftest: af_unix: Make SCM_RIGHTS into OOB data. 2024-05-21 13:42:08 +02:00
nolibc tools/nolibc: add support for uname(2) 2024-04-14 20:28:54 +02:00
nsfs
ntb
openat2 linux_kselftest-next-6.7-rc1 2023-11-01 17:08:10 -10:00
perf_events selftests/perf_events: Test FASYNC with watermark wakeups 2024-04-14 22:26:33 +02:00
pid_namespace
pidfd linux_kselftest-next-6.10-rc1 2024-05-14 11:36:54 -07:00
power_supply selftests: power_supply: Make it POSIX-compliant 2024-05-06 13:57:20 -06:00
powerpc powerpc updates for 6.10 2024-05-17 09:05:46 -07:00
prctl selftests: prctl: Add prctl test for PR_GET_NAME 2023-11-30 16:30:05 -07:00
proc proc: test ProtectionKey in proc-empty-vm test 2023-11-01 12:46:59 -07:00
pstore
ptp ptp: add testptp mask test 2023-10-15 20:07:52 +01:00
ptrace
rcutorture torture: Scale --do-kvfree test time 2024-04-16 11:16:36 +02:00
resctrl selftests/resctrl: fix clang build warnings related to abs(), labs() calls 2024-05-08 16:53:19 -06:00
ring-buffer ring-buffer/selftest: Add ring-buffer mapping test 2024-05-13 18:09:56 -04:00
riscv RISC-V: selftests: cbo: Ensure asm operands match constraints, take 2 2024-04-26 10:21:56 -07:00
rlimits
rseq selftests/rseq: Do not skip !allowed_cpus for mm_cid 2024-01-22 11:40:36 -07:00
rtc
rust kselftest: Add basic test for probing the rust sample modules 2024-03-04 13:13:04 -07:00
safesetid
sched selftests: sched: Fix spelling mistake "hiearchy" -> "hierarchy" 2024-02-20 17:28:19 -07:00
seccomp selftests/seccomp: Try to fit runtime of benchmark into timeout 2024-03-29 13:57:14 -06:00
sgx selftests/sgx: Include KHDR_INCLUDES in Makefile 2024-05-08 17:08:46 -06:00
sigaltstack selftests/sigaltstack: Fix wrong format specifier 2023-10-13 14:08:26 -06:00
size
sparc64
splice
static_keys selftests: static_keys: fix test name in messages 2023-10-05 14:28:59 -06:00
sync selftests: sync: ksft_exit_pass() does not return 2024-05-06 13:57:21 -06:00
syscall_user_dispatch selftests: sud_test: return correct emulated syscall value on RISC-V 2024-04-25 09:46:41 -07:00
sysctl sysclt: Clarify the results of selftest run 2023-12-28 04:57:57 -08:00
tc-testing selftests/tc-testing: require an up to date iproute2 for blockcast tests 2024-03-04 19:30:57 -08:00
tdx selftests/tdx: add gitignore file 2023-10-05 14:22:53 -06:00
thermal/intel selftests: thermal: intel: workload_hint: add missing gitignore 2024-02-20 16:30:55 -07:00
timens
timers selftests: timers: ksft_exit functions do not return 2024-05-06 13:57:21 -06:00
tmpfs
tpm2
tty kselftest/tty: Report a consistent test name for the one test we run 2024-05-06 13:57:19 -06:00
turbostat tools/power turbostat: v2024.04.10 2024-04-10 09:07:57 -04:00
uevent selftests: uevent: add missing gitignore 2024-02-20 16:30:40 -07:00
user
user_events selftests/user_events: Add non-spacing separator check 2024-05-13 12:15:46 -04:00
vDSO selftests/vDSO: fix runtime errors on LoongArch 2024-05-05 17:28:07 -07:00
watchdog
wireguard printk changes for 6.10 2024-05-15 12:34:46 -07:00
x86 linux_kselftest-next-6.10-rc1 2024-05-14 11:36:54 -07:00
zram
.gitignore
gen_kselftest_tar.sh
kselftest_deps.sh selftests: kselftest_deps: fix l5_test() empty variable 2024-05-08 16:46:41 -06:00
kselftest_harness.h linux_kselftest-next-6.10-rc1 2024-05-14 11:36:54 -07:00
kselftest_install.sh
kselftest_module.h
kselftest.h selftests: kselftest: Make ksft_exit functions return void instead of int 2024-05-06 13:57:21 -06:00
lib.mk Networking changes for 6.10. 2024-05-14 19:42:24 -07:00
Makefile Networking changes for 6.10. 2024-05-14 19:42:24 -07:00
run_kselftest.sh kselftest/runner.sh: add netns support 2023-12-23 00:26:32 +00:00