d78a2cbf3d
Hit ratio for "apt-shell <<<unmet" command:
160 slots: hit=46813 miss=22862 67.2%
256 slots: hit=52238 miss=17437 75.0%
So, we've increased the cache size by a factor of 256/160=1.6 or by 60%,
and the number of misses has decreased by a factor of 22862/17437=1.31
or by 1-17437/22862=23.7%. This is not so bad, but it looks like we're
paying more for less. The following analysis shows that this is not
quite true, since the real memory usage has increased by a somewhat
smaller factor.
160 slots, callgrind annotations:
2,406,630,571 PROGRAM TOTALS
795,320,289 lib/set.c:decode_base62_golomb
496,682,547 lib/set.c:rpmsetcmp
93,466,677 sysdeps/x86_64/strcmp.S:__GI_strcmp
91,323,900 sysdeps/x86_64/memcmp.S:bcmp
90,314,290 stdlib/msort.c:msort_with_tmp'2
83,003,684 sysdeps/x86_64/strlen.S:__GI_strlen
58,300,129 sysdeps/x86_64/memcpy.S:memcpy
...
inclusive:
1,458,467,003 lib/set.c:rpmsetcmp
256 slots, callgrind annotations:
2,246,961,708 PROGRAM TOTALS
634,410,352 lib/set.c:decode_base62_golomb
492,003,532 lib/set.c:rpmsetcmp
95,643,612 sysdeps/x86_64/memcmp.S:bcmp
93,467,414 sysdeps/x86_64/strcmp.S:__GI_strcmp
90,314,290 stdlib/msort.c:msort_with_tmp'2
79,217,962 sysdeps/x86_64/strlen.S:__GI_strlen
56,509,877 sysdeps/x86_64/memcpy.S:memcpy
...
inclusive:
1,298,977,925 lib/set.c:rpmsetcmp
So the decoding routine now takes about 20% fewer instructions, and
inclusive rpmsetcmp cost is reduced by about 11%. Note, however, that
bcmp is now the third most expensive routine (due to higher hit ratio).
Since recent glibc versions provide optimized memcmp implementations, I
imply that total/inclusive improvement can be somewhat better than 11%.
As per memory usage, the question "how much the cache takes" cannot be
generally answered with a single number. However, if we simply sum the
size of all malloc'd chunks on each rpmsetcmp invocation, using the
piece of code with a few obvious modifications elsewhere, we can obtain
the following statistics.
if (hc == CACHE_SIZE) {
int total = 0;
for (i = 0; i < hc; i++)
total += ev[i]->msize;
printf("total %d\n", total);
}
160 slots, memory usage:
min=1178583
max=2048701
avg=1330104
dev=94747
q25=1266647
q50=1310287
q75=1369005
256 slots, memory usage:
min=1670029
max=2674909
avg=1895076
dev=122062
q25=1828928
q50=1868214
q75=1916025
This indicates that average cache size is increased by about 42% from
1.27M to 1.81M; however, the third quartile is increased by about 40%,
and the maximum size is increased only by about 31% from 1.95M to 2.55M.
By which I conclude that extra 600K must be available even on low-memory
machines like Raspberry Pi (256M RAM).
* * *
What's a good hit ratio?
$ DepNames() { pkglist-query '[%{RequireName}\t%{RequireVersion}\n]' \
/var/lib/apt/lists/_ALT_Sisyphus_x86%5f64_base_pkglist.classic |
fgrep set: |cut -f1; }
$ DepNames |wc -l
34763
$ DepNames |sort -u |wc -l
2429
$ DepNames |sort |uniq -c |sort -n |awk '$1>1{print$1}' |Sum
33924
$ DepNames |sort |uniq -c |sort -n |awk '$1>1{print$1}' |wc -l
1590
$ DepNames |sort |uniq -c |sort -n |tail -256 |Sum
27079
$
We have 34763 set-versioned dependencies, which refer to 2429 sonames;
however, only 33924 dependencies refer to 1590 sonames more than once,
and the first reference is always a miss. Thus the best possible hit
ratio (if we use at least 1590 slots) is (33924-1590)/34763=93.0%.
What happens if we use only 256 slots? Assuming that dependencies are
processed in random order, the best strategy must spend its cache slots
on sonames with the most references. This way we can serve (27079-256)
dependencies via cache hit, and so the best possible hit ratio for 256
slots is is 77.2%, assuming that dependencies are processed in random
order.
This is RPM, the Red Hat Package Manager.
The latest releases are always available at:
ftp://ftp.rpm.org/pub/rpm
Additional RPM documentation (papers, slides, HOWTOs) can also be
found at the same site, as well as http://www.rpm.org.
There is a mailing list for discussion of RPM issues, rpm-list@redhat.com.
To subscribe, send a message to rpm-list-request@redhat.com with the word
"subscribe" in the subject line.
RPM was originally written by:
Erik Troan <ewt@redhat.com>
Marc Ewing <marc@redhat.com>
See the CREDITS file for a list of folks who have helped us out
tremendously. RPM is Copyright (c) 1998 by Red Hat Software, Inc.,
and may be distributed under the terms of the GPL and LGPL (see the
file COPYING for details).