IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Pushing new elements to the front tends to assign extra weight to that
elements, at the expense of other elements that are already in the cache.
The idea is then to try first-time insertion somewhere in the middle.
Further attempts suggest that the "pivot" should be closer to the end.
Cache performance for "apt-shell <<<unmet", previous commit:
hit=62375 miss=17252
Cache performance for "apt-shell <<<unmet", this commit:
hit=65085 miss=14542
After base62+golomb decoding, most deltas are under 65536 (in Provides
versions, average delta should be around 1536). So the whole version
can be stored using short deltas, effectively halving memory footprint.
However, this seems to be somewhat slower: per-delta copying and
decode_golomb must be invoked to recover hash values. On the other
hand, this allows to increase cache size (128 -> 192). But note that,
with larger cache sizes, LRU linear search will take longer. So this is
a compromise - and apparently a favourable one.
Currently, set.c uses array of chars to store bit vector. Each char
stores one bit: 0 or 1.
Let's use packed bitmap instead. It creates room for optimizations.
Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>
In build.c:checkSpec(), build dependencies are checked by creating
a transaction, adding source header to the transaction and then running
rpmdepCheck(). Source headers have only BuildRequies and BuildConflicts
types of dependencies (no regular Requires and Conflicts). Also, source
packages have no Provides, but they have NAMES. When a self-conflicting
package is installed, its self conflict will be triggered by the source
package name.
To fix the problem, note that binary packages explicitly provide
their N=EVR among Provides; and source packages provides nothing,
even the name. So the solution is as simple as not to check
the dependencies against package names.
Update: also, do not check installed Requires against erasedPackages names.
To check the dependencies of an installed package, a "transaction" is
created, and the package is added to the transaction. The transaction
is then checked with rpmdepCheck(). However, since the installed
package has not been marked for removal, a conflict can be triggered
between the installed and that of transaction copies of the package.
The right thing to do is to mark package for removal, re-add it to
the transaction, and then to check the dependencies.
Header instance is its number in /var/lib/rpm/Packages database.
When a header comes from the database, it is sometimes useful to know
its instance (I need this to adjust verify.c:verifyDependencies() for
self-conflicting packages). On the contrary, setting instance numbers
should happen only within librpmdb, which is why headerSetInstance()
comes with hidden visibility.
Only for the last two weeks or so, the issues has been raised twice.
By specifying "Provides: foo, Conflicts: foo", people expect that
other packages which provide "foo" will not be installed along with
the package. What people don't anticipate is that the package will
conflict with itself, and will not be installed at all. This is where
apt and rpm differ. In apt, "conflicts may never self match". In rpm,
Requires and Conflicts are handled in exactly the same way, except that
Requires should match, and Conflicts should not match (I call this
a symmetry). Both can match against the package they come from.
So, to permit self-conflicting packages, I have to break the symmetry
and pass additional argument which indicates the type of dependency
being processed (either Requires or Conflicts). The code is then
adjusted to discard self-matching Conflicts.
Obsoletes should be handled specially, too. In tsSatisfiesDepend(),
I attempt to handle the Obsoletes case as well. It is rather
unfortunate that, in rpmdepCheck(), Obsoletes are simply not checked
just yet.
Let's try to use recent libbeectypt in our World Best RPM.
Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>
Signed-off-by: Dmitry V. Levin <ldv@altlinux.org>
My previous assumption that strdup() was not needed for dbProvCache keys
was wrong. The keys point to header memory, which is right. However,
those are not only ts->addedPackages and ts->erasedPackages headers.
In checkDependent(), headers come from rpmdb and are disposed immediately
after the check.
For erasedPackages, the dirIndex and provIndex are unused, and
thus should not be created at all. There is arguably a better
option - to provide explicit alMakeIndex and alFreeIndex API.
Based on rpm.org 2e76d0e6 by Panu Matilainen:
> Add in-memory hash for caching rpmdb dependency lookups
> - worst case behavior for uncached dependency lookups can be disastrous,
> eg > 35s vs < 1s on my laptop for trying to remove /bin/sh provider
> - we only bother caching rpmdb lookups, the other cases plenty fast already
> - using in-memory cache avoids nasty in vs out of chroot issues with
> temporary db files, which otherwise were about as fast
However, we do not use full-blown printDepend-based caching (i.e.
we no longer cache depends with versions). This is because, well,
dependency versions are likely to differ. This is especially true
if we consider upcoming set-versions for soname symbols - hashing
symbol sets here will be just a waste of time and memory. And so
now we cache satisfied/unsatisfied depends by just name. Thus,
"yes" hit can be used immediately only for unversioned dependences.
Top 10 dependencies which will be handled by the cache:
$ rpm -qaR |grep -v rpmlib |grep -v = |sort |uniq -c |sort -n |tail
245 /usr/lib/perl5/vendor_perl
311 libm.so.6(GLIBC_2.2.5)(64bit)
386 libpthread.so.0(GLIBC_2.2.5)(64bit)
454 /lib64/ld-linux-x86-64.so.2
548 libc.so.6(GLIBC_2.3)(64bit)
587 /bin/sh
828 libc.so.6(GLIBC_2.3.4)(64bit)
906 libc.so.6(GLIBC_2.4)(64bit)
1128 rtld(GNU_HASH)
1140 libc.so.6(GLIBC_2.2.5)(64bit)
$
Top 10 dependencies which will not be handled by the cache:
$ rpm -qaR |grep -v rpmlib |grep -e = |sort |uniq -c |sort -n |tail
13 python-base = 2.6.5-alt2
14 mono(mscorlib) = 1.0
15 qt4-common = 4.6.2-alt6
16 mono(mscorlib) = 2.0
18 mktemp >= 1:1.3.1
20 koffice-common = 4:2.2.0-alt2
20 perl-base >= 1:5.8.0
23 alternatives >= 0:0.4
49 libqt4-core >= 4.6.2
54 perl-base >= 1:5.6.0
$
Here's a simple test to see if the cache works (using Panu's example -
trying to remove /bin/sh).
(before this change)
$ time LD_LIBRARY_PATH=$PWD/1 rpm -e --test sh 2>&1 |tail
/bin/sh is needed by groff-base-1.20.1-alt0.20091013
/bin/sh is needed by groff-base-1.20.1-alt0.20091013
/bin/sh is needed by groff-base-1.20.1-alt0.20091013
/bin/sh is needed by libgnome-sharp-2.24.1-alt1
/bin/sh is needed by kernel-image-std-def-2.6.32-alt15
/bin/sh is needed by kernel-image-std-def-2.6.32-alt15
/bin/sh is needed by kde4libs-4.4.5-alt1
/bin/sh is needed by kde4base-runtime-core-4.4.5-alt1
/bin/sh is needed by kde4base-konqueror-4.4.5-alt1
/usr/lib/bash is needed by bash-builtin-lockf-0.3.1-alt1
rpm -e --test sh 2>&1 6.18s user 3.44s system 94% cpu 10.182 total
$
(after this change)
$ time LD_LIBRARY_PATH=$PWD/2 rpm -e --test sh 2>&1 |tail
/bin/sh is needed by groff-base-1.20.1-alt0.20091013
/bin/sh is needed by groff-base-1.20.1-alt0.20091013
/bin/sh is needed by groff-base-1.20.1-alt0.20091013
/bin/sh is needed by libgnome-sharp-2.24.1-alt1
/bin/sh is needed by kernel-image-std-def-2.6.32-alt15
/bin/sh is needed by kernel-image-std-def-2.6.32-alt15
/bin/sh is needed by kde4libs-4.4.5-alt1
/bin/sh is needed by kde4base-runtime-core-4.4.5-alt1
/bin/sh is needed by kde4base-konqueror-4.4.5-alt1
/usr/lib/bash is needed by bash-builtin-lockf-0.3.1-alt1
rpm -e --test sh 2>&1 0.11s user 0.09s system 91% cpu 0.218 total
$
Before this change, only rpm headers for addedPackages were loaded
during transaction; for removedPackages, we only stored the list
of rpmdb header numbers (numeric instances). With this change,
removedPackages' headers are getting loaded, too (the list is called
erasedPackages). The reason is that, when a header is loaded,
it is possible to use pointers to point e.g. into header strings
without using strdup. This might come useful as we try to reimplement
the depends cache.
