Commit Graph

1692 Commits

Author SHA1 Message Date
Alexey Tourbin
ce4f05af30 debugedit.c: imported from rpm.org
Recent changes:
7e7ca4b 2010-08-17 Anssi Hannula	debugedit: fix incorrect error messages regarding -b and -d
e6a0b0e 2010-05-10 Roland McGrath	Make debugedit -i recompute build ID only when any DWARF was actually changed.
c4b6362 2010-05-10 Roland McGrath	Whitespace fixups for tools/debugedit.c
71a7dd8 2009-06-16 Panu Matilainen	Add debugedit support for DWARF-3 (RhBug:505774)
2011-01-30 03:05:59 +03:00
Alexey Tourbin
351e5f71be 4.0.4-alt100.13
- Removed --fileid query selector and Filemd5s rpmdb index (rpm.org).
- Removed ancient dependency loop whiteout mechanism (rpm.org).
- rpmdb.c: Do not exclude Requires(pre) dependencies from rpmdb index.
- Implemented %__find_{requires,provides}_filter macros (lower-level) and
  %filter_from_{requires,provides} (higher-level, compatible with Fedora).
2011-01-23 02:31:53 +03:00
Alexey Tourbin
a3ba18acbf build/checkFiles.c: avoid fts warning for no buildroot 2011-01-23 02:31:00 +03:00
Alexey Tourbin
2d3c3cef27 removed ancient dependency loop whiteout mechanism (rpm.org) 2011-01-23 02:30:59 +03:00
Alexey Tourbin
f355ac14b5 platform: simplify %_configure_platform and %_target_libdir macros 2011-01-23 02:30:48 +03:00
Alexey Tourbin
82f7aeb283 implemented %__find_{requires,provides}_filter and %filter_from_{requires,provides}
The first two macros provide general mechanism for find-requires
and find-provides filtering.  Any valid command or shell syntax can
be used - the command will be executed using "eval" within find-*
script.  The implementation in %___build_pre is not particular
beautiful (as it uses here-docs), but there seems to be no easy
way to quote for eval yet.

The second two commands are built on top of the first: they use
sed as a filtering tool.  The implementation is compatible with Fedora:
http://fedoraproject.org/wiki/Packaging:AutoProvidesAndRequiresFiltering
2011-01-23 02:30:08 +03:00
Alexey Tourbin
7247ecbda7 rpmdb.c: do not exclude Requires(pre) dependencies from rpmdb index
It can be observed that, while packages require their dependencies,
the dependencies in turn are not always required by their packages:

$ rpm -qa --requires |awk '{print$1}' |sort -u |xargs rpm -q --whatrequires >/dev/null
warning: no package requires /bin/chgrp
warning: no package requires /bin/chmod
warning: no package requires /bin/mv
warning: no package requires /etc/rc.d/init.d/postfix
warning: no package requires /sbin/pidof
warning: no package requires /sbin/post_ldconfig
warning: no package requires /usr/bin/alt-docs-genextras
warning: no package requires /usr/bin/gpasswd
warning: no package requires /usr/bin/gtk-query-immodules-2.0
warning: no package requires /usr/bin/mkfontdir
warning: no package requires /usr/sbin/alternatives-update
warning: no package requires /usr/sbin/control-dump
warning: no package requires /usr/sbin/control-restore
warning: no package requires /usr/sbin/gconf_install_schema
warning: no package requires /usr/sbin/groupadd
warning: no package requires /usr/sbin/indexhtml-update
warning: no package requires /usr/sbin/post_service
warning: no package requires /usr/sbin/useradd
warning: no package requires /usr/sbin/usermod
warning: no package requires /usr/share/php/scripts/php_postin.sh
warning: no package requires dvdrwtools-control
warning: no package requires libguile-vhttpd
warning: no package requires php5-suhosin
warning: no package requires rpmlib(CompressedFileNames)
warning: no package requires rpmlib(PartialHardlinkSets)
warning: no package requires rpmlib(PayloadFilesHavePrefix)
warning: no package requires rpmlib(PayloadIsLzma)
warning: no package requires rpmlib(SetVersions)
warning: no package requires rpmlib(VersionedDependencies)
warning: no package requires x11presetdrv
$

This is because rpm removes "install prerequisites" from rpmdb index.
As a consequence, rpm will allow (but apt will not) to remove packages
with "not required" dependencies:

$ rpm -e --test x11presetdrv
$

This is possibly wrong, and it makes dependency optimization hard (e.g.
can't merge duplicate names with different flags).  Also, the code was
buggy: testing isInstallPreReq was not enough as it is not mutually
exclusive with isErasePreReq - e.g. Requries(pre,preun) dependencies
can be used in specfile.
2011-01-22 18:38:22 +03:00
Alexey Tourbin
42b139d1eb removed --fileid query selector and Filemd5s rpmdb index (rpm.org) 2011-01-22 17:35:13 +03:00
Alexey Tourbin
ed41c1738f 4.0.4-alt100.12
- build/checkFiles.c: Fixed %exclude vs unpackaged regression.
- header.c: Optimized header loading and access routines.
2011-01-21 21:13:40 +03:00
Alexey Tourbin
5a1601ed3d build/checkFiles.c: fixed %exclude vs unpackaged regression 2011-01-21 20:48:10 +03:00
Alexey Tourbin
335d64cf5e header.c (headerFindI18NString): disable conversion when lang is "C" 2011-01-21 15:41:20 +03:00
Alexey Tourbin
0193cbf8b9 header.c (headerFindI18NString): avoid repeated getenv calls 2011-01-21 15:25:56 +03:00
Alexey Tourbin
a45456493b header.c (findEntry): replaced bsearch with custom routine
Headers normally have around 64 tags, which means bsearch will issue
6 callbacks, only to perform a single instruction (a->tag - b->tag).

This also removes a bogus piece of code.

Here is how these optimizations will impact apt.  I remove
/var/cache/apt/*.bin and run "apt-cache show rpm".

4.0.4-alt100.11:
3,630,479,716  PROGRAM TOTALS
399,285,303  ???:0x0000000000033150'2 [libc-2.11.3.so]
319,106,185  ???:strlen [ld-2.11.3.so]
242,215,488  ???:pkgCache::FindPackage(char const*) [libapt-pkg-libc6.9-6.so.2.1.0]
198,711,478  ???:memcpy [ld-2.11.3.so]
198,594,999  ???:0x0000000000071970 [libc-2.11.3.so]
195,473,223  header.c:regionSwab.clone.2 [librpmdb-4.0.4.so]
135,707,034  ???:strcmp [ld-2.11.3.so]
117,134,836  header.c:dataLength [librpmdb-4.0.4.so]
102,919,244  ???:0x00000000000710f0 [libc-2.11.3.so]
102,291,091  ???:stpcpy [libc-2.11.3.so]
 75,555,843  ???:bsearch [libc-2.11.3.so]
 68,014,424  ???:getenv [libc-2.11.3.so]

this commit:
3,093,300,624  PROGRAM TOTALS
319,106,185  ???:strlen [ld-2.11.3.so]
242,215,488  ???:pkgCache::FindPackage(char const*) [libapt-pkg-libc6.9-6.so.2.1.0]
197,663,758  ???:0x0000000000071970 [libc-2.11.3.so]
185,771,577  header.c:regionSwab.clone.2 [librpmdb-4.0.4.so]
136,263,419  ???:strcmp [ld-2.11.3.so]
111,934,199  ???:memcpy [ld-2.11.3.so]
102,399,027  ???:0x00000000000710f0 [libc-2.11.3.so]
102,291,091  ???:stpcpy [libc-2.11.3.so]
 95,231,406  header.c:dataLength [librpmdb-4.0.4.so]
 76,706,744  ???:0x0000000000033150'2 [libc-2.11.3.so]
 68,359,227  header.c:findEntry [librpmdb-4.0.4.so]
 68,014,424  ???:getenv [libc-2.11.3.so]
2011-01-21 15:24:54 +03:00
Alexey Tourbin
bb9e8fbede header.c (headerLoad): check if the header is already sorted 2011-01-21 03:28:12 +03:00
Alexey Tourbin
1c87e9b34f header.c (dataLength): optimize string array loops 2011-01-21 02:08:49 +03:00
Alexey Tourbin
dfd33329ab header.c (regionSwab): optimize htons and htonl loops 2011-01-21 01:24:06 +03:00
Alexey Tourbin
e80861fd52 4.0.4-alt100.11
- build/files.c (addFile): Replaced myftw() with fts(3).
- build/checkFiles.c: Reimplemented check for unpackaged files using fts(3).
- python: Backported forceArray changes from rpm5.org (Alexander Myltsev).
2011-01-16 10:26:25 +03:00
Alexander Myltsev
d6701e921f python: backport forceArray changes from rpm5. 2011-01-16 10:18:05 +03:00
Alexander Myltsev
909e4fd5f2 Force array return type for RPMTAG_REQUIREFLAGS. 2011-01-16 10:18:02 +03:00
Alexey Tourbin
252c8b7aed build/checkFiles.c: reimplemented checkIntersect() using fiIntersect() 2011-01-15 11:49:07 +03:00
Alexey Tourbin
d3c57d3ee4 build/checkFiles.c: reimplemented checkFiles() using fts(3) 2011-01-15 11:48:33 +03:00
Alexey Tourbin
6528547c06 build/files.c: warn on fts error 2011-01-15 10:58:11 +03:00
Alexey Tourbin
324d08015d build/files.c (addFile): factored addFile1 to save lstat calls
When addFile() is called with file path obtained from specfile,
various checks must be performed, so that only valid paths can
enter the package.  However, when a path is passed down to fts,
it is no longer possible to obtain ill-formed paths.  So a lot
of lstat calls for intermediate path components can be saved.
2011-01-15 10:36:01 +03:00
Alexey Tourbin
161864908a build/files.c (addFile): replaced myftw() with fts(3) 2011-01-15 10:35:58 +03:00
Alexey Tourbin
d3842e8715 build/files.c (addFile): updated from rpm.org 2011-01-15 10:35:18 +03:00
Alexey Tourbin
592b51b3a3 build/files.c: removed ancient fl->prefix stuff 2011-01-14 22:11:52 +03:00
Alexey Tourbin
00ec5a8146 build/files.c: removed obsolete parseForRegexLang 2011-01-14 21:51:45 +03:00
Alexey Tourbin
1db852bde4 4.0.4-alt100.10
- platform.in: fixed %configure options for noarch packages.
2011-01-11 17:01:07 +03:00
Alexey Tourbin
4c9813bac8 platform.in: fixed %configure options for noarch packages
This introduces new macro %_configure_platform which will normally
expand to %_target_platform.  However, for noarch packages, a special
version called %_configure_platform_noarch will be used, which will use
%_host_cpu instead of mock "noarch" cpu.  Suggested by Dmitry V. Levin.

This fixes %configure failures like
checking build system type... Invalid configuration `noarch-alt-linux': machine `noarch-alt' not recognized
2011-01-11 16:28:27 +03:00
Alexey Tourbin
342aef732d 4.0.4-alt100.9
- set.c: Tweak LRU first-time insertion policy.
2011-01-07 07:00:15 +03:00
Alexey Tourbin
fad9df878b set.c: tweak LRU first-time insertion policy
Pushing new elements to the front tends to assign extra weight to that
elements, at the expense of other elements that are already in the cache.
The idea is then to try first-time insertion somewhere in the middle.
Further attempts suggest that the "pivot" should be closer to the end.

Cache performance for "apt-shell <<<unmet", previous commit:
hit=62375 miss=17252

Cache performance for "apt-shell <<<unmet", this commit:
hit=65085 miss=14542
2011-01-07 06:45:38 +03:00
Alexey Tourbin
cc5cafb9d7 4.0.4-alt100.8
- macro.c: Replaced repeated bsearch+qsort calls with custom
  bsearch+memmove-like routine; rpm startup time is now 10x faster.
2011-01-06 06:04:23 +03:00
Alexey Tourbin
05eb59911d macro.c: fixed macroTable corruption in freeArgs 2011-01-06 05:52:17 +03:00
Alexey Tourbin
d6e06cbbd5 build.c: adjust for macroTableSize change 2011-01-06 05:00:53 +03:00
Alexey Tourbin
6370ecd629 macro.c: replaced repeated bsearch+qsort with BSEARCH, 10x faster startup
"rpmquery rpm" callgrind annotations, previous commit:
86,825,432  PROGRAM TOTALS
33,090,616  ???:0x0000000000033080'2
25,635,193  ???:strcmp
11,530,816  ???:compareMacroName
 7,241,913  ???:memcpy
 2,008,600  ???:0x0000000000033080
 1,734,274  ???:sortMacroTable
   774,941  ???:doDefine
   642,062  ???:0x00000000000714c0
   308,897  ???:0x000000000002a7b0
   308,137  ???:do_lookup_x
   259,615  ???:0x0000000000070c50
   245,388  ???:poptReadConfigFile

"rpmquery rpm" callgrind annotations, this commit:
6,249,372  PROGRAM TOTALS
774,941  ???:doDefine
531,291  ???:addMacro
523,434  ???:0x00000000000714c0
388,985  ???:strcmp
308,897  ???:0x000000000002a7b0
308,098  ???:do_lookup_x
245,388  ???:poptReadConfigFile

"rpmquery rpm" repeated 100 times (user time), previous commit:
3.78s

"rpmquery rpm" repeated 100 times (user time), this commit:
0.40s
2011-01-06 04:04:18 +03:00
Alexey Tourbin
f332ec8636 bsearch.h: initial revision 2011-01-06 03:30:42 +03:00
Alexey Tourbin
add6349fea system.h: moved AUTO_REALLOC from depends.h 2011-01-06 03:26:09 +03:00
Alexey Tourbin
523b00db6e depends.c (rpmRangesOverlap): avoid extra strcmp calls 2011-01-06 03:24:14 +03:00
Alexey Tourbin
4d5831af27 strcasecmp.c: enhanced with table lookup 2011-01-06 03:22:55 +03:00
Alexey Tourbin
96d67572ce 4.0.4-alt100.7
- set.c: Reverted Kirill's changes.
- set.c: Applied aggressive optimization techniques (30% speed-up).
2011-01-04 11:14:18 +03:00
Alexey Tourbin
0f99bd9e13 spec: compile set.c with -fprofile-* options
callgrind results for "apt-cache unmet", 4.0.4-alt100.6:
2,215,614,910  PROGRAM TOTALS
1,126,021,045  ???:decode_set
  486,259,420  ???:rpmsetcmp
  147,177,041  ???:strcmp
   79,769,617  ???:strlen
   61,764,819  ???:0x0000000000033080'2
   54,573,169  ???:memcpy
   31,477,402  ???:strcpy
   24,471,167  ???:pkgCache::DepIterator::AllTargets()

callgrind results for "apt-cache unmet", this commit (built in hasher):
1,502,995,357  PROGRAM TOTALS
625,132,308  ???:decode_base62_golomb
374,940,745  ???:rpmsetcmp
 91,031,162  ???:strcmp
 69,615,481  ???:strlen
 61,764,819  ???:0x0000000000033080'2
 30,928,154  ???:memcpy
 26,108,178  ???:strcpy
 24,471,167  ???:pkgCache::DepIterator::AllTargets()

Average user time for "apt-shell <<<unmet", 4.0.4-alt100.6:
2.55s

Average user time for "apt-shell <<<unmet", this commit (built in hasher):
1.72s
2011-01-04 10:51:34 +03:00
Alexey Tourbin
9380d2ec53 tools/Makefile.am: compile setcmp.static, for profiling 2011-01-04 09:31:52 +03:00
Alexey Tourbin
afe9677742 tools/setcmp.c: with argc==1, read versions from stdin 2011-01-04 09:05:29 +03:00
Alexey Tourbin
cf36274cb9 header.h: shut up -Wextra warning
header.h: In function 'headerFreeData':
header.h:705:11: error: comparison between signed and unsigned integer expressions
2011-01-03 09:52:38 +03:00
Alexey Tourbin
42db4e0a9d set.c: final touches 2011-01-03 09:24:15 +03:00
Alexey Tourbin
73adeec07e set.c: cache short deltas
After base62+golomb decoding, most deltas are under 65536 (in Provides
versions, average delta should be around 1536).  So the whole version
can be stored using short deltas, effectively halving memory footprint.
However, this seems to be somewhat slower: per-delta copying and
decode_golomb must be invoked to recover hash values.  On the other
hand, this allows to increase cache size (128 -> 192).  But note that,
with larger cache sizes, LRU linear search will take longer.  So this is
a compromise - and apparently a favourable one.
2011-01-03 09:04:50 +03:00
Alexey Tourbin
b674d2bc31 set.c: tweak delta routines and maskv 2011-01-03 08:21:13 +03:00
Alexey Tourbin
ebe77a46cd set.c: avoid extra strcmp calls in the caching code 2011-01-03 08:21:12 +03:00
Alexey Tourbin
46f3968205 set.c: optimize array access in rpmsetcmp
callgrind results for "apt-cache unmet", 4.0.4-alt100.6:
2,198,298,537  PROGRAM TOTALS
1,115,738,267  lib/set.c:decode_set
  484,035,006  lib/set.c:rpmsetcmp
  143,078,002  ???:strcmp
   79,477,321  ???:strlen
   61,780,572  ???:0x0000000000033080'2
   54,466,947  ???:memcpy
   31,161,399  ???:strcpy
   24,438,336  ???:pkgCache::DepIterator::AllTargets()

callgrind results for "apt-cache unmet", this commit:
1,755,431,664  PROGRAM TOTALS
764,189,271  lib/set.c:decode_base62_golomb
404,493,494  lib/set.c:rpmsetcmp
143,076,968  ???:strcmp
 70,833,953  ???:strlen
 61,780,572  ???:0x0000000000033080'2
 54,466,947  ???:memcpy
 31,161,399  ???:strcpy
 24,438,336  ???:pkgCache::DepIterator::AllTargets()
2011-01-03 08:21:10 +03:00
Alexey Tourbin
d1ad36aef1 set.c: implemented combined base62+golomb decoding routine 2011-01-03 08:20:55 +03:00