IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
This is to address a few problems:
1) When checking RPM_BUILD_ROOT, st_mode test performed by /usr/bin/which
is not quite reliable. Files can be packaged with different %attr mode.
2) When checking RPM_BUILD_ROOT, there could be symbolic links there
which are (not-so-) broken.
3) When checking host system, files like /sbin/init (which is 0700)
are effectively bypassed by /usr/bin/which.
4) There's an ongoing practice of placing shell function libraries
under /usr/bin, e.g. /usr/bin/git-sh-setup. These files are sourced
from within shell scripts and need not be executable at all.
This leads me to the point that permission check, which is performed by
/usr/bin/which, is not needed at all. Note that things are getting more like
contents_index_bin search. And for RPM_BUILD_ROOT, we do not even require
strong stat-wise file existence.
I like this new term: "strong stat-wise file existence".
It's awesome.
There are two possibilities for protection:
1) we should protect at least from very evil shell metacharacters,
like [$*], and also from [:cntrl:] (e.g. newline).
2) we can provide an exhaustive list of characters that are valid
for non-evil pathnames and commands, and issue mandatory warning
if the command or path appears to be evil.
I chose the latter approach.
Valid character range is 'A-Za-z0-9/@=.,:_+-'.
Note that (almost) all files from our base build system
are valid paths:
$ valid='A-Za-z0-9/@=.,:_+-'
$ hsh-run -- rpm -qal |grep "[^$valid]"
/usr/bin/[
/usr/share/man/man1/[.1.bz2
(contains no files)
(contains no files)
$
Later we'll see if the range of valid characters needs to be extended.
(old)
$ /usr/lib/rpm/pkgconfig.req /usr/lib/pkgconfig/gtkextra-2.0.pc
Package gtk+-2.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `gtk+-2.0.pc'
to the PKG_CONFIG_PATH environment variable
Package 'gtk+-2.0', required by 'GtkExtra', not found
$
(new)
$ scripts/pkgconfig.req.in /usr/lib/pkgconfig/gtkextra-2.0.pc
Package gtk+-2.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `gtk+-2.0.pc'
to the PKG_CONFIG_PATH environment variable
Package 'gtk+-2.0', required by 'GtkExtra', not found
pkgconfig.req.in: failed to process /usr/lib/pkgconfig/gtkextra-2.0.pc
$
$ ls /usr/lib/pkgconfig/*.pc >.1
$ ls /usr/lib/pkgconfig/*.pc |file -NF$'\t' -f - |./scripts/pkgconfig.req.files >.2
pkgconfig.req.files: /usr/lib/pkgconfig/libpng.pc: symbolic link to `libpng12.pc'
$ diff -U1 .1 .2
--- .1 2007-08-28 18:26:10 +0400
+++ .2 2007-08-28 18:26:13 +0400
@@ -115,3 +115,2 @@
/usr/lib/pkgconfig/libpcre.pc
-/usr/lib/pkgconfig/libpng.pc
/usr/lib/pkgconfig/libpng12.pc
$
Remember that symlinks are relevant only for find-requires.
There is no such thing as symbolic links in find-provides.
I see that this can add SOME problems if e.g. /usr is relocated
to /storage/usr in the build environment. This is because CanonPath
follows symlinks for dirname.
But I argue that it is safe for hasher, and it fixes some problems
with contents_index search which is used only in the hasher (by default).
I also argue that, even if /usr is relocated, this is not going to be a BIG
problem, because it is not going to produce unmet dependencies (well, most
of the time). This is because 'rpm -qf' will work as expected in that screwed
build environment.
This is actually a DWIM-style hack. It does what we want but I cannot
think of a better name. The idea is that sometimes we want to clean
up path name, possibly following symbolic links, except for the last
component, which we want to keep as is.
$ sh -c '. scripts/functions; CanonPath /etc/init.d/functions'
/etc/rc.d/init.d/functions
$ sh -c '. scripts/functions; CanonPath /usr/bin/../bin/perl'
/usr/bin/perl
$
So actually it does a few different things: 1) prepend $PWD if needed;
2) cleanup dirname; 3) canonicalize dirname with respect to symbolic links.
Now the question is how to process symbolic links which
targets are directories, e.g. /etc/init.d ? My answer is that
both "/etc/init.d" and "/etc/init.d/", as well as "/etc/init.d/."
should yield the same result, which is "/etc/rc.d/init.d".
When something goes not as good as we would like, we should
be able to increase verbosity locally for this particular case.
Old behaviour:
$ sh -efu -c '. /usr/lib/rpm/find-package; RPM_PKG_CONTENTS_INDEX_ALL=/etc/passwd FindPackage myscript /bin/cat'
sh: myscript: checking contents_index_all for /bin/cat
coreutils
$
In this example I pretend that /bin/cat lookup failed against
contents_index_all. The problem here is that, once we know about
a "strange thing" happening, we also want to know the end of the story.
Where does coreutils come from? That is the question.
New behaviour:
$ sh -efu -c '. /usr/lib/rpm/find-package; RPM_PKG_CONTENTS_INDEX_ALL=/etc/passwd FindPackage myscript /bin/cat'
sh: myscript: checking contents_index_all for /bin/cat
sh: myscript: /bin/cat -> coreutils (via rpmdb)
coreutils
$
When "strange things" happen, I increase verbosity locally,
until the "strange thing" is resolved. I use 'local Verbose=Info'
approach to increase verbosity. Since shell has dynamic variables
(with visibility bound to function-call stack), this is just what
we need.
Strange things are:
- in FindByPath:
+ contents_index_all search
- in FindByName:
+ ambiguous contents_index_bin search
(the same command under e.g. /sbin and /usr/bin)
+ ambiguous /usr/bin/which search (ditto)
In FindByPath, we should always check contents_index_bin first.
It is not quite expensive, and we make no assumptions which path
entries it may contain (our contents_index_bin, in addition
to standard *bin/ paths, also has /etc entries).
However, if contents_index_bin lookup fails, we may or may
not want to proceed with very expensive contents_index_all
search. We assume that, if contents_index_bin lookup actually
has take place for some standard *bin/ path, then there is simply
no need to proceed with contents_index_all.
Now we also assume that contents_index_all file can be possibly
gzipped (I use "gzip -cdfq" for "zcat or cat", found in zgrep).
Also increased verbosity (Verbose to Info) for contents_index_all
messages, since it is expensive and it is a means of "last resort"
to finding something strange before giving up, which is quite worth
to note about.
Also increased verbosity for "raw, not found" dependencies,
since they are likely to become unmet.
There's a bug in "sh --rpm-requires" mode:
$ sh --rpm-requires /dev/stdin <<<'exec -a PERL /usr/bin/perl'
executable(-a)
$
Then "-a" is passed to grep as its first argument (search pattern).
Consider R-devel package, which has its own private bin/ directory,
where it keeps certain scripts.
$ rpm -ql R-devel |grep libtool
/usr/lib/R/bin/libtool
$ rpm -ql R-devel |grep texi2dvi
/usr/lib/R/bin/texi2dvi
$
I replace some scripts, such as libtool, with wrappers:
$ tail -1 /usr/lib/R/bin/libtool
exec /usr/bin/libtool "$@"
$
R-devel now has a libtool dependency, generated by shell.req:
$ rpm -qR R-devel |grep libtool
libtool-common
$ rpm -qf /usr/bin/libtool
libtool-common-0.2-alt1
$
However, I want the very same dependency SIMPLY BY PLACING SYMBOLIC LINK
to /usr/bin/libtool. My new script symlinks.req implements this idea.
It works as follows:
1) We check all absolute symbolic links, e.g. -> /usr/bin/libtool.
If symbolic link is absolute, we fetch its value and feed to FindPackage.
Note that FindPackage will first check whether e.g. /usr/bin/libtool is
available under RPM_BUILD_ROOT.
2) We also check all broken symbolic links, e.g. -> ../../../bin/libtool.
The fact that symbolic link is broken means that it cannot be resolved into
buildroot and otherwise must be resolved in the host system after the package
is installed, which is the dependency on where it would point to after install.
This means we must canonicalize the link value, strip RPM_BUILD_ROOT,
and then call FindPackage. Here is what happens:
$ cd `mktemp -d`
$ RPM_BUILD_ROOT=$PWD
$ mkdir -p ./usr/lib/R/bin/
$ ln -s `relative /usr/bin/libtool /usr/lib/R/bin/libtool` $RPM_BUILD_ROOT/usr/lib/R/bin/libtool
$ l $RPM_BUILD_ROOT/usr/lib/R/bin/libtool
lrwxrwxrwx 1 at at 20 Mar 9 22:14 /tmp/.private/at/tmp.AHnBX26473/usr/lib/R/bin/libtool -> ../../../bin/libtool
$ [ -e $RPM_BUILD_ROOT/usr/lib/R/bin/libtool ] || echo broken
broken
$ readlink -vm $RPM_BUILD_ROOT/usr/lib/R/bin/libtool
/tmp/.private/at/tmp.AHnBX26473/usr/bin/libtool
$ to=`!!`; echo ${to#$RPM_BUILD_ROOT}
/usr/bin/libtool
$
And then FindPackage is called with "/usr/bin/libtool".
Now note that if FindPackage is not able to associate e.g. /usr/bin/libtool
with any particular rpm package, it will just place raw dependency on
/usr/bin/libtool. So, this new type of dependencies is a bit dangerous:
basically for each really broken symbolic link there will be an unmet
dependency. On the other hand, this can be viewed as a means of protection
from packages with really broken symbolic links.
If you are frightened with this brand new and dangerous type of dependencies,
please also note that this new type of dependencies is not much more dangerous
than absolute paths to executables in plain shell scripts.
$ /usr/lib/rpm/shell.req -v /dev/stdin <<</i/am/unmet
error: file /i/am/unmet: No such file or directory
shell.req: /dev/stdin: /i/am/unmet -> /i/am/unmet (raw, not found)
/i/am/unmet
$
E.g. if the content index search detected something like
/usr/bin/arpsend -> arpsend
/usr/sbin/arpsend -> vzctl
then we should consult the host system first. There's a good chance
that the right package, either arpsend or vzctl, IS installed, and
other candidate packages are NOT installed. However, if /usr/bin/which
cannot find any candidate, we should reconsult the index again.
There's been a few problems.
1) ~/bin should not be checked.
$ /usr/lib/rpm/shell.req -v /dev/stdin <<<vim
warning: no package provides /home/at/bin/vim
shell.req: /dev/stdin: vim not found (skip)
$
Fortunately /usr/bin/which has --skip-tilde option.
2) Check for /etc/alternatives was missing.
$ PATH=/bin:/usr/bin /usr/lib/rpm/shell.req -v /dev/stdin <<<vim
shell.req: /dev/stdin: vim -> /usr/bin/vim -> vim-X11
vim-console
vim-enhanced (via rpmdb)
vim-X11
vim-console
vim-enhanced
$
This is why FindByName should ultimately call FindByPath.
After this change, it works just fine:
$ scripts/shell.req.in -v /dev/stdin <<<vim
shell.req.in: /dev/stdin: vim -> /usr/bin/vim -> ... (via which)
shell.req.in: /dev/stdin: /usr/bin/vim -> /usr/bin/vim (alternative)
/usr/bin/vim
$
Also enabled /usr/bin/which --all option and added diagnostics
for really ambiguous cases.
Actually if you think about it a few hours or so... you may come to
know that FindByName() should ultimately call FindByPath(). But this
will be the next commit.
Moved old code from find-requires and find-provides to lib.req and lib.prov.
For some reason, also a few python-related lines was removed.
Adapted lib.req and lib.prov to work without RPM_BUILD_ROOT.
In lib.prov, simplified RPM_FINDPROV_LIB_PATH handling (replaced
`lookup_path "$dir" "$PATH"' with `[ -z "${PATH##* $dir *}" ]').
Added lib.req.files and lib.prov.files.
Also enhanced the scripts by using pkg-config --print-errors, because
pkg-config is silent by default. Also added checks for pkg-config exit
status and valid output.
Also changed the order: libdir/pkgconfig:datadir/pkgconfig instead of
datadir/pkgconfig:libdir/pkgconfig, according to pkg-config(1).
See my previous commit. It goes like this:
$ /usr/lib/rpm/shell.req -v /usr/bin/buildreq
shell.req: /usr/bin/buildreq: cat -> /bin/cat -> coreutils (via rpmdb)
shell.req: /usr/bin/buildreq: cmp -> /usr/bin/cmp -> diffutils (via rpmdb)
shell.req: /usr/bin/buildreq: function(Info) not found (skip)
shell.req: /usr/bin/buildreq: function(show_help) not found (skip)
shell.req: /usr/bin/buildreq: function(show_usage) not found (skip)
shell.req: /usr/bin/buildreq: rm -> /bin/rm -> coreutils (via rpmdb)
shell.req: /usr/bin/buildreq: sed -> /bin/sed -> sed (via rpmdb)
coreutils
diffutils
sed
$
I argue that this behaviour, i.e. function(Info) processing, is erroneous.
There are two reasons: 1) it is impossible to resolve function(Info) neither
via FindPackage nor via .provides.sh (the latter has just plain function names);
2) the fact that Info has been detected as function means that Info() function
is defined in the very same file, i.e. /usr/bin/buildreq. This means that
function(Info) is self-provided dependency and thus should not be processed
at all.
Now it works like this:
$ /usr/lib/rpm/shell.req -v /usr/bin/buildreq
shell.req: /usr/bin/buildreq: cat -> /bin/cat -> coreutils (via rpmdb)
shell.req: /usr/bin/buildreq: cmp -> /usr/bin/cmp -> diffutils (via rpmdb)
shell.req: /usr/bin/buildreq: function(Info) not found (skip)
shell.req: /usr/bin/buildreq: function(show_help) not found (skip)
shell.req: /usr/bin/buildreq: function(show_usage) not found (skip)
shell.req: /usr/bin/buildreq: rm -> /bin/rm -> coreutils (via rpmdb)
shell.req: /usr/bin/buildreq: sed -> /bin/sed -> sed (via rpmdb)
coreutils
diffutils
sed
$
It looks like there's a problem with function(Info) here.
Verbosity is a good thing.
This is the dependency generator for #! lines.
shebang.req.files makes sure that only executable scripts are processed.
Here is why I chose the "shebang" name.
From Jargon File (4.3.1, 29 Jun 2001) [jargon]:
shebang /sh*-bang/ n. The character sequence "#!" that frequently
begins executable shell scripts under Unix. Probably derived from "shell
bang" under the influence of American slang "the whole shebang"
(everything, the works).
$ cd /usr/lib/perl5/pod
$ grep -i sharpbang *
perltoc.pod:C<shar>, C<sharpbang>, C<shmattype>, C<shortsize>, C<shrpenv>, C<shsharp>,
$ grep -i shebang *
perl58delta.pod:The command-line options -s and -F are now recognized on the shebang
perlbs2000.pod:BS2000 POSIX doesn't support the shebang notation
perlfaq3.pod:batch file and codify it in C<ALTERNATE_SHEBANG> (see the
perlfaq7.pod:line in your perl script (the "shebang" line) does not contain the
perlfaq7.pod:where you expect it so you need to adjust your shebang line.
perlfaq8.pod:but my shebang line is not the path to perl, so the shell runs the
perlglossary.pod:=item shebang
perljp.pod: "shebang"ѹǡJperlѤscriptΤۤȤɤѹʤѲǽȻפޤ
perlplan9.pod:such as "#!/usr/local/bin/perl". This is known as a shebang
perlplan9.pod:shebang path from config information located in Plan 9
perlport.pod:sub-systems do not support the C<#!> shebang trick for script invocation.
perlport.pod:OS/390 will support the C<#!> shebang trick in release 2.8 and beyond.
perlrun.pod:C<ALTERNATE_SHEBANG> (see the F<dosish.h> file in the source
perltoc.pod:serialization, server, service, setgid, setuid, shared memory, shebang,
perlwin32.pod:Perl scripts on UNIX use the "#!" (a.k.a "shebang") line to
$
1) added scripts/shell.req.files and scripts/shell.prov.files.
scripts/shell.req.files has new patterns for '/usr/bin/env bash'
and '/bin/ash'.
2) adapted scripts/shell.req.in and scripts/shell.prov.in
by using ArgvFileAction. Also made them work with empty RPM_BUILD_ROOT.
Note: now shell.req produces dependencies from .provides.sh even
if the latter is inside RPM_BUILD_ROOT. The reason is that .provides.sh
and the script can reside in two different subpackages (and otherwise,
rpm will optimize out the dependency).
3) removed corresponding old code from autodeps/linux.req.in
and autodeps/linux.prov.in.
This function provides "standard calling conventions" for req/prov methods.
The idea is that the shell script which implements a req/prov method need
not know how exactly it was called and how it should process its arguments.
Instead, it should impelment a function which takes exactly one argument,
which is filename, say MyReq, and finally just call
ArgvFileAction MyReq "$@"
Standard input to argv conversion is then done automatically and transparently.
This will also enable scripts to process real argv arguments, if any,
instead of standard input (but fall back to standard input otherwise).
I also added --help and -v|--verbose options. Surprisingly enough,
the latter increases RPM_SCRIPTS_DEBUG level.
Also added non-intrusive canonicalization of pathnames, hence the name
ArgvFileAction. This is not going to affect rpm-build, but hopefully
this can help when the script is invoked manually.
I want some rpm scripts to be modular, so that e.g. adding new automatic
dependencies does not require ad hoc modifications.
The plan is as follows: I am going to manage methods. Methods
are scripts which e.g. implement particular dependency detection.
The set of methods is defined by its suffix, so that /usr/lib/rpm/*.req
are all "req" methods (the "req" method set).
Hereby I provide two functions which abstract the usage of method sets.
1)
methods=$(SetupMethods SETID WANTED_METHODS)
Returns (prints) the list of active methods identified by SETID.
The SETID argument, according to the above, should be filename suffix.
Performs a check if all WANTED_METHODS are available. WANTED_METHODS is
a space and/or comma separated list of method basenames, e.g. "perl" for
/usr/lib/rpm/perl.req. The list of WANTED_METHODS is also interpreted
according to AutoReqProv tag rules, i.e. "yes" to enable all methods,
"no" to disable all methods, "noperl" to exclude "perl" etc.
When no methods are active, the output string is guaranteed to be empty.
2)
RunMethods SETID "$methods" [CMD...]
Executes method set scripts identified by SETID; the list of methods
"$methods" must be obtained from SetupMethods. CMD can warp method script
execution.
I am going to make certain req/prov scripts work even with empty
RPM_BUILD_ROOT (so that I can analyze the dependencies within the
host system). This probably means that I will remove ValidateBuildRoot
clause from the script. However, since non-empty RPM_BUILD_ROOT still
must be valid, I place the check here.
Just setting -x is very noisy. Here is a better plan. I implement 3
debug levels: verbose (1), debug (2, implies verbose) and -x (3, implies
debug). I move RPM_SCRIPTS_DEBUG test from scripts to scripts/functions,
as well as provide Verbose() and Debug() shell functions for use in scripts.
Furthermore, _scripts_debug macro is now automatically set when rpmbuild
is invoked with --verbose option. Use -vv for debug and -vvv for -x.
