Commit Graph

2069 Commits

Author SHA1 Message Date
Alexey Tourbin
e854592c75 rpmio.c: updated lzma compression set-up 2018-06-26 12:42:44 +03:00
Alexey Tourbin
4d777d01d2 changed a few more vars int -> uint32_t/size_t
This is probably the last change of such kind.  There are many other
instances left, and fixing them all is hopeless.  On the other hand,
the way mod 2^32 arithmetic works, although technically not always
well defined, is to our advantage.  I suggest that only user-visible
discrepancies further be fixed.

The following comparison to st_size looks particularly bad.
The code turns out to be unused!

lib/signature.c:
> verifySizeSignature(const char * datafile, int_32 size, /*@out@*/ char * result)
> [...]
>     if (size != st.st_size) {
>         sprintf(result, "Header+Archive size mismatch.\n"
2018-06-26 12:42:44 +03:00
Alexey Tourbin
4970509cb9 header.c: changed RPM_INT32_TYPE fmt %d -> %u
It seems that no integers stored in a header need to be interpreted
as signed integers; rpm.org uses PRIu64 for all RPM_NUMERIC_CLASS data.
2018-06-26 12:42:44 +03:00
Alexey Tourbin
5eb85a7a5e signature.c: change file size fmt %d -> %zu
Some of the preceding code is probably undefined or unspecified behavior,
but there's no easy way to fix it other than rewriting, which I'm not
going to do.  Surprisingly enough, the code just happens to work, due to
a series of mutual cancellations mod 2^32.  As they say in Russian,
the war will write off all.  Likewise, mod 2^32 arithmetic can write off
a multitude of sins (James 5:20).

> static inline rpmRC checkSize(FD_t fd, int siglen, int pad, int datalen)
[...]
>     int delta;
[...]
>     delta = (sizeof(struct rpmlead) + siglen + pad + datalen) - st.st_size;

Here, the expression in parentheses yields a different numeric value
depending on whether datalen is signed or unsigned.  However, when delta
is finally truncated to 32 bits, the result turns out to be the same.

>     switch (delta) {
>     case -32:	/* XXX rpm-4.0 packages */
>     case 32:	/* XXX Legacy headers have a HEADER_IMAGE tag added. */
>     case 0:
2018-06-26 12:42:44 +03:00
Alexey Tourbin
7db1d19fd8 verify.c: fixed RPMVERIFY_FILESIZE for 2G+ files
The diff context is just big enough to see what happens.
RPMTAG_FILESIZES are interpreted as signed integers, and st_size which
has type off_t is also signed.  So the 32-bit file size form the header
gets sign-extended, after which the equality to st_size does not hold.
2018-06-26 12:42:44 +03:00
Alexey Tourbin
355063e53f domd5: fall back to fd read loop on mmap failure
This completes support for building packages with 2G+ files, more or
less (packaging 4G+ files is still a distant goal).  I am able to build
a valid package with a 3G sparse file on both i586 and x86-64.  On i586,
though, such a package cannot be installed with rpm-4.0:

    error: unpacking of archive failed on file /usr/bin/6;5a70a676:
    cpio: write failed - File too large

I believe there is no obvious way to identify and mark such packages,
since rpmlib(LargeFiles) dependency is meant only for 4G+ files, which
require 07070X cpio entries.

Therefore, limited support for handling such packages shall be provided
for rpm-4.0 in branch p8 (there are issues other than simply enabling
AC_SYS_LARGEFILE).
2018-06-26 12:42:44 +03:00
Alexey Tourbin
d410a87e93 configure.ac: enabled large-file support
This turns out to be a bit easier than I thought yesterday.  When mmap
fails in lib/fsm.c, it falls back to file descriptor IO.  So by simply
enabling AC_SYS_LARGEFILE, I'm already able to package a 3G file, which
can be written like this:

perl -e 'seek STDOUT,3<<30,0 and print 1 or die' >out
2018-06-26 12:42:44 +03:00
Alexey Tourbin
3df35b4d63 build/pack.c: automatically downgrade lzma levels 7-9 on smaller inputs 2018-06-26 12:42:44 +03:00
Alexey Tourbin
d85641818b build/pack.c: pre-calculate cpio archive size, fail gracefully on >= 4G
The nearest goal is to be able to package files up to 4G, on both 32-bit
64-bit architectures, provided that cpio payload size also does not
exceed 4G.  More specifically, the limit is 2^32-2, while 2^32-1 remains
special (this is not my own invention, I adopt it from rpm.org).

If the limit is exceeded, rpmbuild should fail gracefully instead of
writing malformed packages with truncated integer tags.

(Supporting 4G+ files is a distant goal - neither rpm-4.0 nor apt-rpm
can handle them yet.  Besides, it's not clear if producing such big
packages is even a good idea, unless one wishes to package p0rn.)
2018-06-26 12:42:44 +03:00
Alexey Tourbin
a1a075caa2 rpmio: fixed Fwrite() truncated return value
This is the reason why files >= 2G could not be added to cpio, even on
64-bit architectures.  Fwrite was called with a buffer >= 2G, but the
return value was truncated to int (this narrowing conversion is probably
undefined behavior already; what happens next, though, is that after
casting to size_t it assumes a very big numeric value of >= 2^64-2^32).
So rpmbuild was failing with an unrelated errno, "cpio: write failed -
Bad file descriptor".

There are other reasons why this still won't work on 32-bit architectures.
For one, rpmbuild mmaps a file before feeding it to cpio, and 32-bit
architectures have a very limited ability, if any, to mmap 2G+ files.
2018-06-26 12:42:44 +03:00
Ivan Zakharyaschev
c58b3a24d0 4.0.4-alt101.M80P.1 - Merged rpm-build-4.0.4-alt102 into p8 (everything except for separating rpm & -build):
[4.0.4-alt102]
  + verify-elf: fixed passing of LD_PRELOAD in verify_unresolved.
  [4.0.4-alt101]
  + verify-elf: made verify_lfs check shared objects too.
  + Relaxed check for gcc package name in {cpp,pkgconfiglib}.req generators.
  [4.0.4-alt100.99]
  + Updated generator of non-LFS functions list.
  [4.0.4-alt100.97]
  + fixup-libraries: fixed recognition of PIEs (ldv@).
  + verify-elf: treat PIEs as executables in the check for unresolved symbols (ldv@).
2017-01-17 16:16:56 +03:00
cd09d0527e [rpm-build 4.0.4-alt102] cherry-picked (skipped the separation of rpm & -build)
- verify-elf: fixed passing of LD_PRELOAD in verify_unresolved.
2017-01-17 16:16:31 +03:00
88c1020672 verify-elf: fix passing of LD_PRELOAD in verify_unresolved
In verify_unresolved(), do not preload requested objects to $elf_ldd
directly because the latter is not an ELF interpreter but a shell
script.  Instead, pass the list of objects that have to be preloaded
to $elf_ldd script which in turn assigns it to LD_PRELOAD variable
in the ELF interpreter invocation.
2017-01-17 16:16:28 +03:00
9515fd6545 rpmio: fix potential buffer overflow in Fdopen
In file included from /usr/include/string.h:648:0,
                 from ../system.h:76,
                 from rpmio.c:6:
In function 'strncat',
    inlined from 'Fdopen' at rpmio.c:3473:5:
/usr/include/bits/string3.h:158:10: warning: call to __builtin___strncat_chk might overflow destination buffer
   return __builtin___strncat_chk (__dest, __src, __len, __bos (__dest));
2017-01-17 16:16:28 +03:00
Gleb Fotengauer-Malinovskiy
8605fa7393 [rpm-build 4.0.4-alt101] cherry-picked (skipped the separation of rpm & -build)
- verify-elf: made verify_lfs check shared objects too.
- Relaxed check for gcc package name in {cpp,pkgconfiglib}.req generators.
- [not for p8] rpm-build: add R: rpmspec.
2017-01-17 16:16:03 +03:00
Gleb Fotengauer-Malinovskiy
fc627b0ef0 Relax check for gcc package name in {cpp,pkgconfiglib}.req generators 2017-01-17 16:11:35 +03:00
Gleb Fotengauer-Malinovskiy
bea326ff93 verify-elf: make verify_lfs check libraries too
Previously, verify_lfs checked if object has valid interpreter, but it
doesn't make sense in case of shared objects.  verify_lfs check is valid
for all ELFs linked with libraries supporting both LFS and non-LFS API
(libc.so and libz.so in our case).
2017-01-17 16:11:27 +03:00
Gleb Fotengauer-Malinovskiy
e6eb33cf4b [rpm-build 4.0.4-alt100.99] cherry-picked (skipped the separation of rpm & -build)
- Fixed non-LFS check in verify-elf.
- [not for p8] Dropped fake provides made for rpm 4.13.0 bootstrap.
2017-01-17 16:10:06 +03:00
Gleb Fotengauer-Malinovskiy
1b41cd0741 Fix generation of verify-elf-non-lfs-funcs.list
If there were both `func' and `func'64, `func' was considered non-LFS.
Unfortunately, that approach missed functions like fts64_* and
readdir64_r.
2017-01-17 16:07:21 +03:00
Ivan Zakharyaschev
22eb7f0c27 4.0.4-alt100.96.M80P.1 - Build for p8. - verify-elf: correctly handle grep's status
...in practically impossible cases.
2016-12-09 03:27:48 +03:00
Ivan Zakharyaschev
f415265c95 Merge branch 'alt', remote-tracking branch 'ldv@ALT/maint' into altM/80P 2016-12-09 03:27:26 +03:00
Ivan Zakharyaschev
2ac0536ff6 verify_rpath(): distinguish grep's failure and a normal valid case
Previously (according to the exit statuses):

0. if something strange was found in $rpath, an error was reported;
1. if nothing strange was found, no error was reported;
2. if grep failed, it looked like the previous case and no error was
reported (as if $rpath was valid).

Now, the pipe (and the whole script) would abort on grep's failure.
2016-12-09 03:23:12 +03:00
Ivan Zakharyaschev
0445f481cc verify_stack(): distinguish failures and a normal status of grep
Now, if something fails, the pipe (and the function, and the whole
script) will fail.

If the exit codes are normal, they are handled according to our logic.

Previously, a failure (which is an exceptional rare case) would lead to:

1. skipping the further exe_stack check and error_strict STACK "$f" 'STACK entry not found'
2. silently skipping the exe_stack check (which was impossible anyway because of the failure)
2016-12-09 03:23:12 +03:00
Ivan Zakharyaschev
fc56b0be97 verify_unresolved(): correctly treat the status of grep in a very unusual case
The pipe (and the function, and the whole script) would fail when the
resulting constructed rpath was empty. But usually at least
$RPM_VERIFY_ELF_LDD_RPATH is not empty, so the bad behavior has not
been observed.
2016-12-09 03:23:12 +03:00
0e2f86b0fb 4.0.4-alt100.97
- fixup-libraries: fixed recognition of PIEs.
- verify-elf: treat PIEs as executables in the check for unresolved symbols.
2016-12-07 11:30:44 +00:00
a4fb4d42e7 verify-elf (verify_unresolved): treat PIEs as executables 2016-12-07 11:30:44 +00:00
8c1f9d589b Move "is this ELF shared object an executable" check to a separate script
* configure.ac (AC_OUTPUT): Add scripts/fixup-libraries.
* rpm-4_0.spec (%files build): Add %_rpmlibdir/is_elf_so_executable.
* scripts/is_elf_so_executable: New file.
* scripts/Makefile.am (EXTRA_DIST, config_SCRIPTS): Add it.
* scripts/fixup-libraries: Rename to scripts/fixup-libraries.in,
use is_elf_so_executable.
2016-12-07 11:30:42 +00:00
a51ea574e9 scripts/fixup-libraries: fix recognition of PIEs 2016-12-07 11:30:25 +00:00
Ivan Zakharyaschev
d9f7deba30 4.0.4-alt100.95.M80P.1 - Build for p8. 2016-12-01 13:40:39 +03:00
Ivan Zakharyaschev
401d9520ba 4.0.4-alt100.96
- verify-elf: don't confuse the initial verify_rpath() in case
  of two RUNPATH/RPATHs (ALT#32826).
2016-11-30 17:50:04 +03:00
Ivan Zakharyaschev
d362bae600 verify-elf made a bit safer: pipes or command substitutions make it fail
1. set -o pipefail
2. don't use command substitutions inside other complex expressions; just assign to a var -- then it would make the script fail
2016-11-30 17:49:59 +03:00
Ivan Zakharyaschev
00b00565ea verify-elf: don't call verify_rpath() under a pipe because the errors are saved in a var
(A pipe makes a subshell and the var is modified there.)
2016-11-30 17:48:24 +03:00
Ivan Zakharyaschev
c0f11587a5 verify-elf: don't confuse verify_rpath() in case of two RUNPATH/RPATHs (ALT#32826) 2016-11-30 17:30:49 +03:00
Ivan Zakharyaschev
91c923f251 verify-elf: tiny cleanup: unify the awk expression for RPATH extraction
Rewritten equivalently, like in lib.req.
2016-11-30 17:09:09 +03:00
Ivan Zakharyaschev
70369c1a8e verify-elf: tiny cleanup: local var declaration moved to the place of use
The other occurence of "rpath" is in the second part of a pipe and is
local to that subshell anyway.
2016-11-30 17:08:09 +03:00
Ivan Zakharyaschev
00635cb1ac 4.0.4-alt100.95
- verify-elf: honor RUNPATH in addition to RPATH, too (like in lib.req).
- (unnoticeable) shell.req: generalize the shebang regexp w.r.t. other
  hardcoded locations of /usr/bin/env. (No need to be too strict here:
  shebang.req should catch bad locations. It's not our job.)
- %distribution: ALT Linux --> ALT (ALT#32707).
2016-11-29 15:12:34 +03:00
Ivan Zakharyaschev
3172abb8c1 %%distribution: ALT Linux --> ALT (ALT#32707) and some minor corrections of old @altlinux.ru addresses 2016-11-29 15:12:34 +03:00
Ivan Zakharyaschev
66f1a4cb8d verify-elf: RUNPATH overrides RPATH for verify_unresolved
From ld.so's documentation about the search order:

Using the directories specified in the DT_RPATH dynamic section
attribute of the binary if present and DT_RUNPATH attribute does not
exist. Use of DT_RPATH is deprecated.
2016-11-29 15:11:46 +03:00
Ivan Zakharyaschev
65509e2763 verify-elf: honor RUNPATH, too (like in lib.req.in:95 and debuginfo.req.in:76)
in addition to RPATH
2016-11-28 22:37:18 +03:00
Ivan Zakharyaschev
e2810bda5c added me to "Copyright (C)" 2016-11-28 22:37:13 +03:00
Ivan Zakharyaschev
d2e5d7dce7 shell.req: generalize the shebang regexp w.r.t. other locations of env
No need to be too strict here: shebang.req would (or should) catch bad
locations. It's not our work.
2016-05-27 16:02:37 +03:00
Ivan Zakharyaschev
ef66de16d5 shell.req: more idiomatic regexp for reading the shebang (use ?)
Compared with a similar regexp in Emacs sources --
http://git.altlinux.org/gears/e/emacs24.git?p=emacs24.git;a=blob;f=lisp/progmodes/sh-script.el;h=4700324d376db79e27d05e5352fff2f7e841a603;hb=HEAD#l1637
: "#![ \t]?\\([^ \t\n]*/bin/env[ \t]\\)?\\([^ \t\n]+\\)"

Conclusion: mostly identical meaning, with the exception of a more
liberal path of env in Emacs.

But ALT Sisyphus has /usr/bin/env only, so I don't see a good reason
to make the regexp more general. (If a different path is written
there, it won't work in ALT Sisyphus.)
2016-05-27 15:19:10 +03:00
Ivan Zakharyaschev
06f844cd6f 4.0.4-alt100.94
- shell.req: use the version of (ba|)sh from the shebang.
- find-{requires,provides}: run all scripts even for empty lists of files.
2016-05-24 23:08:16 +03:00
Ivan Zakharyaschev
f9bf39c2aa shell.req: use the version of (ba|)sh from the shebang 2016-05-24 23:08:04 +03:00
Ivan Zakharyaschev
d13f11fe4f find-provides: removed optimization for empty lists of files
The same reasons as for find-requires:

1. can be buggy in future;
2. give a chance to the methods to generate some special Provides
2016-04-07 13:58:27 +03:00
Ivan Zakharyaschev
0ebfa8f191 find-requires: removed optimization for empty lists of files
Reasons:

1. code clarity (and more robust w.r.t. to future changes which might
lead to bugs, as before)

2. to give all the special methods a chance to print something in the
case of an empty list, too (when needed)
2016-04-06 17:09:38 +03:00
Ivan Zakharyaschev
edf305d42c 4.0.4-alt100.93
- find-requires fixed for packages consisting completely of symlinks.
2016-04-06 03:30:21 +03:00
Ivan Zakharyaschev
4d23ad4702 percolate: removed loop optimization (for code clarity), result won't change 2016-04-06 03:26:17 +03:00
Ivan Zakharyaschev
95ffed16a6 find-requires: it was wrong to check "$tmpdir"/files for emptiness
Since there appeared a special handling for just some kind of
symlinks, "$tmpdir"/files gave only a part of "$tmpdir"/files+types --
the final list of what to analyze.
2016-04-06 03:22:49 +03:00
Ivan Zakharyaschev
e6e202e522 4.0.4-alt100.92
- find-requires: treat symlinks leading to some real content all way
  through the same package as that content.
- find-{requires,provides}: in case of an error in it, the pipe to it
  will be broken (must not change the visible behavior of rpm-build).
2016-04-05 23:26:03 +03:00