Commit Graph

2412 Commits

Author SHA1 Message Date
bfb4142971 4.0.4-alt159
- rpmio: Lower memory limit on 32-bit systems for xz compression.
- rpmio: Make liblzma errors more verbose.
2020-12-13 02:35:44 +03:00
afe6605589 rpmio: Lower memory limit on 32-bit systems for xz compression
For some systems 128 MiB delta is not enough for `w7T.xzdio`. Found by
experiment that 256 MiB is enough.

Fixes:

  error: create archive failed on file /usr/src/in/source/FlightGear-data-2020.1.2.tar: cpio: write failed - Bad file descriptor
2020-12-13 02:35:37 +03:00
e3d73c95d5 rpmio: Make liblzma errors more verbose
LZMA error codes lost in the depths of IO callbacks. But they could be
informative to user, especially LZMA_MEM_ERROR (5).
2020-12-13 02:35:30 +03:00
4d2baaf881 4.0.4-alt158
- Prevent exceeding 32 bit memory limitations with xz compression.
2020-12-10 19:15:42 +03:00
0b18c0498e prevent exceeding 32 bit memory limitations with multithreaded xz compression
Based on commit a60f36a55 by Per Øyvind Karlsen. Original commit message:

  As 32 bit build suffers under the limitation of 32 bit address space,
  regardless of it's environment would be ie. 64 bit and not have this
  constration, rpm must make sure not to exceed this memory limitation.
  When using multithreaded xz compression, the number of threads used will
  increase the memory usage, making it necessary to check the memory
  required with the number of threads to be used.

  Number of compression threads will therefore be kept reduced untill
  amount of memory required won't exceed this limitation.

  For 32 bit binaries running under 64 bit host environment, where less
  available memory will be reserved for kernel, easing memory constraints,
  determination of this will be done by a combination of checking host
  arch as well as whether 32 bit personality flag is set, thereby still
  allow a sligthly greater memory usage for such cases to avoid
  imposing unnecessatry limitations under such environments.

Changes from the original commit:

- Do not call uname(2).
- Limits are increased.
- Do not call lzma_memlimit_set().
- No message about thread limitation.

Reviewed-by: Dmitry V. Levin <ldv@altlinux.org>
2020-12-10 19:15:42 +03:00
e434333b9a 4.0.4-alt157
- Add support and enable multi-threaded xz payload compression.
- Add affinity aware %getncpus macro.
- platform: change %__nprocs to use %getncpus instead of nproc(1).
2020-12-08 22:49:11 +03:00
117a2e6042 platform.in: change %__nprocs definition to use %getncpus instead of nproc(1)
Both use sched_getaffinity(2), but %getncpus is built-in and saves fork
and execve.

Suggested-by: Dmitry V. Levin <ldv@altlinux.org>
2020-12-08 22:49:11 +03:00
6ffa199751 macros: Enable multi-threaded xz payload compression
`T' (or `T0') meant auto-detect number of threads by `%getncpus' macro.
2020-12-08 22:49:11 +03:00
a4bace1771 Merge branch 'xz-multi-thread' into sisyphus
Multi-thread xz payload compression patches from upstream rpm.
These include new affinity aware %getncpus macro.

* xz-multi-thread:
  Add get_compression_threads and refactor code.
  Add documentation for %getncpus macro
  Add a built-in macro for fetching number of CPUs, affinity aware and all
  Only use multi thread support with liblzma >= 5.2.0
  Add support for multithreaded xz compression
  add support for setting xz memlimit
  add rpmlog(RPMLOG_ERR, ...) for liblzma return codes
2020-12-08 22:48:41 +03:00
Martin Liska
7ebe00643c Add get_compression_threads and refactor code.
Merged-by: Vitaly Chikunov <vt@altlinux.org>

Removed 4 thread limitation for 32-bit systems.
2020-12-08 22:48:19 +03:00
Pavlina Moravcova Varekova
1af22e100d Add documentation for %getncpus macro
Merged-by: Vitaly Chikunov <vt@altlinux.org>
2020-12-08 22:48:05 +03:00
Panu Matilainen
e3bd69c7df Add a built-in macro for fetching number of CPUs, affinity aware and all
There's an increasing number of placing wanting to know the number of
CPU's for parallel processing, and increasingly these things are running
in containers and such where the total host CPU count is not meaningful.

Merged-by: Vitaly Chikunov <vt@altlinux.org>
2020-12-08 22:47:59 +03:00
Florian Festi
0263f6ebbd Only use multi thread support with liblzma >= 5.2.0
Merged-by: Vitaly Chikunov <vt@altlinux.org>
2020-12-08 22:47:50 +03:00
Per Øyvind Karlsen
c23872d713 Add support for multithreaded xz compression
Merged-by: Vitaly Chikunov <vt@altlinux.org>
2020-12-08 22:47:50 +03:00
Per Øyvind Karlsen
222d4ff7fc add support for setting xz memlimit
Merged-by: Vitaly Chikunov <vt@altlinux.org>
2020-12-08 22:47:50 +03:00
Per Øyvind Karlsen
c254b0d63f add rpmlog(RPMLOG_ERR, ...) for liblzma return codes 2020-12-08 22:47:50 +03:00
459b3bc3f8 4.0.4-alt156
- debugedit: Do not 'edit_dwarf2' when just extracting build-id.
- debuginfo: Fix 'warning: File listed twice' for debug sources.
2020-11-15 17:55:01 +03:00
623707e728 debuginfo: Fix 'warning: File listed twice' for debug sources
debugedit sometimes output top level directory, since we do not filter
regular files in process-debuginfo since 309af2d59 ("debuginfo: Create
empty src/debug directories for paths with `..'") we need to filter
them out before passing into %files for debuginfo package.

Co-authored-by: Dmitry V. Levin <ldv@altlinux.org>
Fixes: 309af2d59 ("debuginfo: Create empty src/debug directories for paths with `..'")
2020-11-15 17:54:34 +03:00
0bbb4e84c3 debugedit: Do not 'edit_dwarf2' when just extracting build-id
No need to call edit_dwarf2() if debugedit is invoked just to extract
build-id (with `-i -n'). Otherwise, we will get `DWARF version 0
unhandled' error for compressed .debug files:

  /usr/lib/rpm/debugedit: ./usr/lib/debug/...e.ko.debug: DWARF version 0 unhandled

Partial forward port of: 1f2ad272a ("debugedit: Add -n to avoid recomputing of
build-id").
2020-11-14 18:25:07 +03:00
0993473ed3 4.0.4-alt155
- debugedit: Fix 'Unhandled relocation 0 in .debug_info section' on e2k.
- debuginfo: Fix source paths with `..` by creating appropriate empty dirs
  under `/usr/src/debug` tree (closes: #39175).
2020-11-12 22:24:18 +03:00
309af2d59e debuginfo: Create empty src/debug directories for paths with `..'
Create empty directories under `/usr/src/debug` tree for source paths
containing `..` (for example `../../src` in elfutils). This should fix
debuginfo sources for autotools builds in subdir.

Fixes: https://bugzilla.altlinux.org/39175
2020-11-12 22:08:42 +03:00
bb0b76e759 debugedit: Fix 'Unhandled relocation 0 in .debug_info section' on e2k
Fix parsing Elbrus ELFs. brp-debuginfo error:

  /usr/lib/rpm/debugedit: ./lib/modules/5.4.58-elbrus-def-alt1.5.2/misc/xt_so.ko: Unhandled relocation 0 in .debug_info section

Reported-by: Michael Shigorin <mike@altlinux.org>
Tested-by: Andrey Savchenko <bircoph@altlinux.org>
2020-11-12 18:53:05 +03:00
a2370b4983 4.0.4-alt154
- addReqProv: issue a warning and ignore self-provides,
  providePackageNVR already does the right thing.
2020-11-11 08:00:00 +00:00
128d160548 addReqProv: ignore self-provides
providePackageNVR already does the right thing.
2020-11-11 08:00:00 +00:00
a1f907ca9c 4.0.4-alt153
- debugedit: Copy debugedit with its tests from upstream.
2020-11-10 06:43:20 +03:00
14e97e6b25 tests: Add debugedit tests
Copy reduced tests/Makefile.am from RPM upstream and make tests work.
2020-11-10 06:43:20 +03:00
29ac9fc75a tests/debugedit: Uncompress object files before tests
We have in `*.o` `.debug_*` sections compressed by default, thus,
getting `DWARF version 0 unhandled' error.
2020-11-10 06:43:20 +03:00
018ae0a853 Merge (theirs) branch 'debugedit.at' into sisyphus
rpm:
   git fast-export HEAD -- tests/debugedit.at \
       tests/data/SOURCES/???.c tests/data/SOURCES/*.h > b

rpm-build:
  git fast-import < b
  git br -m master debugedit.at
  git merge -X theirs --allow-unrelated-histories debugedit.at
2020-11-10 06:43:14 +03:00
08c5331f48 debugedit: Build newly imported debugedit 2020-11-09 23:07:25 +03:00
90136608c8 Merge (theirs) branch 'debugedit' into sisyphus
On rpm commit 98b71f7a92e8e9d902e2bfcd6c30af84dd80289b:
  $ git fast-export HEAD -- tools/debugedit.c > a

On rpm-build:
  $ git fast-import < a
  $ git br -m master debugedit
  $ git merge -X theirs --allow-unrelated-histories debugedit
2020-11-09 22:36:32 +03:00
32943fad62 4.0.4-alt152
- debuginfo: Allow kbuilds with the output directory.
- debugedit: Fix 'canonicalization unexpectedly shrank by one character'
  (closes: #39184).
- brp-sign-kmodules: Sign kernel modules after kernel build.
2020-11-06 15:16:45 +03:00
8b68c147ee brp-sign-kmodules: Sign kernel modules after kernel build
Try to work similar to `CONFIG_MODULE_SIG_ALL=y'.
2020-11-06 15:16:45 +03:00
dfc381d732 debugedit: Fix 'canonicalization unexpectedly shrank by one character'
Do not let canonicalization shrink directory name by 1 character. Or else
that would prevent (in-place) changing of Directory Table entries in
.debug_line section, causing following error in brp-debuginfo:

  /usr/lib/rpm/debugedit: canonicalization unexpectedly shrank by one character

Fixes: https://bugzilla.altlinux.org/show_bug.cgi?id=39184
2020-11-06 15:16:45 +03:00
4738619b15 debuginfo: Allow kbuilds with the output directory
Consider another directory level when searching for vmlinux, for kernel
builds in sub-directory using `make O=output_dir'. Also, improve warning
message making it more understandable.
2020-11-06 15:16:45 +03:00
4f5394b5a1 scripts/brp-compress.in: extend the range of autocompressed file types
Autocompress "[^:]*ASCII.* text.*" file types in addition to
"[^:]*troff or preprocessor input text.*".
2020-09-11 08:00:00 +00:00
1357563e49 scripts/brp-compress.in: use sed -rn 2020-09-11 08:00:00 +00:00
0dafa18b37 4.0.4-alt151
- debuginfo: Do not try to use eu-elfcompress if it does not exist.
- debuginfo: Fix adding non-existent files into debuginfo package.
2020-08-28 03:09:00 +03:00
453b0907f5 debuginfo: Do not try to use eu-elfcompress if it does not exist
Reason: ae1ecef8 ("process-debuginfo: Do not call eu-elfcompress if it doesn't exist")
Fixes: 8682814b ("debuginfo: Try to uncompress if debugedit failed to extract sources list")
2020-08-28 03:09:00 +03:00
b9ea232966 debuginfo: Fix adding non-existent files into debuginfo package
Do not add (uncompressed) module (or vmlinux) into debuginfo package if
it does not actually exists.

Due to other bugs there could be compressed kernel module, (which
implies adding uncompressed module into debuginfo), but, uncompressed
version (which brp-debuginfo should create) does not exist, making
build errors such as:

  error: No such file or directory: /usr/src/tmp/kernel-image-std-def-buildroot/usr/lib/debug/lib/modules/5.8.0-std-def-alt1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko

Reported-by: Anton V. Boyarshinov <boyarsh@altlinux.org>
Fixes: 7aa048df ("Generate debuginfo for kernel packages")
2020-08-28 03:09:00 +03:00
506c6c5e4d 4.0.4-alt150
- Removed obsolete ldconfig, update menu, and install info macros.
- Implemented canonicalization of symlink destinations.
- Backported:
  + added a warning on absolute symlinks;
  + disabled calculation of digests for ghost files.
2020-08-24 19:38:47 +00:00
62037838a4 build: canonicalise symlink destinations
The canonicalization is based on the code lifted from tools/debugedit.c

The new behaviour is enabled by default and controlled by
%_canonicalize_symlinks macro.
2020-08-24 19:38:47 +00:00
Panu Matilainen
5f7fe6577f Warn on absolute symlinks at build-time
While supported of course, absolute symlinks are nasty in many ways
so issue a warning to packagers. Update two affected test-cases.
Related to #668.

(cherry picked from rpm.org commit 8359ded6784579b9f2527a09888132034783e1de)
2020-08-24 19:38:47 +00:00
Bernhard M. Wiedemann
7dba4f2bba Do not store digests of ghost files
when the files exist during build time.
The hash will never be used for verification anyway.

This helps making packages build more reproducibly.

(cherry picked from rpm.org commit 8cda485e1150dc6be990a7aa4496da848154cf55)
2020-08-24 19:38:47 +00:00
Panu Matilainen
308bc34ea9 Handle readlink() failure in genCpioListAndHeader() correctly
It might be a rare condition but it /can/ happen, and previously that would cause '\0' written out of bounds (at negative array offset). Also leave room for the terminating '\0' at the end of buffer when calling readlink(), previously a link exactly the size of buffer would've been silently truncated.

(cherry picked from rpm.org commit c499a0b912629eeeabc3554d3e3a95c088e96422)
2020-08-24 19:38:47 +00:00
7bae988692 Remove obsolete install info macros
This removes %install_info and %uninstall_info macros which point
to stabs since texinfo-4.13-alt1 released back in 2009.
2020-08-24 19:38:47 +00:00
c4685d6094 Remove obsolete ldconfig macros
This removes %post_ldconfig, %post_ldconfig_lib, %post_ldconfig_sys,
and %postun_ldconfig macros which were marked as obsolete by commit
da3de4082f back in 2008.
2020-08-24 19:38:47 +00:00
1b3396c94c Remove obsolete update menu macros
This removes %_update_menus_bin, %update_menus, and %clean_menus macros
which were marked as obsolete by commit
c9231cb270 back in 2008.
2020-08-24 19:38:47 +00:00
e66c512280 platform.in: remove @redhat@ and @mandrake@ lines, strip @alt@ prefix
The former are long unused and irrelevant,
the @alt@ prefix is a useless noise.
2020-08-24 19:38:47 +00:00
89f9245180 4.0.4-alt149
- find-debuginfo-files: Fix processing of root directory (closes: #38842).
2020-08-24 22:38:47 +03:00
f946b5c3b5 find-debuginfo-files: Fix processing of root directory
`%dir /' would cause errors (ALT#38842):

  error: No such file or directory: /usr/src/tmp/broken-debug-buildroot/usr/lib/debug/lib/modules
  error: No such file or directory: /usr/src/tmp/broken-debug-buildroot/usr/lib/debug/lib/modules/vmlinux

Reported-by: Vitaly Lipatov <lav@altlinux.org>
BZ: https://bugzilla.altlinux.org/show_bug.cgi?id=38842
2020-08-24 22:11:27 +03:00