IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
The idea is that rpm-macros-python could be installed without python2,
consequently, python2 will no longer be installed by default, but no
python2 req/prov would be lost because of that change because,
if a python2 module package or a package containing a python2 script
would miss to pull in python2, it would fail to build.
When there is number after "T" (suggested number of threads or "0" for
getncpus), lzopen_internal() mode parser would skip one byte, and when
it's at the end of the string it would then parse undesired garbage from
the memory, making intermittent compression failures.
Fixes: c23872d71 ("Add support for multithreaded xz compression")
Fixes-upstream: 7740d1098.
Upstream-PR: https://github.com/rpm-software-management/rpm/pull/1478
Mode flags could change from what is specified in defines and errors
could happen before any other descriptive messages would appear, but
it's useful to know some additional info about xz compression to debug
multi-threading memory failures.
Reviewed-by: Arseny Maslennikov <arseny@altlinux.org>
Liblzma is non-robust on memory allocation problems - while it works
well on 64-bit architectures with plenty of RAM, it's problematic on
32-bit architectures when multi-threading is used.
Try another workaround: instead of just guessing the best delta, move
half of the current (512MiB) delta into 'guard malloc' allocation test
of what LZMA encoder memusage would return.
This is definitely hackish (and vulnerable to toctou), but it's that
much we could do without significantly reworking stream_encoder_mt.
Error message:
Wrote: /usr/src/RPM/RPMS/armh/libmozjs78-tools-78.0.1-alt2.armh.rpm (w5.lzdio)
error: lzwrite: lzma error 5
error: lzclose: lzma error 11
error: create archive failed on file /usr/src/tmp/libmozjs78-buildroot/usr/lib/debug/usr/lib/libmozjs-78.so.debug: cpio: write failed - Bad file descriptor
Amends: f5fcb8f43 ("Lower memory limit on 32-bit arches for xz compression, again")
Amends: afe660558 ("Lower memory limit on 32-bit systems for xz compression")
Amends: 0b18c0498 ("prevent exceeding 32 bit memory limitations with multithreaded xz compression")
Reviewed-by: Dmitry V. Levin <ldv@altlinux.org>
Recent changes left downgradeLzmaLevel dysfunctional (because it does
not recognize 'T'). In my view, the only advantage of XZ over LZMA is
that XZ can can split input into blocks and compress them in parallel
(resulting in a speed-up). Other advantages, such as a checksum, are
immaterial for our purpose (because the package manager must verify the
integrity of a package beforehand). Therefore, downgradeLzmaLevel will
also downgrade XZ->LZMA automatically, for payloads smaller than
5*dictSize (the default blockSize being 3*dictSize, so that there are
at least two blocks, the second block not too small).
rpmio.c: In function 'get_compression_threads':
rpmio.c:2114:12: warning: implicit declaration of function 'rpmExpandNumeric' [-Wimplicit-function-declaration]
2114 | threads = rpmExpandNumeric("%{getncpus}");
| ^~~~~~~~~~~~~~~~
Reported-by: Dmitry V. Levin <ldv@altlinux.org>
`strip' is called by `install -s` that could occur in unpatched
Makefiles, breaking debuginfo generation.
Try to catch `install` invocations via:
- %__install macro used in `make INSTALL=`.
- Prefix PATH with `/usr/libexec/rpm-build` with install and strip
wrappers.
Suggested-by: Dmitry V. Levin <ldv@altlinux.org>
Reviewed-by: Dmitry V. Levin <ldv@altlinux.org>
For some systems 128 MiB delta is not enough for `w7T.xzdio`. Found by
experiment that 256 MiB is enough.
Fixes:
error: create archive failed on file /usr/src/in/source/FlightGear-data-2020.1.2.tar: cpio: write failed - Bad file descriptor
Based on commit a60f36a55 by Per Øyvind Karlsen. Original commit message:
As 32 bit build suffers under the limitation of 32 bit address space,
regardless of it's environment would be ie. 64 bit and not have this
constration, rpm must make sure not to exceed this memory limitation.
When using multithreaded xz compression, the number of threads used will
increase the memory usage, making it necessary to check the memory
required with the number of threads to be used.
Number of compression threads will therefore be kept reduced untill
amount of memory required won't exceed this limitation.
For 32 bit binaries running under 64 bit host environment, where less
available memory will be reserved for kernel, easing memory constraints,
determination of this will be done by a combination of checking host
arch as well as whether 32 bit personality flag is set, thereby still
allow a sligthly greater memory usage for such cases to avoid
imposing unnecessatry limitations under such environments.
Changes from the original commit:
- Do not call uname(2).
- Limits are increased.
- Do not call lzma_memlimit_set().
- No message about thread limitation.
Reviewed-by: Dmitry V. Levin <ldv@altlinux.org>
- Add support and enable multi-threaded xz payload compression.
- Add affinity aware %getncpus macro.
- platform: change %__nprocs to use %getncpus instead of nproc(1).
Multi-thread xz payload compression patches from upstream rpm.
These include new affinity aware %getncpus macro.
* xz-multi-thread:
Add get_compression_threads and refactor code.
Add documentation for %getncpus macro
Add a built-in macro for fetching number of CPUs, affinity aware and all
Only use multi thread support with liblzma >= 5.2.0
Add support for multithreaded xz compression
add support for setting xz memlimit
add rpmlog(RPMLOG_ERR, ...) for liblzma return codes
There's an increasing number of placing wanting to know the number of
CPU's for parallel processing, and increasingly these things are running
in containers and such where the total host CPU count is not meaningful.
Merged-by: Vitaly Chikunov <vt@altlinux.org>
debugedit sometimes output top level directory, since we do not filter
regular files in process-debuginfo since 309af2d59 ("debuginfo: Create
empty src/debug directories for paths with `..'") we need to filter
them out before passing into %files for debuginfo package.
Co-authored-by: Dmitry V. Levin <ldv@altlinux.org>
Fixes: 309af2d59 ("debuginfo: Create empty src/debug directories for paths with `..'")
No need to call edit_dwarf2() if debugedit is invoked just to extract
build-id (with `-i -n'). Otherwise, we will get `DWARF version 0
unhandled' error for compressed .debug files:
/usr/lib/rpm/debugedit: ./usr/lib/debug/...e.ko.debug: DWARF version 0 unhandled
Partial forward port of: 1f2ad272a ("debugedit: Add -n to avoid recomputing of
build-id").
- debugedit: Fix 'Unhandled relocation 0 in .debug_info section' on e2k.
- debuginfo: Fix source paths with `..` by creating appropriate empty dirs
under `/usr/src/debug` tree (closes: #39175).
Create empty directories under `/usr/src/debug` tree for source paths
containing `..` (for example `../../src` in elfutils). This should fix
debuginfo sources for autotools builds in subdir.
Fixes: https://bugzilla.altlinux.org/39175
