Commit Graph

2452 Commits

Author SHA1 Message Date
715065e999 brp: Add brp-strip-lto script
Strip LTO sections and symbols from objects and archives (static
libraries). %brp_strip_none macro is respected.

Reviewed-by: Dmitry V. Levin <ldv@altlinux.org>
2021-08-24 00:52:22 +03:00
8851274e51 4.0.4-alt172
- Fix regression introduced by forgotten file usage.
2021-08-18 01:21:31 +03:00
b893f89a7f Fix regression introduced by forgotten file usage
There was two forgotten file instances. Fix error such as:

  /root/tmp/rpm-tmp.80ZNEF: line 1: /usr/sbin/post_service: No such file or directory
  error: %post(traefik-2.4.14-alt1.x86_64) scriptlet failed, exit status 127
  error: traefik-2.4.14-alt1.x86_64: install failed

Fixes: 5ef5c8ff4 ("Use file4 instead of file")
2021-08-18 01:20:36 +03:00
ffb8342a16 4.0.4-alt171
- Use file4 instead of file.
2021-08-12 13:23:31 +03:00
5ef5c8ff4a Use file4 instead of file
This will allow upgrading system file to v5, leaving rpmbuild using
stable file v4.
2021-08-12 13:23:31 +03:00
Gleb Fotengauer-Malinovskiy
1d03e6395e 4.0.4-alt170
- rpm-build: do not record directory sizes in packages.
2021-07-06 16:18:08 +03:00
Panu Matilainen
0bf0dd0809 Do not record directory sizes in packages
Directory size is an install-time filesystem specific implementation
detail, build-time just doesn't have a clue. If we want to try taking
directory sizes into account, this needs to happen in the disk space
checking at install time.

It also helps making builds a bit more reproducable (PR #229)

(cherry picked from commit 2cf7096ba534b065feb038306c792784458ac9c7)
2021-07-06 16:17:15 +03:00
Arseny Maslennikov
1b8549bf5d 4.0.4-alt169
- Introduced %_smp_build_ncpus macro.
2021-06-08 18:40:15 +03:00
Arseny Maslennikov
87424e1d65 Introduce %_smp_build_ncpus macro
This macro offers %{_smp_mflags}, but without the -j prefix, to signify
the configured number of processor cores to use.
We can use this to pass that number to utilities that do not understand
the -j number syntax, for example:
`cmake --build %_cmake__builddir --verbose --parallel %_smp_build_ncpus'

The name was picked to be compatible with RH:
https://github.com/rpm-software-management/rpm/blob/rpm-4.16.1.3/platform.in#L51
We preserve ALT semantics though.
2021-06-08 18:38:35 +03:00
46c42b83ee 4.0.4-alt168
- rpm-build: added rpm-macros-python3 to requirements.
2021-05-01 08:00:00 +00:00
62370d58b8 spec: add rpm-macros-python3 to rpm-build requirements
This is required to guarantee that no python3 req/prov are accidentally
lost because python3.{req,prov} are not installed.

Unlike rpm-build-python3, rpm-macros-python3 can be installed without
python3, so no extra dependencies are pulled in.

If rpm-build-python3 is not installed but python3.{req,prov}.files
decide that python3.{req,prov} have to be invoked, the latter will issue
an error.
2021-05-01 08:00:00 +00:00
1a750d8e21 4.0.4-alt167
- Use external debugedit.
2021-04-29 22:39:57 +03:00
3a74c95016 Remove internal debugedit
Do not build internal debugedit and its tests. Debugedit is now external
package.
2021-04-29 22:39:57 +03:00
89e3f8b764 Use system provided debugedit
Debugedit is split into a separate package, use it.

References:
  https://sourceware.org/bugzilla/show_bug.cgi?id=27351
  http://lists.rpm.org/pipermail/rpm-ecosystem/2021-February/000734.html
  https://sourceware.org/debugedit/
2021-04-29 22:39:21 +03:00
4ceb43a3b9 4.0.4-alt166
- Moved brp-bytecompile_python and brp-hardlink_pyo_pyc to rpm-build-python.
2021-04-29 12:00:00 +00:00
bb6b349cb4 Move brp-bytecompile_python and brp-hardlink_pyo_pyc to rpm-build-python
If there are .py files to bytecompile, then rpm-build-python has to be
installed, so move these scripts to rpm-build-python where they belong.
2021-04-29 12:00:00 +00:00
efeccf82ff 4.0.4-alt165
- Replaced rpm-build-python with rpm-macros-python in rpm-build requirements.
- Removed python-devel from BuildRequires.
2021-04-28 20:00:00 +00:00
9fd0523dfd spec: apply some specfile hardening
%define _unpackaged_files_terminate_build 1
%define _stripped_files_terminate_build 1
%set_verify_elf_method strict
2021-04-28 20:00:00 +00:00
9299e5cba9 spec: remove python-devel from BR
python-devel is not needed for build.
2021-04-28 20:00:00 +00:00
501ebe3ec0 spec: replace rpm-build-python with rpm-macros-python in rpm-build requirements
The idea is that rpm-macros-python could be installed without python2,
consequently, python2 will no longer be installed by default, but no
python2 req/prov would be lost because of that change because,
if a python2 module package or a package containing a python2 script
would miss to pull in python2, it would fail to build.
2021-04-28 20:00:00 +00:00
0901ca7282 4.0.4-alt164
- ldd.in: fix trace_elf error diagnostics (ldv).
- rpmio: Workaround another liblzma memory allocation failure on armh.
- rpmio+pack: Update rpmio_flags 'T' with actual threads used.
- rpmio: Make xzdio errors more informative.
- rpmio: Fix lzopen_internal mode parsing when 'Tn' is used.
2021-01-16 03:16:55 +03:00
161dbaaeb1 rpmio: Fix lzopen_internal mode parsing when 'Tn' is used
When there is number after "T" (suggested number of threads or "0" for
getncpus), lzopen_internal() mode parser would skip one byte, and when
it's at the end of the string it would then parse undesired garbage from
the memory, making intermittent compression failures.

Fixes: c23872d71 ("Add support for multithreaded xz compression")
Fixes-upstream: 7740d1098.
Upstream-PR: https://github.com/rpm-software-management/rpm/pull/1478
2021-01-16 03:16:55 +03:00
ef8e18f471 rpmio: Make xzdio errors more informative
Mode flags could change from what is specified in defines and errors
could happen before any other descriptive messages would appear, but
it's useful to know some additional info about xz compression to debug
multi-threading memory failures.

Reviewed-by: Arseny Maslennikov <arseny@altlinux.org>
2021-01-16 03:16:55 +03:00
56e441c403 rpmio+pack: Update rpmio_flags 'T' with actual threads used
Useful information for debugging liblzma memory issues.

Reviewed-by: Arseny Maslennikov <arseny@altlinux.org>
Reviewed-by: Dmitry V. Levin <ldv@altlinux.org>
2021-01-16 03:16:55 +03:00
cae4cfe66f rpmio: Fix formatting of liblzma error messages
Add forgotten new lines.

Fixes: c254b0d63 ("add rpmlog(RPMLOG_ERR, ...) for liblzma return codes")
2021-01-16 01:57:20 +03:00
58313570f4 rpmio: Workaround another liblzma memory allocation failure on armh
Liblzma is non-robust on memory allocation problems - while it works
well on 64-bit architectures with plenty of RAM, it's problematic on
32-bit architectures when multi-threading is used.

Try another workaround: instead of just guessing the best delta, move
half of the current (512MiB) delta into 'guard malloc' allocation test
of what LZMA encoder memusage would return.

This is definitely hackish (and vulnerable to toctou), but it's that
much we could do without significantly reworking stream_encoder_mt.

Error message:

  Wrote: /usr/src/RPM/RPMS/armh/libmozjs78-tools-78.0.1-alt2.armh.rpm (w5.lzdio)
  error: lzwrite: lzma error 5
  error: lzclose: lzma error 11
  error: create archive failed on file /usr/src/tmp/libmozjs78-buildroot/usr/lib/debug/usr/lib/libmozjs-78.so.debug: cpio: write failed - Bad file descriptor

Amends: f5fcb8f43 ("Lower memory limit on 32-bit arches for xz compression, again")
Amends: afe660558 ("Lower memory limit on 32-bit systems for xz compression")
Amends: 0b18c0498 ("prevent exceeding 32 bit memory limitations with multithreaded xz compression")
Reviewed-by: Dmitry V. Levin <ldv@altlinux.org>
2021-01-16 01:57:02 +03:00
ca8e8131b4 ldd.in: fix trace_elf error diagnostics 2021-01-11 08:00:00 +00:00
83ce032556 4.0.4-alt163
- Added fixup method: gnuconfig.
2020-12-21 08:00:00 +00:00
5e710ba046 Add fixup method: gnuconfig
Replaces all config.guess and config.sub files outside gnu-config
to symlinks leading to /usr/share/gnu-config/{config.guess,config.sub}.
2020-12-21 08:00:00 +00:00
c2007ceab8 4.0.4-alt162
- Downgrade XZ->LZMA automatically for small payloads (by Alexey Tourbin).
- Added payload compression string diagnostics.
- Fixed exit status of "rpmbuild --version" command.
2020-12-19 22:43:47 +00:00
fde66de855 Fix exit status of rpmbuild --version
Reported-by: Michael Bykov <m.bykov@gmail.com>
2020-12-19 22:43:47 +00:00
b6d1778c72 pack.c: print payload compression string
Given that downgradeLzmaLevel can change the payload compression,
it's useful to have this diagnostics in the output produced by rpmbuild.
2020-12-19 22:43:47 +00:00
Alexey Tourbin
b295de5893 pack.c: downgrade XZ->LZMA automatically for small payloads
Recent changes left downgradeLzmaLevel dysfunctional (because it does
not recognize 'T').  In my view, the only advantage of XZ over LZMA is
that XZ can can split input into blocks and compress them in parallel
(resulting in a speed-up).  Other advantages, such as a checksum, are
immaterial for our purpose (because the package manager must verify the
integrity of a package beforehand).  Therefore, downgradeLzmaLevel will
also downgrade XZ->LZMA automatically, for payloads smaller than
5*dictSize (the default blockSize being 3*dictSize, so that there are
at least two blocks, the second block not too small).
2020-12-19 22:43:47 +00:00
ef75eb0b2e 4.0.4-alt161
- debuginfo: strip --reloc-debug-sections for kernel modules.
- rpmio: Lower memory limit on 32-bit arches for xz compression, again.
- brp-debuginfo: Add debug sources check.
- rpmio: Include rpmmacro.h to fix compilation warning.
2020-12-18 06:27:08 +03:00
9726c71d1e rpmio: Include rpmmacro.h to fix compilation warning
rpmio.c: In function 'get_compression_threads':
rpmio.c:2114:12: warning: implicit declaration of function 'rpmExpandNumeric' [-Wimplicit-function-declaration]
 2114 |  threads = rpmExpandNumeric("%{getncpus}");
      |            ^~~~~~~~~~~~~~~~

Reported-by: Dmitry V. Levin <ldv@altlinux.org>
2020-12-18 06:26:31 +03:00
cd0eca00e9 brp-debuginfo: Add debug sources check
Warn if debuginfo is present, but debug sources aren't.

Reviewed-by: Dmitry V. Levin <ldv@altlinux.org>
2020-12-18 03:45:22 +03:00
457fef1d64 debuginfo: strip --reloc-debug-sections for kernel modules
`--reloc-debug-sections` resolves trivial relocs between debug
sections and designed for use on kernel modules debug (.ko.debug)
reducing their sizes.
2020-12-18 03:45:22 +03:00
f5fcb8f43d rpmio: Lower memory limit on 32-bit arches for xz compression, again
For armh 256 MiB delta is not enough for `w7T.xzdio`.

Fixes:

  [armh] error: lzwrite: lzma error 5
  [armh] error: lzclose: lzma error 11
  [armh] error: create archive failed on file /usr/src/tmp/llvm11.0-buildroot/usr/lib/debug/.build-id/61/30f51d2a9f8b999e986a2f6c027c1266ba2cf9: cpio: write failed - Bad file descriptor

Reviewed-by: Dmitry V. Levin <ldv@altlinux.org>
2020-12-18 03:45:15 +03:00
ac623ad5b7 4.0.4-alt160
- Add a wrapper for install in %install section to skip strip.
2020-12-15 04:05:15 +03:00
a2e3f87824 Add a wrapper for install in %install section to skip strip
`strip' is called by `install -s` that could occur in unpatched
Makefiles, breaking debuginfo generation.

Try to catch `install` invocations via:

- %__install macro used in `make INSTALL=`.
- Prefix PATH with `/usr/libexec/rpm-build` with install and strip
  wrappers.

Suggested-by: Dmitry V. Levin <ldv@altlinux.org>
Reviewed-by: Dmitry V. Levin <ldv@altlinux.org>
2020-12-15 04:05:15 +03:00
bfb4142971 4.0.4-alt159
- rpmio: Lower memory limit on 32-bit systems for xz compression.
- rpmio: Make liblzma errors more verbose.
2020-12-13 02:35:44 +03:00
afe6605589 rpmio: Lower memory limit on 32-bit systems for xz compression
For some systems 128 MiB delta is not enough for `w7T.xzdio`. Found by
experiment that 256 MiB is enough.

Fixes:

  error: create archive failed on file /usr/src/in/source/FlightGear-data-2020.1.2.tar: cpio: write failed - Bad file descriptor
2020-12-13 02:35:37 +03:00
e3d73c95d5 rpmio: Make liblzma errors more verbose
LZMA error codes lost in the depths of IO callbacks. But they could be
informative to user, especially LZMA_MEM_ERROR (5).
2020-12-13 02:35:30 +03:00
4d2baaf881 4.0.4-alt158
- Prevent exceeding 32 bit memory limitations with xz compression.
2020-12-10 19:15:42 +03:00
0b18c0498e prevent exceeding 32 bit memory limitations with multithreaded xz compression
Based on commit a60f36a55 by Per Øyvind Karlsen. Original commit message:

  As 32 bit build suffers under the limitation of 32 bit address space,
  regardless of it's environment would be ie. 64 bit and not have this
  constration, rpm must make sure not to exceed this memory limitation.
  When using multithreaded xz compression, the number of threads used will
  increase the memory usage, making it necessary to check the memory
  required with the number of threads to be used.

  Number of compression threads will therefore be kept reduced untill
  amount of memory required won't exceed this limitation.

  For 32 bit binaries running under 64 bit host environment, where less
  available memory will be reserved for kernel, easing memory constraints,
  determination of this will be done by a combination of checking host
  arch as well as whether 32 bit personality flag is set, thereby still
  allow a sligthly greater memory usage for such cases to avoid
  imposing unnecessatry limitations under such environments.

Changes from the original commit:

- Do not call uname(2).
- Limits are increased.
- Do not call lzma_memlimit_set().
- No message about thread limitation.

Reviewed-by: Dmitry V. Levin <ldv@altlinux.org>
2020-12-10 19:15:42 +03:00
e434333b9a 4.0.4-alt157
- Add support and enable multi-threaded xz payload compression.
- Add affinity aware %getncpus macro.
- platform: change %__nprocs to use %getncpus instead of nproc(1).
2020-12-08 22:49:11 +03:00
117a2e6042 platform.in: change %__nprocs definition to use %getncpus instead of nproc(1)
Both use sched_getaffinity(2), but %getncpus is built-in and saves fork
and execve.

Suggested-by: Dmitry V. Levin <ldv@altlinux.org>
2020-12-08 22:49:11 +03:00
6ffa199751 macros: Enable multi-threaded xz payload compression
`T' (or `T0') meant auto-detect number of threads by `%getncpus' macro.
2020-12-08 22:49:11 +03:00
a4bace1771 Merge branch 'xz-multi-thread' into sisyphus
Multi-thread xz payload compression patches from upstream rpm.
These include new affinity aware %getncpus macro.

* xz-multi-thread:
  Add get_compression_threads and refactor code.
  Add documentation for %getncpus macro
  Add a built-in macro for fetching number of CPUs, affinity aware and all
  Only use multi thread support with liblzma >= 5.2.0
  Add support for multithreaded xz compression
  add support for setting xz memlimit
  add rpmlog(RPMLOG_ERR, ...) for liblzma return codes
2020-12-08 22:48:41 +03:00
Martin Liska
7ebe00643c Add get_compression_threads and refactor code.
Merged-by: Vitaly Chikunov <vt@altlinux.org>

Removed 4 thread limitation for 32-bit systems.
2020-12-08 22:48:19 +03:00