96 Commits

Author SHA1 Message Date
Michael Shigorin
ca6ec25e32 bin/archdep-filter: a debugging note
...just to have it handy when it's in need next time.
2021-12-04 22:12:09 +07:00
Michael Shigorin
b963e9bf47 bin/archdep-filter: cosmetic cleanups
"-a arch" is not requisite either; and having bunches
of empty lines in the resulting pkglists that are user
visible at least within the conventional installer's
alterator-pkg (groups selection) module wouldn't be nice.

I chose to sacrifice empty-line separators for clarity;
the really good cleanup would save *single* empty lines
between chunks of non-empty ones (not at the pkglist's
start or end); feel free to implement that as well.
2021-12-04 22:12:09 +07:00
Michael Shigorin
1b5b309bf8 bin/archdep-filter: implement multi-!matching too
This has been clearly lacking while making the previous commit
but the implementation isn't that clear so let it be a separate
step.

The problem requiring the change in subsequent processors
is that these relied upon "@arch" as a flag to be inspected,
and "pkg@!arch1,arch2" on arch2 needs to take out *all* of that
fragment *including* arch1 mention as well.

Part of the cause is difference in handling: "positive" multi-match
would explode its "client" line into multiple lines to filter down
the pipeline, while "negative" multi-match *has* to keep that line
on a similarly single line (otherwise we'd end up with N-1 of those
slipping past the filter for particular architecture thus defeating
the whole purpose of "negative" matching semantics):

$ echo 'pkg@!E2K,mipsel,riscv64' |
  sed -r  ':loop; s/^((([^@]+@!)[^,]+)+),([a-zA-Z0-9_]+)/\1@!\4/; t loop'
pkg@!E2K@!mipsel@!riscv64

I've tried my best to test this specific change but it still might
introduce a regression in some corner case; feel free to report;
looks like there's a space for improvement in m-p's automated
tests department as well.

So now we can do:

  pkg@!ARCHES1,ARCHES2,arch3,arch4

and have pkg excluded on arches mentioned; the previous approach
could only offer explicit whitelists (not that it was entirely
wrong but then again, we have both ExclusiveArch and ExcludeArch
rpmtags in our spec files).
2021-12-04 22:11:50 +07:00
Michael Shigorin
2bc28a9cae bin/archdep-filter: implement multi-matching
This has been inspired by a few commits that cared
for package availability reasons on a particular
architecture; the problem at hand is that pkglists
might need to include groups of packages that are
(un)available on groups of arches, and tackling that
with plain pkg@arch just results in combinatorial
explosion of that matrix.

Arches are handled one-by-one with a few hardcoded
macro substitutions.

Exploding a "pkg@arch1,arch2" string into a set of:

pkg@arch1
pkg@arch2

with subsequent archdep pruning would do the trick;
so here's another sed oneliner that does just that:

$ echo 'pkg@X86,ARM,ppc64le' |
  sed -r ':loop; s/^((([^@]+@)[^,]+)+),([^,]+)/\1\n\3\4/; t loop'
pkg@X86
pkg@ARM
pkg@ppc64le

See-also: 9601a9e7ce92c7a521fd154f59d8e17524f12a95
See-also: 5581dc91ec6f3330c338995d1cdfbca285298011
See-also: http://stackoverflow.com/a/55781741/561921
2021-11-23 23:49:48 +07:00
Anton Midyukov
b1923f94a3 tar2fs: start partitions from 34 MiB for riscv64
This space is reserved for installation U-Boot.
2021-11-22 22:30:28 +07:00
Dmitriy Terekhin
10b37d45f3 tar2fs: fix syntax of legacy_boot parameter
Parameter "legacyboot" causes an error:
parted: invalid token: legacyboot
2021-09-04 21:52:15 +07:00
Anton Midyukov
7398e449eb bin/check-pkg-list: return error, if lists contain unavailable packages 2021-08-23 20:53:19 +07:00
Anton Midyukov
2c3415ff71 bin/check-pkg-list: clean debug messages
There is little use, but a lot of noise.
2021-08-23 20:53:19 +07:00
Anton Midyukov
d0bc42296d bin/check-pkg-list: only output a list of unavailable packages to build.log 2021-08-23 20:53:19 +07:00
Anton Midyukov
94e25d0ce3 bin/check-pkg-list: fix for use with mkimage-profiles 2021-08-23 20:53:19 +07:00
Anton Midyukov
29349c9aa5 bin/metadep-expander: do not abort build, if metapackage not available
Unavailable metapackages will be removed from the list.
A warning will be displayed in the log.
2021-08-23 20:53:19 +07:00
Anton Midyukov
287adde54c bin/metadep-expander: fix output redirection to /dev/null 2021-08-23 20:53:19 +07:00
Anton Midyukov
edd7cceddb tar2fs: Mark root partition as bootable if extlinux.conf is present
Booting from extlinux.conf is preferred over booting in EFI mode.
2021-07-23 16:02:03 +07:00
Anton Midyukov
1fcd9baad8 tar2fs: Add offset 16 MiB for singleboard PC support
This space can be used by single board for u-boot.
2021-07-23 16:02:03 +07:00
Anton Midyukov
31975900dd pkg.in/lists/Makefile, metadep-expander, doc: Add @META suffix for pkglist items
The @META suffix is used to expand the metapackage into a list.
apt asks for the dependencies of such a package and adds them to the
package list after this metapackage.
2021-07-23 16:01:47 +07:00
Michael Shigorin
ee8892702f tar2fs: fix e2k handling
The missing glob resulted in disk images properly
partitioned only for e2kv3 ("e2k" for historical
reasons in ALT), and v4+ ones only got a single
ext4 partition that wouldn't work with current
MCST's Boot code.
2021-04-28 17:54:19 +07:00
Anton Midyukov
27a675134a tar2fs: Enable secure-boot support for x86_64, add riscv64 support, cleanup 2021-02-01 18:04:40 +07:00
Anton Midyukov
a2c43788d2 tar2fs: Disable os-probe at the time of grub installation
os-probe finds loopback devices.
2021-01-25 19:42:32 +07:00
Anton Midyukov
1782ffdcdc arm-rpi4, build-vm, tar2fs: Add new argument VM_BOOTTYPE for tar2fs
While this argument can only have one "EFI" value.
Thus, the problem of building an image for Raspberry Pi on armh,
where there is no grub-efi, was solved.
2020-11-26 19:52:29 +07:00
Anton Midyukov
b07db07061 tar2fs: Set UUID in extlinux.conf, if exist
extlinux.conf is not tied to the u-boot bootloader and always
needed replace Label with the UUID if the file exist.
2020-09-09 23:20:54 +07:00
Anton Midyukov
b7a390f633 tar2fs: Set flag boot on root part for u-boot bootloader 2020-07-16 02:28:58 +07:00
Anton Midyukov
95ced6d0c4 tar2fs: Added Raspberry Pi 4 support
Since the main goal for aarch64 and armh for us is Raspberry Pi,
it is always worth creating a fat32 partition for these arches.
At least for now.
2020-07-16 02:28:58 +07:00
Anton Midyukov
0f28ee414f build-vm, tar2fs: added VM_PARTTABLE
For Raspberry Pi needed to build grub-efi + MBR images.
2020-05-22 13:34:37 +07:00
Anton Midyukov
f6449ec0ec tar2fs: fixed empty space between section 1 and 2 on ppc64le 2020-04-07 22:35:52 +07:00
Anton Midyukov
fc67e5a8c2 build-vm, tar2fs: get ARCH from command line argument
tar2fs does not run in hasher and without qemu. This causes build
problems for an architecture other than the host.
2020-04-07 22:35:52 +07:00
Anton Midyukov
8117194b02 tar2fs: grub-efi without Secure Boot
If build is performed on a system with EFI Secure Boot enabled,
then grub will not boot even on EFI without Secure Boot.
This problem is caused by the use of the --removable option, which
is not compatible with option --uefi-secure-boot  the current grub.
Option --uefi-secure-boot is enabled automatically if the system is
booted in EFI Secure Boot mode.
2020-04-07 22:35:52 +07:00
Anton Midyukov
4d39e0236d tar2fs: add support efi 2020-04-03 22:56:58 +07:00
Anton Midyukov
d65ae1021b archdep-filter: add @ARM filter for aarch64, arm*
There are use cases when having this common knob would be desirable.
2019-11-01 13:48:54 +03:00
Michael Shigorin
0e9bb9a650 bin/check-pkg-list: imported from m-p-d
I've long wanted to have this nice helper around in m-p too
but might have set the bar too high.

The "prorper integration" idea was to have the script check
just those pkglists which are relevant for a given image IIRC.

Let's toss it in to begin.
2019-10-31 13:15:39 +03:00
Michael Shigorin
fab15ba7a8 bin/archdep-filter: factored out
This code started breeding within pkg.in/*/Makefile,
and it was lacking both E2K (as a generic alias,
not just "e2k or e2kv4") _and_ the ability to negate
selection, as in "everywhere but not on any e2k").

Let's fix all of that at once, and parallelize sed
execution as well; my tests with 1000-line file
containing "a b c d@!E2K e@e2k f g@!X86 h i@IA32"
lines show roughly 3x higher CPU load and lower
execution time (~0.3 sec vs ~0.9 sec on 801-PC).

And turn that code snippet into a proper filter
with inline edit capability (I've dropped the exit
trap as failing to mv signals a disaster anyway).
2019-10-23 14:40:36 +03:00
Anton Midyukov
2ac26880f1 tar2fs: avoid duplicate fstab entry
This commit fixes duplication of root device entry in /etc/fstab
and changes LABEL=ROOT entry with the proper one if found.

Apparently this didn't hurt qemu images much though...
2019-09-23 19:51:57 +03:00
Gleb Fotengauer-Malinovskiy
d4cbbcba76 tar2fs: Add support of ppc64le vm images 2019-08-19 23:31:00 +03:00
Anton Midyukov
0e5e1215f9 tar2fs: switch case for $BOOTLOADER
This allows one not to hope that only the packages of the desired
loader will be installed.
2019-08-19 23:26:54 +03:00
Anton Midyukov
07559bc94a tar2fs: clean fstab
Should be done this way in the first place it seems.
2019-08-19 23:26:22 +03:00
Anton Midyukov
8e1dd12f8e kernel, tar2fs: do not create /boot/.origver 2019-08-19 23:18:31 +03:00
Anton Midyukov
27674e297b build-vm, kernel, tar2fs: make-initrd happens now in build-vm
NB: 07-kernel change breaks multi-kernel setup!

Breaks: 650e92bf7fe7bcec939d4512b275ad843268bba4
2019-08-19 23:16:06 +03:00
Anton Midyukov
d571ab545e tar2fs: implement u-boot support 2019-06-10 20:37:06 +03:00
Michael Shigorin
f8ae619bbb cleanlog, reports.mk: factor the script out
Sometimes REPORT=1 just wasn't passed to make, and the build.log
saved is noisy and harder to compare through diff(1); let's move
the implementation out of the makefile and into a standalone
script so it can be reused like this:

  BUILDDIR=$(sed -rn "s/^mki-image-scripts: export GLOBAL_BUILDDIR='(.*)'$/\1/p" build/build.log
  bin/cleanlog < build/build.log > build/reports/clean.log

See "export GLOBAL_BUILDDIR=" line in the particular log
(grabbing that one from stdin is not exactly trivial though).
2019-06-10 19:47:14 +03:00
Anton Midyukov
12f8d8e2f4 tar2fs: unified partition type selector
Currently, only e2k requires specific disk partitioning.
2019-06-04 17:58:36 +03:00
Anton Midyukov
d55bfec1e9 tar2fs: clean duplicate
INITRD_MODULES is reset further down the script.

Fixes: be75d3c3d24452b05a4ff6cbdfb58092a94a8d32
2019-06-04 17:48:33 +03:00
Mikhail Gordeev
7f1ec09c9b tar2fs: do not chown ROOTFS and WORKDIR
Changing ROOTFS owner causes a problem:
created image's root directory is owned
by user executing make instead of root.

Changing WORKDIR owner is unnecessary
because it will be removed anyways.
2019-03-04 20:23:58 +03:00
Dmitriy Terekhin
be75d3c3d2 need /.host/qemu* in the chroot if qemu is used
The original commit broke system tar2fs use by accidentally
moving TOPDIR definition into a separate shell execution;
thanks iv@ for spotting and fixing it promptly.

Co-authored-by: Ivan A. Melnikov" <iv@altlinux.org>
2019-03-04 19:56:03 +03:00
Ivan A. Melnikov
bedd7cf453 tar2fs: set defaults for mips* 2018-12-25 15:43:21 +03:00
11e1bf73db tar2fs: add grub support
...for vm images; BIOS one tested, EFI one untested yet
but will be required for AArch64 VMs it seems.
2018-12-24 19:33:41 +03:00
Michael Shigorin
c71ff2a39e tar2fs: handle single boot label as default
No sense to "provide" choice when there's none.
2018-10-29 12:46:08 +03:00
Michael Shigorin
27b07b2dca tar2fs: ensure proper cwd when cleaning up
...so that umounts can happen reliably.

Reported-by: Gleb Fotengauer-Malinovskiy <glebfm@altlinux.org>
2018-10-29 12:45:46 +03:00
Michael Shigorin
b31af10c79 tar2fs: generate proper boot.conf for e2k
This one supports multiple kernels (and tweaks boot
labels aiming to simplify target kernel selection
for the CPU at hand) instead of an earlier attempt
to fill in a template.

No need for the template so just drop it.
2018-05-21 22:50:07 +03:00
Michael Shigorin
650e92bf7f build-vm, tar2fs: handle multiple kernels either
This was crucial for e2k rescue image supporting
multiple CPU versions but can be useful elsewhere.
2018-05-21 22:50:00 +03:00
Michael Shigorin
aa7f2d8423 tar2fs: chgrp and failsafe kpartx
The current state made vm images belong to root group,
no reason to not change those to the primary group of
the user building an image.

kpartx -d -s could fail in some circumstances,
make a safety cleanup call more verbose.
2017-08-07 18:00:16 +03:00
Michael Shigorin
35dd200883 tar2fs: add e2k arch support
Somewhat ARM-like with only the bootloader bits
standing out; Elbrus 2000 firmware can read ext2
and boot the kernel directly.
2017-08-02 22:14:09 +03:00