IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Aimed at live images at first but should cover installers as well.
This has been brewing for quite some time and while the proper
implementation is considerably more complex (and hard to do)
looks like there's demand for the particular important use case,
namely LiveCDs for Russian users, so this code has been shared
with a few people before merge.
This function's got its argument order chosen for "aesthetical"
reason of $(2) following $(1) in the macros but the logical order
is exactly the opposite: we care for kernel flavour much more than
for module set (which is dependent upon it).
So while silent dropout of kernel-image if KFLAVOURS is set
but KMODULES is empty could be fixed by testing for $(2) only,
it looks like a good time to fix this discrepancy altogether.
This change is done to reduce ambiguity in some cases;
the previous intention has been to ease navigation when
staying in a particular directory, now it's been changed
in favour of convenient toplevel `git grep' in fact.
Both variants have their pros and cons, I just find myself
leaning to this one by now hence the commit. Feel free to
provide constructive criticism :)
Some path-related bitrot has also been fixed while at that.
It appears that manually specified IMAGEDIR, e.g. by adding
IMAGEDIR = ~/out/$(shell date +%Y%m%d)
to ~/.mkimage/profiles.mk, might be problematic due to
missing globbing. Let's make sure the paths are globbed
and directories are created -- since make's wildcard()
returns an empty string when there's nothing there [yet].
The initial approach required some quite involved postprocessing
as described in http://www.altlinux.org/UEFI#HOWTO; after having
ironed out the kinks so that initial EFI support could be merged
into mkimage proper we're better off just using it, eh?
The issue actually hit image.in/Makefile: "metadata" target
in features.in/metadata/lib/50-metadata.mk wasn't reached
even if features.in/build-distro/lib/90-build-distro.mk
would ACK that the "whatever" actions included "metadata";
thus Metadata/pkg-groups.tar wasn't created and the installer
silently failed to install the .base system.
Let's armour the rest of the cases where the order of inclusion
might be important as well.
hsh-initroot leaves the chroot's root directory permissions
as 1775 while these should really be 755 at most; let's fix it
(important for both VE and VM images, useful for rescue/livecd
ones as well -- especially those with an installer onboard).
Another feature suggested by Michael Radyuk (torabora):
some images are known alpha/beta quality, it's more handy
to just state this at the build time than to rename by hand.
As it happens, adding another architecture required almost no changes;
native 32-bit ppc build took only ARCH and a repo, qemu-ppc one still
has problems (/.host/entry hangs while unpacking setup for fakedata).
Proof of concept on a QS22:
$ make ve/bare.tar.gz
** ARCH: ppc
/bin/sh: rpmvercmp: command not found
21:41:01 cleaning up
21:41:03 initializing BUILDDIR: build/
21:41:03 preparing distro config
21:41:05 starting image build (coffee time)
21:42:48 done (1:42)
** image: $TMP/out/bare-20120716-ppc.tar.gz [21M]
mkimage and hasher can make use of qemu to run
non-native binaries while working on the chroots;
thanks kas@, manowar@ and sbolshakov@ for implementing
this functionality as well as providing nice examples
through mkimage-profiles-arm and mkimage-profile-armrootfs.
This required the architecture check to be added since baking
a tarball with "arm" as its specified arch and x86_64 inside
isn't particularly good thing to let slip through; however
the implementation is quite fragile, bugreports and patches
are seriously welcome.
NB: APTCONF evaluation order between lazy make and nimble shell
turned out to be quite a delicate issue in this particular case.
The only thing to be fixed was setarch(8) symlinks assumption
that is correct for x86 but not for ARM.
There's also some hasher(7) setup to be done:
mkdir -p ~/.hasher
echo >> ~/.hasher/config <<-EOF
def_target=arm
#cache_dir=$HOME/tmp # depends on RAM/storage configuration
EOF
...and of course apt(8) should be properly set up too.
An example PoC build on a CM-A510 board (tmpfs):
$ make BRANDING=altlinux-centaurus ve/bare.tar.gz
** ARCH: arm
18:10:45 initializing BUILDDIR: build/
18:10:45 preparing distro config: build/distcfg.mk
18:10:46 starting image build: tail -f build/build.log
18:14:49 done (4:02)
** image: $TMP/out/bare-20120706-arm.tar.gz [23M]
An initial draft of it was done half a year ago but several tricky
thingies had kept the code from showing up as it was rather brittle
and incomplete.
This implementation involves quite a few changes all over the place
but finally works good enough for live and installer images.
Please pay attention to the versions of these packages:
- installer-feature-setup-plymouth (0.3.2-alt1+)
- branding-altlinux-sisyphus (20110706-alt2+ if used)
- plymouth (0.8.3-alt20.git20110406+)
See also:
- http://www.altlinux.org/Branding
- http://www.altlinux.org/Plymouth
This further refines the modular build by making
metadata being a clearly separated feature rather
than having to rely on runtime tests, and also by
moving the code which cares for kernel bits of base
installation (.base list) in a feature of its own.
There's more to it but let's get the ball rolling first.
As duly noted by glebfm@, branding issues need more attention
by now since only stage1/install2 got some of it so far in this
regard. Hence the dedicated feature comes to the rescue
(well no, it doesn't actually mess with rescue!).
Following m-p-d, a more involved default output directory
structure is feasible now:
~/out/name/date/name-date-arch.type
instead of plain
~/out/name-date-arch.type
This particular behaviour can be achieved by passing
SORTDIR='$(IMAGE_NAME)/$(DATE)'; note the single quotes.
Reports are also saved in this resulting structure
albeit the place is still highly debatable.
From now on, non-empty SAVE_PROFILE variable will indicate
the need to carry the particular generated profile inside
the image built from it.
Thanks gns@ for this feature in liveflash.eeepc.
It was briefly mentioned in QUICKSTART but somehow managed
to evade the commandlines provided. And while at it, let's
make errors like this more explicit to avoid extra lookups.
Oh, and fix QUICKSTART so that readers miss the hassle. :)
Thanks Vladimir Karpinsky for pointing this problem out.
The former toplevel Makefile is now toplevel main.mk;
this change allows for multi-target, multi-arch processing
in the current toplevel Makefile.
As the "build" symlink semantics change quite considerably
when one is doing bulk builds (several pruned builddirs might
be useful for comparison), BUILDDIR is now much more likely
to be recreated: the cases when it will persist are when it's
either a single-image build or when the prefix hasn't changed.
There are some more or less subtle bugfixes and enhancements
all over the map as well.
Done within 20111230..20120102 timeframe, actually...
Here we go with postprocessing priorities along the way
as ISO hybridization has to occur before implanting
final MD5 sum (which must happen earlier than e.g. some
external MD5SUM file generation).
Unfortunately proper dependencies aren't applicable here
(though I'd like to be proved wrong on this one).
Please note that this needs propagator > 20101130-alt9
for automatic mode to work (has also been worked around
in gfxboot case with design-bootloader-source-6.0-alt1).
Thanks rom_as@ for asking about the hybrid image status
and helping out with testing.
We've got some parts of it in build-distro feature,
and some went to dev feature for no real reason.
But a bare installer might go without package base,
and LiveCDs other than live-builder might find local
repository useful given aufs2 root overlay.
Now the overall scheme is more straightforward:
- a distro:
+ asks that a package repo be included
+ cares to further add the packages to it
- a repo feature:
+ pulls in sub/main for it to happen
+ provides genbasedir script to create repo metadata
+ supplements live feature with repo configuration
This is a base for "media check" to become available:
using this feature will implant a checksum into the image
so that it can be verified during install.
Also added a test/demo distro/live-isomd5sum target.
For real distros an alterator module is probably due.
There's a recommended version (0.2.0+ currently) and also
the minimal version, 0.1.7, which received the important fixes.
It was proposed by nice antique@ folks IIRC.
Unfortunately the "suboptimal version" warning is pretty modest,
and "minimal version" error will be apparent with DEBUG enabled;
still the latter will terminate the downstream build and leave
a clear message in build.log at any rate.
- toplevel README received some long-needed refactoring
+ lowlevel detail moved, well, to lowlevel READMEs
- reflected more thoroughly that m-p is not about distros anymore
- dropped features.in/00example/README.en: it's already out-of-date
a bit, and there's no perceived need in thorough English docs so far
- wiki article got split into parts and somewhat rewritten, links updated
- mv doc/{CodingStyle,style.txt}
The bigger goal was being able to set up build in a way
that would allow for images (with configs and logs) be
deposited in per-IMAGE_NAME subdirectories of IMAGEDIR;
that's not done yet but a part of it is ready.
NB: in BUILDDIR, symlinking the just-built image is now
replaced with symlinking the IMAGEDIR since its location
is then predictable thus .gitignore-able for further work
on a generated profile, and more documentable as well.
It's not a hard change though, if you miss the image link
just drop me a note (or a commit).
Essentially some more polishing:
- image path extracted from downstream build log;
- extended error/warning regexp a bit so those with
color grep options get even prettier output.
Notes:
- "1024" a magic number (briefly explained when introduced)
moved to a sort of variable;
- "100 lines" for tail(1) is a rule-of-thumb taking into account
typical amount of hasher/mkimage exhaust given GLOBAL_VERBOSE.
Preferences might be somewhat interesting too: while the official
ones shouldn't influence the build result at all, there's no whitelist
so all kinds of weirdness can be stuffed into local config in principle.
That should be diagnosable at least.
If you make distro/live-builder.iso, the result is an image
containing almost everything (short of actual full enough
repository) to rebuild itself. It will attempt to configure
eth0 with DHCP and reach http://ftp.altlinux.org for packages.
RAM requirements start with 2Gb, self-build is accomplished
on a 4Gb host with "make CLEAN=1 distro/live-builder.iso".
Packages required for "make distro/syslinux.iso" get included.
(some due fixups all over the place too)
If the build is (re-)run withing generated profile directory,
the makefiles no longer have ARCH passed by upstream -- the
upstream might be unreachable at that point indeed.
This is unfortunate but we should cope with that.
NB: might be revisited when architecture-specific packagelists emerge
since currently there's no difference in package lists so we can just
re-run the build for a different architecture but this might change...
This is quite a large-scale change since mkimage-profiles got used to
baking distributions over the last year, and virtual environments are
quite different, so e.g. image.in/Makefile had to be split in two with
the main part of it moved into features.in/iso/lib/.
Short overview:
- features.in/Makefile: lib/ support
(supporting VE images requires dynamic modifications
to image.in/Makefile before starting the build;
the most natural way to achieve that seems to use
features mechanism along with makefile include dir)
- packaging format related part moved into features.in/pack
(should be better prepared for diversity either)
- features.in/iso renamed to features.in/build-distro
- features.in/ve renamed to features.in/build-ve
+ NB: these could not be merged as e.g. features.in/build
due to completely different script hooks
- lib/image.mk renamed to lib/build.mk
- image, config, log postprocessing moved downstream
- added a sort of a topping in the form of lib/sugar.mk
- assorted style fixups (like ifeq usage)
- clean.mk: reliability fix (the problem was observed by Oleg Ivanov
and me too but finally it did get the attention quantum)
- reviewed, updated and extended docs
+ QUICKSTART: should be[come] a step-by-step guide
(thanks Leo-sp50 for prodiving feedback)
install2 cleanups:
- functionally indifferent ones: particularly, install2/*/98system's
"mkdir -p /image" was superfluous as it was done by that time already
by sub.in/stage2/image-scripts.d/00stage1
- taken apart, prepared for tags: so far it's a mostly moot change
since the installer cleanup scripts themselves are mostly the same as
preceding 90cleanup was (with some additions corresponding to recent
kernel development); it's still unclear what the mechanism for
configuring the cleanups in effect will be, either directory/package
regex lists or tagged scripts excluded from execution by yet another tag
fixes:
- image.in/Makefile: fix metadata related test; the actual test was
assuming that stage1 kernel means installer, which is not the case
since generic stage2 introduction; oh well
- 85cleanup-lowmem: a "_" too much was the culprit in destroying the
needed translations along with those deemed superfluous; thanks go to
Oleg Ivanov and Lenar Shakirov for finding the bug and proposing the
fix altogether
additions:
- features.in/Makefile: reworked help target; it was rather inaccessible
due to BUILDDIR normally undefined at the time of direct make
invocation, and BUILDDIR is normally defined during normal builds
anyways so let's try it this way.
- README++
daydreams:
- 01-genbasedir: we should drop bzip2 compressed pkglists some day
but see genbasedir and apt-cdrom first, 90-pkg.sh (alterator-pkg)
will fail miserably otherwise
This was done while debugging GLOBAL_CLEANUP_PACKAGES
getting doubled... as it got no hard initial value but
rather was always added to, it appeared that at least
stage2/Makefile would obtain a once-initialized value
from upstream and double it while including distcfg.mk
itself.
It's way less hassle to just proxy the value once.
It was clear that "common" isn't very apt for packages that
will get *everywhere*, and became apparent when the need for
a "base+live packages" variable arrived with powerbutton feature.
So:
- the former COMMON_PACKAGES are now SYSTEM_PACKAGES;
- COMMON_PACKAGES act as "BASE+LIVE_PACKAGES".
Note that SYSTEM_PACKAGES also got factored out from stage2 based
features into stage2 subprofile itself; cleanups were due as well.
Rather minor fixups for things changed in the meanwhile and not
yet (re)documented properly; and a change for memtest feature
to require syslinux feature (the code's been changed to fit
the updated description, actually, and the change is purely
formal as no syslinux alternative is being used/planned so far).
distcfg.mk is nice for tracing the variable values' build-up,
but it's more useful to have a look at the final result sometimes.
So here it is, logged for each build and callable by hand.
MAIN_GROUPS should align better along with MAIN_PACKAGES
and MAIN_LISTS (even if MAIN_ prefix might be suboptimal
given that these packages are essentially extras within
the particular image).
