IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Overview of the changes:
- ARM support: separate ext2 /boot, no LILO
- avoid race condition with devmapper
- trap ERR so that -e in shebang doesn't result in extra cleanup hassle
- configurable root filesystem type (ext4 by default)
- jumps through parted hoops
Details:
1. LILO is x86-specific while the rest of the script can be used
to prepare e.g. Marvell ArmadaXP or CuBox images; we can generally
count on uboot supporting ext2 for relatively sane platforms but
not ext4 that would be a better root filesystem performance-wise.
2. Apparently /dev/mapper/loopXpY can be still missing at the time
when kpartx returns and pop up a bit later... sit there, wait
and check for it.
3. If something went wrong with any command of the script it would bail out
due to -e in shebang; it is now better to clean up the loopback device
and its mappings in this situation either.
4. One size doesn't fit all, really.
5. The parted sizing was sloppy as in broken, now it's just half insane.
Someone's decision to stick units and auto-alignment knobs into
a single one was apparently hilarious...
http://www.gnu.org/software/parted/manual/parted.html#unit
Manual loop/dm cleanup is described in documentation just in case.
/boot size meter is suboptimal in terms of additional I/O incurred,
will be most likely rewritten to make use of advance "du -s".
The original mkisofs would only care for the proper ISO9660 image
but we've switched to xorriso which is able to perform the hack
to yield UEFI hybrid images; thus no need for the postprocessing.
Requires mkimage >= 0.2.5 and xorriso (obviously).
Folks have been wondering already whether e.g. t6 breakage
qualifies as a bug; it does indeed unless the "broken" part
depends on the features not available with that branch yet,
e.g. EFI support.
That is, p6/t6 continue to use 3.81 (while providing 3.82
alongside with it), and Sisyphus has moved to 3.82 before
p7/t7. We support both versions by now.
The initial sketch did work but was somewhat redundant
while lacking the knob conveniently change output directory
as well as means to get it cleaned up.
See http://www.opennet.ru/openforum/vsluhforumID3/86239.html#1
for a query that has led to this one; in particular,
- xdm dropped (won't log in root and there are no users yet);
- network is brought up and configured via DHCP by default;
- apt-get works out-of-box;
- default image size is twice the chroot size.
This part of docs was pleading to put it into a small shell
script; it was done to facilitate kas@' debugging efforts
so that qemu-system-ppc might eventually get fixed, thus
livecd-qemu-arch package now "obsoletes" this file.
Another feature suggested by Michael Radyuk (torabora):
some images are known alpha/beta quality, it's more handy
to just state this at the build time than to rename by hand.
The prerequisites for a cleanup after a successful build
were somewhat weird at this point; now the rules are:
- if DEBUG level is more than 1 or CHECK is set, don't do it;
- otherwise if at least one of the following conditions is true:
+ there's more than one target being built in a row;
+ the build was run by e.g. alterator-mkimage;
+ metaprofile directory is read only
...then do a distclean.
If these are still weird or feel unsuitable for profile hacking,
drop me a note (or a patch).
As it happens, adding another architecture required almost no changes;
native 32-bit ppc build took only ARCH and a repo, qemu-ppc one still
has problems (/.host/entry hangs while unpacking setup for fakedata).
Proof of concept on a QS22:
$ make ve/bare.tar.gz
** ARCH: ppc
/bin/sh: rpmvercmp: command not found
21:41:01 cleaning up
21:41:03 initializing BUILDDIR: build/
21:41:03 preparing distro config
21:41:05 starting image build (coffee time)
21:42:48 done (1:42)
** image: $TMP/out/bare-20120716-ppc.tar.gz [21M]
mkimage and hasher can make use of qemu to run
non-native binaries while working on the chroots;
thanks kas@, manowar@ and sbolshakov@ for implementing
this functionality as well as providing nice examples
through mkimage-profiles-arm and mkimage-profile-armrootfs.
This required the architecture check to be added since baking
a tarball with "arm" as its specified arch and x86_64 inside
isn't particularly good thing to let slip through; however
the implementation is quite fragile, bugreports and patches
are seriously welcome.
NB: APTCONF evaluation order between lazy make and nimble shell
turned out to be quite a delicate issue in this particular case.
The only thing to be fixed was setarch(8) symlinks assumption
that is correct for x86 but not for ARM.
There's also some hasher(7) setup to be done:
mkdir -p ~/.hasher
echo >> ~/.hasher/config <<-EOF
def_target=arm
#cache_dir=$HOME/tmp # depends on RAM/storage configuration
EOF
...and of course apt(8) should be properly set up too.
An example PoC build on a CM-A510 board (tmpfs):
$ make BRANDING=altlinux-centaurus ve/bare.tar.gz
** ARCH: arm
18:10:45 initializing BUILDDIR: build/
18:10:45 preparing distro config: build/distcfg.mk
18:10:46 starting image build: tail -f build/build.log
18:14:49 done (4:02)
** image: $TMP/out/bare-20120706-arm.tar.gz [23M]
Minor tweaks to toplevel docs as well as some doc/*.txt,
doc/variables.txt renamed to doc/params.txt, and a brand new
doc/pkglists.txt is added (thanks manowar@ for his considerations).
Raw disk images are convenient and universal
but there are custom formats like Qemu's qcow2
providing additional features, e.g. copy-on-write
or space savings. All of this ultimately belongs
to mkimage but in the mean time has been implemented
here as well.
Yes, mkimage-profiles is now able to build VM disk images.
So far the support is pretty basic:
- a single hard drive image with a single partition/FS
- only stock root password is configurable
- LILO is hardwired as a bootloader
The resulting images tend to boot under qemu/kvm though.
Please see doc/vm.txt for the warning regarding additional
privileges and setup required. This was started back in
February but I still hoped to avoid sudo/privileged helper
(and libguestfs is almost as undistributable as can be)...
Thanks:
- http://blog.quinthar.com/2008/07/building-1gb-bootable-qemu-image-using.html
- Alexey Morarash who reworked that as https://github.com/tuxofil/linsygen
- led@, legion@, vitty@, aen@ for providing advice and inspiration
It might be spottable but not immediately obvious that a feature
lives entirely in features.in/FEATURE and every target it provides
is described in the corresponding config.mk; thanks dkr@ for asking.
Following m-p-d, a more involved default output directory
structure is feasible now:
~/out/name/date/name-date-arch.type
instead of plain
~/out/name-date-arch.type
This particular behaviour can be achieved by passing
SORTDIR='$(IMAGE_NAME)/$(DATE)'; note the single quotes.
Reports are also saved in this resulting structure
albeit the place is still highly debatable.
Multiple ARCHES won't just magically work without
the ability to figure out the correct apt.conf;
fortunately there's just the right example handy
in profiles.mk.sample already.
Thanks glebfm@ for feedback.
The existing implementation would handle kernel differences
just fine but a bit too automatically: if it sees xz support,
that's what will end up being used (and if there's -Xbcj binary
compression filter available for the target platform, it will
be applied unequivocally either).
It's perfectly suitabe for getting fine-tuned release images
but is also a bit too resource-consuming while developing the
image configuration which has no business with its compression.
The one and only knob is SQUASHFS (see doc/variables.txt);
to give an idea of the differences, here are some numbers
for a mostly-binary (43% as per 99-elf-stats) webkiosk livecd
and a rather less so (18%) flightgear one on a dual quad-core
X5570 node (each mksquashfs run used up all the cores):
SQUASHFS | live-webkiosk.iso | live-flightgear.iso
---------+-------------------+---------------------
fast | 3:30 / 130M | 5:11 / 852M
normal * | 3:37 / 100M | 5:35 / 688M
tight | 3:50 / 98M | 6:47 / 683M
Thus if the knob isn't fiddled with, the defaults will allow
for a reasonably fast build of a pretty slim image; if one is
building a release or if a particular image is very sensitive
being close to the media capacity then just add SQUASHFS=tight
and see it a percent or two down on size.
Please note that lzo/gzip-compressed images are also quicker
to uncompress thus further helping with test iterations.
Thanks to led@ and glebfm@ for helpful hints and questions.
- incompatible change (to fix the rather broken early style):
use/syslinux/ui-% is now use/syslinux/ui/%;
- default timeout changed to 9 seconds (long enough and keeps
the countdown in a single figure);
- added totaltimeout of 300 seconds;
- provided live kiosk images with almost-instant boot by default;
...and some other assorted tweaks here and there, sorry.
The purpose is being able to examine particular target interdependency
graph for a given image having been configured to avoid convoluted
dependencies (loops in particular).
The implementation is based on SHELL hook hint by John Graham-Cumming:
http://cmcrossroads.com/ask-mr-make/6535-tracing-rule-execution-in-gnu-make
From what I've read so far, most of the code should run on 3.80;
there seem to have been some bits that are dependent on 3.81
features, but there is not a bit that depends on 3.82+ features
so far.
From now on, non-empty SAVE_PROFILE variable will indicate
the need to carry the particular generated profile inside
the image built from it.
Thanks gns@ for this feature in liveflash.eeepc.
doc/variables.txt was missing the already-existing BUILDLOG
variable description, and ARCHES got added during multi-target
toplevel rewrite. Other minor fixes come as appropriate.
Here we go with postprocessing priorities along the way
as ISO hybridization has to occur before implanting
final MD5 sum (which must happen earlier than e.g. some
external MD5SUM file generation).
Unfortunately proper dependencies aren't applicable here
(though I'd like to be proved wrong on this one).
Please note that this needs propagator > 20101130-alt9
for automatic mode to work (has also been worked around
in gfxboot case with design-bootloader-source-6.0-alt1).
Thanks rom_as@ for asking about the hybrid image status
and helping out with testing.
The idea is to check:
- the reachability of every target
used to build the image in question;
- the availability of all the package lists
and subsequently packages for that image;
- the lack of "dangling" intermediate targets,
features, pkglists, hooks etc.
So far only the first step is implemented --
it's easy and somewhat helpful already for
make CHECK=1 all
CLEAN is so useful and fiddling with .work chroots does
demand knowledge (hsh-shell is handy btw); so unless we
really get our hands dirty, let's spare ours preciouss
tmpfss.
- toplevel README received some long-needed refactoring
+ lowlevel detail moved, well, to lowlevel READMEs
- reflected more thoroughly that m-p is not about distros anymore
- dropped features.in/00example/README.en: it's already out-of-date
a bit, and there's no perceived need in thorough English docs so far
- wiki article got split into parts and somewhat rewritten, links updated
- mv doc/{CodingStyle,style.txt}
This one regulates the build wrapper: if the value is non-empty
then nice(1) and ionice(1) will be attempted so that the build
behaves better in regard to other tasks running on the system.
A few doc/variables.txt updates along the way.
Unfortunately apt configuration is not straightforward at all
regarding being overridden: one can't just provide sources.list
but needs a corresponding apt.conf along with it, and that apt.conf
must disable the SysV-style snippet directories to avoid interesting
side effects of all the things getting overlaid.
So it's not surprising that torabora has asked for an example...
(thanks go to boyarsh@ since I asked him for an example long ago)
Implemented opportunistic alarm support as proposed by torabora;
the actual result depends on readline and/or terminal settings
(read up on "visual bell" vs "audible bell" in case it's wrong).
TODO: this ought to be shifted downstream when proper logging
framework is there.
This is quite a large-scale change since mkimage-profiles got used to
baking distributions over the last year, and virtual environments are
quite different, so e.g. image.in/Makefile had to be split in two with
the main part of it moved into features.in/iso/lib/.
Short overview:
- features.in/Makefile: lib/ support
(supporting VE images requires dynamic modifications
to image.in/Makefile before starting the build;
the most natural way to achieve that seems to use
features mechanism along with makefile include dir)
- packaging format related part moved into features.in/pack
(should be better prepared for diversity either)
- features.in/iso renamed to features.in/build-distro
- features.in/ve renamed to features.in/build-ve
+ NB: these could not be merged as e.g. features.in/build
due to completely different script hooks
- lib/image.mk renamed to lib/build.mk
- image, config, log postprocessing moved downstream
- added a sort of a topping in the form of lib/sugar.mk
- assorted style fixups (like ifeq usage)
- clean.mk: reliability fix (the problem was observed by Oleg Ivanov
and me too but finally it did get the attention quantum)
- reviewed, updated and extended docs
+ QUICKSTART: should be[come] a step-by-step guide
(thanks Leo-sp50 for prodiving feedback)
install2 cleanups:
- functionally indifferent ones: particularly, install2/*/98system's
"mkdir -p /image" was superfluous as it was done by that time already
by sub.in/stage2/image-scripts.d/00stage1
- taken apart, prepared for tags: so far it's a mostly moot change
since the installer cleanup scripts themselves are mostly the same as
preceding 90cleanup was (with some additions corresponding to recent
kernel development); it's still unclear what the mechanism for
configuring the cleanups in effect will be, either directory/package
regex lists or tagged scripts excluded from execution by yet another tag
fixes:
- image.in/Makefile: fix metadata related test; the actual test was
assuming that stage1 kernel means installer, which is not the case
since generic stage2 introduction; oh well
- 85cleanup-lowmem: a "_" too much was the culprit in destroying the
needed translations along with those deemed superfluous; thanks go to
Oleg Ivanov and Lenar Shakirov for finding the bug and proposing the
fix altogether
additions:
- features.in/Makefile: reworked help target; it was rather inaccessible
due to BUILDDIR normally undefined at the time of direct make
invocation, and BUILDDIR is normally defined during normal builds
anyways so let's try it this way.
- README++
daydreams:
- 01-genbasedir: we should drop bzip2 compressed pkglists some day
but see genbasedir and apt-cdrom first, 90-pkg.sh (alterator-pkg)
will fail miserably otherwise
Rather minor fixups for things changed in the meanwhile and not
yet (re)documented properly; and a change for memtest feature
to require syslinux feature (the code's been changed to fit
the updated description, actually, and the change is purely
formal as no syslinux alternative is being used/planned so far).
distcfg.mk is nice for tracing the variable values' build-up,
but it's more useful to have a look at the final result sometimes.
So here it is, logged for each build and callable by hand.
- introduced generic stage2 subprofile (non-standalone)
- ported installer and rescue over to stage2/{install2,rescue}
- initial stage2/live (needs more work for sure)
- use make-initrd-propagator
- updated and somewhat extended doc/
NB: mind #26133, #26134
