Commit Graph

60 Commits

Author SHA1 Message Date
Mikhail Efremov
1a7940c86f Change link to the image format.
$(IMAGE_NAME).$(IMAGE_TYPE) -> $(IMAGE_NAME)-latest-$(ARCH).$(IMAGE_TYPE)
2012-06-21 17:12:27 +04:00
Mikhail Efremov
9bd853e88c Add LIBDIR variable.
Allow to put the log files and reports in a custom directory instead
of IMIGEDIR.
2012-06-21 17:11:02 +04:00
Michael Shigorin
8989fc2771 added plymouth feature
An initial draft of it was done half a year ago but several tricky
thingies had kept the code from showing up as it was rather brittle
and incomplete.

This implementation involves quite a few changes all over the place
but finally works good enough for live and installer images.

Please pay attention to the versions of these packages:
- installer-feature-setup-plymouth (0.3.2-alt1+)
- branding-altlinux-sisyphus (20110706-alt2+ if used)
- plymouth (0.8.3-alt20.git20110406+)

See also:
- http://www.altlinux.org/Branding
- http://www.altlinux.org/Plymouth
2012-06-14 17:15:24 +03:00
Michael Shigorin
c83cc111a6 initial kernel and metadata features
This further refines the modular build by making
metadata being a clearly separated feature rather
than having to rely on runtime tests, and also by
moving the code which cares for kernel bits of base
installation (.base list) in a feature of its own.

There's more to it but let's get the ball rolling first.
2012-05-10 21:18:02 +03:00
Michael Shigorin
51f5b88062 initial branding feature
As duly noted by glebfm@, branding issues need more attention
by now since only stage1/install2 got some of it so far in this
regard.  Hence the dedicated feature comes to the rescue
(well no, it doesn't actually mess with rescue!).
2012-05-04 15:57:06 +03:00
Michael Shigorin
6e6f572b43 better output directory and results structuring
Following m-p-d, a more involved default output directory
structure is feasible now:

  ~/out/name/date/name-date-arch.type

instead of plain

  ~/out/name-date-arch.type

This particular behaviour can be achieved by passing
SORTDIR='$(IMAGE_NAME)/$(DATE)'; note the single quotes.

Reports are also saved in this resulting structure
albeit the place is still highly debatable.
2012-04-23 23:35:43 +03:00
Michael Shigorin
74d1e45a1c docs update
Status updates and readability tweaks all over the place.
2012-04-02 18:13:14 +03:00
Michael Shigorin
ece828c0dc image.in/Makefile: output image size too
Checking the image size is one of the usual things
after building it, and there's some media related
testing to be ported from m-p-d as well.
2012-03-29 16:30:05 +03:00
Michael Shigorin
80090d1f1a minor tweaks (no functional changes)
Mostly multiline compactions, but also use/install2/net
got factored out from distro/server-ovz for future reuse.
2012-03-21 15:55:24 +02:00
Michael Shigorin
4355f401db image.in/Makefile: introduced save-profile
From now on, non-empty SAVE_PROFILE variable will indicate
the need to carry the particular generated profile inside
the image built from it.

Thanks gns@ for this feature in liveflash.eeepc.
2012-02-07 22:21:33 +02:00
Michael Shigorin
7a3ab55db4 image.in/Makefile: fixed thinko regression
MKI_VERSION was tested against even before the config.mk
that defines it got included.  Ouch.
2012-02-06 12:58:20 +02:00
Michael Shigorin
0ec7d7cd01 check vigorously for installed mkimage
It was briefly mentioned in QUICKSTART but somehow managed
to evade the commandlines provided.  And while at it, let's
make errors like this more explicit to avoid extra lookups.
Oh, and fix QUICKSTART so that readers miss the hassle. :)

Thanks Vladimir Karpinsky for pointing this problem out.
2012-01-04 22:33:53 +02:00
Michael Shigorin
2f307ffffb main.mk introduced, lib/*.mk partially rewritten
The former toplevel Makefile is now toplevel main.mk;
this change allows for multi-target, multi-arch processing
in the current toplevel Makefile.

As the "build" symlink semantics change quite considerably
when one is doing bulk builds (several pruned builddirs might
be useful for comparison), BUILDDIR is now much more likely
to be recreated: the cases when it will persist are when it's
either a single-image build or when the prefix hasn't changed.

There are some more or less subtle bugfixes and enhancements
all over the map as well.

Done within 20111230..20120102 timeframe, actually...
2012-01-02 21:01:09 +02:00
Michael Shigorin
6fadf7e0e9 initial isohybrid support
Here we go with postprocessing priorities along the way
as ISO hybridization has to occur before implanting
final MD5 sum (which must happen earlier than e.g. some
external MD5SUM file generation).

Unfortunately proper dependencies aren't applicable here
(though I'd like to be proved wrong on this one).

Please note that this needs propagator > 20101130-alt9
for automatic mode to work (has also been worked around
in gfxboot case with design-bootloader-source-6.0-alt1).

Thanks rom_as@ for asking about the hybrid image status
and helping out with testing.
2011-12-16 15:22:28 +02:00
Michael Shigorin
ae7139f8b6 initial repo feature
We've got some parts of it in build-distro feature,
and some went to dev feature for no real reason.

But a bare installer might go without package base,
and LiveCDs other than live-builder might find local
repository useful given aufs2 root overlay.

Now the overall scheme is more straightforward:
- a distro:
  + asks that a package repo be included
  + cares to further add the packages to it
- a repo feature:
  + pulls in sub/main for it to happen
  + provides genbasedir script to create repo metadata
  + supplements live feature with repo configuration
2011-11-19 11:45:59 +02:00
Michael Shigorin
52f6897318 initial isomd5sum feature
This is a base for "media check" to become available:
using this feature will implant a checksum into the image
so that it can be verified during install.

Also added a test/demo distro/live-isomd5sum target.
For real distros an alterator module is probably due.
2011-11-11 19:50:54 +02:00
Michael Shigorin
94d2ca36aa image.in/Makefile: phonytize targets
It is a current convention to prefer clearly phony targets
(see the wiki page) so let's follow it here too.
2011-11-09 21:16:45 +02:00
Michael Shigorin
7a8ac9e7e0 image.in/Makefile: mkimage version check
There's a recommended version (0.2.0+ currently) and also
the minimal version, 0.1.7, which received the important fixes.
It was proposed by nice antique@ folks IIRC.

Unfortunately the "suboptimal version" warning is pretty modest,
and "minimal version" error will be apparent with DEBUG enabled;
still the latter will terminate the downstream build and leave
a clear message in build.log at any rate.
2011-11-08 00:07:38 +02:00
Michael Shigorin
ddf0c5b7c7 full-view docs update
- toplevel README received some long-needed refactoring
  + lowlevel detail moved, well, to lowlevel READMEs
- reflected more thoroughly that m-p is not about distros anymore
- dropped features.in/00example/README.en: it's already out-of-date
  a bit, and there's no perceived need in thorough English docs so far
- wiki article got split into parts and somewhat rewritten, links updated
- mv doc/{CodingStyle,style.txt}
2011-11-07 00:01:36 +02:00
Michael Shigorin
3c1bca535c downstream postprocessing symlinks tweaked
The bigger goal was being able to set up build in a way
that would allow for images (with configs and logs) be
deposited in per-IMAGE_NAME subdirectories of IMAGEDIR;
that's not done yet but a part of it is ready.

NB: in BUILDDIR, symlinking the just-built image is now
replaced with symlinking the IMAGEDIR since its location
is then predictable thus .gitignore-able for further work
on a generated profile, and more documentable as well.
It's not a hard change though, if you miss the image link
just drop me a note (or a commit).
2011-11-07 00:01:36 +02:00
Michael Shigorin
07fdc6dd9b further toplevel logging tweaks
Essentially some more polishing:
- image path extracted from downstream build log;
- extended error/warning regexp a bit so those with
  color grep options get even prettier output.

Notes:
- "1024" a magic number (briefly explained when introduced)
  moved to a sort of variable;
- "100 lines" for tail(1) is a rule-of-thumb taking into account
  typical amount of hasher/mkimage exhaust given GLOBAL_VERBOSE.
2011-11-07 00:01:36 +02:00
Michael Shigorin
cc8982dd7e tweaked variable logging
Preferences might be somewhat interesting too: while the official
ones shouldn't influence the build result at all, there's no whitelist
so all kinds of weirdness can be stuffed into local config in principle.
That should be diagnosable at least.
2011-11-07 00:01:36 +02:00
Michael Shigorin
813571cb10 dump sources.list to build.log
The seemingly obvious feature was proposed by torabora@.
APTCONF complicates things a bit of course...
(and was slightly kludgy btw; fixed)
2011-11-07 00:01:36 +02:00
Michael Shigorin
83392012d4 live-builder distro: your self-hosted livecd
If you make distro/live-builder.iso, the result is an image
containing almost everything (short of actual full enough
repository) to rebuild itself.  It will attempt to configure
eth0 with DHCP and reach http://ftp.altlinux.org for packages.

RAM requirements start with 2Gb, self-build is accomplished
on a 4Gb host with "make CLEAN=1 distro/live-builder.iso".

Packages required for "make distro/syslinux.iso" get included.

(some due fixups all over the place too)
2011-11-06 17:36:09 +02:00
Michael Shigorin
0a8490ad7f logging tweaks for better readability
...and create a symlink to a just-built image from builddir too.
2011-11-04 16:54:41 +02:00
Michael Shigorin
77115ff50b image.in/functions.mk: added ARCH dup
If the build is (re-)run withing generated profile directory,
the makefiles no longer have ARCH passed by upstream -- the
upstream might be unreachable at that point indeed.

This is unfortunate but we should cope with that.

NB: might be revisited when architecture-specific packagelists emerge
since currently there's no difference in package lists so we can just
re-run the build for a different architecture but this might change...
2011-11-04 16:54:41 +02:00
Michael Shigorin
d5a5941f96 official {distro,ve}/* support
This is quite a large-scale change since mkimage-profiles got used to
baking distributions over the last year, and virtual environments are
quite different, so e.g. image.in/Makefile had to be split in two with
the main part of it moved into features.in/iso/lib/.

Short overview:

- features.in/Makefile: lib/ support
  (supporting VE images requires dynamic modifications
  to image.in/Makefile before starting the build;
  the most natural way to achieve that seems to use
  features mechanism along with makefile include dir)

- packaging format related part moved into features.in/pack
  (should be better prepared for diversity either)
- features.in/iso renamed to features.in/build-distro
- features.in/ve  renamed to features.in/build-ve
  + NB: these could not be merged as e.g. features.in/build
    due to completely different script hooks

- lib/image.mk renamed to lib/build.mk
- image, config, log postprocessing moved downstream
- added a sort of a topping in the form of lib/sugar.mk
- assorted style fixups (like ifeq usage)

- clean.mk: reliability fix (the problem was observed by Oleg Ivanov
  and me too but finally it did get the attention quantum)

- reviewed, updated and extended docs
  + QUICKSTART: should be[come] a step-by-step guide
    (thanks Leo-sp50 for prodiving feedback)
2011-11-04 16:54:41 +02:00
Michael Shigorin
720a579690 wash, rinse, repeat
install2 cleanups:

- functionally indifferent ones: particularly, install2/*/98system's
  "mkdir -p /image" was superfluous as it was done by that time already
  by sub.in/stage2/image-scripts.d/00stage1

- taken apart, prepared for tags: so far it's a mostly moot change
  since the installer cleanup scripts themselves are mostly the same as
  preceding 90cleanup was (with some additions corresponding to recent
  kernel development); it's still unclear what the mechanism for
  configuring the cleanups in effect will be, either directory/package
  regex lists or tagged scripts excluded from execution by yet another tag

fixes:

- image.in/Makefile: fix metadata related test; the actual test was
  assuming that stage1 kernel means installer, which is not the case
  since generic stage2 introduction; oh well

- 85cleanup-lowmem: a "_" too much was the culprit in destroying the
  needed translations along with those deemed superfluous; thanks go to
  Oleg Ivanov and Lenar Shakirov for finding the bug and proposing the
  fix altogether

additions:

- features.in/Makefile: reworked help target; it was rather inaccessible
  due to BUILDDIR normally undefined at the time of direct make
  invocation, and BUILDDIR is normally defined during normal builds
  anyways so let's try it this way.

- README++

daydreams:

- 01-genbasedir: we should drop bzip2 compressed pkglists some day
  but see genbasedir and apt-cdrom first, 90-pkg.sh (alterator-pkg)
  will fail miserably otherwise
2011-11-04 16:15:31 +02:00
Michael Shigorin
75fd795210 ensure that distcfg.mk gets included once at most
This was done while debugging GLOBAL_CLEANUP_PACKAGES
getting doubled... as it got no hard initial value but
rather was always added to, it appeared that at least
stage2/Makefile would obtain a once-initialized value
from upstream and double it while including distcfg.mk
itself.

It's way less hassle to just proxy the value once.
2011-11-04 16:15:30 +02:00
Michael Shigorin
774169c6cd COMMON_PACKAGES and SYSTEM_PACKAGES
It was clear that "common" isn't very apt for packages that
will get *everywhere*, and became apparent when the need for
a "base+live packages" variable arrived with powerbutton feature.

So:
- the former COMMON_PACKAGES are now SYSTEM_PACKAGES;
- COMMON_PACKAGES act as "BASE+LIVE_PACKAGES".

Note that SYSTEM_PACKAGES also got factored out from stage2 based
features into stage2 subprofile itself; cleanups were due as well.
2011-11-04 16:15:30 +02:00
Michael Shigorin
fba30e0675 docs update
Rather minor fixups for things changed in the meanwhile and not
yet (re)documented properly; and a change for memtest feature
to require syslinux feature (the code's been changed to fit
the updated description, actually, and the change is purely
formal as no syslinux alternative is being used/planned so far).
2011-11-04 16:15:30 +02:00
Michael Shigorin
69d5133a3b image.in/vars.mk: dump distcfg variables nicely
distcfg.mk is nice for tracing the variable values' build-up,
but it's more useful to have a look at the final result sometimes.
So here it is, logged for each build and callable by hand.
2011-11-04 16:15:30 +02:00
Michael Shigorin
b56df31624 cosmetic fixup (s/GROUPS/MAIN_GROUPS/g)
MAIN_GROUPS should align better along with MAIN_PACKAGES
and MAIN_LISTS (even if MAIN_ prefix might be suboptimal
given that these packages are essentially extras within
the particular image).
2011-11-04 16:15:30 +02:00
Michael Shigorin
3a5b47e712 improved logging during profile creation
- better *_LISTS printout
- somewhat cleaner .base comments
- more comprehensible git log messages
  + special handling for complex subprofiles
  + no need to commit build.log ;-)
2011-11-04 16:15:30 +02:00
Michael Shigorin
2b90dffa6f don't override targets for no reason 2011-11-04 16:15:30 +02:00
Michael Shigorin
f5a8b89381 stage2 based live subprofiles, updated docs
- introduced generic stage2 subprofile (non-standalone)
- ported installer and rescue over to stage2/{install2,rescue}
- initial stage2/live (needs more work for sure)
- use make-initrd-propagator
- updated and somewhat extended doc/

NB: mind #26133, #26134
2011-11-04 16:15:30 +02:00
Michael Shigorin
5569b4fce1 prepare for multiple live images
s,INSTALLER_KFLAVOUR,STAGE1_KFLAVOUR,g
s,INSTALLER_KMODULES,STAGE1_KMODULES,g

install2 isn't the only livecd around;
so far all of these share the same kernel
in m-p-d, and we'd rather need two+ kernels
with dual-arch media or so; let's not complicate
things too much at this point
2011-11-04 16:15:30 +02:00
Michael Shigorin
878abab5ce clean.mk and friends: introduced CLEAN control variable
make CLEAN=1 will prune workdirs after packing their results,
so disk (or preferably tmpfs) usage will be lower
2011-11-04 16:15:30 +02:00
Michael Shigorin
5f88152223 .base rehashed, DISK_LISTS renamed, etc
In particular:
- .base is now generated from pieces (see image.in/Makefile)
- s/DISTROS/IMAGES/g; s/CONFIGS/DISTROS/g (for clarity)
- s/DISK_LISTS/MAIN_LISTS/g ("disk" was early m-p-d legacy)
- introduced BASE_PACKAGES to complement BASE_LISTS
- minor tweaks to Makefile (ARCH and DATE moved elsewhere)
- libdistro.mk: dropped overlooked IMAGE_INIT_LIST copy
- clean.mk: silly cleanup
2011-11-04 16:15:30 +02:00
Michael Shigorin
006c30a9cd 01-genbasedir: dropped dup cleanup
This was pulled from mkimage-profiles-desktop as is,
and so far is useless (addons, ltsp are out of scope yet).

When reintroduced, it'd better be done as a separate script.
2011-11-04 16:15:29 +02:00
Michael Shigorin
5c571b9af4 pass squashfs options from stage1 test to install2
This looks a bit weird (two subprofiles become a bit more
tightly coupled) but in fact install2 does depend on stage1,
and if stage1 doesn't create squashcfg.mk then install2 is
just fine with defaults (provided they fit the installer kernel
used).  A proper mkimage test infrastructure might be handy though.

Also there:
- 01-initfs: partial cleanup (bootsplash is obsolete anyways)
- regarding 03-test-kernel's errorlevel test:
  if 0, 1 and 2 need to be distinguished, "!" MUST NOT be used
  as it negates so that 0 becomes 1 and the rest becomes 0
2011-11-04 16:15:29 +02:00
Michael Shigorin
5d2ca2ff63 image.in/Makefile: workaround for alterator-pkg
It would handle regular expressions but choke with
multiple words per line, ouch!
2011-11-04 16:15:29 +02:00
Michael Shigorin
e53b6acb56 installer firmware support
Thanks go to lakostis@ (see #18047) for all the aic94xx reminders
and the final testing of the proposed image.
2011-11-04 16:15:29 +02:00
Michael Shigorin
faafaf12e5 README updates
...and some more cosmetic fixes
2011-11-04 16:15:29 +02:00
Michael Shigorin
c2f70e5c10 kernel and BUILDDIR fixes
- image.in/functions.mk: rework kpackage()
  + it takes two arguments explicitly now: this adds some noise
    for "generic" invocations but is rather less messy with recently
    introduced STAGE1_KFLAVOUR (which in its turn is rather cleaner
    than messing with KFLAVOURS, especially since soemthing changed
    in presumably apt and we can't rely on kernel packages being
    installed in the order formed).
- BUILDDIR/DEBUG related fixes
  + Makefile: BUILDDIR initialization moved to distro.mk
- build.log += git info
2011-11-04 16:15:29 +02:00
Michael Shigorin
3f012958dc server-ovz; KDEFAULT; syslinux features reworked
Renamed server-light.iso into server-ovz.iso to avoid brand dilution
and confusion (rider@'s server-light rather favours kvm, anyways).

Introduced KDEFAULT: a reliable default kernel chooser knob
since apt's regex ordering proved pretty unreliable.
Spelling things explicitly is better anyways.

SYSLINUX related features undergone pretty major rewrite
(that includes syslinux, hdt and memtest).

The problem to tackle was features.in/syslinux/generate.mk
assuming syslinux and pciids available in build *host* system;
this well might not be the case (or worse yet, those can be
just different).  So now we're a bit less elegant and a bit
more enterprise, stuffing things into chroot and working there.

Bunch of other fixes along the road, including ; to name a few:
- fixed memtest entry (overlooked while renaming SYSLINUX_ITEMS)
- new and shiny doc/CodingStyle
- gfxboot, stage1 target chain, hdt tweaks
- distro.mk rehashed
- README++
- TODO: dropped (integer overflow anyways)
  + actually moved off-tree to reduce commit spam
- s,\.config\.mk,distcfg.mk,g
- doc/profiles.mk.sample: sample ~/.mkimage/profiles.mk
- ...and assorted fixups/additions

Sorry for convoluted commit, this would have been pretty hard to
rework into some really readable shape (and you might be interested
in the original repo's history horrors then, anyways).
2011-11-04 16:15:29 +02:00
Michael Shigorin
d93989a3ff initial logging subsystem
Now we don't dump all the gory details onto thee
but rather log them handsomely and present only
the most important details by default; use DEBUG=1
and/or BUILDLOG=/path/to/file to tweak.

Also introduced IMAGEDIR and IMAGENAME.

The rest:
- ovz-smp is no more
- base-l10n pkglist (suggested by Alexander Kuprin)
- clean.mk:
  + fixed an oversight: the test result erroneously applied
    only to the first command of several ones that should be
    run iff build/ is a symlink
  + drop implicit rules (it's no C code)
- Makefile: more reasonable iso names
- image.in/Makefile: .disk/date too
2011-11-04 16:15:29 +02:00
Michael Shigorin
ace5cb0ff6 assorted fixups
- more thorough build stages inline doc
- quieter BUILDDIR instantiation
- branding-altlinux-sisyphus by default
- prettify installer selection somewhat
- mkimage #24669 fixed upstream
- sub.in/stage1/modules: +drm (as in m-p-d, for KMS)
- tags2lists: generalize a bit
2011-11-04 16:15:29 +02:00
Michael Shigorin
16e763e9ea kernel day
- drop hardwired kernel flavour from pkglists
- today's std-ng lacks aufs, let's switch to un-def
- second assault at KERNEL_PACKAGES_REGEXP
- re-introduced kpackages (builds ok)

PS: base lists: switch to grub, add udev, cleanup
2011-11-04 16:15:29 +02:00
Michael Shigorin
d77e1d8dc8 drop autoconf, a few vars in .mk suffice
A major change in approach largely thanks to discussions
with Alexey Cheusov but also well aligned with my own findings:
autoconf doesn't let the variables to form an inheritance.

And data flow described at http://www.altlinux.org/WhiteLabel
(which in its turn was born thanks to Gavin Henrick of Diva Telecom
and to Alexander Bokovoy of SaM-Solutions) is really dependent on
the existence of such an inheritance.

Also:
- distro.mk += try()
- "hide" special targets
- fixed wrt distro/.{base,init,metaconf}, thx gns@
- README updates
  + added metaconf.mk
  + clarifications
- updated pci.ids location for hdt
2011-11-04 16:15:29 +02:00