Commit Graph

1368 Commits

Author SHA1 Message Date
Michael Shigorin
b52ff4ef91 lib/build.mk: different colours for different outcomes
As was (quite reasonably) asked by someone and me too,
why should a successful build yield a *red* line
(a grep's default)?

So now it's new and improved, 25% free and so forth:
with a successful build you get a green line, while
errors from a broke one result in red ones.

Clinically tested in both b/w and w/b colour schemes;
in case you're not satisfied, please return original
ANSI_OK and ANSI_FAIL values to the colour dealer
and pass your favourite ones instead.
2011-11-21 13:52:41 +02:00
Michael Shigorin
ad68750855 experimental server-ovz-netinst image
It's now possible to:
- make distro/server-ovz.iso;
- make distro/server-ovz-netinst.iso;
- publish the former image's contents on ftp.linux.kiev.ua;
- boot the latter (~17M) image and enjoy the netinstall ;-)

The catch is that the stage2 (altinst file) location has to be
hardwired into syslinux config snippet for things to happen
automatically -- even if it can be specified manually in case
of failure.

The other catch is that currently a netinstall image is somewhat
tied to the particular image it installs since stage1 kernel and
stage2 modules must correspond strictly (the typical symptoms of
the glitch would be missing mouse driver and weird "permission
denied" errors during an attempt to partition the hard drives).

It might be desirable to provide multi-distro netinstall image...
2011-11-19 22:05:11 +02:00
Michael Shigorin
9cedefdba9 introduced add_feature function
The features might get copy-pasted (or even copied-and-pruned)
when initialized; there's an unneccessary duplication of the
function name in the line adding it to FEATURES list, thus
prone to being forgotten and causing some havoc later on.

It was wrong in the first place but tackling this with some
double-colon rules ran into terminality issues, and further
tortures were considered unneccessary.

The current solution isn't perfect (no completely transparent
function name registration upon corresponding target being called)
but at least it is an improvement...
2011-11-19 11:47:29 +02:00
Michael Shigorin
781c98bb3d server-ovz: rebase onto server-mini
The culprit was missing dhcpcd, although the rest
of niceties already in server-mini were welcome to
server-ovz either.
2011-11-19 11:47:19 +02:00
Michael Shigorin
c4a5515b64 tagged scripts breakage partially fixed
It appears that features.in/Makefile functioned a bit
differently by now than was described back then: after
loops and pushds got rearranged for robustness, it stopped
to pick up a cleanup feature tagged script.

That particular script is now better de-tagged and simply
placed as a script to be merged into install2 subprofile.

The tagged scripts still require a bit more comprehension
to understand the use cases (e.g., do we need per-subprofile
tagged script subdirs or just a toplevel one should be looked
at, with script names telling where to put them).

README used to mix up subprofiles and features; fixed.
2011-11-19 11:45:59 +02:00
Michael Shigorin
ae7139f8b6 initial repo feature
We've got some parts of it in build-distro feature,
and some went to dev feature for no real reason.

But a bare installer might go without package base,
and LiveCDs other than live-builder might find local
repository useful given aufs2 root overlay.

Now the overall scheme is more straightforward:
- a distro:
  + asks that a package repo be included
  + cares to further add the packages to it
- a repo feature:
  + pulls in sub/main for it to happen
  + provides genbasedir script to create repo metadata
  + supplements live feature with repo configuration
2011-11-19 11:45:59 +02:00
Michael Shigorin
7964bccb92 initial systemd feature
Thanks shaba@ for both the package and a wiki page,
and dek@ for the inspiration to finally get to it.
2011-11-19 11:45:50 +02:00
Michael Shigorin
7110ba77de minor pkglist/docs tweaks
server-base += openssh-slients
xorg += xorg-drv-wizardpen
2011-11-13 00:37:45 +02:00
Michael Shigorin
52f6897318 initial isomd5sum feature
This is a base for "media check" to become available:
using this feature will implant a checksum into the image
so that it can be verified during install.

Also added a test/demo distro/live-isomd5sum target.
For real distros an alterator module is probably due.
2011-11-11 19:50:54 +02:00
Michael Shigorin
f257b8e9df stage1/modules: added sdhci
This might be needed to install onto an SD card in a "native"
(non-USB-mediated) SD/MMC cardreader; thanks Vladimir Karpinsky
and gns@ for going over it for liveflash.eeepc case.
2011-11-09 21:20:53 +02:00
Michael Shigorin
94d2ca36aa image.in/Makefile: phonytize targets
It is a current convention to prefer clearly phony targets
(see the wiki page) so let's follow it here too.
2011-11-09 21:16:45 +02:00
Michael Shigorin
b85d82c219 features.in/power: renamed++ from powerbutton
This feature was handling powersave already, so the name
should be changed already.  Thanks sem@ for cpufreq-simple,
there's now a compelling reason for that rename.

Tweaked a few distro recipes accordingly.
2011-11-09 17:56:05 +02:00
Michael Shigorin
e2eef6eb71 gear-store-tags 2011-11-08 00:10:48 +02:00
Michael Shigorin
6782808223 0.4.3.2-alt1
- mkimage version required/checked
2011-11-08 00:08:30 +02:00
Michael Shigorin
7a8ac9e7e0 image.in/Makefile: mkimage version check
There's a recommended version (0.2.0+ currently) and also
the minimal version, 0.1.7, which received the important fixes.
It was proposed by nice antique@ folks IIRC.

Unfortunately the "suboptimal version" warning is pretty modest,
and "minimal version" error will be apparent with DEBUG enabled;
still the latter will terminate the downstream build and leave
a clear message in build.log at any rate.
2011-11-08 00:07:38 +02:00
Michael Shigorin
b3e4d9af0c 0.4.3.1-alt1
- CLEAN by default unless DEBUG
2011-11-07 14:56:09 +02:00
Michael Shigorin
ab70588f94 CLEAN by default (unless DEBUG)
CLEAN is so useful and fiddling with .work chroots does
demand knowledge (hsh-shell is handy btw); so unless we
really get our hands dirty, let's spare ours preciouss
tmpfss.
2011-11-07 12:28:09 +02:00
Michael Shigorin
4313629aac 0.4.3-alt2
- include %mpdir/ itself as well
2011-11-07 11:50:23 +02:00
Michael Shigorin
0301f08e87 gear-store-tags 2011-11-07 00:08:29 +02:00
Michael Shigorin
a91acf3de7 0.4.3-alt1
- enhancements to logging
- NICE variable: employ nice(1) and ionice(1) if available
- features.in/syslinux: banner tweaked to include target name
- features.in/live: set up unicode locale/consolefont
2011-11-07 00:06:54 +02:00
Michael Shigorin
ddf0c5b7c7 full-view docs update
- toplevel README received some long-needed refactoring
  + lowlevel detail moved, well, to lowlevel READMEs
- reflected more thoroughly that m-p is not about distros anymore
- dropped features.in/00example/README.en: it's already out-of-date
  a bit, and there's no perceived need in thorough English docs so far
- wiki article got split into parts and somewhat rewritten, links updated
- mv doc/{CodingStyle,style.txt}
2011-11-07 00:01:36 +02:00
Michael Shigorin
73f676f92c live: set up basic unicode locale
This only deals with console output (a text livecd doesn't ask
the user regarding the preferred language so far).

Refer to these discussions if needed:
https://bugzilla.altlinux.org/show_bug.cgi?id=25225
https://lists.altlinux.org/pipermail/devel-conf/2005-August/001785.html
2011-11-07 00:01:36 +02:00
Michael Shigorin
3c1bca535c downstream postprocessing symlinks tweaked
The bigger goal was being able to set up build in a way
that would allow for images (with configs and logs) be
deposited in per-IMAGE_NAME subdirectories of IMAGEDIR;
that's not done yet but a part of it is ready.

NB: in BUILDDIR, symlinking the just-built image is now
replaced with symlinking the IMAGEDIR since its location
is then predictable thus .gitignore-able for further work
on a generated profile, and more documentable as well.
It's not a hard change though, if you miss the image link
just drop me a note (or a commit).
2011-11-07 00:01:36 +02:00
Michael Shigorin
07fdc6dd9b further toplevel logging tweaks
Essentially some more polishing:
- image path extracted from downstream build log;
- extended error/warning regexp a bit so those with
  color grep options get even prettier output.

Notes:
- "1024" a magic number (briefly explained when introduced)
  moved to a sort of variable;
- "100 lines" for tail(1) is a rule-of-thumb taking into account
  typical amount of hasher/mkimage exhaust given GLOBAL_VERBOSE.
2011-11-07 00:01:36 +02:00
Michael Shigorin
cc8982dd7e tweaked variable logging
Preferences might be somewhat interesting too: while the official
ones shouldn't influence the build result at all, there's no whitelist
so all kinds of weirdness can be stuffed into local config in principle.
That should be diagnosable at least.
2011-11-07 00:01:36 +02:00
Michael Shigorin
cd3e47c562 introduced NICE variable
This one regulates the build wrapper: if the value is non-empty
then nice(1) and ionice(1) will be attempted so that the build
behaves better in regard to other tasks running on the system.

A few doc/variables.txt updates along the way.
2011-11-07 00:01:36 +02:00
Michael Shigorin
102aa3851d syslinux *.cfg: replace banner
Back then I didn't come up with anything smarter than
"mkimage-profiles 2.0" (with my tongue in a cheek),
but as m-p has grown up to 0.4 it's time to fix this.

When done properly, all of the string should be brandable
(with some sane default value inheriting from image name),
but let's do it at least bit by bit.
2011-11-07 00:01:36 +02:00
Michael Shigorin
30b1aa9be6 toplevel build report now timestamped
Thanks torabora@ for yet again seemingly obvious feature request
which strangely managed to evade implementation before.

On an afterthought, mass builds would suggest too much coffee
instead of a progress indicator -- so implemented the latter.

NB: the actual downstream-make-calling rule would expand the "naive"
$(shell date) too early: the rule is evaluated before starting its
execution, and as it's the time consuming one the shell evaluation
was in need, not make's.  The result is less generally available
(needs to be double quoted and won't work inside e.g. awk programs)
but way more precise.
2011-11-07 00:01:36 +02:00
Michael Shigorin
813571cb10 dump sources.list to build.log
The seemingly obvious feature was proposed by torabora@.
APTCONF complicates things a bit of course...
(and was slightly kludgy btw; fixed)
2011-11-07 00:01:36 +02:00
Michael Shigorin
67a7f68e79 .gear/mkimage-profiles.spec: initial package
Also added to the live-builder ISO which is now self-hosted
(sans full repo): one can build an image capable of rebuilding
itself (which is not that useful) and of building other goodies
on some temporarily unused RAM-filled hardware (which is the goal).
2011-11-07 00:01:36 +02:00
Michael Shigorin
83392012d4 live-builder distro: your self-hosted livecd
If you make distro/live-builder.iso, the result is an image
containing almost everything (short of actual full enough
repository) to rebuild itself.  It will attempt to configure
eth0 with DHCP and reach http://ftp.altlinux.org for packages.

RAM requirements start with 2Gb, self-build is accomplished
on a 4Gb host with "make CLEAN=1 distro/live-builder.iso".

Packages required for "make distro/syslinux.iso" get included.

(some due fixups all over the place too)
2011-11-06 17:36:09 +02:00
Michael Shigorin
b6438b9c8f bin/mktmpdir: lower the space bar
With not-that-recent mkimage-profiles development,
it's no longer apparent that at least a gigabyte
of free space is required to build something useful
(at least for the tests, like syslinux.iso).

In short, the guesser cutoff margin is now 256M.
2011-11-06 17:36:09 +02:00
Michael Shigorin
08de714970 added doc/apt.conf.sample
Unfortunately apt configuration is not straightforward at all
regarding being overridden: one can't just provide sources.list
but needs a corresponding apt.conf along with it, and that apt.conf
must disable the SysV-style snippet directories to avoid interesting
side effects of all the things getting overlaid.

So it's not surprising that torabora has asked for an example...
(thanks go to boyarsh@ since I asked him for an example long ago)
2011-11-04 16:54:41 +02:00
Michael Shigorin
895fae7d82 BELL support
Implemented opportunistic alarm support as proposed by torabora;
the actual result depends on readline and/or terminal settings
(read up on "visual bell" vs "audible bell" in case it's wrong).

TODO: this ought to be shifted downstream when proper logging
framework is there.
2011-11-04 16:54:41 +02:00
Michael Shigorin
e8306860f1 introduced conf.d/ for distro, ve config snippets
This was asked for by Leo-sp50 and torabora, and seems quite reasonable:
let's provide means to keep at least some distribution configurations
a bit apart, so that these can be considered more standalone in terms
of hard warranted functionality but at the same time enjoying the common
infrastructure.

Considering lib/distro.mk: it's now experimentally pulled apart so that
parallel development of different distro families can go on without
major merge hassles.  *Please* don't abuse with massive copy-paste.

And before you ask: this might get extended to allow for "private"
out-of-tree configurations being included since apparently there
are goals with no meaning outside of some very particular context...
but otherwise I'd like to encourage getting reusable bits in-tree.
2011-11-04 16:54:41 +02:00
Michael Shigorin
6a280d758d stage2-based debug fixup
Employed double colon targets to eliminate
  warning: ignoring old commands for target `debug'
2011-11-04 16:54:41 +02:00
Michael Shigorin
912d86c4af features.in/syslinux: tweak 00gfxboot.cfg
The same change as in m-p-d f1c5dd0 (discussed in #22486).
Seems it wasn't the culprit regarding the "BIOS won't boot
off this DVD" but is also recommended in README.gfxboot.
2011-11-04 16:54:41 +02:00
Michael Shigorin
812883facc features.in/syslinux: new 10removable.cfg
Borrowed from wmsmall iso by kostyalamer@.
2011-11-04 16:54:41 +02:00
Michael Shigorin
ace2ee1fc1 features.in/Makefile: fixups
src/dst tags might have been empty confusing tags2lists;
the current implementation is more robust (along with
slightly better debug within bin/tags2lists itself).

pushd/popd spam tamed too (replaced by nice log messages).
2011-11-04 16:54:41 +02:00
Michael Shigorin
0a8490ad7f logging tweaks for better readability
...and create a symlink to a just-built image from builddir too.
2011-11-04 16:54:41 +02:00
Michael Shigorin
77115ff50b image.in/functions.mk: added ARCH dup
If the build is (re-)run withing generated profile directory,
the makefiles no longer have ARCH passed by upstream -- the
upstream might be unreachable at that point indeed.

This is unfortunate but we should cope with that.

NB: might be revisited when architecture-specific packagelists emerge
since currently there's no difference in package lists so we can just
re-run the build for a different architecture but this might change...
2011-11-04 16:54:41 +02:00
Michael Shigorin
7919df829c avoid "nothing to commit" in build.log
Typical (to-be-refactored when having settled down)
"cd/git .../cd -" sequences are tweaked to safeguard
against changing back without having actually changed to,
just in case.

features.in/Makefile left with pushd/popd due to its
three-level diving course (which somewhat asks to be
refactores in functions either but is intrinsically
somewhat complex OTOH).
2011-11-04 16:54:41 +02:00
Michael Shigorin
d5a5941f96 official {distro,ve}/* support
This is quite a large-scale change since mkimage-profiles got used to
baking distributions over the last year, and virtual environments are
quite different, so e.g. image.in/Makefile had to be split in two with
the main part of it moved into features.in/iso/lib/.

Short overview:

- features.in/Makefile: lib/ support
  (supporting VE images requires dynamic modifications
  to image.in/Makefile before starting the build;
  the most natural way to achieve that seems to use
  features mechanism along with makefile include dir)

- packaging format related part moved into features.in/pack
  (should be better prepared for diversity either)
- features.in/iso renamed to features.in/build-distro
- features.in/ve  renamed to features.in/build-ve
  + NB: these could not be merged as e.g. features.in/build
    due to completely different script hooks

- lib/image.mk renamed to lib/build.mk
- image, config, log postprocessing moved downstream
- added a sort of a topping in the form of lib/sugar.mk
- assorted style fixups (like ifeq usage)

- clean.mk: reliability fix (the problem was observed by Oleg Ivanov
  and me too but finally it did get the attention quantum)

- reviewed, updated and extended docs
  + QUICKSTART: should be[come] a step-by-step guide
    (thanks Leo-sp50 for prodiving feedback)
2011-11-04 16:54:41 +02:00
Michael Shigorin
720a579690 wash, rinse, repeat
install2 cleanups:

- functionally indifferent ones: particularly, install2/*/98system's
  "mkdir -p /image" was superfluous as it was done by that time already
  by sub.in/stage2/image-scripts.d/00stage1

- taken apart, prepared for tags: so far it's a mostly moot change
  since the installer cleanup scripts themselves are mostly the same as
  preceding 90cleanup was (with some additions corresponding to recent
  kernel development); it's still unclear what the mechanism for
  configuring the cleanups in effect will be, either directory/package
  regex lists or tagged scripts excluded from execution by yet another tag

fixes:

- image.in/Makefile: fix metadata related test; the actual test was
  assuming that stage1 kernel means installer, which is not the case
  since generic stage2 introduction; oh well

- 85cleanup-lowmem: a "_" too much was the culprit in destroying the
  needed translations along with those deemed superfluous; thanks go to
  Oleg Ivanov and Lenar Shakirov for finding the bug and proposing the
  fix altogether

additions:

- features.in/Makefile: reworked help target; it was rather inaccessible
  due to BUILDDIR normally undefined at the time of direct make
  invocation, and BUILDDIR is normally defined during normal builds
  anyways so let's try it this way.

- README++

daydreams:

- 01-genbasedir: we should drop bzip2 compressed pkglists some day
  but see genbasedir and apt-cdrom first, 90-pkg.sh (alterator-pkg)
  will fail miserably otherwise
2011-11-04 16:15:31 +02:00
Michael Shigorin
b474bff468 iso.mk: low space hint on build failure
Just in case the build.log will be inobvious, and it's easy to diagnose
automatically.  Thanks Andrey Stroganov for this use case.

Thanks for improving the initial implementation go to raorn@ for kind
commit lynch and to Yuri Bushmelev for actually suggesting something
more concise.

BTW the "1024" magic number was taken out of thin air:
the "no free space" errors are most likely to happen while
forming/populating a chroot (apt/rpm errors out) and chroots are
roughly two orders of magnitude heftier than a megabyte.
2011-11-04 16:15:31 +02:00
Michael Shigorin
368cfd26bd syslinux: trivial cleanup
"echo ... | xargs cp -t" is a bit more complicated
compared to plain "copy ..."
2011-11-04 16:15:31 +02:00
Michael Shigorin
f5178dee01 features.in/cleanup: auto-add feature tag
The extra tag to filter on was omitted while
moving the code from features.in/cleanup/generate.mk
into the more generic features.in/Makefile; fixed.

Auto-added tags will definitely see an overhaul
when it's more clear what can be done with them.
2011-11-04 16:15:31 +02:00
Michael Shigorin
08b935545a server-ovz.iso: employ proper cleanup
The reason not to add server cleanup right into
distro/server-base is the possibility to build
combined installers (read centaurus).
Gotta sort it out some day...
2011-11-04 16:15:30 +02:00
Michael Shigorin
5d37a0c54d avoid setting GLOBAL_* altogether
Applying set() and only set() to a GLOBAL_* is safe
but still a potentially confusing example; so let's
just do it right (and warn unsuspecting folks too).
2011-11-04 16:15:30 +02:00
Michael Shigorin
75fd795210 ensure that distcfg.mk gets included once at most
This was done while debugging GLOBAL_CLEANUP_PACKAGES
getting doubled... as it got no hard initial value but
rather was always added to, it appeared that at least
stage2/Makefile would obtain a once-initialized value
from upstream and double it while including distcfg.mk
itself.

It's way less hassle to just proxy the value once.
2011-11-04 16:15:30 +02:00