Commit Graph

39 Commits

Author SHA1 Message Date
Michael Shigorin
305e97d5a2 introduced lib/kernel.mk helper makefile
Sometimes it's desirable to provide the kernel supporting
maximal amount of RAM on the system; bad news is that x86
has a kludge named PAE, good news is that x86_64 doesn't
need it at all; but now we must be able to choose between
those.

BIGRAM will hold the flavour needed.
2012-04-23 23:10:29 +03:00
Michael Shigorin
6ada32dc91 reports.mk: added reports/scripts
This one extracts the exact time-ordered list of script hooks
run while building the image.

Some commons refactoring is also due.
2012-04-23 23:10:29 +03:00
Michael Shigorin
4858791145 virtualbox feature renamed to vm, extended
Initial SPICE support has been added for kvm/libvirt installation
and boot-up using qxl and spice by default as proposed by shaba@.

VirtualBox part is shifted a level deeper correspondingly
but otherwise stays the same.
2012-04-18 21:04:11 +03:00
Michael Shigorin
a6cc9fa8c3 lib/build.mk: tweaked wreckage filter
A pretty common issue breaking the image build is inter-package
file conflict resulting in hsh-install failure down there.
Let's bring that back to attention conveniently.
2012-04-10 23:39:53 +04:00
Michael Shigorin
665c70a444 use PATH instead of hardwired pathlets
bin/ prefix tried to become a permatemp(tm).  No way.
2012-04-07 15:45:40 +03:00
Michael Shigorin
88050c9742 lib/log.mk: check hasher version
There is at least one known deficiency for mkimage-profiles:
build.log will be truncated if verbose mode is enabled and
hasher version is lower than 1.3.22.

The check is done here since it's where the logging is arranged,
and doing it in image.in/Makefile would result in the warning
about log-truncating software being truncated by the said software.

Thanks Max Kosmach for reporting this inobviousity.
2012-04-03 16:26:25 +03:00
Michael Shigorin
9fd0bc143e more readable `make help', finally
The output was still somewhat ragged in 80x24 terminal window
with fmt(1) which wasn't anticipating the word length difference
subsequent column(1) would have to cope with later on.

Thanks Loic Cattani for his shell columnizer implementation:
https://github.com/Arko/Columnize
2012-04-02 17:16:27 +03:00
Michael Shigorin
587eac45ec tweak the fatal errorlevel
127 is also returned by dot(1), and we're toasted
if unable to create $(BUILDDIR).
2012-03-27 08:36:23 +03:00
Michael Shigorin
ed7bf0db9f initial ISO9660 metadata support
As was noted by Alexey Shabalin in libosinfo context,
current ALT Linux images tend to lack ISO9660 metadata
-- which they did have back in the day of Master 2.4.

Please note that the data collection occurs this way
due to mkimage's config.mk resetting the values to be
empty; this was worked around by using another config
file, $(BUILDDIR)lib/iso.mk, and including it later
but that would require a separate target with per-target
CONFIG variable which isn't elegant at all given the need
to actually build up the metadata set.

So the variables were changed (to be more readable anyways)
and then proxied back to BOOT_*.  This might be cleaned up
some day after the inclusion order is tweaked or mkimage
defaults get set-if-unset-yet (?=).
2012-03-26 22:23:28 +03:00
Michael Shigorin
17bd880e19 ve/generic += openssh suite
openssh-server is in need indeed on almost any server instance;
thanks Aleksey Cheusov for reporting the shortage.

This might be amended in the future but is reasonable right now.
2012-03-25 21:46:45 +03:00
Michael Shigorin
5dba27d011 main.mk: factored out help targets
These are better off in their own lib/help.mk
2012-03-19 19:23:38 +02:00
Michael Shigorin
79bb3f8d6e plant SHELL beacons into all config-forming rules
As noted in doc/assumptions.txt, the SHELL based target tracing
only works for rules with recipes, even empty but present ones.

The simplest thing to do is hooking "; @:" onto the rule's tail
(one-liner with a non-printing shell builting "true" command).
2012-03-19 19:23:38 +02:00
Michael Shigorin
788cad885e initial target tracing support and reporting
The purpose is being able to examine particular target interdependency
graph for a given image having been configured to avoid convoluted
dependencies (loops in particular).

The implementation is based on SHELL hook hint by John Graham-Cumming:
http://cmcrossroads.com/ask-mr-make/6535-tracing-rule-execution-in-gnu-make
2012-03-19 19:23:38 +02:00
Michael Shigorin
62ab13cc77 distro/installer related graph cleanup
As was duly noted by Leo-sp50, both server.mk and desktop.mk
duplicate a few bits layered over bare distro/installer which
happened to be both a dependency (thus should reduce redundancy)
and a "real distro" target (well, it doesn't just work yet, need
to provide networking and sources.list in install2 by hand).

Fixed by moving a "node" to distro/.installer along with typical
additions and leaving a bare installer as is by now; there's a
need to get it working at least for DHCP/ftp.altlinux.org case.
2012-02-09 09:33:39 +02:00
Michael Shigorin
b95caa9b73 lib/ve.mk: introduced ve/bare
A minimal chroot supporting extension via apt-get;
vitals if built on Sisyphus as of Jan 16, 2012:

  i586: 13M tar.xz, 58M chroot (33M w/o /usr/share/{doc,locale,man})
x86_64: 14M tar.xz, 60M chroot (35M w/o /usr/share/{doc,locale,man})

Trivial fixups (extra checks) added to two script hooks.
2012-01-16 20:10:16 +04:00
Michael Shigorin
5a1065c33e bin/mktmpdir: check for filesystem mount options
As was found out by Vladimir Karpinsky (thanks for patience!),
the autochosen directory might still have too restrictive mount
options -- nodev and/or noexec.  Hopefully the diags are a bit
better and faster by now.
2012-01-10 16:00:56 +02:00
Michael Shigorin
0ec7d7cd01 check vigorously for installed mkimage
It was briefly mentioned in QUICKSTART but somehow managed
to evade the commandlines provided.  And while at it, let's
make errors like this more explicit to avoid extra lookups.
Oh, and fix QUICKSTART so that readers miss the hassle. :)

Thanks Vladimir Karpinsky for pointing this problem out.
2012-01-04 22:33:53 +02:00
Michael Shigorin
e93d6cabca docs: minor updates
doc/variables.txt was missing the already-existing BUILDLOG
variable description, and ARCHES got added during multi-target
toplevel rewrite.  Other minor fixes come as appropriate.
2012-01-02 21:01:09 +02:00
Michael Shigorin
2f307ffffb main.mk introduced, lib/*.mk partially rewritten
The former toplevel Makefile is now toplevel main.mk;
this change allows for multi-target, multi-arch processing
in the current toplevel Makefile.

As the "build" symlink semantics change quite considerably
when one is doing bulk builds (several pruned builddirs might
be useful for comparison), BUILDDIR is now much more likely
to be recreated: the cases when it will persist are when it's
either a single-image build or when the prefix hasn't changed.

There are some more or less subtle bugfixes and enhancements
all over the map as well.

Done within 20111230..20120102 timeframe, actually...
2012-01-02 21:01:09 +02:00
Michael Shigorin
3d2f69f112 lib/{clean,profile}.mk: robustness improvements
First, let's not do rsync --delete on an unverified target dir
again: the lesson was learned during a subway hacking session
and I must say that SSDs are frightening fast (even if it was
more than a second to realize what happens and terminate the
extermination before it got /home, thanks xterm).

Second, let's use a variable for common name and make's own
realpath function instead of external binary.
2012-01-02 21:01:09 +02:00
Michael Shigorin
96e89d0062 s/subst/sed -i/g
Initial openSUSE package base taming effort has shown that
relatively few things should be fixed; subst has been generalized
as -i option to sed(1) since its introduction, so let's just fix it.
2011-12-26 18:21:22 +02:00
Michael Shigorin
16a06f51d0 cleanup dependency dups
Several targets were used in a duplicated way
that made no harm but no sense either.
2011-12-16 15:28:41 +02:00
Michael Shigorin
e89a0f1c62 lib/build.mk: tweak the optimizer
$(SHORTEN) is better in the midst of the pipe
so that greps are terminal and can sense the tty
(or the lack thereof in case we're logging further).
2011-12-08 09:30:06 +02:00
Michael Shigorin
390f4f6bc8 lib/build.mk: add basic CHECK support
The idea is to check:
- the reachability of every target
  used to build the image in question;
- the availability of all the package lists
  and subsequently packages for that image;
- the lack of "dangling" intermediate targets,
  features, pkglists, hooks etc.

So far only the first step is implemented --
it's easy and somewhat helpful already for
  make CHECK=1 all
2011-12-06 14:22:20 +02:00
Michael Shigorin
7e219c14f1 lib/distro.mk: don't do localboot by default
distro/.base target used to pull in localboot syslinux config
snippet which might be too early for some of the further distros;
it's a quite fragile equilibrium which was shifted a bit by imz@
(see #26606).  Feel free to reopen the discussion though, things
might be tweaked so that localboot might be desirable on almost
every image even if with lower priority...
2011-11-24 15:44:19 +02:00
Michael Shigorin
b52ff4ef91 lib/build.mk: different colours for different outcomes
As was (quite reasonably) asked by someone and me too,
why should a successful build yield a *red* line
(a grep's default)?

So now it's new and improved, 25% free and so forth:
with a successful build you get a green line, while
errors from a broke one result in red ones.

Clinically tested in both b/w and w/b colour schemes;
in case you're not satisfied, please return original
ANSI_OK and ANSI_FAIL values to the colour dealer
and pass your favourite ones instead.
2011-11-21 13:52:41 +02:00
Michael Shigorin
9cedefdba9 introduced add_feature function
The features might get copy-pasted (or even copied-and-pruned)
when initialized; there's an unneccessary duplication of the
function name in the line adding it to FEATURES list, thus
prone to being forgotten and causing some havoc later on.

It was wrong in the first place but tackling this with some
double-colon rules ran into terminality issues, and further
tortures were considered unneccessary.

The current solution isn't perfect (no completely transparent
function name registration upon corresponding target being called)
but at least it is an improvement...
2011-11-19 11:47:29 +02:00
Michael Shigorin
ab70588f94 CLEAN by default (unless DEBUG)
CLEAN is so useful and fiddling with .work chroots does
demand knowledge (hsh-shell is handy btw); so unless we
really get our hands dirty, let's spare ours preciouss
tmpfss.
2011-11-07 12:28:09 +02:00
Michael Shigorin
ddf0c5b7c7 full-view docs update
- toplevel README received some long-needed refactoring
  + lowlevel detail moved, well, to lowlevel READMEs
- reflected more thoroughly that m-p is not about distros anymore
- dropped features.in/00example/README.en: it's already out-of-date
  a bit, and there's no perceived need in thorough English docs so far
- wiki article got split into parts and somewhat rewritten, links updated
- mv doc/{CodingStyle,style.txt}
2011-11-07 00:01:36 +02:00
Michael Shigorin
07fdc6dd9b further toplevel logging tweaks
Essentially some more polishing:
- image path extracted from downstream build log;
- extended error/warning regexp a bit so those with
  color grep options get even prettier output.

Notes:
- "1024" a magic number (briefly explained when introduced)
  moved to a sort of variable;
- "100 lines" for tail(1) is a rule-of-thumb taking into account
  typical amount of hasher/mkimage exhaust given GLOBAL_VERBOSE.
2011-11-07 00:01:36 +02:00
Michael Shigorin
cc8982dd7e tweaked variable logging
Preferences might be somewhat interesting too: while the official
ones shouldn't influence the build result at all, there's no whitelist
so all kinds of weirdness can be stuffed into local config in principle.
That should be diagnosable at least.
2011-11-07 00:01:36 +02:00
Michael Shigorin
cd3e47c562 introduced NICE variable
This one regulates the build wrapper: if the value is non-empty
then nice(1) and ionice(1) will be attempted so that the build
behaves better in regard to other tasks running on the system.

A few doc/variables.txt updates along the way.
2011-11-07 00:01:36 +02:00
Michael Shigorin
30b1aa9be6 toplevel build report now timestamped
Thanks torabora@ for yet again seemingly obvious feature request
which strangely managed to evade implementation before.

On an afterthought, mass builds would suggest too much coffee
instead of a progress indicator -- so implemented the latter.

NB: the actual downstream-make-calling rule would expand the "naive"
$(shell date) too early: the rule is evaluated before starting its
execution, and as it's the time consuming one the shell evaluation
was in need, not make's.  The result is less generally available
(needs to be double quoted and won't work inside e.g. awk programs)
but way more precise.
2011-11-07 00:01:36 +02:00
Michael Shigorin
813571cb10 dump sources.list to build.log
The seemingly obvious feature was proposed by torabora@.
APTCONF complicates things a bit of course...
(and was slightly kludgy btw; fixed)
2011-11-07 00:01:36 +02:00
Michael Shigorin
83392012d4 live-builder distro: your self-hosted livecd
If you make distro/live-builder.iso, the result is an image
containing almost everything (short of actual full enough
repository) to rebuild itself.  It will attempt to configure
eth0 with DHCP and reach http://ftp.altlinux.org for packages.

RAM requirements start with 2Gb, self-build is accomplished
on a 4Gb host with "make CLEAN=1 distro/live-builder.iso".

Packages required for "make distro/syslinux.iso" get included.

(some due fixups all over the place too)
2011-11-06 17:36:09 +02:00
Michael Shigorin
895fae7d82 BELL support
Implemented opportunistic alarm support as proposed by torabora;
the actual result depends on readline and/or terminal settings
(read up on "visual bell" vs "audible bell" in case it's wrong).

TODO: this ought to be shifted downstream when proper logging
framework is there.
2011-11-04 16:54:41 +02:00
Michael Shigorin
e8306860f1 introduced conf.d/ for distro, ve config snippets
This was asked for by Leo-sp50 and torabora, and seems quite reasonable:
let's provide means to keep at least some distribution configurations
a bit apart, so that these can be considered more standalone in terms
of hard warranted functionality but at the same time enjoying the common
infrastructure.

Considering lib/distro.mk: it's now experimentally pulled apart so that
parallel development of different distro families can go on without
major merge hassles.  *Please* don't abuse with massive copy-paste.

And before you ask: this might get extended to allow for "private"
out-of-tree configurations being included since apparently there
are goals with no meaning outside of some very particular context...
but otherwise I'd like to encourage getting reusable bits in-tree.
2011-11-04 16:54:41 +02:00
Michael Shigorin
0a8490ad7f logging tweaks for better readability
...and create a symlink to a just-built image from builddir too.
2011-11-04 16:54:41 +02:00
Michael Shigorin
d5a5941f96 official {distro,ve}/* support
This is quite a large-scale change since mkimage-profiles got used to
baking distributions over the last year, and virtual environments are
quite different, so e.g. image.in/Makefile had to be split in two with
the main part of it moved into features.in/iso/lib/.

Short overview:

- features.in/Makefile: lib/ support
  (supporting VE images requires dynamic modifications
  to image.in/Makefile before starting the build;
  the most natural way to achieve that seems to use
  features mechanism along with makefile include dir)

- packaging format related part moved into features.in/pack
  (should be better prepared for diversity either)
- features.in/iso renamed to features.in/build-distro
- features.in/ve  renamed to features.in/build-ve
  + NB: these could not be merged as e.g. features.in/build
    due to completely different script hooks

- lib/image.mk renamed to lib/build.mk
- image, config, log postprocessing moved downstream
- added a sort of a topping in the form of lib/sugar.mk
- assorted style fixups (like ifeq usage)

- clean.mk: reliability fix (the problem was observed by Oleg Ivanov
  and me too but finally it did get the attention quantum)

- reviewed, updated and extended docs
  + QUICKSTART: should be[come] a step-by-step guide
    (thanks Leo-sp50 for prodiving feedback)
2011-11-04 16:54:41 +02:00