Commit Graph

85 Commits

Author SHA1 Message Date
Michael Shigorin
f8af1c9243 docs updated
Minor tweaks to toplevel docs as well as some doc/*.txt,
doc/variables.txt renamed to doc/params.txt, and a brand new
doc/pkglists.txt is added (thanks manowar@ for his considerations).
2012-06-25 19:29:38 +03:00
Michael Shigorin
8a04d9b656 initial specific VM formats support
Raw disk images are convenient and universal
but there are custom formats like Qemu's qcow2
providing additional features, e.g. copy-on-write
or space savings.  All of this ultimately belongs
to mkimage but in the mean time has been implemented
here as well.
2012-06-25 19:29:38 +03:00
Michael Shigorin
75f7c62bd1 initial build-vm feature
Yes, mkimage-profiles is now able to build VM disk images.
So far the support is pretty basic:

- a single hard drive image with a single partition/FS
- only stock root password is configurable
- LILO is hardwired as a bootloader

The resulting images tend to boot under qemu/kvm though.

Please see doc/vm.txt for the warning regarding additional
privileges and setup required.  This was started back in
February but I still hoped to avoid sudo/privileged helper
(and libguestfs is almost as undistributable as can be)...

Thanks:

- http://blog.quinthar.com/2008/07/building-1gb-bootable-qemu-image-using.html
- Alexey Morarash who reworked that as https://github.com/tuxofil/linsygen
- led@, legion@, vitty@, aen@ for providing advice and inspiration
2012-06-18 21:56:46 +03:00
Michael Shigorin
5dc8001a8a doc/features.txt: clarified the convention
It might be spottable but not immediately obvious that a feature
lives entirely in features.in/FEATURE and every target it provides
is described in the corresponding config.mk; thanks dkr@ for asking.
2012-05-21 21:04:00 +03:00
Michael Shigorin
8cc8d5ff36 doc/variables.txt: tweak CHECK description
It wasn't exactly obvious that the actual image build
isn't started with CHECK=1.
2012-05-21 21:04:00 +03:00
Michael Shigorin
dc93c137d9 docs: more hints on features
It wasn't articulated clearly enough that features are strictly
incremental (while variables can be overridden).
2012-05-21 21:04:00 +03:00
Michael Shigorin
6e6f572b43 better output directory and results structuring
Following m-p-d, a more involved default output directory
structure is feasible now:

  ~/out/name/date/name-date-arch.type

instead of plain

  ~/out/name-date-arch.type

This particular behaviour can be achieved by passing
SORTDIR='$(IMAGE_NAME)/$(DATE)'; note the single quotes.

Reports are also saved in this resulting structure
albeit the place is still highly debatable.
2012-04-23 23:35:43 +03:00
Michael Shigorin
6ada32dc91 reports.mk: added reports/scripts
This one extracts the exact time-ordered list of script hooks
run while building the image.

Some commons refactoring is also due.
2012-04-23 23:10:29 +03:00
Michael Shigorin
acaf12c34e doc/variables.txt: ARCHES explained
Multiple ARCHES won't just magically work without
the ability to figure out the correct apt.conf;
fortunately there's just the right example handy
in profiles.mk.sample already.

Thanks glebfm@ for feedback.
2012-04-09 22:08:00 +03:00
Michael Shigorin
fe58c46ead stage2: tunable squashfs compression
The existing implementation would handle kernel differences
just fine but a bit too automatically: if it sees xz support,
that's what will end up being used (and if there's -Xbcj binary
compression filter available for the target platform, it will
be applied unequivocally either).

It's perfectly suitabe for getting fine-tuned release images
but is also a bit too resource-consuming while developing the
image configuration which has no business with its compression.

The one and only knob is SQUASHFS (see doc/variables.txt);
to give an idea of the differences, here are some numbers
for a mostly-binary (43% as per 99-elf-stats) webkiosk livecd
and a rather less so (18%) flightgear one on a dual quad-core
X5570 node (each mksquashfs run used up all the cores):

SQUASHFS | live-webkiosk.iso | live-flightgear.iso
---------+-------------------+---------------------
fast     | 3:30 / 130M       | 5:11 / 852M
normal * | 3:37 / 100M       | 5:35 / 688M
tight    | 3:50 / 98M        | 6:47 / 683M

Thus if the knob isn't fiddled with, the defaults will allow
for a reasonably fast build of a pretty slim image; if one is
building a release or if a particular image is very sensitive
being close to the media capacity then just add SQUASHFS=tight
and see it a percent or two down on size.

Please note that lzo/gzip-compressed images are also quicker
to uncompress thus further helping with test iterations.

Thanks to led@ and glebfm@ for helpful hints and questions.
2012-04-09 22:03:29 +03:00
Michael Shigorin
74d1e45a1c docs update
Status updates and readability tweaks all over the place.
2012-04-02 18:13:14 +03:00
Michael Shigorin
3d649db656 live: introduce homepage redirector knobs
There are three more variables to adjust the redirect page:
HOMEPAGE, HOMENAME and HOMEWAIT; e.g. live-webkiosk is now
quite a bit more whitelabel.
2012-03-31 21:08:09 +03:00
Michael Shigorin
b5e6906b56 syslinux configuration revisited
- incompatible change (to fix the rather broken early style):
  use/syslinux/ui-% is now use/syslinux/ui/%;

- default timeout changed to 9 seconds (long enough and keeps
  the countdown in a single figure);

- added totaltimeout of 300 seconds;

- provided live kiosk images with almost-instant boot by default;

...and some other assorted tweaks here and there, sorry.
2012-03-31 21:08:09 +03:00
Michael Shigorin
788cad885e initial target tracing support and reporting
The purpose is being able to examine particular target interdependency
graph for a given image having been configured to avoid convoluted
dependencies (loops in particular).

The implementation is based on SHELL hook hint by John Graham-Cumming:
http://cmcrossroads.com/ask-mr-make/6535-tracing-rule-execution-in-gnu-make
2012-03-19 19:23:38 +02:00
Michael Shigorin
cff036c212 doc/assumptions.txt: specify minimal gmake version
From what I've read so far, most of the code should run on 3.80;
there seem to have been some bits that are dependent on 3.81
features, but there is not a bit that depends on 3.82+ features
so far.
2012-02-18 09:48:38 +02:00
Michael Shigorin
4355f401db image.in/Makefile: introduced save-profile
From now on, non-empty SAVE_PROFILE variable will indicate
the need to carry the particular generated profile inside
the image built from it.

Thanks gns@ for this feature in liveflash.eeepc.
2012-02-07 22:21:33 +02:00
Michael Shigorin
e93d6cabca docs: minor updates
doc/variables.txt was missing the already-existing BUILDLOG
variable description, and ARCHES got added during multi-target
toplevel rewrite.  Other minor fixes come as appropriate.
2012-01-02 21:01:09 +02:00
Michael Shigorin
6fadf7e0e9 initial isohybrid support
Here we go with postprocessing priorities along the way
as ISO hybridization has to occur before implanting
final MD5 sum (which must happen earlier than e.g. some
external MD5SUM file generation).

Unfortunately proper dependencies aren't applicable here
(though I'd like to be proved wrong on this one).

Please note that this needs propagator > 20101130-alt9
for automatic mode to work (has also been worked around
in gfxboot case with design-bootloader-source-6.0-alt1).

Thanks rom_as@ for asking about the hybrid image status
and helping out with testing.
2011-12-16 15:22:28 +02:00
Michael Shigorin
390f4f6bc8 lib/build.mk: add basic CHECK support
The idea is to check:
- the reachability of every target
  used to build the image in question;
- the availability of all the package lists
  and subsequently packages for that image;
- the lack of "dangling" intermediate targets,
  features, pkglists, hooks etc.

So far only the first step is implemented --
it's easy and somewhat helpful already for
  make CHECK=1 all
2011-12-06 14:22:20 +02:00
Michael Shigorin
ab70588f94 CLEAN by default (unless DEBUG)
CLEAN is so useful and fiddling with .work chroots does
demand knowledge (hsh-shell is handy btw); so unless we
really get our hands dirty, let's spare ours preciouss
tmpfss.
2011-11-07 12:28:09 +02:00
Michael Shigorin
ddf0c5b7c7 full-view docs update
- toplevel README received some long-needed refactoring
  + lowlevel detail moved, well, to lowlevel READMEs
- reflected more thoroughly that m-p is not about distros anymore
- dropped features.in/00example/README.en: it's already out-of-date
  a bit, and there's no perceived need in thorough English docs so far
- wiki article got split into parts and somewhat rewritten, links updated
- mv doc/{CodingStyle,style.txt}
2011-11-07 00:01:36 +02:00
Michael Shigorin
cd3e47c562 introduced NICE variable
This one regulates the build wrapper: if the value is non-empty
then nice(1) and ionice(1) will be attempted so that the build
behaves better in regard to other tasks running on the system.

A few doc/variables.txt updates along the way.
2011-11-07 00:01:36 +02:00
Michael Shigorin
08de714970 added doc/apt.conf.sample
Unfortunately apt configuration is not straightforward at all
regarding being overridden: one can't just provide sources.list
but needs a corresponding apt.conf along with it, and that apt.conf
must disable the SysV-style snippet directories to avoid interesting
side effects of all the things getting overlaid.

So it's not surprising that torabora has asked for an example...
(thanks go to boyarsh@ since I asked him for an example long ago)
2011-11-04 16:54:41 +02:00
Michael Shigorin
895fae7d82 BELL support
Implemented opportunistic alarm support as proposed by torabora;
the actual result depends on readline and/or terminal settings
(read up on "visual bell" vs "audible bell" in case it's wrong).

TODO: this ought to be shifted downstream when proper logging
framework is there.
2011-11-04 16:54:41 +02:00
Michael Shigorin
d5a5941f96 official {distro,ve}/* support
This is quite a large-scale change since mkimage-profiles got used to
baking distributions over the last year, and virtual environments are
quite different, so e.g. image.in/Makefile had to be split in two with
the main part of it moved into features.in/iso/lib/.

Short overview:

- features.in/Makefile: lib/ support
  (supporting VE images requires dynamic modifications
  to image.in/Makefile before starting the build;
  the most natural way to achieve that seems to use
  features mechanism along with makefile include dir)

- packaging format related part moved into features.in/pack
  (should be better prepared for diversity either)
- features.in/iso renamed to features.in/build-distro
- features.in/ve  renamed to features.in/build-ve
  + NB: these could not be merged as e.g. features.in/build
    due to completely different script hooks

- lib/image.mk renamed to lib/build.mk
- image, config, log postprocessing moved downstream
- added a sort of a topping in the form of lib/sugar.mk
- assorted style fixups (like ifeq usage)

- clean.mk: reliability fix (the problem was observed by Oleg Ivanov
  and me too but finally it did get the attention quantum)

- reviewed, updated and extended docs
  + QUICKSTART: should be[come] a step-by-step guide
    (thanks Leo-sp50 for prodiving feedback)
2011-11-04 16:54:41 +02:00
Michael Shigorin
720a579690 wash, rinse, repeat
install2 cleanups:

- functionally indifferent ones: particularly, install2/*/98system's
  "mkdir -p /image" was superfluous as it was done by that time already
  by sub.in/stage2/image-scripts.d/00stage1

- taken apart, prepared for tags: so far it's a mostly moot change
  since the installer cleanup scripts themselves are mostly the same as
  preceding 90cleanup was (with some additions corresponding to recent
  kernel development); it's still unclear what the mechanism for
  configuring the cleanups in effect will be, either directory/package
  regex lists or tagged scripts excluded from execution by yet another tag

fixes:

- image.in/Makefile: fix metadata related test; the actual test was
  assuming that stage1 kernel means installer, which is not the case
  since generic stage2 introduction; oh well

- 85cleanup-lowmem: a "_" too much was the culprit in destroying the
  needed translations along with those deemed superfluous; thanks go to
  Oleg Ivanov and Lenar Shakirov for finding the bug and proposing the
  fix altogether

additions:

- features.in/Makefile: reworked help target; it was rather inaccessible
  due to BUILDDIR normally undefined at the time of direct make
  invocation, and BUILDDIR is normally defined during normal builds
  anyways so let's try it this way.

- README++

daydreams:

- 01-genbasedir: we should drop bzip2 compressed pkglists some day
  but see genbasedir and apt-cdrom first, 90-pkg.sh (alterator-pkg)
  will fail miserably otherwise
2011-11-04 16:15:31 +02:00
Michael Shigorin
ef0d5e1581 doc/: minor update
Somewhat more uniform style and a few additional bits.
2011-11-04 16:15:30 +02:00
Michael Shigorin
fba30e0675 docs update
Rather minor fixups for things changed in the meanwhile and not
yet (re)documented properly; and a change for memtest feature
to require syslinux feature (the code's been changed to fit
the updated description, actually, and the change is purely
formal as no syslinux alternative is being used/planned so far).
2011-11-04 16:15:30 +02:00
Michael Shigorin
69d5133a3b image.in/vars.mk: dump distcfg variables nicely
distcfg.mk is nice for tracing the variable values' build-up,
but it's more useful to have a look at the final result sometimes.
So here it is, logged for each build and callable by hand.
2011-11-04 16:15:30 +02:00
Michael Shigorin
3d10c59d2b test-based ramdisk size for live squashfs images
Implementation based on m-p-d::profiles/scripts.d/03-syslinux
(but heavily modified for m-p of course)
2011-11-04 16:15:30 +02:00
Michael Shigorin
f5a8b89381 stage2 based live subprofiles, updated docs
- introduced generic stage2 subprofile (non-standalone)
- ported installer and rescue over to stage2/{install2,rescue}
- initial stage2/live (needs more work for sure)
- use make-initrd-propagator
- updated and somewhat extended doc/

NB: mind #26133, #26134
2011-11-04 16:15:30 +02:00
Michael Shigorin
878abab5ce clean.mk and friends: introduced CLEAN control variable
make CLEAN=1 will prune workdirs after packing their results,
so disk (or preferably tmpfs) usage will be lower
2011-11-04 16:15:30 +02:00
Michael Shigorin
c4e0a2490d README updates
...including hasher note, thx m-p-d.
2011-11-04 16:15:29 +02:00
Michael Shigorin
3f012958dc server-ovz; KDEFAULT; syslinux features reworked
Renamed server-light.iso into server-ovz.iso to avoid brand dilution
and confusion (rider@'s server-light rather favours kvm, anyways).

Introduced KDEFAULT: a reliable default kernel chooser knob
since apt's regex ordering proved pretty unreliable.
Spelling things explicitly is better anyways.

SYSLINUX related features undergone pretty major rewrite
(that includes syslinux, hdt and memtest).

The problem to tackle was features.in/syslinux/generate.mk
assuming syslinux and pciids available in build *host* system;
this well might not be the case (or worse yet, those can be
just different).  So now we're a bit less elegant and a bit
more enterprise, stuffing things into chroot and working there.

Bunch of other fixes along the road, including ; to name a few:
- fixed memtest entry (overlooked while renaming SYSLINUX_ITEMS)
- new and shiny doc/CodingStyle
- gfxboot, stage1 target chain, hdt tweaks
- distro.mk rehashed
- README++
- TODO: dropped (integer overflow anyways)
  + actually moved off-tree to reduce commit spam
- s,\.config\.mk,distcfg.mk,g
- doc/profiles.mk.sample: sample ~/.mkimage/profiles.mk
- ...and assorted fixups/additions

Sorry for convoluted commit, this would have been pretty hard to
rework into some really readable shape (and you might be interested
in the original repo's history horrors then, anyways).
2011-11-04 16:15:29 +02:00
Michael Shigorin
9c883d39b6 docs day
ToC:
- introduced doc/
- features.in/00example/
- READMEs and TODO for more
- code comments clarified

Some unrelated minor fixes bundled either.
2011-11-04 16:15:29 +02:00