Commit Graph

107 Commits

Author SHA1 Message Date
Michael Shigorin
9414bde4f5 added sub.in/{main,stage2}/lib/.gitignore
Whoops, forgot to feed placeholders for git to grok
the empty directories that are needed later...
2012-05-21 21:04:00 +03:00
Michael Shigorin
c20e820ac7 features.in: generalize lib/ handling
From now on, subprofiles can include the lib/*.mk
brought in by features just like image.in/ long did.
2012-04-25 14:31:26 +03:00
Michael Shigorin
50bfea9d5c cleanup: *support* livecd-install
livecd-install support wasn't added properly;
fixed (thanks glebfm@ yet again).

Something like fine-grained KEEP_* might be due...
2012-04-23 23:10:29 +03:00
Michael Shigorin
d0a8ea6ba8 85cleanup-legacy: less cruel one
The early version considered ISO and KOI encoding families
as obsolete; the current one is a bit more wise and knows
these are just /rare/.  Thanks glebfm@ for #27168 research
and cinnamon by slava@ for ISO-related noises at startup.
2012-04-23 23:10:29 +03:00
Michael Shigorin
86c42e2db6 implemented {THE,BASE,LIVE}_PACKAGES_REGEXP
There were STAGE1_PACKAGES_REGEXP and MAIN_PACKAGES_REGEXP
but adding more of those was postponed to avoid bloat and
bitrot; THE_PACKAGES_REGEXP is needed for use/firmware now
and looks like BASE_PACKAGES_REGEXP and LIVE_PACKAGES_REGEXP
will be useful before too long either.

Docs updated to include stage-specific package related vatiables.
2012-04-10 23:39:53 +04:00
Gleb Fotengauer-Malinovskiy
1432bb1607 add dummy distro/livecd-install
sub.in/stage2/image-scripts.d/99cleanup-boot,
livecd-install need /boot files too
2012-04-09 22:44:11 +03:00
Michael Shigorin
fe58c46ead stage2: tunable squashfs compression
The existing implementation would handle kernel differences
just fine but a bit too automatically: if it sees xz support,
that's what will end up being used (and if there's -Xbcj binary
compression filter available for the target platform, it will
be applied unequivocally either).

It's perfectly suitabe for getting fine-tuned release images
but is also a bit too resource-consuming while developing the
image configuration which has no business with its compression.

The one and only knob is SQUASHFS (see doc/variables.txt);
to give an idea of the differences, here are some numbers
for a mostly-binary (43% as per 99-elf-stats) webkiosk livecd
and a rather less so (18%) flightgear one on a dual quad-core
X5570 node (each mksquashfs run used up all the cores):

SQUASHFS | live-webkiosk.iso | live-flightgear.iso
---------+-------------------+---------------------
fast     | 3:30 / 130M       | 5:11 / 852M
normal * | 3:37 / 100M       | 5:35 / 688M
tight    | 3:50 / 98M        | 6:47 / 683M

Thus if the knob isn't fiddled with, the defaults will allow
for a reasonably fast build of a pretty slim image; if one is
building a release or if a particular image is very sensitive
being close to the media capacity then just add SQUASHFS=tight
and see it a percent or two down on size.

Please note that lzo/gzip-compressed images are also quicker
to uncompress thus further helping with test iterations.

Thanks to led@ and glebfm@ for helpful hints and questions.
2012-04-09 22:03:29 +03:00
Michael Shigorin
c2996cc493 stage2: 99-elf-stats for squashfs tuning
This kind of test was proposed by led@ to gather statistics
on chroot's contents going to become squashfs (the script
optimizations lowering added overhead from ~10 sec down
to a subsecond range were also proposed by him).

Intentionally not documented in doc/variables.txt due to
the rather lowlevel nature of the probe (at least so far).

The knobs involved are SQUASHFS (the additional effort kicks
in only for "tight" case) and GLOBAL_SQUASHFS_SORT (must be
non-empty for this extra overhead to occur).

Additional experimentation is needed to find out whether
the difference in squashfs size and performance is worth
the trouble (seems the impact is non-zero but pretty minor).
2012-04-09 22:03:29 +03:00
Michael Shigorin
74d1e45a1c docs update
Status updates and readability tweaks all over the place.
2012-04-02 18:13:14 +03:00
Michael Shigorin
1b885b3d24 cleanup scripts rehashed
There's much reason for reuse instead of duplication
among the different stage2-based subprofiles.

In particular, the rather monolithic driver cleanup script
of the ancient is better done in several clear pieces with
the final depmod run.

Scripts dropping apt/rpm databases will dump pkglist first.

A script purging /boot/* will honour live-install if present.

Minor inno^Wfixups all over the map too.
2012-03-31 21:08:09 +03:00
Michael Shigorin
efa73b97f0 sub.in/stage1/Makefile: comment fixup
Referring to install2 subprofile after it got generalized
and renamed to stage2 is an omission, better fix it late
than never.
2012-01-11 15:54:59 +02:00
Michael Shigorin
426c985da3 syslinux: piggybacking cleanups
Avoid extra files where a variable would suffice; and use
the more specific "syslinux.list" rather than a "files.list".
2012-01-04 15:50:01 +02:00
Michael Shigorin
180ed5359c "if test ..." replaced by "if [ ... ]"
Fixed up the remnants of the early style mix
to correspond to the proposed doc/style.txt;
the rationale being that

	if [ ... ]; then
		...
		...
	fi

is the more readable construct among itself,

	if test ...; then
		...
		...
	fi

and

	[ ... ] && {
		...
		...
	}

due to the condition being more distinguishable
when bracketed and the body more apparent as the
one inside "if" and not any other block; the less
obvious difference is that the final construct of
the latter form is prone to the whole script exit
status being non-zero if the condition isn't met.
2011-12-21 12:14:03 +02:00
Michael Shigorin
d6972a39bf introduced THE_{KMODULES,PACKAGES,LISTS,GROUPS}
As too many things started duplicating between distros proper
and (e.g. corresponding) LiveCDs, it became apparent that a class
of entities which end up working for THE_USER (not a sysadmin,
and not a developer, just a Linux user) is in need.

So THE_KMODULES will power installed basesystem and live image,
while THE_PACKAGES, THE_LISTS and THE_GROUPS will participate
in building those.
2011-12-19 22:32:59 +02:00
Michael Shigorin
1b3b38b7c5 subprofiles: minor docs extension
"sub/stage2/live" might seem a bit cryptic and not readily
traceable, so let's expand on explanations accordingly.
2011-12-16 15:28:41 +02:00
Michael Shigorin
ae7139f8b6 initial repo feature
We've got some parts of it in build-distro feature,
and some went to dev feature for no real reason.

But a bare installer might go without package base,
and LiveCDs other than live-builder might find local
repository useful given aufs2 root overlay.

Now the overall scheme is more straightforward:
- a distro:
  + asks that a package repo be included
  + cares to further add the packages to it
- a repo feature:
  + pulls in sub/main for it to happen
  + provides genbasedir script to create repo metadata
  + supplements live feature with repo configuration
2011-11-19 11:45:59 +02:00
Michael Shigorin
7110ba77de minor pkglist/docs tweaks
server-base += openssh-slients
xorg += xorg-drv-wizardpen
2011-11-13 00:37:45 +02:00
Michael Shigorin
f257b8e9df stage1/modules: added sdhci
This might be needed to install onto an SD card in a "native"
(non-USB-mediated) SD/MMC cardreader; thanks Vladimir Karpinsky
and gns@ for going over it for liveflash.eeepc case.
2011-11-09 21:20:53 +02:00
Michael Shigorin
ddf0c5b7c7 full-view docs update
- toplevel README received some long-needed refactoring
  + lowlevel detail moved, well, to lowlevel READMEs
- reflected more thoroughly that m-p is not about distros anymore
- dropped features.in/00example/README.en: it's already out-of-date
  a bit, and there's no perceived need in thorough English docs so far
- wiki article got split into parts and somewhat rewritten, links updated
- mv doc/{CodingStyle,style.txt}
2011-11-07 00:01:36 +02:00
Michael Shigorin
83392012d4 live-builder distro: your self-hosted livecd
If you make distro/live-builder.iso, the result is an image
containing almost everything (short of actual full enough
repository) to rebuild itself.  It will attempt to configure
eth0 with DHCP and reach http://ftp.altlinux.org for packages.

RAM requirements start with 2Gb, self-build is accomplished
on a 4Gb host with "make CLEAN=1 distro/live-builder.iso".

Packages required for "make distro/syslinux.iso" get included.

(some due fixups all over the place too)
2011-11-06 17:36:09 +02:00
Michael Shigorin
6a280d758d stage2-based debug fixup
Employed double colon targets to eliminate
  warning: ignoring old commands for target `debug'
2011-11-04 16:54:41 +02:00
Michael Shigorin
7919df829c avoid "nothing to commit" in build.log
Typical (to-be-refactored when having settled down)
"cd/git .../cd -" sequences are tweaked to safeguard
against changing back without having actually changed to,
just in case.

features.in/Makefile left with pushd/popd due to its
three-level diving course (which somewhat asks to be
refactores in functions either but is intrinsically
somewhat complex OTOH).
2011-11-04 16:54:41 +02:00
Michael Shigorin
d5a5941f96 official {distro,ve}/* support
This is quite a large-scale change since mkimage-profiles got used to
baking distributions over the last year, and virtual environments are
quite different, so e.g. image.in/Makefile had to be split in two with
the main part of it moved into features.in/iso/lib/.

Short overview:

- features.in/Makefile: lib/ support
  (supporting VE images requires dynamic modifications
  to image.in/Makefile before starting the build;
  the most natural way to achieve that seems to use
  features mechanism along with makefile include dir)

- packaging format related part moved into features.in/pack
  (should be better prepared for diversity either)
- features.in/iso renamed to features.in/build-distro
- features.in/ve  renamed to features.in/build-ve
  + NB: these could not be merged as e.g. features.in/build
    due to completely different script hooks

- lib/image.mk renamed to lib/build.mk
- image, config, log postprocessing moved downstream
- added a sort of a topping in the form of lib/sugar.mk
- assorted style fixups (like ifeq usage)

- clean.mk: reliability fix (the problem was observed by Oleg Ivanov
  and me too but finally it did get the attention quantum)

- reviewed, updated and extended docs
  + QUICKSTART: should be[come] a step-by-step guide
    (thanks Leo-sp50 for prodiving feedback)
2011-11-04 16:54:41 +02:00
Michael Shigorin
c5bf0529bc initial features.in/cleanup
This is a sort of anti-feature which removes and not builds;
still with mkimage-profiles' approach we can at least build
up the removal procedures as well.

It's what triggered the tagged scripts, BTW.
2011-11-04 16:15:30 +02:00
Michael Shigorin
774169c6cd COMMON_PACKAGES and SYSTEM_PACKAGES
It was clear that "common" isn't very apt for packages that
will get *everywhere*, and became apparent when the need for
a "base+live packages" variable arrived with powerbutton feature.

So:
- the former COMMON_PACKAGES are now SYSTEM_PACKAGES;
- COMMON_PACKAGES act as "BASE+LIVE_PACKAGES".

Note that SYSTEM_PACKAGES also got factored out from stage2 based
features into stage2 subprofile itself; cleanups were due as well.
2011-11-04 16:15:30 +02:00
Michael Shigorin
fba30e0675 docs update
Rather minor fixups for things changed in the meanwhile and not
yet (re)documented properly; and a change for memtest feature
to require syslinux feature (the code's been changed to fit
the updated description, actually, and the change is purely
formal as no syslinux alternative is being used/planned so far).
2011-11-04 16:15:30 +02:00
Michael Shigorin
b56df31624 cosmetic fixup (s/GROUPS/MAIN_GROUPS/g)
MAIN_GROUPS should align better along with MAIN_PACKAGES
and MAIN_LISTS (even if MAIN_ prefix might be suboptimal
given that these packages are essentially extras within
the particular image).
2011-11-04 16:15:30 +02:00
Michael Shigorin
cf81cc6785 live, rescue, and use tags
It's still very immature -- base lists should be really base,
and functionality should be sorted out in more consistent way.
So far moved to tagged lists (which need some more experimentation
anyways to get both lists and their use more elegant).

- fixed live.iso (now actually useful):
  + extra cleanup was being done (coming from install2 case)
  + root user was password blocked
  + there was no unprivileged user (added "altlinux" w/o password)
  + added xdm setup hook for future X-based livecd flavours
  + several picks for a less slim "base" list
  + xdm login
- tweaked rescue.iso (added ext3grep)
2011-11-04 16:15:30 +02:00
Michael Shigorin
3a5b47e712 improved logging during profile creation
- better *_LISTS printout
- somewhat cleaner .base comments
- more comprehensible git log messages
  + special handling for complex subprofiles
  + no need to commit build.log ;-)
2011-11-04 16:15:30 +02:00
Michael Shigorin
2b90dffa6f don't override targets for no reason 2011-11-04 16:15:30 +02:00
Michael Shigorin
2b47da9852 fixed stage1 for syslinux.iso
squashcfk.mk is not to be attempted to copy unconditionally,
if we've got no kernel for stage2 there's no data (and hook)
to create it there
2011-11-04 16:15:30 +02:00
Michael Shigorin
f5a8b89381 stage2 based live subprofiles, updated docs
- introduced generic stage2 subprofile (non-standalone)
- ported installer and rescue over to stage2/{install2,rescue}
- initial stage2/live (needs more work for sure)
- use make-initrd-propagator
- updated and somewhat extended doc/

NB: mind #26133, #26134
2011-11-04 16:15:30 +02:00
Michael Shigorin
0e57149de9 initial rescue image
NB: this implementation is awfully cut-n-paste one,
there are lots of common code with install2
which are subject to careful surgery
2011-11-04 16:15:30 +02:00
Michael Shigorin
5569b4fce1 prepare for multiple live images
s,INSTALLER_KFLAVOUR,STAGE1_KFLAVOUR,g
s,INSTALLER_KMODULES,STAGE1_KMODULES,g

install2 isn't the only livecd around;
so far all of these share the same kernel
in m-p-d, and we'd rather need two+ kernels
with dual-arch media or so; let's not complicate
things too much at this point
2011-11-04 16:15:30 +02:00
Michael Shigorin
878abab5ce clean.mk and friends: introduced CLEAN control variable
make CLEAN=1 will prune workdirs after packing their results,
so disk (or preferably tmpfs) usage will be lower
2011-11-04 16:15:30 +02:00
Michael Shigorin
20844de700 rough bootloader selection draft (grub/lilo)
NB: needs installer-distro-altlinux-generic = 6.0-alt2
    as it doesn't require alterator-grub indefinitely
2011-11-04 16:15:30 +02:00
Michael Shigorin
5f88152223 .base rehashed, DISK_LISTS renamed, etc
In particular:
- .base is now generated from pieces (see image.in/Makefile)
- s/DISTROS/IMAGES/g; s/CONFIGS/DISTROS/g (for clarity)
- s/DISK_LISTS/MAIN_LISTS/g ("disk" was early m-p-d legacy)
- introduced BASE_PACKAGES to complement BASE_LISTS
- minor tweaks to Makefile (ARCH and DATE moved elsewhere)
- libdistro.mk: dropped overlooked IMAGE_INIT_LIST copy
- clean.mk: silly cleanup
2011-11-04 16:15:30 +02:00
Michael Shigorin
b69a8a63b7 README update
reword 'em, structurize 'em, facelift 'em too
2011-11-04 16:15:30 +02:00
Michael Shigorin
0b0ad61b34 install2: split 85cleanup-legacy off 85cleanup-cjk
We have several categories of overhead data:
- legacy 8-bit fonts, locales and gconv modules;
- illegible fonts (e.g. 5x7 or 6x10);
- obsolete vendor specific stuff;
- CJK (which would require proper support anyways).

Notes:
- JIS and things like VISCII are 8-bit either
  and thus might be reconsidered as "legacy";
- proper CJK support would probably include
  scalable fonts and an input method helper;
- main target audience is rather ru/uk/be by now.

(dropped wireless from install2 either)
2011-11-04 16:15:29 +02:00
Michael Shigorin
50e0779f9d sub.in/install2/Makefile: enforce a kernel
A bit earlier the situation where there was a stage1 fallback
for INSTALLER_KFLAVOUR as the last of KFLAVOURS, so we'd have
an installer kernel or build would break; now the situration
got somewhat twisted: there could be a vmlinuz in stage1 but
no corresponding modules in install2 -- which can lead to
different surprises but at least alterator-vm would complain
about "Operation not permitted" on partition layout commit.

The fallback is a copypaste from sub.in/stage1/Makefile though
and should be redone properly somewhere.  The question so far
is, where exactly?
2011-11-04 16:15:29 +02:00
Michael Shigorin
1a8d74e834 moved stage1 scripts into feature.in/installer
All the three scripts depend on installer kernel presence in fact.
When live/rescue are in place, these should be adjusted correspondingly,
probably by moving them to some commonly used intermediate place.

Changed stage1 banner too.
2011-11-04 16:15:29 +02:00
Michael Shigorin
5c571b9af4 pass squashfs options from stage1 test to install2
This looks a bit weird (two subprofiles become a bit more
tightly coupled) but in fact install2 does depend on stage1,
and if stage1 doesn't create squashcfg.mk then install2 is
just fine with defaults (provided they fit the installer kernel
used).  A proper mkimage test infrastructure might be handy though.

Also there:
- 01-initfs: partial cleanup (bootsplash is obsolete anyways)
- regarding 03-test-kernel's errorlevel test:
  if 0, 1 and 2 need to be distinguished, "!" MUST NOT be used
  as it negates so that 0 becomes 1 and the rest becomes 0
2011-11-04 16:15:29 +02:00
Michael Shigorin
71c59a458a stage1/Makefile: serialize prereqs
The problem rarely manifested itself on 8-way server
in the form of attempt to copy ./syslinux while it
wasn't there yet.  The proper fix should include
interdependencies but this kluge works either;
didn't observe this with other subprofiles.
2011-11-04 16:15:29 +02:00
Michael Shigorin
e53b6acb56 installer firmware support
Thanks go to lakostis@ (see #18047) for all the aic94xx reminders
and the final testing of the proposed image.
2011-11-04 16:15:29 +02:00
Michael Shigorin
faafaf12e5 README updates
...and some more cosmetic fixes
2011-11-04 16:15:29 +02:00
Michael Shigorin
c2f70e5c10 kernel and BUILDDIR fixes
- image.in/functions.mk: rework kpackage()
  + it takes two arguments explicitly now: this adds some noise
    for "generic" invocations but is rather less messy with recently
    introduced STAGE1_KFLAVOUR (which in its turn is rather cleaner
    than messing with KFLAVOURS, especially since soemthing changed
    in presumably apt and we can't rely on kernel packages being
    installed in the order formed).
- BUILDDIR/DEBUG related fixes
  + Makefile: BUILDDIR initialization moved to distro.mk
- build.log += git info
2011-11-04 16:15:29 +02:00
Michael Shigorin
3f012958dc server-ovz; KDEFAULT; syslinux features reworked
Renamed server-light.iso into server-ovz.iso to avoid brand dilution
and confusion (rider@'s server-light rather favours kvm, anyways).

Introduced KDEFAULT: a reliable default kernel chooser knob
since apt's regex ordering proved pretty unreliable.
Spelling things explicitly is better anyways.

SYSLINUX related features undergone pretty major rewrite
(that includes syslinux, hdt and memtest).

The problem to tackle was features.in/syslinux/generate.mk
assuming syslinux and pciids available in build *host* system;
this well might not be the case (or worse yet, those can be
just different).  So now we're a bit less elegant and a bit
more enterprise, stuffing things into chroot and working there.

Bunch of other fixes along the road, including ; to name a few:
- fixed memtest entry (overlooked while renaming SYSLINUX_ITEMS)
- new and shiny doc/CodingStyle
- gfxboot, stage1 target chain, hdt tweaks
- distro.mk rehashed
- README++
- TODO: dropped (integer overflow anyways)
  + actually moved off-tree to reduce commit spam
- s,\.config\.mk,distcfg.mk,g
- doc/profiles.mk.sample: sample ~/.mkimage/profiles.mk
- ...and assorted fixups/additions

Sorry for convoluted commit, this would have been pretty hard to
rework into some really readable shape (and you might be interested
in the original repo's history horrors then, anyways).
2011-11-04 16:15:29 +02:00
Michael Shigorin
ace5cb0ff6 assorted fixups
- more thorough build stages inline doc
- quieter BUILDDIR instantiation
- branding-altlinux-sisyphus by default
- prettify installer selection somewhat
- mkimage #24669 fixed upstream
- sub.in/stage1/modules: +drm (as in m-p-d, for KMS)
- tags2lists: generalize a bit
2011-11-04 16:15:29 +02:00
Michael Shigorin
16e763e9ea kernel day
- drop hardwired kernel flavour from pkglists
- today's std-ng lacks aufs, let's switch to un-def
- second assault at KERNEL_PACKAGES_REGEXP
- re-introduced kpackages (builds ok)

PS: base lists: switch to grub, add udev, cleanup
2011-11-04 16:15:29 +02:00
Michael Shigorin
d77e1d8dc8 drop autoconf, a few vars in .mk suffice
A major change in approach largely thanks to discussions
with Alexey Cheusov but also well aligned with my own findings:
autoconf doesn't let the variables to form an inheritance.

And data flow described at http://www.altlinux.org/WhiteLabel
(which in its turn was born thanks to Gavin Henrick of Diva Telecom
and to Alexander Bokovoy of SaM-Solutions) is really dependent on
the existence of such an inheritance.

Also:
- distro.mk += try()
- "hide" special targets
- fixed wrt distro/.{base,init,metaconf}, thx gns@
- README updates
  + added metaconf.mk
  + clarifications
- updated pci.ids location for hdt
2011-11-04 16:15:29 +02:00
Michael Shigorin
9c883d39b6 docs day
ToC:
- introduced doc/
- features.in/00example/
- READMEs and TODO for more
- code comments clarified

Some unrelated minor fixes bundled either.
2011-11-04 16:15:29 +02:00
Michael Shigorin
a9e161b40d gfxboot brought to reasonable shape
Also included:
- reworked syslinux feature inclusion
- syslinux: s/textprompt/prompt/
- README++
2011-11-03 20:41:49 +02:00
Michael Shigorin
30bb813793 initial groups support/selection
...along with obituary fixes.
2011-11-03 20:39:01 +02:00
Michael Shigorin
c8b578dd9a features' day
- get rid of $dest
- work on features.in/syslinux
- README++
2011-11-03 20:39:01 +02:00
Michael Shigorin
4cb23d7ccd drop the dummy configure
It wasn't serving any good by now...
README++ and cleanups all over the place
2011-11-03 20:39:00 +02:00
Michael Shigorin
9a8f8ef7cd distro.mk: functionalization
- set(), add(), log()
  + special thanks to rider@ for log format feedback
- simple debug facility
- builds, installs and runs... again
2011-11-03 20:39:00 +02:00
Michael Shigorin
c91a2a0994 syslinux and features
- syslinux: menu.c32 works
- features: initial draft
  + memtest
- mv pkg/ pkg.in/
  + server-base: include openssh-blacklist by default
- mv isodata/ copy/
- README: changed to utf8, added an URL
2011-11-03 20:38:35 +02:00