Commit Graph

58 Commits

Author SHA1 Message Date
Michael Shigorin
f9f1437bfe stage1: added ext4.ko to modules
It's been found out that live_rw isn't going to work
without ext4 module being available; let's ensure that.
2015-04-20 22:34:46 +03:00
Michael Shigorin
559f80adf3 a few modules.d test drives
These are aimed to test the modules.d/ and auto-pickup
implementation as well as to present an example.

At least 50-net might change (or just get renamed to avoid
auto-pickup) some day as the "net" feature's meaning is
to provide networking upon bootup and these modules are
only needed within stage1 if we're going to netboot;
and that's quite different thing.

armh-cubox bits are prone to get renamed/generalized too
since e.g. ArmadaXP based server images are going to need
this as well.
2015-04-20 13:35:28 +03:00
Michael Shigorin
e139a5e091 stage2: added broken-down module lists
These were produced off the single sub.in/stage1/modules
file using this scriptlet to prefix/annotate the names:

  grep '\.ko$' modules \
  | grep -v / \
  | while read m; do \
  	echo "$(find /lib/modules/$(uname -r)/kernel/{drivers,fs} \
		-name "$m" -printf %P $m $(modinfo -d "${m%.ko}" 2>&1)"; \
    done

...with subsequent sorting and manual separation.

This is meant to be the second stage in monolithic modules
file split, so the lists themselves are largely unmolested
otherwise.  The plan is to further split those into prefix-
and module-specific ones.

Add a note clarifying 10-stage2's status, by the way.
2015-04-20 13:35:28 +03:00
Michael Shigorin
1ee01498d5 stage1, stage2: moved propagator modules file
What was a static sub.in/stage1/modules (and the only one)
is now features.in/stage2/stage1/modules.d/10-stage2
(basically a compatibility file that might go some day).

It will be auto-picked as its name corresponds to the
NN-SUFFIX pattern specified in stage1 subprofile now
with $(FEATURES) going into default STAGE1_MODLISTS.
2015-04-20 13:35:27 +03:00
Michael Shigorin
a36d0236a4 stage1, stage2: initial modlists support
stage1's got prepare-modules target collecting
modules file snippets all over stage1/modules.d/
subdirectories within individual features.

stage2 now adds names of all the features going into
a particular image as snippet file suffix list so that
individual features don't have to register themselves
twice (as a feature and as a propagator modules.d
snippet carrier).

This is going to allow both "uncommon" modules getting
included with no problem (sin@ has wanted cifs ones
for quite some time, for example, and some want e.g.
infiniband modules) *and* to reduce the actual list
below the common mark as well (which is the case with
live-privacy image, for one).

And stage1 memory consumption does matter in some cases
as it's highly critical with no chance to use swap yet.
2015-04-20 13:35:24 +03:00
Michael Shigorin
08d802ab0d stage1: drop dummy.ko
Looks like it's been dumped in along with the rest but not
actually used in {make-initrd-,}propagator; the problem with it
is that snd-dummy.ko matches and pulls a bunch of unrelated
modules where these don't belong (grep -w wouldn't match
snd_dummy.ko though).
2015-03-29 20:41:29 +03:00
Michael Shigorin
bc0f8a45ee stage1: re-add a few modules
These can be found in (semi-)supported branches still:

- loop.ko:
  + 3.0.101-std-def-alt0.M60P.1
  + 3.4.96-led-ws-alt0.M70P.1
- aufs.ko:
  + 2.6.32-el-smp-alt31
  + 3.4.96-led-ws-alt0.M70P.1

ehci_marvell.ko isn't found in contemporary sisyphus/armh
kernels but let's purge it later during archdep rewrite.

NB: libusual.ko has been renamed to usb-libusual.ko as of p6
    (not to be found in p7 anymore), and nls_base.ko was in
    2.6.32 kernels as of p6 but not there in p7; purge these
    somewhere down the road.
2015-03-29 20:41:29 +03:00
Michael Shigorin
09a9255f2f stage1: purge nonexistent modules
This file has been floating around for quite some time,
and some of its contents are pure bit rot by now...

Drop the modules that don't exist as of 3.19.2-un-def-alt1
upon manual diff examination.
2015-03-29 20:41:29 +03:00
Michael Shigorin
71ad34a9bf stage1: fix module name
It wasn't sdhci_pci.ko back in 2.6.32 actually and it's
sdhci-pci.ko; so grep used by mkmodpack won't pick it up.
2015-03-29 20:41:28 +03:00
Michael Shigorin
f112a3358e stage1: add USB3 support
This has been missing for *so* long somehow, and adding some 200k
of modules for fast hardware that's widely available by now
looks like a deal.

Added USB Attached SCSI module just in case (or rather for weak
crc_t10dif symbols?).
2015-03-29 20:41:26 +03:00
Michael Shigorin
6f452182fd fix kpackages() API
This function's got its argument order chosen for "aesthetical"
reason of $(2) following $(1) in the macros but the logical order
is exactly the opposite: we care for kernel flavour much more than
for module set (which is dependent upon it).

So while silent dropout of kernel-image if KFLAVOURS is set
but KMODULES is empty could be fixed by testing for $(2) only,
it looks like a good time to fix this discrepancy altogether.
2014-09-19 20:53:25 +04:00
Michael Shigorin
730e791fec stage1: fix for non-propagator stage2
It's been a given that any stage2 is propagator-based
but that's not neccessarily so; the "run X as PID 1"
sort of contest has sparkled interest in some others.
2014-09-19 20:52:19 +04:00
Michael Shigorin
5b0697393d stage1: READMEdability fix
Kernel modules come as both *.ko and kernel-modules-*.rpm
and it wasn't clear enough whether to mention "ahci" or "v4l".
2014-08-18 22:30:53 +04:00
Michael Shigorin
49b6291a40 kernel, stage1: don't provide r8168.ko by default
It conflicts with r8169.ko inobviously.

The whole mess looks like this:
- r8169.ko doesn't work for all of Realtek 8111/8168/8169 mutations
- r8168.ko works with some of the chips r8169.ko doesn't
- r8168.ko also works with many chips r8169.ko works with
- r8169.ko is provided by kernel-image package (thus default)
- r8168.ko is provided by kernel-modules-r8168 package (optional)
- kernel-modules-r8168 package requires r8168-blacklist package
- r8168-blacklist package is a one-liner that blacklists r8169.ko
- STAGE1_KMODULES wouldn't include r8168 (std-def) or rtl8168 (led-ws)
- sub.in/stage1/modules would mention r8168.ko (m-p-d: r8169.ko)

So a LiveCD built with use/kernel/net might work with RTL8111/8110
just fine when booted live but fail to automatically load the module
when installed onto hard drive; manual modprobe r8169 would work though.
NB: some of the chips (those available to me) would work just fine
both ways -- this has contributed to fixing this *that* late.

Bottom line:

  do not install backup/kludge drivers overriding main ones by default!

Thanks sem@ for providing the crucial hint.
2014-03-17 21:22:29 +04:00
Michael Shigorin
3f547e2504 documentation: use paths relative to toplevel dir
This change is done to reduce ambiguity in some cases;
the previous intention has been to ease navigation when
staying in a particular directory, now it's been changed
in favour of convenient toplevel `git grep' in fact.

Both variants have their pros and cons, I just find myself
leaning to this one by now hence the commit.  Feel free to
provide constructive criticism :)

Some path-related bitrot has also been fixed while at that.
2014-03-05 21:36:30 +04:00
Evgeny Sinelnikov
d385a7864c stage1: add CIFS installation method 2014-01-11 22:51:19 +04:00
Michael Shigorin
ef02d479f7 stage1: add ehci_marvell to modules
This is ARM-specific (more specifically, Marvell ArmadaXP-specific
for me right now) but doesn't hurt if the module is absent.
2013-06-17 19:29:11 +04:00
Michael Shigorin
5b21100bed README update
These have been proofread somewhat to correspond
to the current state of affairs; a missing one
was added for fonts feature.
2013-06-10 19:43:31 +04:00
Michael Shigorin
429ce58608 added live_rw support
There's a need for a separate boot target since
persistent storage is way slower than tmpfs indeed;
usbflash has a tendency for huge performance drops
given simultaneous writes in addition to reads which
are the bottleneck already.

make-initrd-propagator 0.18 introduced ext4 rw slice,
so the corresponding kernel module needs to be included
into stage1; see also #28289.

NB: not available on x86_64-efi (or hybrid GPT to be strict)
due to fragility of the hack being made: parted(8) panics
upon seeing that, and good ol' fdisk is unable to treat it.

NB: use/live/rw use/rescue/rx use/syslinux/ui/gfxboot
are unlikely to play very nice together due to the latter's
magic l10n: "session" label is taken by live_rw config snippet
and *is* translated in design-bootloader-source;
OTOH "rescue_session" is *not*.
2013-03-25 14:13:42 +04:00
Michael Shigorin
35689d158d stage1: replace r8169 with r8168
The issue is that r8169 is rather broken nowadays while
r8168 tends to work on the same hardware; see also #28473.

Thanks zerg@ for having hinted that it's stage1 modules,
not the root squashfs.
2013-03-06 01:20:36 +02:00
Michael Shigorin
393ef79235 stage1: added virtio drivers
I will test more in kvm
I will test more in kvm
I will test more in kvm
...
2012-12-17 14:47:42 +04:00
Michael Shigorin
5a61e2d4cc efi: employ mkimage
The initial approach required some quite involved postprocessing
as described in http://www.altlinux.org/UEFI#HOWTO; after having
ironed out the kinks so that initial EFI support could be merged
into mkimage proper we're better off just using it, eh?
2012-12-17 14:40:55 +04:00
Michael Shigorin
45cddfd25a stage1: added nfsv*.ko to modules
The newer kernels have versioned NFS support code moved
into a few separate modules with nice self-explanatory
messages reading "Protocol not supported" if one has
managed to overlook this; thanks boyarsh@ for heads-up
(based on f545923271f9d1938d1887632ab4697c4c009039 m-p-d).
2012-11-09 16:14:38 +04:00
Michael Shigorin
ccf3b2220a asciidoc: avoid images/icons/note.png
It was actually an experiment leftover and spoils
toplevel completion so let's drop that one, just
as mentioned by torabora@.
2012-11-05 12:26:26 +02:00
Mike Radyuk
85217cd11d added asciidoc support 2012-10-31 21:22:06 +02:00
Michael Shigorin
6f5be91827 make-3.82 adaptation
The issue actually hit image.in/Makefile: "metadata" target
in features.in/metadata/lib/50-metadata.mk wasn't reached
even if features.in/build-distro/lib/90-build-distro.mk
would ACK that the "whatever" actions included "metadata";
thus Metadata/pkg-groups.tar wasn't created and the installer
silently failed to install the .base system.

Let's armour the rest of the cases where the order of inclusion
might be important as well.
2012-10-10 16:18:35 +03:00
Michael Shigorin
c20e820ac7 features.in: generalize lib/ handling
From now on, subprofiles can include the lib/*.mk
brought in by features just like image.in/ long did.
2012-04-25 14:31:26 +03:00
Michael Shigorin
86c42e2db6 implemented {THE,BASE,LIVE}_PACKAGES_REGEXP
There were STAGE1_PACKAGES_REGEXP and MAIN_PACKAGES_REGEXP
but adding more of those was postponed to avoid bloat and
bitrot; THE_PACKAGES_REGEXP is needed for use/firmware now
and looks like BASE_PACKAGES_REGEXP and LIVE_PACKAGES_REGEXP
will be useful before too long either.

Docs updated to include stage-specific package related vatiables.
2012-04-10 23:39:53 +04:00
Michael Shigorin
fe58c46ead stage2: tunable squashfs compression
The existing implementation would handle kernel differences
just fine but a bit too automatically: if it sees xz support,
that's what will end up being used (and if there's -Xbcj binary
compression filter available for the target platform, it will
be applied unequivocally either).

It's perfectly suitabe for getting fine-tuned release images
but is also a bit too resource-consuming while developing the
image configuration which has no business with its compression.

The one and only knob is SQUASHFS (see doc/variables.txt);
to give an idea of the differences, here are some numbers
for a mostly-binary (43% as per 99-elf-stats) webkiosk livecd
and a rather less so (18%) flightgear one on a dual quad-core
X5570 node (each mksquashfs run used up all the cores):

SQUASHFS | live-webkiosk.iso | live-flightgear.iso
---------+-------------------+---------------------
fast     | 3:30 / 130M       | 5:11 / 852M
normal * | 3:37 / 100M       | 5:35 / 688M
tight    | 3:50 / 98M        | 6:47 / 683M

Thus if the knob isn't fiddled with, the defaults will allow
for a reasonably fast build of a pretty slim image; if one is
building a release or if a particular image is very sensitive
being close to the media capacity then just add SQUASHFS=tight
and see it a percent or two down on size.

Please note that lzo/gzip-compressed images are also quicker
to uncompress thus further helping with test iterations.

Thanks to led@ and glebfm@ for helpful hints and questions.
2012-04-09 22:03:29 +03:00
Michael Shigorin
efa73b97f0 sub.in/stage1/Makefile: comment fixup
Referring to install2 subprofile after it got generalized
and renamed to stage2 is an omission, better fix it late
than never.
2012-01-11 15:54:59 +02:00
Michael Shigorin
426c985da3 syslinux: piggybacking cleanups
Avoid extra files where a variable would suffice; and use
the more specific "syslinux.list" rather than a "files.list".
2012-01-04 15:50:01 +02:00
Michael Shigorin
180ed5359c "if test ..." replaced by "if [ ... ]"
Fixed up the remnants of the early style mix
to correspond to the proposed doc/style.txt;
the rationale being that

	if [ ... ]; then
		...
		...
	fi

is the more readable construct among itself,

	if test ...; then
		...
		...
	fi

and

	[ ... ] && {
		...
		...
	}

due to the condition being more distinguishable
when bracketed and the body more apparent as the
one inside "if" and not any other block; the less
obvious difference is that the final construct of
the latter form is prone to the whole script exit
status being non-zero if the condition isn't met.
2011-12-21 12:14:03 +02:00
Michael Shigorin
d6972a39bf introduced THE_{KMODULES,PACKAGES,LISTS,GROUPS}
As too many things started duplicating between distros proper
and (e.g. corresponding) LiveCDs, it became apparent that a class
of entities which end up working for THE_USER (not a sysadmin,
and not a developer, just a Linux user) is in need.

So THE_KMODULES will power installed basesystem and live image,
while THE_PACKAGES, THE_LISTS and THE_GROUPS will participate
in building those.
2011-12-19 22:32:59 +02:00
Michael Shigorin
7110ba77de minor pkglist/docs tweaks
server-base += openssh-slients
xorg += xorg-drv-wizardpen
2011-11-13 00:37:45 +02:00
Michael Shigorin
f257b8e9df stage1/modules: added sdhci
This might be needed to install onto an SD card in a "native"
(non-USB-mediated) SD/MMC cardreader; thanks Vladimir Karpinsky
and gns@ for going over it for liveflash.eeepc case.
2011-11-09 21:20:53 +02:00
Michael Shigorin
ddf0c5b7c7 full-view docs update
- toplevel README received some long-needed refactoring
  + lowlevel detail moved, well, to lowlevel READMEs
- reflected more thoroughly that m-p is not about distros anymore
- dropped features.in/00example/README.en: it's already out-of-date
  a bit, and there's no perceived need in thorough English docs so far
- wiki article got split into parts and somewhat rewritten, links updated
- mv doc/{CodingStyle,style.txt}
2011-11-07 00:01:36 +02:00
Michael Shigorin
774169c6cd COMMON_PACKAGES and SYSTEM_PACKAGES
It was clear that "common" isn't very apt for packages that
will get *everywhere*, and became apparent when the need for
a "base+live packages" variable arrived with powerbutton feature.

So:
- the former COMMON_PACKAGES are now SYSTEM_PACKAGES;
- COMMON_PACKAGES act as "BASE+LIVE_PACKAGES".

Note that SYSTEM_PACKAGES also got factored out from stage2 based
features into stage2 subprofile itself; cleanups were due as well.
2011-11-04 16:15:30 +02:00
Michael Shigorin
fba30e0675 docs update
Rather minor fixups for things changed in the meanwhile and not
yet (re)documented properly; and a change for memtest feature
to require syslinux feature (the code's been changed to fit
the updated description, actually, and the change is purely
formal as no syslinux alternative is being used/planned so far).
2011-11-04 16:15:30 +02:00
Michael Shigorin
2b90dffa6f don't override targets for no reason 2011-11-04 16:15:30 +02:00
Michael Shigorin
2b47da9852 fixed stage1 for syslinux.iso
squashcfk.mk is not to be attempted to copy unconditionally,
if we've got no kernel for stage2 there's no data (and hook)
to create it there
2011-11-04 16:15:30 +02:00
Michael Shigorin
f5a8b89381 stage2 based live subprofiles, updated docs
- introduced generic stage2 subprofile (non-standalone)
- ported installer and rescue over to stage2/{install2,rescue}
- initial stage2/live (needs more work for sure)
- use make-initrd-propagator
- updated and somewhat extended doc/

NB: mind #26133, #26134
2011-11-04 16:15:30 +02:00
Michael Shigorin
5569b4fce1 prepare for multiple live images
s,INSTALLER_KFLAVOUR,STAGE1_KFLAVOUR,g
s,INSTALLER_KMODULES,STAGE1_KMODULES,g

install2 isn't the only livecd around;
so far all of these share the same kernel
in m-p-d, and we'd rather need two+ kernels
with dual-arch media or so; let's not complicate
things too much at this point
2011-11-04 16:15:30 +02:00
Michael Shigorin
878abab5ce clean.mk and friends: introduced CLEAN control variable
make CLEAN=1 will prune workdirs after packing their results,
so disk (or preferably tmpfs) usage will be lower
2011-11-04 16:15:30 +02:00
Michael Shigorin
1a8d74e834 moved stage1 scripts into feature.in/installer
All the three scripts depend on installer kernel presence in fact.
When live/rescue are in place, these should be adjusted correspondingly,
probably by moving them to some commonly used intermediate place.

Changed stage1 banner too.
2011-11-04 16:15:29 +02:00
Michael Shigorin
5c571b9af4 pass squashfs options from stage1 test to install2
This looks a bit weird (two subprofiles become a bit more
tightly coupled) but in fact install2 does depend on stage1,
and if stage1 doesn't create squashcfg.mk then install2 is
just fine with defaults (provided they fit the installer kernel
used).  A proper mkimage test infrastructure might be handy though.

Also there:
- 01-initfs: partial cleanup (bootsplash is obsolete anyways)
- regarding 03-test-kernel's errorlevel test:
  if 0, 1 and 2 need to be distinguished, "!" MUST NOT be used
  as it negates so that 0 becomes 1 and the rest becomes 0
2011-11-04 16:15:29 +02:00
Michael Shigorin
71c59a458a stage1/Makefile: serialize prereqs
The problem rarely manifested itself on 8-way server
in the form of attempt to copy ./syslinux while it
wasn't there yet.  The proper fix should include
interdependencies but this kluge works either;
didn't observe this with other subprofiles.
2011-11-04 16:15:29 +02:00
Michael Shigorin
e53b6acb56 installer firmware support
Thanks go to lakostis@ (see #18047) for all the aic94xx reminders
and the final testing of the proposed image.
2011-11-04 16:15:29 +02:00
Michael Shigorin
faafaf12e5 README updates
...and some more cosmetic fixes
2011-11-04 16:15:29 +02:00
Michael Shigorin
c2f70e5c10 kernel and BUILDDIR fixes
- image.in/functions.mk: rework kpackage()
  + it takes two arguments explicitly now: this adds some noise
    for "generic" invocations but is rather less messy with recently
    introduced STAGE1_KFLAVOUR (which in its turn is rather cleaner
    than messing with KFLAVOURS, especially since soemthing changed
    in presumably apt and we can't rely on kernel packages being
    installed in the order formed).
- BUILDDIR/DEBUG related fixes
  + Makefile: BUILDDIR initialization moved to distro.mk
- build.log += git info
2011-11-04 16:15:29 +02:00
Michael Shigorin
3f012958dc server-ovz; KDEFAULT; syslinux features reworked
Renamed server-light.iso into server-ovz.iso to avoid brand dilution
and confusion (rider@'s server-light rather favours kvm, anyways).

Introduced KDEFAULT: a reliable default kernel chooser knob
since apt's regex ordering proved pretty unreliable.
Spelling things explicitly is better anyways.

SYSLINUX related features undergone pretty major rewrite
(that includes syslinux, hdt and memtest).

The problem to tackle was features.in/syslinux/generate.mk
assuming syslinux and pciids available in build *host* system;
this well might not be the case (or worse yet, those can be
just different).  So now we're a bit less elegant and a bit
more enterprise, stuffing things into chroot and working there.

Bunch of other fixes along the road, including ; to name a few:
- fixed memtest entry (overlooked while renaming SYSLINUX_ITEMS)
- new and shiny doc/CodingStyle
- gfxboot, stage1 target chain, hdt tweaks
- distro.mk rehashed
- README++
- TODO: dropped (integer overflow anyways)
  + actually moved off-tree to reduce commit spam
- s,\.config\.mk,distcfg.mk,g
- doc/profiles.mk.sample: sample ~/.mkimage/profiles.mk
- ...and assorted fixups/additions

Sorry for convoluted commit, this would have been pretty hard to
rework into some really readable shape (and you might be interested
in the original repo's history horrors then, anyways).
2011-11-04 16:15:29 +02:00