Commit Graph

34 Commits

Author SHA1 Message Date
Colin Walters
e726d008fc compose: Add an automatic-version-suffix key
This allows replacing the `.` in automatic version increments
with whatever one wants (as long as it's a single ASCII character)
right now.

The specific motivation here is for at least RHEL CoreOS to use
`version-suffix: "-"` so that its versions can become valid
semantic versions.

Related: https://github.com/coreos/rpm-ostree/issues/1954
2019-12-13 17:11:16 +01:00
Colin Walters
7db30fec13
treefile: Support arch-include (#1886)
* treefile: Rework test helper

Have each test clearly hold its tempdir; this way we can have a simple
helper function rather than a struct with an unreferenced parameter.

Also use `utils::write_file`.

Prep for further test work.

* treefile: Add a basic test for includes

Prep for adding an arch include test.

* treefile: Support `arch-include`

A long time ago we added architecture-specific package lists
via e.g. `packages-ppc64le`.  Much more recently we added
support for having the `include` key be a list - multiple includes.

By combining these two and supporting architecture-conditional includes,
we've effectively added architecture-conditionals to *all* keys.

Notably we want this for Fedora CoreOS today which is using
`remove-from-packages` on `grub2-tools` which isn't present on
s390x.

* ci: Bump cosa build timeout

Looks like another OpenStack perf regression.
2019-08-21 15:25:08 -04:00
Colin Walters
00bd491fe2 treefile: Support multiple includes
I'm working on having Silverblue inherit from Fedora CoreOS.  But
conceptually it also inherits from (parts of) Workstation.
It is just easier if we support multiple inheritance, then I don't
need to think too hard about how to make it a single inheritance chain.

Closes: #1870
Approved by: jlebon
2019-07-18 22:40:59 +00:00
Jonathan Lebon
fc49fb0089 treefile: Add new add-commit-metadata key
Add support for a new `add-commit-metadata` key in the treefile so that
we can directly specify commit metadata we want to inject from there.

This will be useful in Fedora CoreOS, where we'll have separate
treefiles for each streams, each with stream-specific metadata values
required.

Closes: #1865
Approved by: cgwalters
2019-07-09 20:28:18 +00:00
Colin Walters
5f6578ef3c compose: Add ostree-layers and ostree-override-layers
The use case for `ostree-layers` is to support injecting non-RPM
content in a more flexible way than can be done with `add-files`,
and also without dropping all the way to split composes.

This starts with support on the `compose tree` side but down the
line I'd like to make it more convenient to do *client* side too.

For `ostree-override-layers` this is mainly a development thing
for tools like coreos-assembler.  Rather than building an RPM
we just `make install DESTDIR` then commit and add to
`ostree-override-layers`.

Closes: #1830
Approved by: jlebon
2019-06-07 17:42:32 +00:00
Colin Walters
adff1e98b6 compose: Add boot-location: modules
And this (for now at least) completes the epic journey of the
"where's the kernel"?  With this it's found solely in
`/usr/lib/modules/$kver`.

There are a few reasons to do this; most prominent is that
it avoids duplicating the content as the locations may have
different SELinux labels.

Closes: #1773
Approved by: jlebon
2019-03-08 22:49:40 +00:00
Colin Walters
f37426f1ad compose: Remove support for boot-location: both
I'd like to add a new `boot-location: modules`.  In prep
for that, let's remove the legacy `both` which drops into
`/boot`.

The libostree support for handling `/usr/lib/ostree-boot` has
existed for over 4 years:

```
commit 37a059925f6b96d30190b65bee6bdde0ae1c6915
Commit:     Colin Walters <walters@verbum.org>
CommitDate: Sun Nov 30 23:14:05 2014 -0500

    deploy: Ensure that we can deploy using only /usr/lib/ostree-boot

```

I think we assume now that no one is now making *new* treecomposes and needs
a newer rpm-ostree and that they expect people to be able to use as an
upgrade target from a libostree that predates that.

Closes: #1773
Approved by: jlebon
2019-03-08 22:49:40 +00:00
Jonathan Lebon
1fe66e91d0 rust/treefile: Support dash convention for all options
Now that we support YAML, it's a gotcha/eyesore that some of our options
use underscores rather than dashes. Let's be nice and switch those few
options over, while of course still supporting the previous name.

Co-authored-by: Colin Walters <walters@verbum.org>

Closes: #1749
Approved by: cgwalters
2019-03-02 19:20:21 +00:00
Jonathan Lebon
3fda356dd5 docs/treefile: Add some more details to mutate-os-release
Make it easier to understand what the field actually represents.

Closes: #1751
Approved by: ashcrow
2019-02-01 18:43:29 +00:00
rfairley
a20d10e3d5 libpriv/util: Add date field in auto-versioning
This adds an optional date field to the prefix
passed by automatic_version_prefix. An example of specifying
the field is as follows:

10.<date:%Y>

And the fields progress like:

10.2018.0
10.2018.1
10.2018.2
10.2019.0

The date format creates a new "current date" string using
valid date directives passed into g_date_time_format().

If there is a problem reading the given date format,
an error is given and the next version is returned as NULL.

If no <date:...> tag is detected in the auto version prefix,
the same behavior as before (appending .1 and incrementing) occurs.

This may be helpful to avoid writing glue code to auto-update
the version if a date string in the commit version is desired.
Otherwise, --add-metadata-string=version= is an alternative for
complete customization.

Fixes: #1712

Closes: #1721
Approved by: jlebon
2019-01-14 18:49:41 +00:00
Jonathan Lebon
f8f47480f1 docs/treefile: Add info about paths for add-files
Add a note that only certain paths actually make sense in the
`add-files` directive and link to the OSTree docs. Let's also the
examples given to more realistic and correct ones.

Closes: #1643
Approved by: cgwalters
2018-10-26 20:48:07 +00:00
Colin Walters
8df776e367 compose: Remove support for ex-rojig-spec in treefile
In https://github.com/projectatomic/rpm-ostree/pull/1484
AKA commit 235e8f82da
we introduced a new `rojig` treefile entry that is nicer and
more maintainable than a split spec file.

In prep for more "pure rojig" work, drop support for this key.
Note we can also only now do this because we hard require Rust, and
the new model is only implemented in Rust.

The only known user of this today is FAHC, but that is now
[pinned to an assembler build](https://pagure.io/fedora-atomic-host-continuous/c/cdcb6ae943bee643983d4bd22d00030f2eb930d2?branch=master).

Closes: #1558
Approved by: jlebon
2018-09-13 21:10:21 +00:00
Colin Walters
279e7c4f1b treefile: Support inline postprocess element (for inheritance)
I'm trying to have a more opinionated model where custom builds
use inheritance, and currently one can only have a single
`postprocess-script`.

Further, in YAML it's very convenient to use inline vs external
data.

Closes: #1527
Approved by: jlebon
2018-09-05 15:27:07 +00:00
Colin Walters
40be3fb1cf Add recommends to treefile (default on)
This is for: https://github.com/projectatomic/rpm-ostree/issues/718
But I'm not going to close that issue as this only does the server
side, and I think we should support it client side too.

Since I wrote that issue, we ended up skipping the `dnf_transaction_depsolve()`
API, and hence we don't need to block on a libdnf change.  So
this was quite simple.

Closes: #1513
Approved by: jlebon
2018-08-23 04:24:07 +00:00
Colin Walters
588a0327db postprocess: Error if units with machineid-compat: false
Rather than silently ignoring it.

In theory...we could write to /usr/lib/systemd instead of `/etc`
but eh...I feel like what we really want to do is make it convenient
to write a preset file from the YAML.

(We could have an `add-files` content that takes values literally
 which would be nice in YAML and suck in JSON)

A general thread running through this is that for people making
*derivatives* of a CoreOS-like system, having to create their
own `exampleos-release` package is an annoying hurdle.

Anyways for now we're fixing the bug that we were silently ignoring
it.

Closes: #1488
Approved by: jlebon
2018-08-06 00:33:56 +00:00
Jonathan Lebon
be055179db docs/treefile: Mention rw in for machineid-compat option
See discussions in https://github.com/systemd/systemd/issues/8268 for
more information.

Closes: #1487
Approved by: cgwalters
2018-08-03 16:54:55 +00:00
Colin Walters
344aee1d76 rust: Add support for inline rojig spec files
The rojig spec is almost entirely rpm-ostree implementation details;
let's not have lots of people fork/duplicate it.  Rather add the bits
of rojig to the treefile that people need to define (most notably
the name).

Prep for stabilizing rojig.

I had a few false starts with this PR; managing ownership/lifetimes
across C/Rust is just complicated.  I got bit hard by the fact that
the workdir in `--unified-core` is really dfd-relative, and had to
do a dance to propagate the dfd into rust, as well as down into
the rojig builder.

Closes: #1484
Approved by: jlebon
2018-08-03 16:54:47 +00:00
Colin Walters
d7342731ab Add a treefile option to stop creating /etc/machine-id
We actually want systemd's `ConditionFirstBoot` to fire.  The
primary rationale here is that we're adopting Ignition for Fedora CoreOS,
and having `ConditionFirstBoot=` function will help a lot, as the idea
is it only runs once.

However, I discovered that this breaks the `units` directive for example,
as systemd blows away all the unit state in `/etc`.  The correct thing
to do from the start is to use presets.  We could add an implementation of
`units` which works with this on and instead writes a preset file but...eh.
My plan is to at some point introduce an "epoch" and flip various defaults,
this one, `tmp-is-dir`, the passwd file handling, etc.

See: https://github.com/dustymabe/bootengine/pull/11

Closes: #1425
Approved by: jlebon
2018-06-25 16:19:00 +00:00
Micah Abbott
0399e185e4 docs: add additional details about 'tmp-is-dir'
Expanded the details around using 'tmp-is-dir' and what happens when
it is used.

Closes: #1315
Approved by: dustymabe
2018-03-26 17:26:54 +00:00
Colin Walters
3dd132285a Initial renaming pass of "jigdo" to "rojig"
I noticed that Ubuntu also uses the original "jigdo", so let's start
pulling off the band-aid here and do a mass rename.

For this first pass I'm focusing on CLI entrypoints and docs, as that's what
people are going to see; renaming all of the internal C functions, structure
variables etc. can come later.

Closes: #1269
Approved by: jlebon
2018-02-26 15:32:50 +00:00
Colin Walters
79f4b4b861 compose: Add --ex-jigdo-output-rpm
Part of the goal of jigdo ♲📦 is to support organizations switching to *only*
providing RPMs. An intermediate step there is to "lock" the repo and jigdo
together; we don't want to update the ref if building the jigdoRPM fails.

Add an option to perform `rpm-ostree compose tree` and `rpm-ostree ex
commit2jigdo` together; notably we generate a commit, but only update the ref
once the jigdoRPM is built.

Closes: #1165
Approved by: jlebon
2018-01-10 19:18:40 +00:00
Colin Walters
d02bc4b03c compose: Accept NULL treefile for "use defaults" postprocessing
This is prep for split-compose.  We have some options in the
treefile, like `boot_location` and `tmp-is-dir` etc.  While those
are useful options, I don't want to force everyone using
`rpm-ostree compose postprocess` to write a treefile.

Change the code then to accept a `NULL` treefile to mean
"use the defaults".

Closes: #1070
Approved by: jlebon
2017-10-23 20:35:41 +00:00
Colin Walters
f113fc5e27 Rework treecompose kernel processing
Prep for changing `boot_location: new` to use `/usr/lib/ostree-boot`
and `/usr/lib/modules`.  Rework our kernel postprocessing
so that we unify the `boot_location` handling with initramfs generation.

Instead of doing the initramfs first in postprocessing, we do it nearly last,
after e.g. `etc` is renamed to `usr/etc`. This has some consequences, such as
the fact that `run_bwrap_mutably()` is now called in both situations. In
general, our handling of `etc` is inconsistent, although understandably so.

As part of this, I finally got around to implementing the bit from
https://github.com/systemd/systemd/pull/4174 however suboptimal it is; need the
unified core so we can cleanly ignore the posttrans like we do others.  We
intentionally keep the file around in the generated tree so that installing a
kernel RPM per client doesn't try to do any of this either.

This all gets folded together so that the logic for handling the bootloader gets
simpler - in the Fedora case, we now know to find kernels in `/usr/lib/modules`
and can ignore `/boot`.

Closes: #959
Approved by: jlebon
2017-09-12 14:26:44 +00:00
Colin Walters
3ff48589b3 compose: Make boot_location: legacy be an alias for both
We have 3 locations to find kernels now; I can't think of
a reason to support placing kernels *only* in `/boot`.  The
original commit
15ecaacd36
doesn't give a reason, and I certainly can't think of one now.

This makes `legacy` be an alias for `both`, which should be fully compatible.

Prep for further refactoring towards changing `new` to mean both
`/usr/lib/ostree-boot` *and* `/usr/lib/modules`.

Closes: #959
Approved by: jlebon
2017-09-12 14:26:44 +00:00
Colin Walters
b46fc35901 Add releasever opt, avoid opening up host's rpmdb in treecompose
Closes: https://github.com/projectatomic/rpm-ostree/issues/546

Previously, we'd open up the host's rpmdb for both `compose tree`
and `ex container`.  In the first case, because we require root, we'd
succeed.  For `ex container`, we'd spew an error.

Fixing this was trickier than I thought.  First because there was
*also* a libdnf bug here: https://github.com/rpm-software-management/libdnf/pull/307

Second, there's a compatibility hazard here for anyone using `.repo` files that
reference `$releasever`. This actually happened to me with `ex container` as I'd
just done a `ln -s /etc/yum.repos.d/fedora.repo rpmmd.repos.d`.  I fixed
that first by doing a `sed -i -e 's,$releasever,26,' rpmmd.repos.d/*.repo`.

As far as I can see today, none of Fedora Atomic or CentOS AH rely on this. But
in order to enhance compatibility, let's add a "releasever" option. This makes
it easier again to reuse stock `.repo` files if we wanted to do so.

(Also, I realized we can just use `/usr/share/empty` as *the* canonical immutable
 empty directory)

Closes: #875
Approved by: jlebon
2017-07-21 15:02:59 +00:00
Colin Walters
d2bd8500da compose: Add tmp-is-dir option to make /tmp a directory
There are a few reasons to do this. First, systemd changed to refuse mounts on
symlinks, and hence if one *wants* "/tmp-on-tmpfs", one would need to write a
different `sysroot-tmp.mount` unit.

Second, the original rationale for having this symlink was that if you had
multiple ostree stateroots ("osnames"), it's nicer if they had the same `/tmp`
to avoid duplication. But in practice today that's already an issue due to
`/var/tmp`, and further the multiple-stateroot case is pretty unusual. And that
case is *further* broken by SELinux (if one wanted to have e.g. an Ubuntu and
Fedora) stateroots.  So let's fully decouple this and make `/tmp` a plain
old directory by default, so systemd's `tmp.mount` can become useful.

Now, things get interesting for the case where someone wants a physical `/tmp`
that *does* persist across reboots. Right now, if one just did a `systemctl mask
tmp.mount` as we do in Fedora Atomic Host's cloud images, you'd get a semantic
where `/tmp` stays per-deployment, which is weird.  Our recommendation for
that should likely be to set up a bind mount for `/tmp` → `/var/tmp`.

For now, this stays an option to ensure compatibility; if FAH Cloud images
want to stay with "physical /tmp", then we'd have to change the kickstart.

Closes: https://github.com/projectatomic/rpm-ostree/issues/669

Closes: #778
Approved by: jlebon
2017-05-24 17:49:36 +00:00
Jonathan Lebon
1c1dba8eee treefile.md: clarify boot_location documentation
The default is "both", not "legacy".

Closes: #722
Approved by: cgwalters
2017-04-04 18:47:31 +00:00
Jonathan Lebon
68acb1d2be compose-tests: add more tests
Add a few more tests to exercise some of the treefile options. We do
need to also expand test-basic.sh itself to sanity-check the structure
of a normal ostree compose. That's up next on the list.

Closes: #548
Approved by: cgwalters
2016-12-09 16:18:11 +00:00
Jonathan Lebon
0fa6f0be82 treefile.md: fix wrong field name copy-files
Closes: #548
Approved by: cgwalters
2016-12-09 16:18:11 +00:00
Colin Walters
fec4b885ea docs: Document mutate-os-release
Since I actually forgot the semantics when working on
CentOS AH, and noticed we were missing docs.

Closes: #528
Approved by: jlebon
2016-11-30 17:19:57 +00:00
Colin Walters
4852543ecc compose: Support packages-$basearch
See
https://mail.gnome.org/archives/ostree-list/2016-April/msg00020.html

Closes: #305
Approved by: jlebon
2016-10-04 15:10:32 +00:00
Giuseppe Scrivano
94a2804b0a compose: support adding external files
This will allow to copy arbitrary files into the rootfs, specifying something like:

"add-files": [["service.template", "/exports/service.template"],
              ["config.json.template", "/exports/config.json.template"]]

It is quite useful when building a container image.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #253
Approved by: cgwalters
2016-04-27 20:21:12 +00:00
Colin Walters
484c74ae04 doc: Explain bootstrap_packages a bit more
Closes: #242

Closes: #252
Approved by: miabbott
2016-03-29 19:10:07 +00:00
Colin Walters
d06c5d694b docs: Start using mkdocs
Matching https://git.gnome.org/browse/ostree/commit/docs/CONTRIBUTING.md?id=64ebe2b82ac25f1933f39a5cf8a24a5e0096cf6f
2016-03-09 11:10:58 -05:00