IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
I'm working on having Silverblue inherit from Fedora CoreOS. But
conceptually it also inherits from (parts of) Workstation.
It is just easier if we support multiple inheritance, then I don't
need to think too hard about how to make it a single inheritance chain.
Closes: #1870
Approved by: jlebon
When manually writing lockfile overrides (see previous commit), it's
sometimes easier to not have to specify the SHA256 of the package. For
example, in FCOS, all packages on development and production streams
will be sourced uniquely from coreos-pool, so there's no question of
where the package will come from. It's of course also easier in the
context of local development.
Another motivation for this though is a subtle interaction between
Fedora infra and the way we'd like to implement lockfile management: we
want the override process to be PR-based, with a privileged bot in the
backend tagging new overrides into the pool as necessary on merge.
However, packages built in Koji are initially unsigned, and so we can't
actually *know* what the SHA256 of the package will be until it's signed
and tagged into the pool by the bot.
Closes: #1867
Approved by: cgwalters
There are two reasons for this:
1. I'd like to add overrides semantics to lockfiles, and keying by the
package name only makes this much easier.
2. I'd like to make the digest optional, and keeping it as a tuple makes
this awkward.
A map seems natural too since it makes it more clear that we don't
expect multiple specifications for the same package name.
Another tiny advantage is that it's easier to process with e.g. `jq`.
Closes: #1867
Approved by: cgwalters
We were hitting the classic "negative test passes for the wrong reason".
It was failing not because it didn't have a parent, but because we
didn't pass `--repo`. Fix this and also explicitly check for the error
message we expect.
Closes: #1865
Approved by: cgwalters
Add support for a new `add-commit-metadata` key in the treefile so that
we can directly specify commit metadata we want to inject from there.
This will be useful in Fedora CoreOS, where we'll have separate
treefiles for each streams, each with stream-specific metadata values
required.
Closes: #1865
Approved by: cgwalters
The same way we abbreviate ReplacedBasePackages when there are matching
EVR diffs, let's do something similar for RemovedBasePackages for
matching EVRs.
Solves #1784
Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
Closes: #1852
Approved by: jlebon
Drop the use of Ansible everywhere. In the few cases where we really
Python, just spawn a container instead.
This is required to be able to hack on Fedora CoreOS.
Closes: #1850
Approved by: jlebon
The use case for `ostree-layers` is to support injecting non-RPM
content in a more flexible way than can be done with `add-files`,
and also without dropping all the way to split composes.
This starts with support on the `compose tree` side but down the
line I'd like to make it more convenient to do *client* side too.
For `ostree-override-layers` this is mainly a development thing
for tools like coreos-assembler. Rather than building an RPM
we just `make install DESTDIR` then commit and add to
`ostree-override-layers`.
Closes: #1830
Approved by: jlebon
One problem with how we use lockfiles right now is that we don't enforce
them for dependencies. That is, if `foo` requires `bar`, but only `foo`
is in the manifest, then while `foo` will be locked, `bar` will never
be checked against the lockfile because it was never explicitly
requested.
Higher-level though, I don't like how indirect the locking here feels.
See some comments about that in:
https://github.com/projectatomic/rpm-ostree/pull/1745#discussion_r288772527https://github.com/projectatomic/rpm-ostree/pull/1745#discussion_r289419017
Essentially, the manifest is an input file of patterns, and all we
really know from the lockfile output is that the set of packages in
there satisfies this input in some way. But:
1. there are multiple ways to satisfy the same input (hence why hints
like `SOLVER_FAVOR` exist)
2. the solution is dependent on how the solver is implemented (i.e.
different libsolv versions might yield different solutions)
3. the solution is dependent on flags fed to the solver (i.e. different
libdnf versions might yield different solutions)
So any attempt at cross-checking between the input file and the lockfile
is going to be very hard. Using a stricter mode as I suggested in #1745
of only allowing pure pkgnames or NEVRAs would help, but it wouldn't
address the dependency issue. (Though I'm still thinking about possibly
doing this anyway.)
The solution I propose here is instead to take the nuclear approach: we
completely exclude from the sack all packages of the same name as
packages in our lockfiles, but which do not match the NEVRA. Therefore,
any possible solution has to also satisfy our lockfile (or error out).
Closes: #1849
Approved by: cgwalters
Fixes#1670
This patch introduces a new `compose tree
--ex-write-lockfile-to=manifest.lock` argument and a new `compose tree
--ex-lockfile=manifest.lock` to read it back for subsequent invocations.
Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
Closes: #1745
Approved by: jlebon
Of course, update agents driving rpm-ostree know exactly to which commit
they want the system to upgrade, so `upgrade --lock-finalization` is not
helpful. Teach `deploy` the `--lock-finalization` switch too.
Closes: #1846
Approved by: lucab
Add a new "json" output format. The "diff" format is also a mostly
machine-compatible one. But JSON is much more ubiquitous and easier to
consume.
Closes: #1844
Approved by: cgwalters
1. Allow deleting keys without values (e.g. `nosmt`) if such a key
variant exists (i.e. this won't work if there are only e.g.
`nosmt=foo` and `nosmt=bar` variants).
2. Allow deleting duplicate `keys[=val]` kargs.
Closes: #1834Closes: #1835
Approved by: cgwalters
This allows one to run the tests from a container using overlay +
SELinux protection by running the actual compose into a non-overlay
bind-mount. Otherwise, we'll hit `ENOTSUP` when trying to set labels on
various checkouts.
Closes: #1829
Approved by: cgwalters
There are cases where we do want all the things that specifying a ref
provides (e.g. change detection, version incrementing, SELinux labeling
optimizations, and of course writing the ref) but we *don't* want the
new commit to have a parent. Add a new `--no-parent` option to
accommodate this.
This will be used by coreos-assembler. See discussions at
https://github.com/coreos/coreos-assembler/issues/159.
Closes: #1829
Approved by: cgwalters
We had a subtest that wasn't actually part of the `basic_test()` and so
was being executed when the file gets sourced instead of the function
being explicitly called.
Closes: #1829
Approved by: cgwalters
In the app, rebuild the exact command-line that the client used and pass
that to the daemon to be used as the transaction title. Especially in
transactions like `UpdateDeployment()`, we can avoid reverse-engineering
what the original command used was.
This will be used by the upcoming history feature to record the
command-line used in the journal.
Closes: #1824
Approved by: rfairley
This bumps the requirement on the controlling host to Python 3 only.
It also bumps the requirement on the target host to Python 3 as well
since FCOS doesn't ship Python 2 right now.
Though we'll need to eventually drop all Python usage anyway, but at
least let's get tests passing on FCOS first. (See related previous
patch).
Closes: #1828
Approved by: cgwalters
Also switch to using `jq` on the controlling host instead of Python.
This is also prep for switching CI to FCOS which is likely to not ship
Python at all. There are still spots a bit everywhere where we currently
assume Python on the target host. We'll have to address those soon.
Closes: #1828
Approved by: cgwalters
In Fedora 29, and Fedora 30 Silverblue, I have come across the
following error when executing `make vmsync` from my build container
(also on Fedora 29 and Fedora 30 images respectively):
```
...
Failed to connect to new control master
...
Control socket connect(/var/tmp/ssh-vmcheck-1556768111752693879.sock): Connection refused
Failed to connect to new control master
...
```
Previously this worked with Fedora 28 as the host.
After changing the socket to be in /dev/shm, the SSH connection to
the `vmcheck` VM is successful and the sources sync over.
The cause of this seems to be a problem with overlayfs and unix
sockets: https://github.com/moby/moby/issues/12080
Since overlayfs is the default graph driver in Fedora now, work
around this by switching the socket to be in /dev/shm.
Closes: #1827
Approved by: jlebon
Teach `UpdateDeployment` to make use of libostree's staging lock and
then add a `FinalizeDeployment` API to perform the final unlock &
reboot.
I also added a hidden CLI to make testing this easier, but also because
it's likely the FCOS-agent-yet-to-be-named will just end up using the
CLI to keep it simple.
Closes: #1748Closes: #1814
Approved by: lucab
Prep for making `--unified-core` the only path. It turns
out our compose testsuite has a lot of hardcoded ideas about
how the two paths work. The rojig tests in particular need
cached RPMs, so we can't just rely on caching the pkgcache repo.
Add a `--download-only-rpms` that always returns RPMs, and doesn't
import into the pkgcache repo.
Closes: #1798
Approved by: jlebon
Note this patch only touches the *new* APIs that aren't part of
libostree.
Now that we can use `g_ptr_array_find_with_equal_func`, we can drop our
custom `_ostree_ptr_array_find`.
Also strengthen our handling of values everywhere to handle the `NULL`
case and properly support `KEYWORD` args. I ended up getting rid of
`_ostree_kernel_arg_query_status` in the process since it made that
assumption a lot and overall added more complexity than necessary.
Closes: #1796
Approved by: cgwalters
When we expect a test to not fail, let's `g_assert_no_error` first so
that if it fails, we get an exact printout of what error we encountered
when we expected none.
Closes: #1796
Approved by: cgwalters
Right now we only print a diff of the pending deployment if we have a
cached update (which only happens if user just did an `upgrade`
operation). But really, we can just always print this for the pending
deployment regardless of whether there's a cached update calculated.
This is prep for changing chained operations to only show the diff
between the previous pending deployment to the new pending deployment.
With this patch, the full diff from booted to pending will always be
available through `status` (and `db diff` too though it's not as nice).
Closes: #1760
Approved by: cgwalters
This brings us back in sync with the latest libdnf git master. This
required a bunch of work both on the libdnf and rpm-ostree side to get
working. See e.g.
https://github.com/rpm-software-management/libdnf/issues/645.
A few things to adapt to:
- soname bump to `libdnf.so.2`
- `DnfAdvisory` is no longer a `GObject` (annoyingly it's not replaced
by something we can keep a ref on, so this requires some hacks to
steal from the `GPtrArray` -- could enhance libdnf for this later)
- disable SWDB history writing
- use new reldep public API
- update for latest `hy_subject_get_best_selector()` API
This now unlocks the possibility to add support for modules. (One can
see hints of this in the diff by the fact that `libdnf` links to
`libmodulemd1`.)
Update submodule: libdnf
Closes: #1404
Approved by: cgwalters
Let's make this test work across major version rebases of the FAHC
treecompose job by not hardcoding a specific `%{dist}` here. It's strong
enough to check that the `deploy` operation has the expected previous
version.
I did bump the FAHC buildroot so that next version of the rojig RPM will
be f29, but I don't want to wait until the job has composed at least two
of them.
Closes: #1787
Approved by: cgwalters
Let's output to `ex-container-logs`, which matches the directory we
currently have specified in `artifacts:` so that we actually get test
logs uploaded.
Closes: #1787
Approved by: cgwalters
Otherwise, glib will complain if the array is empty since it can't infer
the type of the item.
While we're here, just `git grep` all instances of
`G_VARIANT_TYPE_ARRAY` and make sure they use a fully-specified format.
I added a test to sanity check that glib is happy to synthesize empty
`GVariant` arrays from `g_variant_builder()` if the format string is
specified.
Closes: #1783
Approved by: cgwalters
And this (for now at least) completes the epic journey of the
"where's the kernel"? With this it's found solely in
`/usr/lib/modules/$kver`.
There are a few reasons to do this; most prominent is that
it avoids duplicating the content as the locations may have
different SELinux labels.
Closes: #1773
Approved by: jlebon
I'd like to add a new `boot-location: modules`. In prep
for that, let's remove the legacy `both` which drops into
`/boot`.
The libostree support for handling `/usr/lib/ostree-boot` has
existed for over 4 years:
```
commit 37a059925f6b96d30190b65bee6bdde0ae1c6915
Commit: Colin Walters <walters@verbum.org>
CommitDate: Sun Nov 30 23:14:05 2014 -0500
deploy: Ensure that we can deploy using only /usr/lib/ostree-boot
```
I think we assume now that no one is now making *new* treecomposes and needs
a newer rpm-ostree and that they expect people to be able to use as an
upgrade target from a libostree that predates that.
Closes: #1773
Approved by: jlebon
Add a `basearch` key to the manifest. This can be used at compose time
to assert the architecture the compose is running on. Though my
motivation is for the common case where it gets omitted from the input
manifest and gets automatically added by rpm-ostree into
`/usr/share/rpm-ostree/treefile.json` for introspection on the client.
(The crucial part here is that the treefile created by rpm-ostree
remains deserializable into a `TreeComposeConfig`).
Closes: https://github.com/coreos/fedora-coreos-tracker/issues/154Closes: #1766
Approved by: cgwalters
Manually patch `file_contexts.subs_dist` so that `/home` is equivalent
to `/var/home`. This is required now that the generated homedirs rules
use `/var/home`. Otherwise, `matchpathcon` for example will return wrong
results.
This patch also includes the *removal* of `/var/home -> /home` so that
we're not dependent on this selinux-policy patch making it at the same
time as downstream:
https://src.fedoraproject.org/rpms/selinux-policy/pull-request/14
(See the conversation there for more information.)
Closes: #1754
Approved by: cgwalters
It's possible for some postprocessing scripts to affect the final
SELinux policy. This is the case for the new `/etc/default/useradd` edit
we now do (#1726), but it could've been the case beforehand too with
user scripts modifying e.g. booleans (though ideally all these
modifications would be part of RPMs).
Do a final `semodule -nB` during postprocessing so that the final policy
we commit is "up to date". Otherwise, users may only see changes take
effect if they layer packages that trigger a rebuild.
The motivation for this is specifically for `/etc/default/useradd`.
There is magic in `selinux-policy` that parses the file and generates
templated rules from the value of `HOME`.
For more info, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1669982https://src.fedoraproject.org/rpms/selinux-policy/pull-request/14Closes: #1754
Approved by: cgwalters
RPM-OSTree has been pretty good so far at consuming the exact same RPMs
used for traditional OSes without modifications. This is important,
because shielding RPMs from the OSTree abstraction means we remain
compatible with a large portion of the ecosystem.
However, there are some apps that definitely require rethinking their
approach. The example right now is akmods, which has a patch proposed to
build kmods at `%post` time on OSTree systems instead of from the
daemon.[1]
In such situations, scriptlets need something to key off of for the
OSTree-specific approach. The `/run/ostree-booted` file is the de facto
API to determine if we're running on an OSTree system or not. This patch
simply extends this API so that scriptlets can naturally make use of
them.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1667014Closes: #1750
Approved by: cgwalters
There's lots of gyrations here for unified-core vs not; it's
been broken in the case of `--unified-core` for a while I think.
In that case our workdir is tmpdir, so rename that directory.
Closes: #1743
Approved by: jlebon
Teach rpm-ostree to interpret rebases where the remote component is a
path to a local repo, e.g.:
rpm-ostree rebase /mnt/ostree/repo:my/target/ref
Essentially, the local remote in this case is considered "ephemeral".
It's kind of the equivalent of, on traditional systems:
dnf install --repofrompath repo,/path/to/repodata ...
The use case for this is in OpenShift v4, in which upgrades are done
from containers containing the OSTree commit. There, we want to point
RPM-OSTree directly at the repo in the mounted container and rebase to
the checksum.
For now, the option is marked experimental. One major reason for this is
that the way we pass the repo differs on RHEL7 vs other platforms. (See
comment block in `rpmostree-dbus-helpers.c` for details).
Related: https://github.com/openshift/machine-config-operator/issues/314
Co-authored-by: Colin Walters <walters@verbum.org>
Closes: #1732
Approved by: cgwalters
This adds an optional date field to the prefix
passed by automatic_version_prefix. An example of specifying
the field is as follows:
10.<date:%Y>
And the fields progress like:
10.2018.0
10.2018.1
10.2018.2
10.2019.0
The date format creates a new "current date" string using
valid date directives passed into g_date_time_format().
If there is a problem reading the given date format,
an error is given and the next version is returned as NULL.
If no <date:...> tag is detected in the auto version prefix,
the same behavior as before (appending .1 and incrementing) occurs.
This may be helpful to avoid writing glue code to auto-update
the version if a date string in the commit version is desired.
Otherwise, --add-metadata-string=version= is an alternative for
complete customization.
Fixes: #1712Closes: #1721
Approved by: jlebon
We were using `g_strfreev()` to free the string array, but the strings
themselves were owned by the `modifiers` GVariantDict. Fix this and make
the comments about it more explicit. On my computer (and at least
Dusty's), this was only actually tripping up libc when passing more than
just one package on the CLI.
Closes: #1707Closes: #1709
Approved by: cgwalters
When handling `GetDeploymentBootConfig()`, we would trip an assertion
when trying to read the full bootconfig from a staged deployment, which
of course doesn't have a full bootconfig yet. Rework this to add a new
`staged` key to the returned dict, in which case only `options` is
included. (Which is all `rpm-ostree kargs` needs anyway).
Closes: #1708
Approved by: cgwalters
If no cache dir is given in the workdir, we would alias the cache dir fd
to the workdir fd. But of course, this meant that we'd try to close the
same fd twice when freeing the compose context. Instead, let's just copy
the fd as is also done in the non-unified path.
Closes: #1697Closes: #1698
Approved by: lucab
One question I often have when looking at the output of `status -a`:
```
AvailableUpdate:
Version: 29.20181202.0 (2018-12-02T08:37:50Z)
Commit: dece5737a087d5c6038efdb86cb4512f867082ccfc6eb0fa97b2734c1f6d99c3
GPGSignature: Valid signature by 5A03B4DD8254ECA02FDA1637A20AA56B429476B4
SecAdvisories: FEDORA-2018-042156f164 Unknown net-snmp-libs-1:5.8-3.fc29.x86_64
FEDORA-2018-87ba0312c2 Moderate kernel-4.19.5-300.fc29.x86_64
FEDORA-2018-87ba0312c2 Moderate kernel-core-4.19.5-300.fc29.x86_64
FEDORA-2018-87ba0312c2 Moderate kernel-modules-4.19.5-300.fc29.x86_64
FEDORA-2018-87ba0312c2 Moderate kernel-modules-extra-4.19.5-300.fc29.x86_64
FEDORA-2018-f467c36c2b Moderate git-core-2.19.2-1.fc29.x86_64
Diff: 67 upgraded, 1 removed, 16 added
```
is "How serious and relevant are these advisories to me? How soon should
I reboot?". For the packages that I'm most familiar with, e.g. `kernel`
and `git-core`, I usually look up the advisory and check why it was
marked as a security update, mentioned CVEs, and how those affect me.
The updateinfo metadata includes a wealth of information that could be
useful here. In Fedora, CVEs treated by the security response team
result in RHBZs, which end up attached to the advisories and thus make
it into that metadata.
This patch tries to reduce friction in answering some of those questions
above by checking for those CVEs and printing a short description in the
output of `status -a`. Example:
```
AvailableUpdate:
Version: 29.20181202.0 (2018-12-02T08:37:50Z)
Commit: dece5737a087d5c6038efdb86cb4512f867082ccfc6eb0fa97b2734c1f6d99c3
GPGSignature: Valid signature by 5A03B4DD8254ECA02FDA1637A20AA56B429476B4
SecAdvisories: FEDORA-2018-042156f164 Unknown net-snmp-libs-1:5.8-3.fc29.x86_64
CVE-2018-18065 CVE-2018-18066 net-snmp: various flaws [fedora-all]
https://bugzilla.redhat.com/show_bug.cgi?id=1637573
FEDORA-2018-87ba0312c2 Moderate kernel-4.19.5-300.fc29.x86_64
FEDORA-2018-87ba0312c2 Moderate kernel-core-4.19.5-300.fc29.x86_64
FEDORA-2018-87ba0312c2 Moderate kernel-modules-4.19.5-300.fc29.x86_64
FEDORA-2018-87ba0312c2 Moderate kernel-modules-extra-4.19.5-300.fc29.x86_64
CVE-2018-16862 kernel: cleancache: Infoleak of deleted files after reuse of old inodes
https://bugzilla.redhat.com/show_bug.cgi?id=1649017
CVE-2018-19407 kernel: kvm: NULL pointer dereference in vcpu_scan_ioapic in arch/x86/kvm/x86.c
https://bugzilla.redhat.com/show_bug.cgi?id=1652656
FEDORA-2018-f467c36c2b Moderate git-core-2.19.2-1.fc29.x86_64
CVE-2018-19486 git: Improper handling of PATH allows for commands to executed from current directory
https://bugzilla.redhat.com/show_bug.cgi?id=1653143
Diff: 67 upgraded, 1 removed, 16 added
```
Including the CVE name and RHBZ link also makes it easier to look for
more details if desired.
Closes: #1695
Approved by: rfairley
Minor regression from #1587. There were places that were still doing
`dnf_context_set_cache_age()` manually, but those calls didn't exactly
have the intended effect since the core now handled caching itself.
The actual result was that the metadata was still being updated, but not
during the `dnf_repo_check` pass that the core does, but rather the
`Importing rpm-md` pass it does right after. So then, we were
incorrectly printing `(cached)` even though we'd update it afterwards.
Switch to the new way of doing things.
Closes: #1686
Approved by: cgwalters
This is relatively uncontroversial functionality that has already proved
useful when helping folks debug their stuff. Let's promote it to the
stable interface.
Closes: #1682
Approved by: rfairley
To make it more obvious what the difference between "Importing metadata"
and "Importing" is, add "rpm-md" to the first and "packages" to the
second.
Closes: #1681
Approved by: cgwalters
In general our current CI/test system is susceptible to drift
between the container and AH. The direction we should be
going is to have coreos-assembler solve this problem with
a SDK, but for now, let's ensure that the container's libsolv
makes it to the host, same thing we do for libostree.
Closes: #1676
Approved by: cgwalters
This turned out to be messier than I thought, because of two primary
factors; the biggest mess here of course is the indirection
through the DBus API.
The other problem is that previously we passed the string to render
each time, and with current indicatif that'd trigger a rerender.
Since (usually) don't change the "prefix string", rework the API.
Change the "percent/n_items" bits to use autocleanups as well, and
to take the prefix string as an initial argument.
Since the state expands to multiple components, also change the
API to use the `0-initialized` pattern rather than trying to
return an aggregate.
We also gain a "sub message" which we use to display e.g.
package names as we're doing checkouts. Note this ends up
at the end, since otherwise everything else jumps around.
Closes: #1661
Approved by: rfairley
External tools often want to parse the ref; for example coreos-assembler
currently does so. Let's ensure `${basearch}` is expanded with
`--print-only` so they can parse that JSON to get the expanded version
reliably.
Implementation note: this is the first Rust code which exposes a
"GLib-like" C API, notably with GHashTable, so we're making more use
of the glib-rs bindings.
Closes: #1653Closes: #1655
Approved by: jlebon
This currently requires a `--i-know-this-is-experimental` flag;
I know it'd be a bit more consistent to have it under `ex`, but
what feels weird about that is *most* of the `ex` commands people
use are client side. This is where we want it to ultimately end
up.
We've landed a lot of prep patches, but I know there's still
a notable amount of code duplication with `compose tree`. What's
left is about ~700 lines but it's mostly not hard/complex code
anymore.
In the future, I'd like to extract more of the compose code
to a `rust/src/compose.rs` or so, but I think this is sustainable
fow now.
My high level goal is to get this into coreos-assembler and stand
up a Silverblue build that uses it.
Closes: #1512
Approved by: jlebon
While serde gives us type checking, it of course doesn't understand
semantics beyond that. One example is checking the compatibility of
`add-files` entries with the OSTree model. This is something we can do
upfront early on to avoid surprises for users.
Also tweak the docs to reflect this new check.
Related: #1642Closes: #1643
Approved by: cgwalters
Add a `selinux` verb to treespec, and bind it from treefile. If
set, use it in the core to load an initial policy before import,
if we didn't already set a policy.
In practice right now this is only used from the compose path
since the SysrootUpgrader uses the policy from the merge deployment.
Unset the policy if rojig mode is enabled.
Now, non-SELinux use cases are required to set `selinux: false`
in the treespec. For `ex container` I just set it in our example
specs. Probably that should forcibly disable it in the
treespec but eh, it's experimental.
The other case I can think of is client-side layering; before
we would create a policy using the target root, but it
wasn't a *hard* requirement, i.e. we didn't error out if
`policy_get_name() == NULL`. Let's preserve that semantic by
hooking off of whether `_new_system()` was used.
Prep for sharing code with `compose rojig`.
Closes: #1630
Approved by: jlebon
This way when debugging if I want to preserve the state so I can
go there and rerun command under gdb for exmaple, all I need
to do is set `env TEST_SKIP_CLEANUP=1` rather than also needing
to find a separate dir and also set `env RPMOSTREE_PRESERVE_TMPDIR=1`.
Closes: #1630
Approved by: jlebon
In preparations for https://github.com/ostreedev/ostree/pull/1740, just
hard enable this path unit for now since centrally-maintained distro
presets still need to be updated.
Closes: #1617
Approved by: cgwalters
Try to tease out a bit more info from the journal by looking at the
systemd message when the service transitions to the dead state or even
looking at the OSTree output itself.
Example outputs:
```
[root@f28-ros ~]# rpm-ostree status
State: idle
Warning: failed to finalize previous deployment
error: opendir(ostree/deploy/fedora-atomic/deploy/887c95887a3047a60372016a0d84536530755b60df3cca33c819f7606e220adf.0): No such file or directory
check `journalctl -b -1 -u ostree-finalize-staged.service`
AutomaticUpdates: disabled
...
```
```
[root@f28-ros ~]# rpm-ostree status
State: idle
Warning: failed to finalize previous deployment
ostree-finalize-staged.service: Failed with result 'timeout'.
check `journalctl -b -1 -u ostree-finalize-staged.service`
AutomaticUpdates: disabled
...
```
Closes: #1601
Approved by: cgwalters
Sample output:
```
$ rpm-ostree status
State: idle
Warning: failed to finalize previous deployment
check `journalctl -b -1 -u ostree-finalize-staged.service`
AutomaticUpdates: disabled
...
```
(Though open to tweaking it).
I also played with directly invoking `journalctl` for the user, but that
can get really spammy with e.g. `os-prober` output and such.
I wrote this in Rust using journal API wrappers because I also plan to
implement the `history` command in Rust and will also enhance that new
`journal` module there for that.
Requires: https://github.com/ostreedev/ostree/pull/1750
Requires: https://github.com/jmesmon/rust-systemd/pull/54
(Though I've pointed the manifest at my branch for now for CI).
Closes: #1567Closes: #1601
Approved by: cgwalters
This is prep for running inside (unprivileged) Kube containers
as they exist today: https://github.com/projectatomic/rpm-ostree/issues/1329
Sadly FUSE today uses a suid binary that ends up wanting CAP_SYS_ADMIN.
I think there's some work on FUSE-in-containers but I'm not sure of
the current status.
What rofiles-fuse here is doing here is protecting is the hardlinked
repo imports. But if `--cachedir` isn't specified, that repository
gets thrown away anyways. So there's no real value to using FUSE
here.
Also since nothing is cached, disable the devino cache.
We also make use of --force-copy-zerosized that just landed
in libostree: https://github.com/ostreedev/ostree/pull/1752
Down the line ideally we gain the capability to detect if either
unprivileged overlayfs/FUSE are available. Then if `--cachedir`
is specified we can make things work.
Closes: #1591
Approved by: jlebon
Split out of supporting "pure rojig" work. We also want
to support this for doing "oscontainers" as is planned for
Red Hat CoreOS. The user experience in both cases is oriented
around versioning of the external wrapper, not the inner ref/commit.
Note for users/builders who want to make use of this feature:
You probably want to mirror the changes in our test suite here to
use the compose JSON and parse the resulting `ostree-commit` out of that.
Closes: #1603
Approved by: jlebon
Drop the `rpmostree.rpmdb.pkglist` keys from the deployment metadata
since users shouldn't need it and it greatly increases the size of the
output.
Closes: #1577
Approved by: cgwalters
Currently `compose tree` supports writing to both an OSTree
repository and optionally simultaneously generating a rojig
RPM or rojig set.
We are instead going to split off a separate `compose rojig` verb,
since we want to support a "pure rojig" mode, and having all
3 options would make this code far too complex.
For now anyone who wants *both* formats can use `compose tree` then
follow up with `ex commit2rojig`.
This is also inline with the new coreos-assembler push, where
we'll treat `rpm-ostree compose` as more of a "low level" API,
and wrap it in a more opinionated model there.
Closes: #1570
Approved by: jlebon
Often, after rebooting from an upgrade, I want to check what was just
updated. This patch makes `db diff` do the right thing in those cases.
Specifically, before `db diff` without arguments would default to
diff'ing the pending deployment with the booted deployment and error out
otherwise. This patch extends the logic so that if there's a rollback
deployment, we default to diff'ing against that.
Closes: #1565
Approved by: cgwalters
This completes the removal of externally-provided rojig spec files.
In the current flow then `compose tree --ex-rojig-output-*` will
end up parsing the manifest twice. But this is prep for defaulting
rojig to "pure" mode without a separate ostree repo. Then
anyone who wants to do both can run `commit2rojig` after.
Closes: #1561
Approved by: jlebon
This is prep for fully dropping the separate spec file, which
is now just used by `commit2rojig`.
A compose test can now specify it wants YAML instead of JSON.
Closes: #1561
Approved by: jlebon
Our trick of using layered commits as base commit updates doesn't jive
well with the new opportunistic reuse of the base rpmdb. The issue is
that a layered commit includes the rpmdb of *its* base commit at the
`/usr/lib/sysimage` location. So to convert it into a proper base commit
means that the layered rpmdb should move there.
Closes: #1502
Approved by: cgwalters
Check if we can reuse the base rpmdb from the pending deployment if it
matches the base rev we're targeting. This allows us to avoid checking
out the tree early only to later on discard it. Such cases include
layering existing packages and inactive requests.
Closes: #1502
Approved by: cgwalters
This change gives an error when --arg is specified on its own.
Previously, if adding --arg without --enable, no changes occur and
only the initramfs regeneration status is printed. Since --arg only
takes effect when used with --enable, this should also give an error
(or let the user know in some way).
Closes: #1548
Approved by: jlebon
Previously, after enabling initramfs regeneration, the initramfs
command would indicate that initramfs regeneratin is disabled,
suggesting that the initramfs had not actually been enabled. This
happened because the initramfs regeneration status for the current
deployment was printed; initramfs regeneration is only enabled for a
pending deployment.
This change prints the regeneration for the pending deployment,
rather than the current, when no options are given to the initramfs
command.
Fixes: #1526Closes: #1548
Approved by: jlebon
It's possible to run the `vmcheck` tests against an existing host that
has `rpm-ostree` already present. We don't overlay the built binaries
in this situation, so we should not check for the presence of the
overlay in the commit meta.
Closes: #1555
Approved by: cgwalters
This removes the logic around supporting opting out of the staging
feature. We don't want to support multiple configurations here, and at
this point, staging should be considered stable.
Closes: #1546
Approved by: cgwalters
We had two `libcomposetest.sh` which I always found confusing.
Fix the naming of the one that's shared with `ex-container`
to be more obvious.
Closes: #1543
Approved by: jlebon
Accumulate the list of requested packages that were not found and error
out with the full list rather than failing early. This fixes a small UX
papercut in certain situations.
Closes: #1540Closes: #1541
Approved by: cgwalters
For higher level tools driving rpm-ostree, the command line arguments
are mostly OK as inputs, but the addition of `--write-commitid-to`
shows that we really want structured data that these tools can parse.
JSON is a good enough interchange format, let's use that.
Most notably this does a pkgdiff in the JSON which several higher
level tools do too.
Closes: #1529
Approved by: jlebon
Since `/tmp` might be on tmpfs, so we'd lose it on reboot. But we have
tests that need it to persist across reboots.
Closes: #1531
Approved by: miabbott
It's possible to run the `vmcheck` tests against an existing VM which
may not have any existing `/etc/yum.repos.d` directory. Since the
tests are providing their own repo files, we should be able to run
them in this uncommon configuration.
Closes: #1530
Approved by: jlebon
Prep for rojig-only compose support. We want to be able
to quickly detect whether or not there were changes in the
input, just like we do when targeting an OSTree repository.
Closes: #1528
Approved by: jlebon
I'm trying to have a more opinionated model where custom builds
use inheritance, and currently one can only have a single
`postprocess-script`.
Further, in YAML it's very convenient to use inline vs external
data.
Closes: #1527
Approved by: jlebon
Similar to the passwd conversion, this commit
implements conversion of group entries into sysuser struct entries
Tests are also included in the commit
Closes: #1519
Approved by: cgwalters
It turns out we basically have to slap an `Option<T>` around
everything, (in particular `bool` etc.) we need to be able
to distinguish in (I believe) all the cases between
"value unspecified" and "value provided".
Concretely it didn't work to try to set `machineid-compat: false`
in an included yaml treefile becuase it was just defaulted to `true`
by the toplevel.
Down the line we should move all of the parsing into Rust
and have two different `struct` types for "YAML we load" versus
"verified treefile".
Closes: https://github.com/projectatomic/rpm-ostree/issues/1524Closes: #1525
Approved by: lucab
This teaches the client to fetch packages from URLs directly so that one
doesn't have to `curl` first and then install. Supported anywhere
package filenames are allowed (notably: `install` and
`override replace`).
One neat things about this is that we download the file into an
`O_TMPFILE` and then pass on ownership of that fd directly to the
daemon. So at no point are the packages actually laying visible on the
system. (Assuming the filesystem supports `O_TMPFILE` that is).
This adds direct linking to libcurl and openssl, two libraries which we
were already pulling in indirectly.
Closes: #1508
Approved by: cgwalters
This is for: https://github.com/projectatomic/rpm-ostree/issues/718
But I'm not going to close that issue as this only does the server
side, and I think we should support it client side too.
Since I wrote that issue, we ended up skipping the `dnf_transaction_depsolve()`
API, and hence we don't need to block on a libdnf change. So
this was quite simple.
Closes: #1513
Approved by: jlebon
We've put a lot of work into staged deployments, it's time
to pull the trigger and turn them on by default. This is
a key step for enabling `stage` mode automatic updates by
default in e.g. Fedora CoreOS/Silverblue.
We add a new `--disable-staged` build-time option to flip
things back.
Closes: #1430
Approved by: jlebon
Staging fixes the `/etc` bug for livefs. There's actually more
we could do here around taking advantage of staging for livefs;
for example, I think once the livefs is complete, we could just delete
the staged deployment. And then we don't need to render on the next
boot the live status, etc.
Anyways, all that can come in the future. This is prep for
enabling staging by default.
Closes: #1430
Approved by: jlebon
Fix `--booted` was printing an extra line after `Deployments:`. While
we're there, also update the section name to `BootedDeployment:` in that
case to be more correct. (We're not printing all the deployments, only
the booted deployment.)
Closes: #1503
Approved by: cgwalters
The `f28-compose` test keeps timing out. Some time recently, I/O
performance of the internal OpenStack instance used for testing has
degraded. I have a ticket open to investigate the regression though
haven't had any luck so far.
Let's just take the easy way out and split the test into two testsuites.
This is obviously hacky, and sad, and unfortunate. But the PRs must keep
flowing until we finally wean off of OpenStack.
Closes: #1498
Approved by: cgwalters
And don't mention "replace" in the error texts.
This is (now) known broken with kernel updates; making that not
dangerous isn't going to be easy. I debated entirely removing it
but for now let's just make it harder to use.
Ref: https://github.com/projectatomic/rpm-ostree/issues/1495Closes: #1497
Approved by: jlebon
We should be expecting testpkg-1.1-1 here, not 1.0-1. This was passing
before because of the nondeterministic `find` output (fixed in the
previous commit) which could spit out the older rojig RPM.
Closes: #1491
Approved by: cgwalters
This test was relying on the order in which `find` reports matching path
names to find the right RPM. This was failing for me locally sometimes
because it matched the wrong RPM file. Fix this by just directly
referencing the full path name since we can.
Closes: #1491
Approved by: cgwalters
My fix to the testsuite in #1488 in which I made the `machineid-compat`
test part of `test-basic.sh` wasn't correct since the basic tests in
`libbasic-test.sh` also check that the default behaviour without the
`machineid-compat` option is to include it.
Let's just do this right and split out the `machineid-compat` test into
its own run.
Closes: #1491
Approved by: cgwalters
I noticed that the latest Fedora Atomic Host 28 and Silverblue did not
have an `OSTREE_VERSION` line in `/etc/os-release` even though both
specified `mutate-os-release` in their manifests. This turned out to be
due to the fact that `/usr/lib/os-release` is now a symlink to a
variant-specific file (e.g. `os-release-atomichost`), so we would
fallback to mutating `/usr/lib/os.release.d/os-release-fedora` instead.
Fix this by just taking the nuclear option of running `realpath` in the
rootfs directly. This is more maintainable than trying to keep up with
changes in variants/naming/etc. There's related discussions to this in
the original [PR](https://github.com/projectatomic/rpm-ostree/pull/410)
which introduced the feature re. resolving symlinks within the rootfs.
Closes: #1481
Approved by: jlebon
Rather than silently ignoring it.
In theory...we could write to /usr/lib/systemd instead of `/etc`
but eh...I feel like what we really want to do is make it convenient
to write a preset file from the YAML.
(We could have an `add-files` content that takes values literally
which would be nice in YAML and suck in JSON)
A general thread running through this is that for people making
*derivatives* of a CoreOS-like system, having to create their
own `exampleos-release` package is an annoying hurdle.
Anyways for now we're fixing the bug that we were silently ignoring
it.
Closes: #1488
Approved by: jlebon
The rojig spec is almost entirely rpm-ostree implementation details;
let's not have lots of people fork/duplicate it. Rather add the bits
of rojig to the treefile that people need to define (most notably
the name).
Prep for stabilizing rojig.
I had a few false starts with this PR; managing ownership/lifetimes
across C/Rust is just complicated. I got bit hard by the fact that
the workdir in `--unified-core` is really dfd-relative, and had to
do a dance to propagate the dfd into rust, as well as down into
the rojig builder.
Closes: #1484
Approved by: jlebon
Make logging work the same as it does for the vmcheck-STI work
(at some point I'll try to unify the 3 parallel+script implementions
we have). This fixes the problem that when the test times out,
the filename won't have `.txt` and S3 won't have the right MIME type.
Closes: #1479
Approved by: jlebon
This is analogous to `upgrade --upgrade-unchanged-exit-77`, but for
`install`/`uninstall`. This way, one can determine whether the command
truly had an effect on the default deployment or not.
Since this works by comparing against the previous default deployment,
this works correctly even if we already had a pending deployment at the
beginning of the transaction.
Closes: #1478
Approved by: cgwalters
Add a new `install/uninstall --idempotent` option to make it easier to
interact with the CLI through scripts. E.g. one doesn't have to check
first if a request has already been installed/uninstalled.
Closes: #1467Closes: #1478
Approved by: cgwalters
This makes it easier for scripts to determine whether there is a pending
deployment instead of using `--json/--jsonpath`.
Closes: #1478
Approved by: cgwalters
The `cached-update` variant would mark a bunch of RPMs as upgraded even
if they didn't actually change. The issue turned out to be we were doing
the diff all wrong in the staged deployment case. I'm not sure what I
was thinking in #1344, but essentially, we were marking all layered RPMs
in the staged deployment as updates instead of only marking those
layered RPMs which were actually changed EVR.
We just simplify the approach here by directly doing a pkglist diff
between the booted and staged deployments and consuming that. That's
really all there is to it! Reduces the code quite a bit too.
Closes: #1446Closes: #1455
Approved by: cgwalters
This one snuck in as part of #1205. I had initially started writing
`inject-pkglist.c` in Python and somehow inadvertedly made it part of
a commit.
Closes: #1455
Approved by: cgwalters
In line with the recent trend of marking things stable, and in
preparation for stabilizing `rojig://` - Let's stabilize the `--unified-core`
option for `compose tree`.
I'm not sure we could make it the default anytime soon; today it trips
over bugs in the PAM package in RHEL7 for example. But it
works fine for Fedora, and I think the code/design are good enough to be stable.
Closes: #1465
Approved by: jlebon
Follow up to: https://github.com/projectatomic/rpm-ostree/pull/1459
We now honor arch-specific packages in YAML, and reject unknown
architectures. I looked a little bit at how to avoid having hardcoded
arch lists, but it doesn't seem worth it right now.
Closes: #1468
Approved by: jlebon
Let's not make the same mistake we did with JSON where typoing a
field means it's silently ignored. This actually caught a bug
in a YAML usage we had:
```
error: Failed to load YAML treefile: unknown field `install_langs`, expected one of ... `install-langs` ...
```
Yes, this is a compatibility break with the feature we just announced
but...I seriously doubt anyone (that isn't known to me) has converted
yet, and if they are excited enough to start using a two-week-old feature
they can adjust.
Closes: #1459
Approved by: cgwalters
We're looking to embed an ostree commit inside a container image,
to make it easier to transport around with other images.
Conceptually here the host system is tracking a container (just
like for rojig we're tracking an RPM). This is the first step
towards making that support nicer; tooling can do
`rebase --custom-origin-url oscontainer://quay.io/exampleos@sha256:...`
and have that show up in `rpm-ostree status`.
There are two values, one intended to be machine readable (like
the `ostree://` and `rojig://` and one for humans which we
display when an admin types `rpm-ostree upgrade`.
This builds on prior work in
27bd7b97bb from #1396 .
Closes: #1406
Approved by: jlebon
This test was hard to adapt for staging. I did the obvious change
of just rebooting after most of the `kargs` invocations, but much
later in the suite there was a test that compared vs the "pristine"
kargs. Supporting that would be a bit hard...I briefly thought
about pinning etc. but it gets so messy to do in both staged and not-staged
defaults. So I just deleted those bits.
Closes: #1453
Approved by: jlebon
Noticed this while looking at the logs for #1432.
Because --check and --preview exit with rc=77 when there are no updates,
we would actually stop early on in the test and marking it as SKIPPED.
Fix this by making sure we explicitly check for the $rc we expected when
using those switches.
I also added a final grep pass to make it easy to inspect whether we
skipped any tests. I was about to do this nicely in `multitest.py`
instead, though it may not be of this world much longer, so meh...
Closes: #1450
Approved by: cgwalters
First, split it into its own section; it's important enough to merit it.
Second, explicitly reference the systemd timer/service units. For
example, a question I often have is "when is the next run" and of course
you can get that rpm `systemctl status rpm-ostreed-automatic.timer` but
you have to know that, and the reminder helps.
(I briefly looked at implementing the `Trigger` line from `systemctl status`
but it's not entirely trivial...tempting to just fork off a `systemctl status | grep `)
Prep for unifying this text with the message we print when one does
`rpm-ostree upgrade` when auto-updates are enabled.
Closes: #1432
Approved by: jlebon
First the pinning tests would try to pin a staged deployment,
and some of the later tests here depend on a subtle way on the
state of the system. It's tempting to do a `reset` before each one
and reboot but this makes things work.
There's some additional assertions here as I went through and
was debugging.
Prep for making staging the default.
Closes: #1438
Approved by: jlebon
Add a new `reset` command that makes it easy to blow away all
customizations: overlays, overrides, and initramfs. One can use flags to
only reset some of the customizations.
I placed this under `ex` out of conservatism. It's a pretty simple
command with simple behaviour, though the features it relies on
(no-layering, no-initramfs) are brand new. We can move it out of there
in a release or two?
Closes: #1387Closes: #1419
Approved by: cgwalters
It's been in experimental for a while now, and we've had good feedback
that it's working. With #1392 fixed, it seems in a good position to
declare stable and commit to that API. This also helps empty out `ex` a
bit.
Closes: #1428
Approved by: cgwalters
Follow-up improvement after
https://github.com/openshift/os/pull/135
This should ensure it survives systemd's preset run on firstboot.
Although honestly...what we should *really* do is check whether
the `default.target` symlink target exists, and if not reset it
to `multi-user.target` so no one would have to care, but that'd
be conceptually separate from this, so I may do it later.
Closes: #1427
Approved by: jlebon
We actually want systemd's `ConditionFirstBoot` to fire. The
primary rationale here is that we're adopting Ignition for Fedora CoreOS,
and having `ConditionFirstBoot=` function will help a lot, as the idea
is it only runs once.
However, I discovered that this breaks the `units` directive for example,
as systemd blows away all the unit state in `/etc`. The correct thing
to do from the start is to use presets. We could add an implementation of
`units` which works with this on and instead writes a preset file but...eh.
My plan is to at some point introduce an "epoch" and flip various defaults,
this one, `tmp-is-dir`, the passwd file handling, etc.
See: https://github.com/dustymabe/bootengine/pull/11Closes: #1425
Approved by: jlebon
We added the `no-layering` option, but made it conflicting with
`--install`. This loosens that requirement so that one can do e.g.
rpm-ostree uninstall --all -install foobar
to essentially remove all layered packages and then add back `foobar`.
Prep for `reset` command.
Closes: #1424
Approved by: cgwalters
Kernel arguments added from `rpm-ostree ex kargs` can be lost
after doing multiple deploy related operations (e.g upgrade, install etc).
The root cause of that is because when doing the deployment
(ostree_sysroot_deploy_tree), the kernel arguments from the
pending deployment were never passed in.
This commit adds logic to pass in kernel arguments from pending deployment
to fix the above mentioned problem
Closes: https://github.com/projectatomic/rpm-ostree/issues/1392Closes: #1420
Approved by: cgwalters
This is one part of #1387 and is prep for a single higher level command
that makes it easier to remove all customizations (layering *and*
overrides).
Closes: #1408
Approved by: cgwalters
