Commit Graph

2728 Commits

Author SHA1 Message Date
Colin Walters
c8113bde32 Add hidden coreos-rootfs seal command
All this does is put the immutable bit on the target directory.
The intention is to replace this bit to start:
8b205bfbb9/src/create_disk.sh (L229)

However, the real goal here is to add code in this file
to handle redeploying the rootfs for Fedora CoreOS which
combines OSTree+Ignition:
https://github.com/coreos/fedora-coreos-tracker/issues/94

Basically doing this in proper Rust is going to be a lot
nicer than shell script in dracut modules.  Among other
details, coreutils `mv` doesn't seem to do the right thing
for SELinux labels when policy isn't loaded.
2019-10-04 08:03:03 -07:00
Jonathan Lebon
68750b6894 ci/papr: Drop f29-codestyle and rust-min-version-check
These are also already covered by the Jenkins pipeline.
2019-10-03 13:39:11 -07:00
Jonathan Lebon
289af613a9 ci/jenkins: don't pass GIT_COMMIT to ci-commitmessage-submodules.sh
Jenkins is tricky: it does an initial checkout, merges the PR head into
the target branch, then creates the pod. Once in the pod, we do a
`checkout scm` which *also* merges the PR head into the target branch.
However, the `change.GIT_COMMIT` variable we get from that is set to the
SHA of the first merge, not the second one. Which... yeah is super
confusing since we explicitly assign `change` from that `checkout scm`
operation. So that's probably a valid bug.

This was then throwing off `ci-commitmessage-submodules.sh` since it
didn't find the merge commit in the graph.

Anyway, not going to spend more time on this. Let's just not pass any
commit at all. The git range `origin/master..HEAD` already does what we
want (go through all the commits in HEAD *not* in master).
2019-10-03 13:39:11 -07:00
Colin Walters
010f269492 ci/papr: Drop cosa build
It's currently covered by the Jenkins job.
2019-10-03 10:32:40 -07:00
Rafael Fonseca
4d7af0b49b rust/utils: move common code to a function
Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
2019-10-02 13:45:40 -07:00
Rafael Fonseca
6dab08b646 rust/lockfile: Add YAML support
Fixes #1904

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
2019-10-02 13:45:40 -07:00
Colin Walters
75a2f8bb33 ci/prow: Use cosa buildroot
See https://github.com/coreos/coreos-assembler/pull/730

- Skip installing deps, hooray!
- Also skip the C test suite for now because of the linked fuse-overlayfs issue
2019-10-02 08:16:10 -07:00
Colin Walters
226b939d29 ci/prow: Use build-check.sh
We want to run the unit tests after a build, so do that.
Furthermore, this actually installs the binaries, which
means if we do it as part of a `Dockerfile` build, we
can use that image for further testing.
2019-10-02 08:16:10 -07:00
Jonathan Lebon
ea28cf95c5 app/reset: Fix argc range check
Otherwise, e.g. `rpm-ostree reset foobar` doesn't actually error out.
Some users have hit this as a typo for
`rpm-ostree override reset foobar`.
2019-10-02 04:36:13 -07:00
Jonathan Lebon
677c3c8b29 ci: Also bump MSRV to 1.37.0 for CCI Jenkins
Just split it out into a separate script for easier sharing.
2019-10-01 11:26:29 -04:00
Colin Walters
8b6cfd6f23 OWNERS: New file for Prow integration
This will help us integrate with OpenShift Prow:
https://github.com/coreos/fedora-coreos-tracker/issues/263
2019-09-27 14:58:21 -04:00
Colin Walters
553be6e165 rust: cargo update
Been a while.  I think we may be clear to do this now.  A lot
of stuff like newer rand, etc.

Closes: #1912
Approved by: jlebon
2019-09-26 13:44:27 +00:00
Colin Walters
a846a25dc9 Bump minimum rust to 1.37
We should be clear to do this now.

Closes: #1912
Approved by: jlebon
2019-09-26 13:44:27 +00:00
Jonathan Lebon
8615a964e7 app/status: Fix printf format string for 32-bit
Hit this when compiling in Koji.

Closes: #1910
Approved by: cgwalters
2019-09-25 17:48:47 +00:00
Jonathan Lebon
ccace4c56c Release 2019.6
Last release was almost two months ago. Not a lot of groundbreaking
features, though let's get the various fixes out.

Closes: #1907
Approved by: lucab
2019-09-24 16:23:27 +00:00
Jonathan Lebon
10755592ea Add new ex history command
This is the rpm-ostree equivalent of `dnf history`. As opposed to the
history of the refspec (i.e. `ostree log`), this shows the history of
the system, i.e. the refspecs the host deployed, checksums, versions,
layered packages, etc... The amount of details remembered is similar to
what shows up in `status`.

There's definitely some further enhancements possible (e.g. printing
package diffs, displaying rollbacks), though this seems in good enough
shape as a first cut.

Closes: #1489

Closes: #1813
Approved by: cgwalters
2019-09-24 14:17:12 +00:00
Jonathan Lebon
9e2ceca06f app/deploy: Gate 77 exit behind --unchanged-exit-77
This has a bit of history, but essentially in 1c01141e, we made both
`upgrade` and `deploy` automatically exit 77 if there were no changes.
Then in c3f1e7c8, we only changed `upgrade` so that it became gated
behind `--upgrade-unchanged-exit-77`.

I think we should carry this forward into `deploy` as well. The way I
look at this is: the default UX shouldn't require users to care about
special exit codes. That's something scripts care about. In its vanilla
form, either a command should error out or succeed.

This patch tries to add some consistency by introducing a new
`--unchanged-exit-77` in both `deploy` and `upgrade` (where it just
replaces the previous switch). The naming here matches what `install`
has too.

So... this does break backwards compatibility for any scripts which
relied on that behaviour. Though the only app I know today which wants
deploy semantics and doesn't use the D-Bus API is Zincati, which
actually hit this issue. There's also RHCOS, though the `pivot` there
uses `rebase`, not `deploy`. So overall, I think this is worth breaking
now while we're still in a transitionary period in the downstreams?

Closes: #1906
Approved by: cgwalters
2019-09-24 08:07:04 +00:00
Jonathan Lebon
2943bcaf3a rust/treefile: Various treefile merging tweaks
We were merging some keys with list values by overriding the full list
instead of appending. Fix those.

Also add some missing keys we weren't merging at all.

Closes: #1908
Approved by: cgwalters
2019-09-23 14:28:57 +00:00
Jonathan Lebon
46ab7d1ae8 ci: Add Jenkins pipeline
This is an experiment in using Jenkins pipelines for our CI. See similar
initiatives in coreos-assembler[1] and fedora-coreos-config[2].

For now, this only does the following testing:
- checks commit for unintended submodule bumps
- checks the minimum Rust version
- builds RPMs
- builds FCOS (with the new RPMs both for executing the build
  itself, as well as included in the built OS)

There are dummy placeholders for where we'd actually run the vmcheck
and the compose testsuites. Let's address those trickier parts as
follow-ups.

[1] https://github.com/coreos/coreos-assembler/pull/667
[2] https://github.com/coreos/fedora-coreos-config/pull/131

Closes: #1899
Approved by: cgwalters
2019-09-18 15:15:28 +00:00
worldofpeace
a8ee788e61 Move D-Bus conf file to $(datadir)/dbus-1/system.d
Since D-Bus 1.9.18 configuration files installed by third-party should
go in share/dbus-1/system.d. The old location is for sysadmin overrides.

Closes: #1903
Approved by: jlebon
2019-09-17 04:37:59 +00:00
Colin Walters
13377d4bab postprocess: Ensure dirs are 0755 regardless of umask
`umask` is one of those really evil Unix things...it's pretty
crazy actually there's still no threadsafe way to "`mkdir` ignoring umask".

This surfaced in someone using coreos-assembler with a working
directory of mode `0750` and having that surface in the target
rootfs.

Ref: https://github.com/coreos/fedora-coreos-tracker/issues/272

Closes: #1902
Approved by: jlebon
2019-09-11 20:56:52 +00:00
Jonathan Lebon
680cc10077 spec: Fix documentation files
Follow-up to the recent licensing changes.

This is the first spec file change since moving it upstream.

Closes: #1900
Approved by: cgwalters
2019-09-09 23:50:32 +00:00
Jonathan Lebon
40f6b4bdc9 vmcheck: Adapt test-override-kernel.sh
This is one of the tests right now that assumes it's running on f29.
We might be sort of in this awkward dual path for a while where we want
tests to run on both f29 (i.e. FAH) and f30 (i.e. FCOS).

Closes: #1900
Approved by: cgwalters
2019-09-09 23:50:32 +00:00
Jonathan Lebon
801de7fdae packaging: Adapt to cargo vendor now being built-in
From Rust v1.37, `cargo vendor` is now baked. Stop building it, and
tweak the vendoring script to adapt to the new UX.

Closes: #1900
Approved by: cgwalters
2019-09-09 23:50:32 +00:00
Jonathan Lebon
e11d426f99 packaging: Move canonical spec file here
So this is a somewhat significant change, but I'd like to try having the
canonical spec file upstream. A few reasons for this:

1. We integrate tightly with the distros we're destined for, and so
   we're in a pretty good position for knowing how the software should
   be packaged.
2. We can atomically change packaging along with the rest of the code.
   This has important ramifications, including that it'll be easier to
   integrate with continuous build services like Packit, but releases
   will also be less fraught with last-minute packaging fixes.
3. I'm playing with Jenkins pipelines and there I'd like to make RPMs
   the "artifact" that gets moved down the pipeline into later stages
   (e.g. `cosa build`). We could even eventually make it an actual
   external artifact so that anyone can easily download RPMs from any
   random PR for testing. (And in fact, with a thin yumrepo layer on
   top, it could be used to replace Packit/rdgo entirely).

Not that this approach doesn't have issues as well (e.g. on the dist-git
side, we'll need some minimal tooling to merge in the changelog), though
I think it's worth trying out.

Closes: #1900
Approved by: cgwalters
2019-09-09 23:50:32 +00:00
Jonathan Lebon
146fe31620 packaging: Fix repomanage usage
Not actually planning to use this for now. Noticed it in passing.

Closes: #1900
Approved by: cgwalters
2019-09-09 23:50:32 +00:00
Jonathan Lebon
4824410cc2 packaging: Drop support for dist-snapshot w/o compose
The build system doesn't support this anymore. Also add `-T0` for
multi-threading.

Closes: #1900
Approved by: cgwalters
2019-09-09 23:50:32 +00:00
Jonathan Lebon
e589161c56 ci/installdeps.sh: Pull ostree from rdgo on f30 too
This is a bit of a hack, but does the trick now. Eventually, we'll need
to either bump rdgo to f30, or set up continuous builds some other way
(e.g. Packit or Jenkins pipelines).

Closes: #1900
Approved by: cgwalters
2019-09-09 23:50:32 +00:00
Jonathan Lebon
bc50cacde3 ci/installdeps.sh: Nuke temporary libdnf dep workaround
These are part of the latest release now, so we can nuke them and skip
one `dnf install` invocation.

Closes: #1900
Approved by: cgwalters
2019-09-09 23:50:32 +00:00
Jonathan Lebon
c7814dd0ed ci: Allow passing HEAD commit to ci-commitmessage-submodules.sh
Jenkins' `checkout scm` gives us the exact commit that we're testing.
Allow passing that to the submodule commit message checker so it doesn't
use `HEAD` (which is likely GitHub's merge commit).

Closes: #1900
Approved by: cgwalters
2019-09-09 23:50:32 +00:00
Jonathan Lebon
6d017644d7 ci: Some more el7 path pruning
Use `dnf` consistently. Drop a couple of other conditionals.

Closes: #1900
Approved by: cgwalters
2019-09-09 23:50:32 +00:00
Colin Walters
b8cff948ea ci: Remove papr ex-container check
The command isn't interesting right now (the YAML treefile stuff mostly obsoletes it)
and the CI context costs money/time.

Closes: #1898
Approved by: jlebon
2019-09-06 01:19:24 +00:00
Colin Walters
d34e41cc05 Clarify license situation to include GPLv2, relicense Rust code
We have contacted all contributors to the code in `rust/` and
that code is now all relicensed under the "standard Rust license"
of `Apache 2.0 OR MIT`.

[Due to an accident](https://github.com/projectatomic/rpm-ostree/issues/1890),
some GPLv2+ code was imported in the C side, and we're unlikely to
easily change that now.  Make this more official by adding the GPLv2.

I'd like to go through the C code and add SPDX and possibly investigate
relicensing some of the GPLv2+ code to LGPLv2+ but, not right now.

For a bit more about Rust and SPDX, see [this issue](https://github.com/rust-lang/cargo/issues/2039).

Closes: https://github.com/projectatomic/rpm-ostree/issues/1890

Closes: #1897
Approved by: jlebon
2019-09-05 20:49:18 +00:00
Colin Walters
80fd2e883b README.md: Add a "why" section, update background.md
Point at FCOS and not Project Atomic.  Add an inline "Why"
section since people will want to know that right away.

(An great thing about Github is the prevalence it gives to `README.md`;
 projects should use that as an "elevator pitch")

Drop outdated bits in `background.md`.

Closes: #1895
Approved by: jlebon
2019-09-04 20:09:13 +00:00
Colin Walters
3d6b33b6d4 rust: Add nix as a dependency
I have multiple outstanding PRs that use it and they conflict on
this, so let's just upstream it now.

It helps avoid some use of `unsafe` talking to libc directly.
2019-08-30 10:29:23 -04:00
Colin Walters
2232edb6b0 ci: Add prow Dockerfile
This is an exact copy of https://github.com/ostreedev/ostree/pull/1906
for now.  From that commit message:

I'd like to add OpenShift's prow to this repository. Let's start
by adding a Dockerfile - it doesn't really do anything besides build.

However...I've lately been thinking about e.g. shipping the ostree tests
as an image, and then e.g. we could test FCOS by running that container
(which would orchestrate the host's ostree).

Anyways, not doing that right now but this is a start.
2019-08-28 16:28:22 -04:00
Colin Walters
0d841b2e2f ci: More fixes for cosa build
Use --entrypoint and also rename it while we're here.

Closes: #1889
Approved by: rfairley
2019-08-24 00:35:44 +00:00
Colin Walters
0b7a1f31a7 importer: Add some error prefixing
I was debugging a failure in cosa when running in unprivileged podman,
with this patch the error is:
`error: Importing package 'alternatives': Writing tmpfiles mtree: llistxattr: Is a directory`

And that finally made it obvious to me that the problem is not using
tmpfs for `/tmp` (well, this is a bug in fuse-overlayfs).

Anyways, let's prefix errors some more on general principle.

Closes: #1889
Approved by: rfairley
2019-08-24 00:35:44 +00:00
Colin Walters
1027ade4db ci: Just use cosa as a buildroot too
Since we have to pull it anyways to do a build, let's use it
as a buildroot.  This should fix CI which broke because we were
doing a f29 build but cosa is f30, and libostree differed.
2019-08-22 20:13:41 +00:00
Rafael Fonseca
cac5aa413d app,daemon: Use public libostree's kargs API
This deduplicates the code between the two repositories.

Closes: #1862

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
2019-08-21 16:47:52 -04:00
Colin Walters
acdd04dac6 Move openat_utils to its own crate
I wanted to use this in another project, and I don't think
we're the only ones.  I extracted the code, added better docs
and tests, and published to crates.io:

https://crates.io/crates/openat-ext
2019-08-21 16:44:04 -04:00
Jonathan Lebon
c279f921cb daemon: Tweak default flag handling logic
When hoisting deploy flags from the option dict, we want the default
values to take effect *only if* the option wasn't specified in the dict.
Instead, because we initialized the return value with the default flags,
the option set couldn't override a flag which was part of the defaults
(IOW, a flag that's on by default couldn't be turned off through the
options dict).

Came upon this issue by trying to use the older `Deploy()` and
`Rebase()` D-Bus APIs which pass default values like `ALLOW_DOWNGRADE`,
but the option dict specifies `allow-downgrade=false`. (We could say,
"you should use the newer `UpdateDeployment()` API", but the cat is
mostly out of the bag at this point on options that get shoved into the
dict and take effect on both APIs).
2019-08-21 16:43:37 -04:00
Jonathan Lebon
313b3e655d app/deploy: Add --disallow-downgrade switch
In FCOS, we want to make sure that Zincati is always deploying a newer
tree to prevent downgrade attacks in certain threat models.

For completeness, also add the option to `rebase`.
2019-08-21 16:43:37 -04:00
Colin Walters
7db30fec13
treefile: Support arch-include (#1886)
* treefile: Rework test helper

Have each test clearly hold its tempdir; this way we can have a simple
helper function rather than a struct with an unreferenced parameter.

Also use `utils::write_file`.

Prep for further test work.

* treefile: Add a basic test for includes

Prep for adding an arch include test.

* treefile: Support `arch-include`

A long time ago we added architecture-specific package lists
via e.g. `packages-ppc64le`.  Much more recently we added
support for having the `include` key be a list - multiple includes.

By combining these two and supporting architecture-conditional includes,
we've effectively added architecture-conditionals to *all* keys.

Notably we want this for Fedora CoreOS today which is using
`remove-from-packages` on `grub2-tools` which isn't present on
s390x.

* ci: Bump cosa build timeout

Looks like another OpenStack perf regression.
2019-08-21 15:25:08 -04:00
Colin Walters
016c1c5e62 cli: Add RPMOSTREE_CLIENT_ID environment variable
Currently [zincati](https://github.com/coreos/zincati/) executes
`rpm-ostree deploy` via the CLI - that just shows up as `cli` as
an agent.  This makes it easier to highlight the fact that it's
actually zincati driving things.

Closes: #1880
Approved by: lucab
2019-08-09 19:46:28 +00:00
Timothée Ravier
241c9f508f postprocess: Quote OSTREE_VERSION value
Use g_shell_quote to quote the value set for OSTREE_VERSION in
/etc/os-release as this is an arbitrary string set at compose time that
may contain whitespace or other special characters.

This edge case was found in [0] as the compose are built with the
following command:

  $ rpm-ostree compose tree \
      --repo="repo" \
      --cachedir="cache" \
      --add-metadata-string="version=Kinoite 30.23" \
      "fedora-kinoite.yaml"

[0] https://discussion.fedoraproject.org/t/kinoite-a-kde-and-now-xfce-version-of-fedora-silverblue/147/95

Signed-off-by: Timothée Ravier <tim@siosm.fr>

Closes: #1873
Approved by: jlebon
2019-07-25 07:44:43 +00:00
Jonathan Lebon
71108521f6 Release 2019.5
I think we're in a good position now for FCOS enablement, and there are
a bunch of fixes we should get out, such as the zstd one.

Closes: #1875
Approved by: cgwalters
2019-07-24 16:04:36 +00:00
William Caban
06b4b40701 core: Update to recognize kernel-rt as kernel packages
This will enable to support of real-time kernel on rpm-ostree based
distributions.

Closes: #1868

Closes: #1872
Approved by: jlebon
2019-07-23 18:39:35 +00:00
Jonathan Lebon
8d90d03d59 composeutil: Add cancellable arg to write_composejson
We have more cancellable functions in there now. No reason not to.

Closes: #1871
Approved by: cgwalters
2019-07-19 13:20:42 +00:00
Jonathan Lebon
1577754176 compose: Add --parent option
This may seem like a backflip on #1829, but there's a common theme here:
in a promotion workflow, the parent (or lack of parent) of a commit is
an important parameter, so we need full flexibility in configuring it.

But again, like #1829, we still want e.g. change detection, versioning,
and various optimizations to happen on whatever the latest commit on
that ref is in the build repo.

Closes: #1871
Approved by: cgwalters
2019-07-19 13:20:42 +00:00