Commit Graph

59 Commits

Author SHA1 Message Date
Jonathan Lebon
bc50cacde3 ci/installdeps.sh: Nuke temporary libdnf dep workaround
These are part of the latest release now, so we can nuke them and skip
one `dnf install` invocation.

Closes: #1900
Approved by: cgwalters
2019-09-09 23:50:32 +00:00
Jonathan Lebon
c7814dd0ed ci: Allow passing HEAD commit to ci-commitmessage-submodules.sh
Jenkins' `checkout scm` gives us the exact commit that we're testing.
Allow passing that to the submodule commit message checker so it doesn't
use `HEAD` (which is likely GitHub's merge commit).

Closes: #1900
Approved by: cgwalters
2019-09-09 23:50:32 +00:00
Jonathan Lebon
6d017644d7 ci: Some more el7 path pruning
Use `dnf` consistently. Drop a couple of other conditionals.

Closes: #1900
Approved by: cgwalters
2019-09-09 23:50:32 +00:00
Colin Walters
2232edb6b0 ci: Add prow Dockerfile
This is an exact copy of https://github.com/ostreedev/ostree/pull/1906
for now.  From that commit message:

I'd like to add OpenShift's prow to this repository. Let's start
by adding a Dockerfile - it doesn't really do anything besides build.

However...I've lately been thinking about e.g. shipping the ostree tests
as an image, and then e.g. we could test FCOS by running that container
(which would orchestrate the host's ostree).

Anyways, not doing that right now but this is a start.
2019-08-28 16:28:22 -04:00
Colin Walters
0d841b2e2f ci: More fixes for cosa build
Use --entrypoint and also rename it while we're here.

Closes: #1889
Approved by: rfairley
2019-08-24 00:35:44 +00:00
Colin Walters
1027ade4db ci: Just use cosa as a buildroot too
Since we have to pull it anyways to do a build, let's use it
as a buildroot.  This should fix CI which broke because we were
doing a f29 build but cosa is f30, and libostree differed.
2019-08-22 20:13:41 +00:00
Jonathan Lebon
035ac2eaa6 tests/vmcheck: Fully drop python 3 requirement
Drop the use of Ansible everywhere. In the few cases where we really
Python, just spawn a container instead.

This is required to be able to hack on Fedora CoreOS.

Closes: #1850
Approved by: jlebon
2019-06-10 15:36:06 +00:00
Jonathan Lebon
206ae24d4e tests: Bump to Python 3 only
This bumps the requirement on the controlling host to Python 3 only.
It also bumps the requirement on the target host to Python 3 as well
since FCOS doesn't ship Python 2 right now.

Though we'll need to eventually drop all Python usage anyway, but at
least let's get tests passing on FCOS first. (See related previous
patch).

Closes: #1828
Approved by: cgwalters
2019-05-08 19:02:32 +00:00
Jonathan Lebon
ad1451fac2 Rebase to latest libdnf
This brings us back in sync with the latest libdnf git master. This
required a bunch of work both on the libdnf and rpm-ostree side to get
working. See e.g.
https://github.com/rpm-software-management/libdnf/issues/645.

A few things to adapt to:

- soname bump to `libdnf.so.2`
- `DnfAdvisory` is no longer a `GObject` (annoyingly it's not replaced
  by something we can keep a ref on, so this requires some hacks to
  steal from the `GPtrArray` -- could enhance libdnf for this later)
- disable SWDB history writing
- use new reldep public API
- update for latest `hy_subject_get_best_selector()` API

This now unlocks the possibility to add support for modules. (One can
see hints of this in the diff by the fact that `libdnf` links to
`libmodulemd1`.)

Update submodule: libdnf

Closes: #1404
Approved by: cgwalters
2019-03-19 14:29:15 +00:00
Jonathan Lebon
6cdcd474b6 Makefile-libs.am: Work around g-ir-scanner issues with clang
Right now there's an issue in Fedora with `g-ir-scanner` picking up
`-fstack-clash-protection` from the `sysconfig` Python module and
passing it to `clang`, which doesn't understand this flag yet.

Just work around this by (1) not even building GIR bindings for our
bundled libdnf since there's no need, and (2) overridding the compiler
used by `g-ir-scanner` so it's always `gcc`.

See: https://github.com/projectatomic/rpm-ostree/pull/1787#issuecomment-473971585

Closes: #1787
Approved by: cgwalters
2019-03-19 12:19:38 +00:00
Jonathan Lebon
d34f24581a ci: Fix repo editing in installdeps.sh
- Use `IFS=` so that we don't drop leading whitespace when reading in,
  which can be used to specify multiple baseurls on multiple lines.
- Use `-r` as good practice so we don't try to interpret anything.
- Use `echo "$line"` so that we don't drop leading whitespace when
  writing out.

Closes: #1787
Approved by: cgwalters
2019-03-19 12:19:38 +00:00
Jonathan Lebon
d0fdd9ac1d ci: Drop c7 support for installdeps.sh
Closes: #1787
Approved by: cgwalters
2019-03-19 12:19:38 +00:00
Jonathan Lebon
4ae3b174f5 ci: Bump to f29
Better late than never!

Closes: #1787
Approved by: cgwalters
2019-03-19 12:19:38 +00:00
Jonathan Lebon
6b928e8f0e ci: Build the whole tree, not just rust bits
Otherwise, we're not testing everything. E.g. the cbindgen bits.

Closes: #1778
Approved by: cgwalters
2019-03-07 21:57:08 +00:00
Colin Walters
b63f3720b8 ci: Add a f29 FCOS build
Let's start doing testing using coreos-assembler + Fedora CoreOS.

Closes: #1774
Approved by: jlebon
2019-03-07 19:16:12 +00:00
Colin Walters
387da3a420 ci: Verify rustfmt
Split out the command into a separate `Makefile` that doesn't
use Automake so we can invoke it early.

Closes: #1674
Approved by: dustymabe
2018-11-21 21:16:03 +00:00
Jonathan Lebon
dbdbaf87ee ci: Make adduser invocation idempotent
This is a bit of a selfish request, though I also really like
`ci/build.sh` for hacking in a throwaway environment. This patch allows
one to rerun `ci/build.sh` without erroring out on `testuser` already
existing.

Closes: #1621
Approved by: cgwalters
2018-10-16 16:03:06 +00:00
Jonathan Lebon
cc69168aaf ci: Drop CI_PKGS and epel-release in .papr.yml
Move specific package mentions from the `.papr.yml` file to
`ci/installdeps.sh`. I find the latter script super useful for
bootstrapping a pet container for rpm-ostree development and this helps
with that.

Closes: #1621
Approved by: cgwalters
2018-10-16 16:03:06 +00:00
Jonathan Lebon
e15baebc68 ci: Split out script to install deps
This should also make it useful for bootstrapping a test container
locally.

Closes: #1606
Approved by: cgwalters
2018-10-11 20:39:57 +00:00
Jonathan Lebon
8f504ec5ce ci/build.sh: Drop unnecessary pkg_install cargo
The latest releases in the stable repos now include the new cargo BR, so
we can drop this now.

Closes: #1606
Approved by: cgwalters
2018-10-11 20:39:57 +00:00
Colin Walters
08ad9d49c8 ci: Stop installing python-{sphinx,devel}
We should no longer need it. Just noticed this while I was in the file recently.

Closes: #1551
Approved by: jlebon
2018-09-11 15:35:32 +00:00
Colin Walters
6a274b831d build-sys: Hard require Rust
As something that manages your base operating system, we care
about reliability, predictability, as well as performance and
low-level access to native operating system facilities.  The
C programming language is great for the latter two, but fails
at providing a truly memory-safe environment.  Rust is fairly
unique in providing a language that doesn't carry a runtime,
so we can gradually "oxidize" and convert our C code without
imposing additional overhead.  It's also got a lot of modern
design niceties, like not having a null pointer.

Let's pull the trigger here and hard require Rust.  It's the
programming language I personally want to be primarily writing in for
years to come.

This is also in line with a recent trend of reducing our
experimental/optional matrix.

Closes: #1509
Approved by: jlebon
2018-08-21 14:49:26 +00:00
Jonathan Lebon
b66337e0cb ci: Pepper in date markers to help debug timing
This should help out with determining what steps take the most time.

Closes: #1459
Approved by: cgwalters
2018-07-21 14:43:48 +00:00
Colin Walters
0cc002fd8d build-sys: Use python3 for libdnf by default if available
Probably at some point libdnf will drop py2 support, but the
main reason I'm doing this is avoids a python2 dependency
for rpm-ostree for distributions that don't want that.

Note of course rpm-ostree itself doesn't use python, libdnf does.
And only for the python bindings, which we don't use either.  So
this is mostly just to DTRT automatically for the libdnf bits; down
the line we could probably add a patch to make the python fully
conditional.

Closes: #1460
Approved by: jlebon
2018-07-20 22:27:34 +00:00
Jonathan Lebon
201b020f60 ci: Bump version_id match to pick up ostree FAHC
Closes: #1433
Approved by: jlebon
2018-06-28 19:33:11 +00:00
Jonathan Lebon
592d6052b9 ci: Bump to F28
Closes: #1358
Approved by: cgwalters
2018-05-23 14:18:41 +00:00
Jonathan Lebon
169fab6f4f ci: Drop BDB1539 hack
This shouldn't be needed anymore.

Closes: #1365
Approved by: cgwalters
2018-05-14 18:14:54 +00:00
Colin Walters
d25588722b ci: Ensure we exclude ostree from base fedora too
rdgo uses an earlier git tag due to branches, until we have proper
repo priorities that don't care about versions, let's do this
hack.

Same thing as https://pagure.io/fedora-atomic-host-continuous/c/a7145410fbc7e73188fde449628fa0343b05e4a4?branch=master

Closes: #1326
Approved by: jlebon
2018-04-05 15:26:23 +00:00
Colin Walters
bb86912de9 tests: Dip our toes into using Ansible
This adds a shell primitive to make it easy to execute a playbook
task list.

The big picture idea is to sync with https://github.com/ostreedev/ostree/pull/1462
and rewrite some of the libvm shell stuff as playbooks, allowing easier
code sharing with a-h-t and just in general being a better library for
talking ssh and executing commnads.

Closes: #1297
Approved by: jlebon
2018-03-09 19:04:32 +00:00
Jonathan Lebon
0729487ae5 Check and display pending security advisories
Pick up security advisories when checking for pending updates and
include them in the `cached-update` property. On the client-side,
display them in the output of `status`.

This was part of the original vision for how useful a smart `check` mode
could be. It directly impacts how one manages their individual system
(e.g. when to reboot), and paves the way for integration into
higher-level apps that act at the cluster level.

Closes: #1249
Approved by: cgwalters
2018-02-15 15:30:26 +00:00
Colin Walters
b85ae9e1d6 jigdo: V4: Use archful provides for jigdoRPM Requires
When I tried to use my WIP client patches to do:
`rpm-ostree rebase rojig://fahc:fedora-atomic-host`,
I got a missing file object which turned out to
be the client importing the i686 RPMs.

This was passing in the test suite because we don't mirror i686 of course, but
on the client side right now we end up using all enabled repos, and since Fedora
is multiarch, the behavior is going to be...not predictable.

Thinking a bit about on this problem I actually happened to recall
the RPM `%{_isa}` macro which is used in Fedora in various places;
for example to "arch bind" `-devel` packages to their base.  See
for example [this case](33c7dc02bc/f/ostree.spec (_79)) in libostree.

As I noted at first, the core problem here is that the "final"
RPM architecture field is not symmetric in any way with the definition
of that `%{_isa}` macro.  See:

d9d47e0114/installplatform (L25)

The *third* solution I ended up on here is to iterate over the
`Provides` on the server side and we take the first thing
that matches `Provides: %{name}(whatever)`.

I briefly thought about trying to somehow drive into libsolv the
logic to prefer the jigdoRPM's native architecture...IIRC yum did
something like that in the past but it was never done in libsolv?
Anyways the dependencies here are now more correct, so other tools
will also handle it.

Closes: #1213
Approved by: jlebon
2018-01-19 14:18:19 +00:00
Jonathan Lebon
83fb40593f ci: explicitly pass --sysconfdir=/etc to configure
I spent some time trying to figure out why on CentOS my changes to
`/etc/rpm-ostreed.conf` didn't seem to have any effect. It turns out
that for some reason I didn't spend too much time looking into, the
autoconf in CentOS defaults to `${prefix}/etc` when `--prefix` is given.

This is also why I was under the erroneous assumption that D-Bus config
files were in `/usr/etc` in CentOS. This patch fixes that as well.

Closes: #1215
Approved by: cgwalters
2018-01-18 20:44:07 +00:00
Jonathan Lebon
5c9a9c30f7 ci/build-check: add -Wno-error=deprecated-declarations
For libdnf.

Closes: #1114
Approved by: jlebon
2018-01-16 14:14:33 +00:00
Colin Walters
90f9fe80e4 scripts: Drop most capabilities
Note this PR requires [bubblewrap 0.2.0](https://github.com/projectatomic/bubblewrap/releases/tag/v0.2.0).

Change our bwrap invocations drop truly dangerous capabilities like
`cap_sys_admin` and `cap_sys_module` just like Docker does today. Because of the
popularity of Docker, we can be pretty sure that most RPM scripts should have
adapted to this (although a problematic area here is that traditional librpm
doesn't actually error out if scripts fail).

There are two reasons to do this:

 - We want "offline" updates by default; updates shouldn't affect the
   running system.  If we prepare the new root in the background, a
   %post shouldn't restart a service for example.  We already "handle"
   this by making `systemctl` a symlink to `/bin/true`, but this approach
   also shuts off `%post`s that do e.g. `insmod`.
 - Protection against accidental system damage

Closes: #1099
Approved by: jlebon
2017-12-05 02:54:23 +00:00
Colin Walters
fcc30ffe5d ci/build.sh: s/26/27/ for FAHC check
Missed this in https://github.com/projectatomic/rpm-ostree/pull/1125

Closes: #1123
Approved by: jlebon
2017-12-04 20:13:34 +00:00
Colin Walters
6bf4206223 compose: Add --ex-unified-core
The "--ex" prefix here means it's an experimental option. A tremendous change
here is that start to support non-uid 0, but there are various things to fix there;
the unpacker for example needs to learn to set imported objects fully based
on the rpmfi information (i.e. default to uid 0, since libarchive gives the
current uid by default).

And even when run as uid 0, there are some bugs, though I'm not sure
of any showstoppers yet.  For example, dracut's `dracut-install` calls
`cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs
from a checkout (it shouldn't be copying that anyways...)

Nevertheless, the infrastructure behind this really helps (is almost a hard
requirement for) the [jigdo effort](https://github.com/projectatomic/rpm-ostree/issues/1081).
Which is really only true due to SELinux - we need to import the packages,
then generate the final tree to get the final policy, then use that policy
to relabel all of the packages.

Closes: #940
Approved by: jlebon
2017-11-17 18:59:34 +00:00
Jonathan Lebon
235f2945bf ci: don't use CentOS Alpha anymore
It's no longer being built and is now older than the latest CentOS AH
release. This should help us no longer see messages like:

(rpm-ostree pkg-add:5662): GLib-CRITICAL **: g_variant_dict_lookup:
assertion 'is_valid_dict (dict)' failed

which happen because in #1034, we started using `G_VARIANT_DICT_INIT`,
whose special magic values only make sense in glib2 >= 2.50. (The alpha
image stopped at 2.46).

Saw this while debugging #1035.

Closes: #1040
Approved by: cgwalters
2017-10-06 00:42:55 +00:00
Jonathan Lebon
5db2389fbb ci: run clang build with -Werror
Start running `clang` with `-Werror` like ostree. We can only run this
on Fedora right now because the CentOS `clang` doesn't support
`-Wno-error=macro-redefined`, though that's fine.

Closes: #1036
Approved by: cgwalters
2017-10-04 21:10:55 +00:00
Jonathan Lebon
a9c38d33b8 vmcheck: add SELinux labeling tests
There's a lot of paths in the core related to SELinux policy changes and
relabeling packages. We currently have no test coverage for them. We add
support in the test libraries here to build such packages.

We also add a test that checks both that we correctly relabel RPMs when
the policy changes and that we handle layered packages that install
SELinux packages properly.

Closes: #999
Approved by: cgwalters
2017-10-03 01:01:14 +00:00
Colin Walters
1acd834104 Add test infra for ex container, and one test for bash
We have some unit-style tests that run `ex container`, but
they aren't "real"; they don't use scripts for example.  Let's
add tests for this similar to `tests/compose`.

We use a 26 base, but the target repos need to be 27
to pick up the fix for: https://bugzilla.redhat.com/show_bug.cgi?id=1478172

Add some bits to share infra between `tests/compose` and `tests/ex-container`;
basically handling the rpmmd repos. I tweaked things to be more streamlined
there between the `.papr.yml` and the test script.

Right now this is just one test for `bash`, but lays some of the infrastructure
for doing more. One thing that we need to do to improve more here is to better
cache RPMs, a bit like the compose tests do.

Closes: #1024
Approved by: jlebon
2017-10-02 18:04:24 +00:00
Colin Walters
b7733510bd ci: Reduce metadata expiration for rdgo
At some point I'll fix rdgo to retain multiple, but I think this may help CI
reliability in the short term.

Closes: #1015
Approved by: jlebon
2017-09-26 19:41:08 +00:00
Colin Walters
2ebf43f4a9 ci: Sync with ostree
We have some drift; keep this more in sync so we maintain the rpm/yum
hackarounds for example in one place.

I backed out the ASAN bits though pending at least a newer gpgme:
<https://github.com/projectatomic/rpm-ostree/pull/1000#issuecomment-331278758>

Closes: #1000
Approved by: jlebon
2017-09-24 13:32:00 +00:00
Jonathan Lebon
2c514f446c ci: don't run clang on CentOS
The CentOS tester is currently failing because clang is being passed an
argument it doesn't know about:

  clang: error: unknown argument: '-fstack-protector-strong'

We get our version of clang from EPEL, which currently has 3.4.2. The
Fedora clang does have support for the flag (4.0.1).

Anyway, the point of this check is just to find unused/uninitialized
vars and other gotchas. In that sense, we're more interested in what the
latest version of clang has to say.

Closes: #992
Approved by: cgwalters
2017-09-14 21:02:46 +00:00
Jonathan Lebon
48ddca9280 ci: workaround broken fedora:26 image
This is essentially the same workaround as
https://github.com/ostreedev/ostree/pull/1143.

See https://bugzilla.redhat.com/show_bug.cgi?id=1483553.

Closes: #975
Approved by: cgwalters
2017-09-07 12:27:54 +00:00
Jonathan Lebon
5294c1ea69 scripts: squash a -Wmaybe-uninitialized
Closes: #968
Approved by: cgwalters
2017-09-01 19:58:55 +00:00
Jonathan Lebon
44ea9f35a1 ci: also compile tests
This should make sure we catch test errors such as
https://github.com/GNOME/libglnx/pull/76 when bumping submodules (until
we eventually get a PR tester on libglnx).

Requires: https://github.com/GNOME/libglnx/pull/76

Update submodule: libglnx

Closes: #954
Approved by: cgwalters
2017-08-29 13:07:21 +00:00
Jonathan Lebon
a2878de382 ci: no longer install polkit-devel
This should get pulled in automatically now.

Closes: #896
Approved by: cgwalters
2017-07-26 19:22:51 +00:00
Colin Walters
fee6d06bf4 lib: Expose new API around basearch
For https://pagure.io/atomic-wg/issue/299 we need to make it
more convenient to substitute the architecture in an installation
context.  I plan to use this API inside `rpmostreepayload` in Anaconda,
so we can substitute the same value of `${basearch}` we use in treefiles
since https://github.com/projectatomic/rpm-ostree/pull/305

Now, you might wonder - why do we need an API wrapping libdnf? It's because
libdnf is not API stable yet. We're just exposing a tiny subset. In theory we
could use the Python dnf bindings in Anaconda, but things get slightly weird if
rpmostreepayload depends on dnf. Perhaps we'll do that down the road, but for
now this a small API surface to maintain (forever).

This change reworks the internal `varsubst` bits to take a pure `DnfContext`,
since we don't want to spin up a whole `RpmOstreeContext` just to do some
string substitutions.

Closes: #877
Approved by: jlebon
2017-07-21 16:02:41 +00:00
Jonathan Lebon
f25444554d ci: no longer run gnome-desktop-testing
We don't currently install any installed tests usable by
gnome-desktop-testing. This conveniently works around not having it
readily packaged on CentOS, though we could always build from source if
the need comes.

Closes: #871
Approved by: cgwalters
2017-07-18 13:58:38 +00:00
Jonathan Lebon
cf19d83502 ci: unite testsuites and run vmcheck on centos
We can be a bit less wasteful here by merging the check and vmcheck
suites into a single suite. The check suite today takes a negligible
amount of time to run, so we're not gaining much by parallelizing them.
It's more of a sanity check at this point before we start vmcheck.

Also start running vmcheck on CentOS 7. We adapt the ci scripts to
accomodate both Fedora and CentOS target machines.

This commit also switches to Fedora 26 as the primary test base.

Closes: #871
Approved by: cgwalters
2017-07-18 13:58:38 +00:00