Commit Graph

41 Commits

Author SHA1 Message Date
Jonathan Lebon
cc69168aaf ci: Drop CI_PKGS and epel-release in .papr.yml
Move specific package mentions from the `.papr.yml` file to
`ci/installdeps.sh`. I find the latter script super useful for
bootstrapping a pet container for rpm-ostree development and this helps
with that.

Closes: #1621
Approved by: cgwalters
2018-10-16 16:03:06 +00:00
Jonathan Lebon
7c2b9c3276 ci: Add rust-min-version-check
Add a check to make sure we stay within the minimum version required to
build with DTS, which is updated frequently, but may still lag behind in
comparison to Fedora stable.

Closes: #1606
Approved by: cgwalters
2018-10-11 20:39:57 +00:00
Jonathan Lebon
16b005bd56 ci: Use continuous image instead of rebasing to it
That should save us a bunch of time. We're also seeing some flakes right
now from rebasing.

Closes: #1585
Approved by: jlebon
2018-10-02 13:14:08 +00:00
Jonathan Lebon
0b14c18a7b ci/compose: Use host repos for building rpm-ostree too
We want to use the host repos not just for downloading the packages
we'll compose trees with, but also for the rpm-ostree build itself
leading up to the tests.

Closes: #1585
Approved by: jlebon
2018-10-02 13:14:08 +00:00
Jonathan Lebon
640c1d74c9 ci: Stop rebasing FAH on updates branch
The stable branch should have the updated librpm now.
Also bump slightly the compose tests timeout.

Closes: #1585
Approved by: jlebon
2018-10-02 13:14:08 +00:00
Jonathan Lebon
f6c0788b4e ci: Use FAH updates branch for newer librpm
Right now, we're compiling against librpm 4.14.2, but running against
4.14.1, so we're hitting missing symbol issues from #1469. Just use the
updates ref for now until the next stable FAH update.

Closes: #1538
Approved by: cgwalters
2018-09-08 21:51:22 +00:00
Colin Walters
6a274b831d build-sys: Hard require Rust
As something that manages your base operating system, we care
about reliability, predictability, as well as performance and
low-level access to native operating system facilities.  The
C programming language is great for the latter two, but fails
at providing a truly memory-safe environment.  Rust is fairly
unique in providing a language that doesn't carry a runtime,
so we can gradually "oxidize" and convert our C code without
imposing additional overhead.  It's also got a lot of modern
design niceties, like not having a null pointer.

Let's pull the trigger here and hard require Rust.  It's the
programming language I personally want to be primarily writing in for
years to come.

This is also in line with a recent trend of reducing our
experimental/optional matrix.

Closes: #1509
Approved by: jlebon
2018-08-21 14:49:26 +00:00
Jonathan Lebon
45e162fb69 ci: Split compose test into two
The `f28-compose` test keeps timing out. Some time recently, I/O
performance of the internal OpenStack instance used for testing has
degraded. I have a ticket open to investigate the regression though
haven't had any luck so far.

Let's just take the easy way out and split the test into two testsuites.
This is obviously hacky, and sad, and unfortunate. But the PRs must keep
flowing until we finally wean off of OpenStack.

Closes: #1498
Approved by: cgwalters
2018-08-13 21:06:18 +00:00
Jonathan Lebon
1a82f210ae ci: Bump f28-compose by 10 minutes
We're consistently seeing timeouts on the `f28-compose` test. It's
likely due to a performance regression in the backend rather than
anything else. I filed a ticket to get an investigation going, though
for now at least, let's bump the timeout some more to keep patches
flowing.

Closes: #1463
Approved by: jlebon
2018-07-24 20:15:10 +00:00
Jonathan Lebon
ec19d67d74 ci: Stop rebasing Fedora onto updates ref
We can drop this hack now since there have been updates since GA. Also
add a note that trying to drop the rebase to continuous causes
`layering-relabel` to fail.

Closes: #1406
Approved by: jlebon
2018-07-20 18:47:51 +00:00
Jonathan Lebon
fbee05bcc5 vmcheck/autoupdate-check.sh: Fix --check/--preview tests
Noticed this while looking at the logs for #1432.

Because --check and --preview exit with rc=77 when there are no updates,
we would actually stop early on in the test and marking it as SKIPPED.
Fix this by making sure we explicitly check for the $rc we expected when
using those switches.

I also added a final grep pass to make it easy to inspect whether we
skipped any tests. I was about to do this nicely in `multitest.py`
instead, though it may not be of this world much longer, so meh...

Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Jonathan Lebon
479406e6a5 Add support for YAML treefiles
Let's modernize and start supporting YAML treefiles. I'll dare make the
sweeping generalization that most people would prefer reading and
writing YAML over JSON.

This takes bits from coreos-assembler[1] that know how to serialize a
YAML file and spit it back out as a JSON and makes it into a shared lib
that we can link against. We could use this eventually for JSON inputs
as well to force a validation check before composing.

If we go this route, we could then turn on `--enable-rust` in FAHC for
now and drop the duplicate code in coreos-assembler.

[1] https://github.com/cgwalters/coreos-assembler

Closes: #1377
Approved by: cgwalters
2018-06-05 13:08:33 +00:00
Jonathan Lebon
592d6052b9 ci: Bump to F28
Closes: #1358
Approved by: cgwalters
2018-05-23 14:18:41 +00:00
Jonathan Lebon
377b1674da ci: Drop f27-sanity testsuite
This started out as an experiment to see how useful it'd be in
rpm-ostree. Again, the goal wasn't to leverage the rpm-ostree-specific
tests of a-h-t here, but rather running the *other* tests to ensure that
we weren't inadvertedly impacting userspace. In the end, as far as I
remember, it hasn't brought up anything since adding it here, so let's
bring the experiment to an end.

It might still be worth doing this same experiment in the ostree CI
instead, though I don't want to add more load there until we fully
switch over to the new CI infra.

Closes: #1369
Approved by: cgwalters
2018-05-16 15:19:13 +00:00
Colin Walters
e57504a88a ci: Split codestyle checks into separate context
Prep for reworking the primary test to do vm-in-container, which
will temporarily be vm-in-container-in-vm.

See https://github.com/projectatomic/rpm-ostree/pull/1362

Closes: #1364
Approved by: jlebon
2018-05-14 19:50:36 +00:00
Jonathan Lebon
38b11d34e5 ci: Use continuous branch for now
We are hitting issues with a-h-t on CAHC right now failing due to an
`atomic scan` issue and preventing promotion of continuous content to
the smoketested branch. This is then blocking our CI which needs a newer
tree with the latest glib2 baked in. Let's unblock our CI for now by
rebasing on the continuous branch. We'll drop that hack once the a-h-t
issues are ironed out.

Closes: #1365
Approved by: cgwalters
2018-05-14 18:14:54 +00:00
Colin Walters
bb86912de9 tests: Dip our toes into using Ansible
This adds a shell primitive to make it easy to execute a playbook
task list.

The big picture idea is to sync with https://github.com/ostreedev/ostree/pull/1462
and rewrite some of the libvm shell stuff as playbooks, allowing easier
code sharing with a-h-t and just in general being a better library for
talking ssh and executing commnads.

Closes: #1297
Approved by: jlebon
2018-03-09 19:04:32 +00:00
Jonathan Lebon
99aa6f89d2 ci: expand rootfs on CentOS
The new kernel tests require a little bit more space.

Closes: #1247
Approved by: cgwalters
2018-02-14 16:16:52 +00:00
Colin Walters
a3e0b47e6b ci/papr: Bump compose tests to 4 min vCPUs
Compose is a slow test right now.  Down the line what I'd
like to do is: https://github.com/projectatomic/papr/pull/70
Since this job can be scheduled as a container, not a VM.  There's
no reason to grab a whole 8GB of RAM for it, but we *do* want multiple
CPUs.  Containers do that by default.

Closes: #1187
Approved by: jlebon
2018-01-09 16:26:32 +00:00
Colin Walters
0e16be8273 tests: Run compose tests in parallel ∥
Over a year later, the "opening the host rpmdb" bug is fixed,
so we can do composes in parallel ∥, hooray!

I'm dusting this off since we were running into CI (PAPR) timeouts
when I was adding more to the compose tests.

Closes: #545
Approved by: jlebon
2018-01-09 14:19:31 +00:00
Colin Walters
90f9fe80e4 scripts: Drop most capabilities
Note this PR requires [bubblewrap 0.2.0](https://github.com/projectatomic/bubblewrap/releases/tag/v0.2.0).

Change our bwrap invocations drop truly dangerous capabilities like
`cap_sys_admin` and `cap_sys_module` just like Docker does today. Because of the
popularity of Docker, we can be pretty sure that most RPM scripts should have
adapted to this (although a problematic area here is that traditional librpm
doesn't actually error out if scripts fail).

There are two reasons to do this:

 - We want "offline" updates by default; updates shouldn't affect the
   running system.  If we prepare the new root in the background, a
   %post shouldn't restart a service for example.  We already "handle"
   this by making `systemctl` a symlink to `/bin/true`, but this approach
   also shuts off `%post`s that do e.g. `insmod`.
 - Protection against accidental system damage

Closes: #1099
Approved by: jlebon
2017-12-05 02:54:23 +00:00
Colin Walters
b84e795c96 ci/papr: Update the other contexts to F27
Let's take a stab at this.

Closes: #1125
Approved by: jlebon
2017-12-04 17:45:56 +00:00
Jonathan Lebon
e35f867494 papr: fix sanity test and bump to f27
It was recently changed to use virtualenv to install and invoke ansible.

Closes: #1112
Approved by: miabbott
2017-11-22 16:55:17 +00:00
Colin Walters
d87970966b ci/papr: Update to f27-primary
I didn't try porting some of the other contexts yet...I want to see
if this works.

Note that FAHC was switched to 27, so it's only luck that things
work on 26.

Closes: #1108
Approved by: jlebon
2017-11-17 19:46:38 +00:00
Colin Walters
cdfba8e131 ci: Update CentOS container to current 7
The rpmmd repo issues seem to be fixed, and the old container was (for some
reason) apparently removed.

Closes: #1073
Approved by: jlebon
2017-10-24 16:26:32 +00:00
Jonathan Lebon
235f2945bf ci: don't use CentOS Alpha anymore
It's no longer being built and is now older than the latest CentOS AH
release. This should help us no longer see messages like:

(rpm-ostree pkg-add:5662): GLib-CRITICAL **: g_variant_dict_lookup:
assertion 'is_valid_dict (dict)' failed

which happen because in #1034, we started using `G_VARIANT_DICT_INIT`,
whose special magic values only make sense in glib2 >= 2.50. (The alpha
image stopped at 2.46).

Saw this while debugging #1035.

Closes: #1040
Approved by: cgwalters
2017-10-06 00:42:55 +00:00
Colin Walters
1acd834104 Add test infra for ex container, and one test for bash
We have some unit-style tests that run `ex container`, but
they aren't "real"; they don't use scripts for example.  Let's
add tests for this similar to `tests/compose`.

We use a 26 base, but the target repos need to be 27
to pick up the fix for: https://bugzilla.redhat.com/show_bug.cgi?id=1478172

Add some bits to share infra between `tests/compose` and `tests/ex-container`;
basically handling the rpmmd repos. I tweaked things to be more streamlined
there between the `.papr.yml` and the test script.

Right now this is just one test for `bash`, but lays some of the infrastructure
for doing more. One thing that we need to do to improve more here is to better
cache RPMs, a bit like the compose tests do.

Closes: #1024
Approved by: jlebon
2017-10-02 18:04:24 +00:00
Colin Walters
2ebf43f4a9 ci: Sync with ostree
We have some drift; keep this more in sync so we maintain the rpm/yum
hackarounds for example in one place.

I backed out the ASAN bits though pending at least a newer gpgme:
<https://github.com/projectatomic/rpm-ostree/pull/1000#issuecomment-331278758>

Closes: #1000
Approved by: jlebon
2017-09-24 13:32:00 +00:00
Colin Walters
aab7571fc1 ci: Fixate centos container image to 7.3 until rpm-md repos update
Sigh.

Closes: #985
Approved by: jlebon
2017-09-12 13:45:50 +00:00
Jonathan Lebon
48ddca9280 ci: workaround broken fedora:26 image
This is essentially the same workaround as
https://github.com/ostreedev/ostree/pull/1143.

See https://bugzilla.redhat.com/show_bug.cgi?id=1483553.

Closes: #975
Approved by: cgwalters
2017-09-07 12:27:54 +00:00
Jonathan Lebon
f5bcd62364 ci: run the improved-sanity-test
Start running the improved-sanity-test from atomic-host-tests to make
sure the system works. This is an easy way to get PR-level comprehensive
integration tests for free.

But note that we *don't* mark it as required since the tests are not
stored here and it can happen that they need to be adjusted for new
rpm-ostree behaviours. In this way, this added check also allows us to
give a heads-up that breaking changes are coming.

Closes: #899
Approved by: cgwalters
2017-08-24 20:43:16 +00:00
Colin Walters
b46fc35901 Add releasever opt, avoid opening up host's rpmdb in treecompose
Closes: https://github.com/projectatomic/rpm-ostree/issues/546

Previously, we'd open up the host's rpmdb for both `compose tree`
and `ex container`.  In the first case, because we require root, we'd
succeed.  For `ex container`, we'd spew an error.

Fixing this was trickier than I thought.  First because there was
*also* a libdnf bug here: https://github.com/rpm-software-management/libdnf/pull/307

Second, there's a compatibility hazard here for anyone using `.repo` files that
reference `$releasever`. This actually happened to me with `ex container` as I'd
just done a `ln -s /etc/yum.repos.d/fedora.repo rpmmd.repos.d`.  I fixed
that first by doing a `sed -i -e 's,$releasever,26,' rpmmd.repos.d/*.repo`.

As far as I can see today, none of Fedora Atomic or CentOS AH rely on this. But
in order to enhance compatibility, let's add a "releasever" option. This makes
it easier again to reuse stock `.repo` files if we wanted to do so.

(Also, I realized we can just use `/usr/share/empty` as *the* canonical immutable
 empty directory)

Closes: #875
Approved by: jlebon
2017-07-21 15:02:59 +00:00
Jonathan Lebon
1806259cc1 ci: stop working around updates-source mirrors
I initially did this because mirrors weren't stable before the F26
release. But now, the canonical source itself is unstable, so let's try
our luck again with mirrors. Might have better luck now that it's
released.

Closes: #881
Approved by: mbarnes
2017-07-18 17:38:43 +00:00
Jonathan Lebon
cf19d83502 ci: unite testsuites and run vmcheck on centos
We can be a bit less wasteful here by merging the check and vmcheck
suites into a single suite. The check suite today takes a negligible
amount of time to run, so we're not gaining much by parallelizing them.
It's more of a sanity check at this point before we start vmcheck.

Also start running vmcheck on CentOS 7. We adapt the ci scripts to
accomodate both Fedora and CentOS target machines.

This commit also switches to Fedora 26 as the primary test base.

Closes: #871
Approved by: cgwalters
2017-07-18 13:58:38 +00:00
Jonathan Lebon
760110596b ci: start testing on F26
Now that PAPR has support for pre-release images of Fedora Atomic Host
26, let's start testing there. We mark it as not required for the time
being.

Closes: #860
Approved by: cgwalters
2017-07-06 18:24:47 +00:00
Jonathan Lebon
87b978c088 ci: make compose test required
This has been around for a while now and has found regressions in the
past, so let's make it required. Also rename the contexts to make it
clear that what version of Fedora they're running on.

Closes: #860
Approved by: cgwalters
2017-07-06 18:24:47 +00:00
Colin Walters
7d3b72c814 Remove all tabs ⭾ in *.[ch] and add a CI check for them
There weren't many left, and I happened to have one of these files open and it
was distracting.

Closes: #839
Approved by: jlebon
2017-06-22 01:20:55 +00:00
Kalev Lember
41a8c7a02e ci: Install polkit-devel for c7-build
We need this until the updated package with the polkit support added
makes it into a repo and yum-builddep picks up the new dep.

Closes: #825
Approved by: cgwalters
2017-06-19 21:19:42 +00:00
Jonathan Lebon
5d8783b62d ci: add centos build checker
We should make sure that we can build on CentOS 7, so that we don't
inadvertently break the rdgo c7 stream. This will be enforced even more
explicitly when we were integrate RPM building in PAPR.

Closes: #824
Approved by: cgwalters
2017-06-08 21:14:58 +00:00
Jonathan Lebon
ac9c3c1635 compose: use test env fedora.repo file instead
During provisioning, PAPR injects a fedora.repo pointing at a much
better & faster mirror than dl.fp.o. Let's use that to make the compose
test less flaky. Hoping to make these sorts of optimizations more
discoverable in upstream PAPR.

Closes: #799
Approved by: cgwalters
2017-05-30 14:17:38 +00:00
Jonathan Lebon
8b8bdcc600 PAPR: migrate to the new name
The redhat-ci service has been renamed to PAPR. Previous values are
still supported though should be considered deprecated.

Closes: #790
Approved by: cgwalters
2017-05-25 16:27:10 +00:00