Commit Graph

57 Commits

Author SHA1 Message Date
Jonathan Lebon
450948a9f6 ci/papr: Drop required commit status context
This was useful in combination with Homu since it only had to watch one
context. Since we're not using Homu anymore (and Tide instead looks at
all statuses by default), let's just drop it. This brings down the
number of statuses on PRs by one more (and so one less context to
override when needed).

Relatedly, also just test on `master` now.
2019-10-09 10:22:58 -07:00
Jonathan Lebon
68750b6894 ci/papr: Drop f29-codestyle and rust-min-version-check
These are also already covered by the Jenkins pipeline.
2019-10-03 13:39:11 -07:00
Colin Walters
010f269492 ci/papr: Drop cosa build
It's currently covered by the Jenkins job.
2019-10-03 10:32:40 -07:00
Jonathan Lebon
677c3c8b29 ci: Also bump MSRV to 1.37.0 for CCI Jenkins
Just split it out into a separate script for easier sharing.
2019-10-01 11:26:29 -04:00
Colin Walters
a846a25dc9 Bump minimum rust to 1.37
We should be clear to do this now.

Closes: #1912
Approved by: jlebon
2019-09-26 13:44:27 +00:00
Colin Walters
b8cff948ea ci: Remove papr ex-container check
The command isn't interesting right now (the YAML treefile stuff mostly obsoletes it)
and the CI context costs money/time.

Closes: #1898
Approved by: jlebon
2019-09-06 01:19:24 +00:00
Colin Walters
0d841b2e2f ci: More fixes for cosa build
Use --entrypoint and also rename it while we're here.

Closes: #1889
Approved by: rfairley
2019-08-24 00:35:44 +00:00
Colin Walters
1027ade4db ci: Just use cosa as a buildroot too
Since we have to pull it anyways to do a build, let's use it
as a buildroot.  This should fix CI which broke because we were
doing a f29 build but cosa is f30, and libostree differed.
2019-08-22 20:13:41 +00:00
Colin Walters
7db30fec13
treefile: Support arch-include (#1886)
* treefile: Rework test helper

Have each test clearly hold its tempdir; this way we can have a simple
helper function rather than a struct with an unreferenced parameter.

Also use `utils::write_file`.

Prep for further test work.

* treefile: Add a basic test for includes

Prep for adding an arch include test.

* treefile: Support `arch-include`

A long time ago we added architecture-specific package lists
via e.g. `packages-ppc64le`.  Much more recently we added
support for having the `include` key be a list - multiple includes.

By combining these two and supporting architecture-conditional includes,
we've effectively added architecture-conditionals to *all* keys.

Notably we want this for Fedora CoreOS today which is using
`remove-from-packages` on `grub2-tools` which isn't present on
s390x.

* ci: Bump cosa build timeout

Looks like another OpenStack perf regression.
2019-08-21 15:25:08 -04:00
Jonathan Lebon
95d1c6ea29 ci: Bump minimum Rust version to v1.31.0
Now that we no longer support el7, let's peg our minimum `rustc` version
to the latest Rust module available in el8 instead. That's 1.31.0 right
now.

And drop use of deprecated `trim_left_matches()` (it was deprecated in
1.33.0, which is currently used in the other testsuites that pull
`rustc` from the Fedora repos; we might want to eventually just use the
`RUST_MIN_VERSION` in all the testsuites to not get caught in the middle
like this again).

Closes: #1787
Approved by: cgwalters
2019-03-19 12:19:38 +00:00
Jonathan Lebon
4ae3b174f5 ci: Bump to f29
Better late than never!

Closes: #1787
Approved by: cgwalters
2019-03-19 12:19:38 +00:00
Jonathan Lebon
bdf12cced6 ci: Stop testing on CentOS 7
Let's stop actively testing on CentOS 7 from this point on. The latest
stable version v2019.2 will be the last fully supported release on this
platform. We may backport patches there if needed.

This will allow us to shed a lot of legacy and dual-paths code that
rendered the code more complex. It'll also reduce friction on new
initiatives like unified core by default and sysusers.

Closes: #1785
Approved by: cgwalters
2019-03-13 21:16:00 +00:00
Jonathan Lebon
6b928e8f0e ci: Build the whole tree, not just rust bits
Otherwise, we're not testing everything. E.g. the cbindgen bits.

Closes: #1778
Approved by: cgwalters
2019-03-07 21:57:08 +00:00
Colin Walters
b63f3720b8 ci: Add a f29 FCOS build
Let's start doing testing using coreos-assembler + Fedora CoreOS.

Closes: #1774
Approved by: jlebon
2019-03-07 19:16:12 +00:00
Jonathan Lebon
1f72fbb055 ci: Bump minimum Rust version to 1.29.2
This matches the latest DTS release on el7.

Closes: #1716
Approved by: cgwalters
2018-12-17 22:06:49 +00:00
Jonathan Lebon
71f0a30d83 ci: Go back to centos/7/atomic/smoketested
We should prefer the smoketested ref in general since it's more stable.
There is an updated glib2 there now. I also tested that layering-relabel
works (at least locally).

Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Jonathan Lebon
cc69168aaf ci: Drop CI_PKGS and epel-release in .papr.yml
Move specific package mentions from the `.papr.yml` file to
`ci/installdeps.sh`. I find the latter script super useful for
bootstrapping a pet container for rpm-ostree development and this helps
with that.

Closes: #1621
Approved by: cgwalters
2018-10-16 16:03:06 +00:00
Jonathan Lebon
7c2b9c3276 ci: Add rust-min-version-check
Add a check to make sure we stay within the minimum version required to
build with DTS, which is updated frequently, but may still lag behind in
comparison to Fedora stable.

Closes: #1606
Approved by: cgwalters
2018-10-11 20:39:57 +00:00
Jonathan Lebon
16b005bd56 ci: Use continuous image instead of rebasing to it
That should save us a bunch of time. We're also seeing some flakes right
now from rebasing.

Closes: #1585
Approved by: jlebon
2018-10-02 13:14:08 +00:00
Jonathan Lebon
0b14c18a7b ci/compose: Use host repos for building rpm-ostree too
We want to use the host repos not just for downloading the packages
we'll compose trees with, but also for the rpm-ostree build itself
leading up to the tests.

Closes: #1585
Approved by: jlebon
2018-10-02 13:14:08 +00:00
Jonathan Lebon
640c1d74c9 ci: Stop rebasing FAH on updates branch
The stable branch should have the updated librpm now.
Also bump slightly the compose tests timeout.

Closes: #1585
Approved by: jlebon
2018-10-02 13:14:08 +00:00
Jonathan Lebon
f6c0788b4e ci: Use FAH updates branch for newer librpm
Right now, we're compiling against librpm 4.14.2, but running against
4.14.1, so we're hitting missing symbol issues from #1469. Just use the
updates ref for now until the next stable FAH update.

Closes: #1538
Approved by: cgwalters
2018-09-08 21:51:22 +00:00
Colin Walters
6a274b831d build-sys: Hard require Rust
As something that manages your base operating system, we care
about reliability, predictability, as well as performance and
low-level access to native operating system facilities.  The
C programming language is great for the latter two, but fails
at providing a truly memory-safe environment.  Rust is fairly
unique in providing a language that doesn't carry a runtime,
so we can gradually "oxidize" and convert our C code without
imposing additional overhead.  It's also got a lot of modern
design niceties, like not having a null pointer.

Let's pull the trigger here and hard require Rust.  It's the
programming language I personally want to be primarily writing in for
years to come.

This is also in line with a recent trend of reducing our
experimental/optional matrix.

Closes: #1509
Approved by: jlebon
2018-08-21 14:49:26 +00:00
Jonathan Lebon
45e162fb69 ci: Split compose test into two
The `f28-compose` test keeps timing out. Some time recently, I/O
performance of the internal OpenStack instance used for testing has
degraded. I have a ticket open to investigate the regression though
haven't had any luck so far.

Let's just take the easy way out and split the test into two testsuites.
This is obviously hacky, and sad, and unfortunate. But the PRs must keep
flowing until we finally wean off of OpenStack.

Closes: #1498
Approved by: cgwalters
2018-08-13 21:06:18 +00:00
Jonathan Lebon
1a82f210ae ci: Bump f28-compose by 10 minutes
We're consistently seeing timeouts on the `f28-compose` test. It's
likely due to a performance regression in the backend rather than
anything else. I filed a ticket to get an investigation going, though
for now at least, let's bump the timeout some more to keep patches
flowing.

Closes: #1463
Approved by: jlebon
2018-07-24 20:15:10 +00:00
Jonathan Lebon
ec19d67d74 ci: Stop rebasing Fedora onto updates ref
We can drop this hack now since there have been updates since GA. Also
add a note that trying to drop the rebase to continuous causes
`layering-relabel` to fail.

Closes: #1406
Approved by: jlebon
2018-07-20 18:47:51 +00:00
Jonathan Lebon
fbee05bcc5 vmcheck/autoupdate-check.sh: Fix --check/--preview tests
Noticed this while looking at the logs for #1432.

Because --check and --preview exit with rc=77 when there are no updates,
we would actually stop early on in the test and marking it as SKIPPED.
Fix this by making sure we explicitly check for the $rc we expected when
using those switches.

I also added a final grep pass to make it easy to inspect whether we
skipped any tests. I was about to do this nicely in `multitest.py`
instead, though it may not be of this world much longer, so meh...

Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Jonathan Lebon
479406e6a5 Add support for YAML treefiles
Let's modernize and start supporting YAML treefiles. I'll dare make the
sweeping generalization that most people would prefer reading and
writing YAML over JSON.

This takes bits from coreos-assembler[1] that know how to serialize a
YAML file and spit it back out as a JSON and makes it into a shared lib
that we can link against. We could use this eventually for JSON inputs
as well to force a validation check before composing.

If we go this route, we could then turn on `--enable-rust` in FAHC for
now and drop the duplicate code in coreos-assembler.

[1] https://github.com/cgwalters/coreos-assembler

Closes: #1377
Approved by: cgwalters
2018-06-05 13:08:33 +00:00
Jonathan Lebon
592d6052b9 ci: Bump to F28
Closes: #1358
Approved by: cgwalters
2018-05-23 14:18:41 +00:00
Jonathan Lebon
377b1674da ci: Drop f27-sanity testsuite
This started out as an experiment to see how useful it'd be in
rpm-ostree. Again, the goal wasn't to leverage the rpm-ostree-specific
tests of a-h-t here, but rather running the *other* tests to ensure that
we weren't inadvertedly impacting userspace. In the end, as far as I
remember, it hasn't brought up anything since adding it here, so let's
bring the experiment to an end.

It might still be worth doing this same experiment in the ostree CI
instead, though I don't want to add more load there until we fully
switch over to the new CI infra.

Closes: #1369
Approved by: cgwalters
2018-05-16 15:19:13 +00:00
Colin Walters
e57504a88a ci: Split codestyle checks into separate context
Prep for reworking the primary test to do vm-in-container, which
will temporarily be vm-in-container-in-vm.

See https://github.com/projectatomic/rpm-ostree/pull/1362

Closes: #1364
Approved by: jlebon
2018-05-14 19:50:36 +00:00
Jonathan Lebon
38b11d34e5 ci: Use continuous branch for now
We are hitting issues with a-h-t on CAHC right now failing due to an
`atomic scan` issue and preventing promotion of continuous content to
the smoketested branch. This is then blocking our CI which needs a newer
tree with the latest glib2 baked in. Let's unblock our CI for now by
rebasing on the continuous branch. We'll drop that hack once the a-h-t
issues are ironed out.

Closes: #1365
Approved by: cgwalters
2018-05-14 18:14:54 +00:00
Colin Walters
bb86912de9 tests: Dip our toes into using Ansible
This adds a shell primitive to make it easy to execute a playbook
task list.

The big picture idea is to sync with https://github.com/ostreedev/ostree/pull/1462
and rewrite some of the libvm shell stuff as playbooks, allowing easier
code sharing with a-h-t and just in general being a better library for
talking ssh and executing commnads.

Closes: #1297
Approved by: jlebon
2018-03-09 19:04:32 +00:00
Jonathan Lebon
99aa6f89d2 ci: expand rootfs on CentOS
The new kernel tests require a little bit more space.

Closes: #1247
Approved by: cgwalters
2018-02-14 16:16:52 +00:00
Colin Walters
a3e0b47e6b ci/papr: Bump compose tests to 4 min vCPUs
Compose is a slow test right now.  Down the line what I'd
like to do is: https://github.com/projectatomic/papr/pull/70
Since this job can be scheduled as a container, not a VM.  There's
no reason to grab a whole 8GB of RAM for it, but we *do* want multiple
CPUs.  Containers do that by default.

Closes: #1187
Approved by: jlebon
2018-01-09 16:26:32 +00:00
Colin Walters
0e16be8273 tests: Run compose tests in parallel ∥
Over a year later, the "opening the host rpmdb" bug is fixed,
so we can do composes in parallel ∥, hooray!

I'm dusting this off since we were running into CI (PAPR) timeouts
when I was adding more to the compose tests.

Closes: #545
Approved by: jlebon
2018-01-09 14:19:31 +00:00
Colin Walters
90f9fe80e4 scripts: Drop most capabilities
Note this PR requires [bubblewrap 0.2.0](https://github.com/projectatomic/bubblewrap/releases/tag/v0.2.0).

Change our bwrap invocations drop truly dangerous capabilities like
`cap_sys_admin` and `cap_sys_module` just like Docker does today. Because of the
popularity of Docker, we can be pretty sure that most RPM scripts should have
adapted to this (although a problematic area here is that traditional librpm
doesn't actually error out if scripts fail).

There are two reasons to do this:

 - We want "offline" updates by default; updates shouldn't affect the
   running system.  If we prepare the new root in the background, a
   %post shouldn't restart a service for example.  We already "handle"
   this by making `systemctl` a symlink to `/bin/true`, but this approach
   also shuts off `%post`s that do e.g. `insmod`.
 - Protection against accidental system damage

Closes: #1099
Approved by: jlebon
2017-12-05 02:54:23 +00:00
Colin Walters
b84e795c96 ci/papr: Update the other contexts to F27
Let's take a stab at this.

Closes: #1125
Approved by: jlebon
2017-12-04 17:45:56 +00:00
Jonathan Lebon
e35f867494 papr: fix sanity test and bump to f27
It was recently changed to use virtualenv to install and invoke ansible.

Closes: #1112
Approved by: miabbott
2017-11-22 16:55:17 +00:00
Colin Walters
d87970966b ci/papr: Update to f27-primary
I didn't try porting some of the other contexts yet...I want to see
if this works.

Note that FAHC was switched to 27, so it's only luck that things
work on 26.

Closes: #1108
Approved by: jlebon
2017-11-17 19:46:38 +00:00
Colin Walters
cdfba8e131 ci: Update CentOS container to current 7
The rpmmd repo issues seem to be fixed, and the old container was (for some
reason) apparently removed.

Closes: #1073
Approved by: jlebon
2017-10-24 16:26:32 +00:00
Jonathan Lebon
235f2945bf ci: don't use CentOS Alpha anymore
It's no longer being built and is now older than the latest CentOS AH
release. This should help us no longer see messages like:

(rpm-ostree pkg-add:5662): GLib-CRITICAL **: g_variant_dict_lookup:
assertion 'is_valid_dict (dict)' failed

which happen because in #1034, we started using `G_VARIANT_DICT_INIT`,
whose special magic values only make sense in glib2 >= 2.50. (The alpha
image stopped at 2.46).

Saw this while debugging #1035.

Closes: #1040
Approved by: cgwalters
2017-10-06 00:42:55 +00:00
Colin Walters
1acd834104 Add test infra for ex container, and one test for bash
We have some unit-style tests that run `ex container`, but
they aren't "real"; they don't use scripts for example.  Let's
add tests for this similar to `tests/compose`.

We use a 26 base, but the target repos need to be 27
to pick up the fix for: https://bugzilla.redhat.com/show_bug.cgi?id=1478172

Add some bits to share infra between `tests/compose` and `tests/ex-container`;
basically handling the rpmmd repos. I tweaked things to be more streamlined
there between the `.papr.yml` and the test script.

Right now this is just one test for `bash`, but lays some of the infrastructure
for doing more. One thing that we need to do to improve more here is to better
cache RPMs, a bit like the compose tests do.

Closes: #1024
Approved by: jlebon
2017-10-02 18:04:24 +00:00
Colin Walters
2ebf43f4a9 ci: Sync with ostree
We have some drift; keep this more in sync so we maintain the rpm/yum
hackarounds for example in one place.

I backed out the ASAN bits though pending at least a newer gpgme:
<https://github.com/projectatomic/rpm-ostree/pull/1000#issuecomment-331278758>

Closes: #1000
Approved by: jlebon
2017-09-24 13:32:00 +00:00
Colin Walters
aab7571fc1 ci: Fixate centos container image to 7.3 until rpm-md repos update
Sigh.

Closes: #985
Approved by: jlebon
2017-09-12 13:45:50 +00:00
Jonathan Lebon
48ddca9280 ci: workaround broken fedora:26 image
This is essentially the same workaround as
https://github.com/ostreedev/ostree/pull/1143.

See https://bugzilla.redhat.com/show_bug.cgi?id=1483553.

Closes: #975
Approved by: cgwalters
2017-09-07 12:27:54 +00:00
Jonathan Lebon
f5bcd62364 ci: run the improved-sanity-test
Start running the improved-sanity-test from atomic-host-tests to make
sure the system works. This is an easy way to get PR-level comprehensive
integration tests for free.

But note that we *don't* mark it as required since the tests are not
stored here and it can happen that they need to be adjusted for new
rpm-ostree behaviours. In this way, this added check also allows us to
give a heads-up that breaking changes are coming.

Closes: #899
Approved by: cgwalters
2017-08-24 20:43:16 +00:00
Colin Walters
b46fc35901 Add releasever opt, avoid opening up host's rpmdb in treecompose
Closes: https://github.com/projectatomic/rpm-ostree/issues/546

Previously, we'd open up the host's rpmdb for both `compose tree`
and `ex container`.  In the first case, because we require root, we'd
succeed.  For `ex container`, we'd spew an error.

Fixing this was trickier than I thought.  First because there was
*also* a libdnf bug here: https://github.com/rpm-software-management/libdnf/pull/307

Second, there's a compatibility hazard here for anyone using `.repo` files that
reference `$releasever`. This actually happened to me with `ex container` as I'd
just done a `ln -s /etc/yum.repos.d/fedora.repo rpmmd.repos.d`.  I fixed
that first by doing a `sed -i -e 's,$releasever,26,' rpmmd.repos.d/*.repo`.

As far as I can see today, none of Fedora Atomic or CentOS AH rely on this. But
in order to enhance compatibility, let's add a "releasever" option. This makes
it easier again to reuse stock `.repo` files if we wanted to do so.

(Also, I realized we can just use `/usr/share/empty` as *the* canonical immutable
 empty directory)

Closes: #875
Approved by: jlebon
2017-07-21 15:02:59 +00:00
Jonathan Lebon
1806259cc1 ci: stop working around updates-source mirrors
I initially did this because mirrors weren't stable before the F26
release. But now, the canonical source itself is unstable, so let's try
our luck again with mirrors. Might have better luck now that it's
released.

Closes: #881
Approved by: mbarnes
2017-07-18 17:38:43 +00:00
Jonathan Lebon
cf19d83502 ci: unite testsuites and run vmcheck on centos
We can be a bit less wasteful here by merging the check and vmcheck
suites into a single suite. The check suite today takes a negligible
amount of time to run, so we're not gaining much by parallelizing them.
It's more of a sanity check at this point before we start vmcheck.

Also start running vmcheck on CentOS 7. We adapt the ci scripts to
accomodate both Fedora and CentOS target machines.

This commit also switches to Fedora 26 as the primary test base.

Closes: #871
Approved by: cgwalters
2017-07-18 13:58:38 +00:00