IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Right now, rebuilding ostree into the continuous tag is manual, so we've
only been doing it when necessary to fast-track something e.g. for
rpm-ostree or cosa (see [1] for the long-term goal).
Which means when finding an ostree to use to override in our CI-built
FCOS, we should just let dnf find whatever the latest version is, even
if it's just from the regular Fedora repos.
[1] https://github.com/packit-service/packit/issues/264
We haven't been consistent about doing this; I personally
think rustfmt is a big aggressive with the line wrapping
but eh, consistency is more important.
And heh so I tried to `git push --set-upstream cgwalters` and
that failed because there was an already extant `rustfmt`
branch from a while ago...looking at that code it got lost
in the CI refactoring - we're not running `build-check.sh`
at the moment.
Move the rustfmt bits into `codestyle.sh` which is closer
to where it should be anyways.
The current `rpm-ostree-2020.1-1.fc31.x86_64` in Fedora
was [built with a truly ancient libostree](https://kojipkgs.fedoraproject.org//packages/rpm-ostree/2020.1/1.fc31/data/logs/x86_64/root.log)
because Fedora's build system is weird and only adds packages
released after "gold" into the buildroot via an override
that times out.
This actively breaks things because rpm-ostree isn't
detecting the read-only sysroot.
Let's bump our hard requirement.
Mostly minor tweaks to adapt to the new custom steps. We have a pretty
involved pipeline here so we don't actually use the higher-level steps
like `fcosBuild`.
We need `cargo` in our `PATH` and we already do the `PATH=...` dance in
`ci/msrv.sh`. This only worked before because we were inadvertedly
re-installing cargo when calling `ci/build.sh`, which was fixed in the
previous commit.
Build FCOS and run vmcheck in the same container, since it's only used
for that anyway right now. The main advantage is that we save time
provisioning another container and not having to stash and unstash the
FCOS image.
Also, since the compose tests don't actually need to wait for the FCOS
image, start running them in parallel with the FCOS + vmcheck branch.
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
There's a lot going on here, but essentially:
1. We change the `vmcheck` model so that it always operates on an
immutable base image. It takes that image and dynamically launches a
separate VM for each test using `kola spawn`. This means we can drop
a lot of hacks around re-using the same VMs.
2. Following from 1., `vmoverlay` now takes as input a base image,
overlays the built rpm-ostree bits, then creates a new base image. Of
course, we don't have to do this in CI, because we build FCOS with
the freshly built RPMs (so it uses `SKIP_VMOVERLAY=1`). `vmoverlay`
then will be more for the developer case where one doesn't want to
iterate via `cosa build` to test rpm-ostree changes. I say "will"
because the functionality doesn't exist yet; I'd like to enhance
`cosa dev-overlay` to do this. (Note `vmsync` should still works just
as before too.)
3. `vmcheck` can be run without building the tree first, as
`tests/vmcheck.sh`. The `make vmcheck` target still exists though for
finger compatibility and better meshing with `vmoverlay` in the
developer case.
What's really nice about using kola spawn is that it takes care of a lot
of things for us, such as the qemu command, journal and console
gathering, and SSH.
Similarly to the compose testsuites, we're using parallel here to run
multiple vmcheck tests at once. (On developer laptops, we cap
parallelism at `$(nproc) - 1`).
That way, anyone can easily download the latest built RPMs from master
or a specific PR. This isn't a replacement for automated builds in Koji
though since it's not multi-arch.
Also fetch the tags so that the NEVRA derived from `git describe` is
nicer.
Jenkins is tricky: it does an initial checkout, merges the PR head into
the target branch, then creates the pod. Once in the pod, we do a
`checkout scm` which *also* merges the PR head into the target branch.
However, the `change.GIT_COMMIT` variable we get from that is set to the
SHA of the first merge, not the second one. Which... yeah is super
confusing since we explicitly assign `change` from that `checkout scm`
operation. So that's probably a valid bug.
This was then throwing off `ci-commitmessage-submodules.sh` since it
didn't find the merge commit in the graph.
Anyway, not going to spend more time on this. Let's just not pass any
commit at all. The git range `origin/master..HEAD` already does what we
want (go through all the commits in HEAD *not* in master).
This is an experiment in using Jenkins pipelines for our CI. See similar
initiatives in coreos-assembler[1] and fedora-coreos-config[2].
For now, this only does the following testing:
- checks commit for unintended submodule bumps
- checks the minimum Rust version
- builds RPMs
- builds FCOS (with the new RPMs both for executing the build
itself, as well as included in the built OS)
There are dummy placeholders for where we'd actually run the vmcheck
and the compose testsuites. Let's address those trickier parts as
follow-ups.
[1] https://github.com/coreos/coreos-assembler/pull/667
[2] https://github.com/coreos/fedora-coreos-config/pull/131Closes: #1899
Approved by: cgwalters
