IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
There's a huge difference between live updates that change
existing things, versus simply adding new packages (files).
The latter is really quite safe, and live layering is one
of the most requested features.
On traditional rpm systems this can hang because the outer
process may have an rpmdb lock, and the inner one wants
to acquire a lock. Here we're sandboxing the `%post` script
and it's targeting a separate temporary filesystem compared to
the booted one (so there's no double locking). Plus we don't
create the rpmdb in the target until all scripts have run.
Inspired by https://twitter.com/_msw_/status/1335981558717587473
Now always based on an overlayfs:
f2773c1b55
This fixes a whole swath of problems with the previous design,
including the danger in replacing `/usr/lib/ostree-boot` which
broke booting for some people.
Further, we don't need to push a rollback deployment; the livefs
changes are always transient. So now we store livefs state
in `/run` instead of in the origin file.
Since we're doing a rewrite, it's now in Rust for much more safety.
We also always work in terms of incremental diffs between commits;
the previous huge hammer of swapping `/usr` was way too dangerous.
There's a lot going on here, but essentially:
1. We change the `vmcheck` model so that it always operates on an
immutable base image. It takes that image and dynamically launches a
separate VM for each test using `kola spawn`. This means we can drop
a lot of hacks around re-using the same VMs.
2. Following from 1., `vmoverlay` now takes as input a base image,
overlays the built rpm-ostree bits, then creates a new base image. Of
course, we don't have to do this in CI, because we build FCOS with
the freshly built RPMs (so it uses `SKIP_VMOVERLAY=1`). `vmoverlay`
then will be more for the developer case where one doesn't want to
iterate via `cosa build` to test rpm-ostree changes. I say "will"
because the functionality doesn't exist yet; I'd like to enhance
`cosa dev-overlay` to do this. (Note `vmsync` should still works just
as before too.)
3. `vmcheck` can be run without building the tree first, as
`tests/vmcheck.sh`. The `make vmcheck` target still exists though for
finger compatibility and better meshing with `vmoverlay` in the
developer case.
What's really nice about using kola spawn is that it takes care of a lot
of things for us, such as the qemu command, journal and console
gathering, and SSH.
Similarly to the compose testsuites, we're using parallel here to run
multiple vmcheck tests at once. (On developer laptops, we cap
parallelism at `$(nproc) - 1`).
Drop the use of Ansible everywhere. In the few cases where we really
Python, just spawn a container instead.
This is required to be able to hack on Fedora CoreOS.
Closes: #1850
Approved by: jlebon
This bumps the requirement on the controlling host to Python 3 only.
It also bumps the requirement on the target host to Python 3 as well
since FCOS doesn't ship Python 2 right now.
Though we'll need to eventually drop all Python usage anyway, but at
least let's get tests passing on FCOS first. (See related previous
patch).
Closes: #1828
Approved by: cgwalters
This is relatively uncontroversial functionality that has already proved
useful when helping folks debug their stuff. Let's promote it to the
stable interface.
Closes: #1682
Approved by: rfairley
This removes the logic around supporting opting out of the staging
feature. We don't want to support multiple configurations here, and at
this point, staging should be considered stable.
Closes: #1546
Approved by: cgwalters
Staging fixes the `/etc` bug for livefs. There's actually more
we could do here around taking advantage of staging for livefs;
for example, I think once the livefs is complete, we could just delete
the staged deployment. And then we don't need to render on the next
boot the live status, etc.
Anyways, all that can come in the future. This is prep for
enabling staging by default.
Closes: #1430
Approved by: jlebon
Our test suite originated when package layering was still being
developed, but now that that's mature, the logic where layering
tests are distinct makes less sense.
The `basic` test had grown to really be a collection of many
miscellaneous things. Let's make that more explicit. Further,
let's avoid having each test suite grow too large; when a single
test fails we don't have an easy way to rerun just that test,
so a crude way to have faster local iteration is to split into groups.
My plan is to reintroduce a `basic` test that covers the basics
of all functionality - update, deploy, layering, etc. The advanced/corner
cases of layering like the `rm -rf /` test would still live in a
`test-layering.sh` or so.
Closes: #1336
Approved by: jlebon
I was trying a `--ex-unified-core` compose of FAW, and things fell over
on `urw-base35-fonts` which does a dance of setting a stamp file in
`%post` and checking it in `%posttrans`.
This whole pattern should be considered deprecated by file triggers. But let's
support it for now.
Note there's a lot of parameter passing as we need a single directory which is
held across multiple script invocations.
Closes: #1319
Approved by: jlebon
It obviously shouldn't block the ability to install, and anyways
the single use in `ksh` is not compelling. If someone comes up with
one we can revisit supporting this.
Closes: https://github.com/projectatomic/rpm-ostree/issues/1216Closes: #1218
Approved by: jlebon
This fixes a large swath of compatibility issues, for the same reasons as
overlayfs makes a lot of things Just Work. The ugly part of course is
doing hidden copyups inside the filesystem.
We've gone quite a long time with the "pure rofiles" mode, and have made changes
to various bits of userspace to be compatible with it. But what finally made me
give up on that is glibc's locale-archive; there's a patch for it that
is stalled, but even if it was applied we would still need to work with
older glibc.
This issue comes to the fore in unified core 🌐 mode, as without this
we won't get a correct locale archive.
Closes: #1171
Approved by: jlebon
Right now the fact that one can only cancel via `Ctrl-C` of an existing client
process is rather frustrating if for example one's ssh connection to a machine
drops. Now, upon reconnecting, one can easily `rpm-ostree cancel` a hung update
or whatever rather than doing the more forcible `systemctl stop rpm-ostreed`
(which is safe of course, unless livefs is involved).
Closes: #1019
Approved by: jlebon
Note this PR requires [bubblewrap 0.2.0](https://github.com/projectatomic/bubblewrap/releases/tag/v0.2.0).
Change our bwrap invocations drop truly dangerous capabilities like
`cap_sys_admin` and `cap_sys_module` just like Docker does today. Because of the
popularity of Docker, we can be pretty sure that most RPM scripts should have
adapted to this (although a problematic area here is that traditional librpm
doesn't actually error out if scripts fail).
There are two reasons to do this:
- We want "offline" updates by default; updates shouldn't affect the
running system. If we prepare the new root in the background, a
%post shouldn't restart a service for example. We already "handle"
this by making `systemctl` a symlink to `/bin/true`, but this approach
also shuts off `%post`s that do e.g. `insmod`.
- Protection against accidental system damage
Closes: #1099
Approved by: jlebon
Just taking what I learned from #1035 and applying it here. What's nice
about this is that there's no cleanup needed. Once the process is killed
(or worst case, we reboot the VM), there's no traces left at all.
Also added a few extra "ok" outputs.
Closes: #1043
Approved by: cgwalters
Prep for implementing `rpm-ostree cancel`, but this works with the way we handle
`Ctrl-C` interactively on a client as well. Being able to cancel a script
execution is quite nice; some of them are expensive, and having one loop forever
has been known to happen.
Closes: #1025
Approved by: jlebon
We were directly bind mounting the checked out `/usr/etc` onto `/etc`
which was exposing us to corruption from scriptlets. Since we already
have an rofiles-fuse mount for `/usr`, let's just re-use its `etc/`
subdir and bind mount that instead.
Closes: #1003
Approved by: cgwalters
Today in Fedora the `glibc-all-langpacks.posttrans` is implemented
in lua, for no good reason. See:
https://bugzilla.redhat.com/show_bug.cgi?id=1367585
Since that's stalled out, let's add support for overrides. This
is obviously a much bigger step with more long term maintenance
implications over our current "ignore scripts" list. But we can't
block either.
This is needed for unified core work:
https://github.com/projectatomic/rpm-ostree/issues/729
(We also override `fedora-release-atomichost` but I'll likely
submit a patch for that upstream)
Closes: #980
Approved by: jlebon
While working on unified core and the Fedora Atomic Host content set, I hit a
dependency between `docker.posttrans` which tries to read `/etc/os-release`, and
`fedora-release-atomichost.post` which creates that symlink.
It seems best practice to me to run `%post`s strictly before
`%posttrans`; we're not likely to do parallelization anytime
soon anyways.
While here I cleaned things up by having an enum for the script kind,
rather than multiple functions, otherwise we would have had another
wrapper in core.c.
Closes: #963
Approved by: jlebon
File triggers are a post-RHEL7 thing; more information at
http://rpm.org/user_doc/file_triggers.html
There are two notable users I've been testing this with;
`glib2` and `vagrant`. The `vagrant` one is more immediately urgent,
since it makes `vagrant-libvirt` work, which I currently rely on
for my workstation dev.
I've tested things successfully with `vagrant`, and I did verify that we run the
`glib2` ones when doing `rpm-ostree ex container`.
Long term, more transaction file triggers are likely to live in
"base" packages like `glib2`. We don't implement those yet, but
extending this to do that shouldn't be too hard.
There was *significant* what I'd call reverse engineering of the
implementation in librpm. The file triggers code there is spread out
and abstracted in a few different places in the code. I found
trying to understand what header values were involved to be quite
tricky.
There are some corner cases like multiple patterns that I *think*
this does correctly, but could use more validation. The main
question I had was - is it required that the patterns for e.g.
`%transfiletriggerin` and `%transfiletriggerun` be identical?
Closes: https://github.com/projectatomic/rpm-ostree/issues/648Closes: #869
Approved by: jlebon
This is a followup to https://github.com/projectatomic/rpm-ostree/pull/888
but more comprehensive; in the layering case, the sanitycheck runs
after all the `%posttrans` scripts, so we'll get a consistent error message
for the `rm -rf /` test.
We also do the sanitycheck for the "pure ostree" case, as well as cases
where we didn't actually layer packages (including `ex override remove` as
well as simply regenerating an initrd).
There's obviously a lot more we could do in a sanitycheck; as I say in the
comment it's tempting to consider trying to boot systemd (in a fully volatile
config), but for now let's do this. In the end of course the admin has rollback
too.
Closes: #892
Approved by: jlebon
I was thinking today about our script handling, and I realized
an excellent way to showcase the advancement rpm-ostree makes
over traditional package managers is the fact that we survive a
`%post` script that does `rm -rf /`!
See e.g. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=858521
It's been this way ever since we first implemented package layering;
the fact that we construct a new root and use bubblewrap to sandbox
makes us very resilient to this type of thing.
But, let's add a test case for this to be sure we preserve this behavior; for
example, if in the future we for some reason we decide to leak some host state
into the scripts.
Closes: #888
Approved by: jlebon
This is required for glibc-all-langpacks at least:
https://bugzilla.redhat.com/show_bug.cgi?id=1367585
Otherwise, its usage is...extraordinarily rare. In fact looking at a snapshot of
`rpm-specs-20170518.tar.xz` from Fedora, the only other use is in
`postfix.spec`, and it appears bogus (the value is already expanded at build
time).
But the glibc case is special, as the value of `install_langs` is indeed
potentially dynamic per system.
Closes: #873
Approved by: jlebon
Seen in the wild with `vagrant`'s use of `%post -p /usr/bin/ruby`. This was a
very easy fix, and actually makes the code a little bit nicer, as we no longer
need to explicitly make the script executable, since we now pass it as
`argv[1]`, the same way librpm does. That in turn would make it possible to fix
the TODO and use `bwrap --file`, but that can come later.
Closes: https://github.com/projectatomic/rpm-ostree/issues/856Closes: #858
Approved by: jlebon
Nuke all the previous goop that was used to create RPMs at `make check`
time and transition all the tests to use the new `build_rpm` function.
It definitely feels cleaner to use. It's also really nice to have the
spec live in the same file as the test that uses it.
Closes: #854
Approved by: cgwalters
We don't want to expose the host's `/tmp` since that means scripts could
potentially find things like the X11 socket or whatever.
To debug things better, add a quick bash script to run bwrap like the C code
does. Perhaps down the line we can add `rpm-ostree internals run-bwrap` or so.
Closes: #647
Approved by: jlebon
Thought it'd be fun to write a test for verifying proper handling of
scriptlets during package layering. There's obviously a lot more that
could go in here (patches welcome!), but it's a start.
Closes: #434
Approved by: cgwalters
