Commit Graph

4765 Commits

Author SHA1 Message Date
Colin Walters
1101c02c2a tests/inst: Add destructive test framework
This adds infrastructure to the Rust test suite for destructive
tests, and adds a new `transactionality` test which runs
rpm-ostree in a loop (along with `ostree-finalize-staged`) and
repeatedly uses either `kill -9`, `reboot` and  `reboot -ff`.

The main goal here is to flush out any "logic errors".

So far I've validated that this passes a lot of cycles
using
```
$ kola run --qemu-image=fastbuild-fedora-coreos-ostree-qemu.qcow2 ext.ostree.destructive-rs.transactionality --debug --multiply 8 --parallel 4
```
a number of times.
2020-08-17 14:34:04 +00:00
OpenShift Merge Robot
ab95c0264c
Merge pull request #2168 from arithx/ci_pxe_offline
ci: add pxe-offline-install testiso scenario
2020-08-13 16:43:38 +02:00
Stephen Lowrie
82c0b4a8b8 ci: add pxe-offline-install testiso scenario 2020-08-11 00:02:12 -05:00
OpenShift Merge Robot
012af18ceb
Merge pull request #2103 from cgwalters/underlay-live
Add "transient" unlock
2020-08-07 17:29:31 -04:00
Colin Walters
f2773c1b55 Add "transient" unlock
I was thinking a bit more recently about the "live" changes
stuff https://github.com/coreos/rpm-ostree/issues/639
(particularly since https://github.com/coreos/rpm-ostree/pull/2060 )
and I realized reading the last debates in that issue that
there's really a much simpler solution; do exactly the same
thing we do for `ostree admin unlock`, except mount it read-only
by default.

Then, anything that wants to modify it does the same thing
libostree does for `/sysroot` and `/boot` as of recently; create
a new mount namespace and do the modifications there.

The advantages of this are numerous.  First, we already have
all of the code, it's basically just plumbing through a new
entry in the state enumeration and passing `MS_RDONLY` into
the `mount()` system call.

"live" changes here also naturally don't persist, unlike what
we are currently doing in rpm-ostree.
2020-08-07 18:57:56 +00:00
OpenShift Merge Robot
621e1d7394
Merge pull request #2166 from pwithnall/summary-sig-downloads
pull: Add summary-{,sig-}bytes options to ostree_repo_pull()
2020-08-07 13:24:58 -04:00
Philip Withnall
f5da67d78a pull: Add summary-{,sig-}bytes options to ostree_repo_pull()
These allow the `summary` and `summary.sig` files to be cached at a
higher layer (for example, flatpak) between related pull operations (for
example, within a single flatpak transaction). This avoids
re-downloading `summary.sig` multiple times throughout a transaction,
which increases the transaction’s latency and introduces the possibility
for inconsistency between parts of the transaction if the server changes
its `summary` file part-way through.

In particular, this should speed up flatpak transactions on machines
with high latency network connections, where network round trips have a
high impact on the latency of an overall operation.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2020-08-07 16:12:15 +01:00
Philip Withnall
bd68c7dfd7 pull: Improve formatting of pull options in documentation
Backticks improve all things.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2020-08-07 16:11:44 +01:00
OpenShift Merge Robot
13e44a6aac
Merge pull request #2163 from cgwalters/fix-ci
ci: Barf on unset umask
2020-08-05 21:20:27 +02:00
Colin Walters
1e127f2dcc ci: Barf on unset umask
Since it's just not a sane thing to do and will cause various
failures in our test suite.
2020-08-05 16:34:52 +00:00
OpenShift Merge Robot
1dedc69a46
Merge pull request #2159 from agners/show-parent
Show commit checksum of parent, if present
2020-08-04 21:54:28 +02:00
OpenShift Merge Robot
9832a36a8c
Merge pull request #2160 from cgwalters/sysroot-ro
remount: Still remount /sysroot writable if not configured ro
2020-08-02 20:53:41 -04:00
OpenShift Merge Robot
4fb5b8f85d
Merge pull request #2162 from cgwalters/fix-repo-finder
tests/repo-finder: Explicitly commit empty dir
2020-08-02 10:29:45 -04:00
OpenShift Merge Robot
769aaea725
Merge pull request #2161 from cgwalters/fix-ci
ci: Fix ISO testing
2020-08-02 09:02:06 -04:00
Colin Walters
98137403aa ci: Fix ISO testing
Regression from
5d7f897908

I'm not sure how (or if) this passed before, the job logs have
been GC'd.

This is a bit confusing but basically right now ostree/rpm-ostree's
CI jobs don't use `/srv/fcos` - it might make sense to port
these to `fcosBuild` but that needs investigation.
2020-08-02 12:55:35 +00:00
Colin Walters
f3c7834f1e tests/repo-finder: Explicitly commit empty dir
We were committing the whole tempdir, which seems to fail
in Travis because the GPG agent Unix domain socket ends up there too,
and ostree refuses to commit sockets.
2020-08-02 12:54:11 +00:00
OpenShift Merge Robot
725b287e3e
Merge pull request #2156 from jlebon/pr/clarify-ref-rev
app: Fix various CLI metavariable names
2020-08-01 15:02:23 -04:00
Colin Walters
33eeb7b9eb remount: Still remount /sysroot writable if not configured ro
Regression from 3564225917
BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1862568

We still need to remount writable if it's not configured on;
because it may need OS adjustments it needs to be opt-in.
2020-08-01 17:27:18 +00:00
Stefan Agner
512b4e6313 Show commit checksum of parent, if present
This is useful for ostree log on client side where often not the
full history of a branch is available. It is also helpful for
ostree show to show if a particular commit has a parent.
2020-07-31 21:46:58 +02:00
Jonathan Lebon
af140266d5 app: Fix various CLI metavariable names
- Use `REV` instead of `REF` in places where we meant it.
- Fix `commit --parent` actually taking a commit checksum and not a ref.
- Fix `ostree admin switch` using `REF` instead of `REFSPEC`.
2020-07-31 15:02:47 -04:00
OpenShift Merge Robot
a5409b872f
Merge pull request #2157 from agners/improve-man-pages
Improve man pages
2020-07-31 14:12:42 -04:00
OpenShift Merge Robot
2905de89c6
Merge pull request #2158 from bgilbert/testiso
ci: test FCOS PXE and ISO install
2020-07-31 13:52:59 -04:00
Benjamin Gilbert
5d7f897908 ci: test FCOS PXE and ISO install
Make sure we don't break the FCOS live image.  PXE is probably sufficient,
but also test the ISO image for good measure.
2020-07-31 06:58:04 -04:00
Stefan Agner
b94c3ae79f man: add missing options to the ostree-commit man page
Add missing parameter to the ostree-commit man page.

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
2020-07-31 11:54:19 +02:00
Stefan Agner
999f9a2b2d man: add glossary to main man page
Add glossary to define some commonly used literals throughout the ostree
man pages.

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
2020-07-31 11:49:38 +02:00
OpenShift Merge Robot
21c9840d13
Merge pull request #2153 from cgwalters/release-2020-4
Release 2020.4
2020-07-22 16:48:44 +02:00
Colin Walters
0d91206a62 Post-release version bump 2020-07-22 14:09:26 +00:00
Colin Walters
901747f985 Release 2020.4
A lot of stuff here, new signing API is the biggest.  Let's
get a release out.
2020-07-21 21:48:51 +00:00
OpenShift Merge Robot
12a9161a79
Merge pull request #2152 from cgwalters/pull-fdatasync
pull: Add --per-object-fsync
2020-07-20 13:55:26 -04:00
OpenShift Merge Robot
d5968f2cb2
Merge pull request #2150 from cgwalters/pull-append-written
pull: Also append bytes written
2020-07-18 11:00:49 -04:00
Colin Walters
a615d35762 pull: Add --per-object-fsync
This is the opposite of
https://github.com/ostreedev/ostree/issues/1184

Motivated by OpenShift seeing etcd performance issues during
OS updates: https://github.com/openshift/machine-config-operator/issues/1897

Basically, if we switch to invoking `fsync()` as we go, it makes
ostree performance worse (in my tests, 31s to write 2G versus 7s if we
delay sync) but it avoids *huge* outliers in `fsync()` time for etcd.
2020-07-18 14:59:01 +00:00
Colin Walters
5b75358357 pull: Also append bytes written
This is very useful information that we get from the transaction
stats.  Append it to the final display if we're not inheriting
the transaction.
2020-07-17 17:03:08 +00:00
OpenShift Merge Robot
848411c9e3
Merge pull request #2151 from jlebon/pr/too-parallel
ci: Constrain parallel build jobs
2020-07-17 12:00:24 -04:00
Colin Walters
6a5f97c145 tests: Add needs-internet tag for webserver bits
Fixes the tests, see https://github.com/coreos/coreos-assembler/pull/1600
TODO: provide a webserver binary via virtio or so
2020-07-17 01:37:46 +00:00
Jonathan Lebon
92284f9b81 ci: Constrain parallel build jobs
The default `_NPROCESSORS_ONLN` heuristic we have isn't cgroups aware.
So it thinks it has e.g. 40 CPUs when running in a k8s pod. This can
then blow through our allocated resource limits.

Declare some modest amount of RAM and CPU resources and override `make`
parallelism.

This matches what rpm-ostree now does in
https://github.com/coreos/rpm-ostree/pull/2155.
2020-07-16 15:44:34 -04:00
OpenShift Merge Robot
4752dd02cf
Merge pull request #2146 from stb-tester/owners-uncomment-wmanley
OWNERS: Uncomment @wmanley
2020-07-10 19:20:27 +02:00
William Manley
f924b0bbe2 OWNERS: Uncomment @wmanley
I've made my affiliation public now thanks to @cgwalters:

https://github.com/ostreedev/ostree/pull/1678#issuecomment-653148139
2020-07-08 15:54:22 +01:00
OpenShift Merge Robot
cb2ecd1459
Merge pull request #2144 from cgwalters/deltas-new-style
lib/deltas: convert ostree_repo_static_delta_generate to new style
2020-07-03 11:31:45 -04:00
Frédéric Danis
892e9acf25 lib/deltas: convert ostree_repo_static_delta_generate to new style
The "new style" code generally avoids `goto err` because it conflicts
with `__attribute__((cleanup))`.  This fixes a compiler warning.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-07-03 13:35:39 +00:00
OpenShift Merge Robot
1b770c5e24
Merge pull request #2143 from jlebon/pr/eacces-lock
lib/repo: Handle EACCES for POSIX locking
2020-07-01 17:05:40 -04:00
Jonathan Lebon
1d755f62af lib/repo: Handle EACCES for POSIX locking
If `glnx_make_lock_file` falls back to `flock`, on NFS this uses POSIX
locks (`F_SETLK`). As such, we need to be able to handle `EACCES` as
well as `EAGAIN` (see `fnctl(2)`).

I think this is what coreos-ostree-importer has been hitting, which runs
on RHEL7 in the Fedora infra and does locking over an NFS share where
multiple apps could concurrently pull things into the repo.
2020-06-30 15:09:41 -04:00
OpenShift Merge Robot
fd8ecdf047
Merge pull request #2131 from cgwalters/sign-success
signapi: Change API to also return a success message
2020-06-24 18:24:41 -04:00
OpenShift Merge Robot
adcd261596
Merge pull request #2001 from agners/multi-device-tree
deploy: support devicetree directory
2020-06-24 03:02:18 -04:00
OpenShift Merge Robot
ba7a4461be
Merge pull request #2138 from cgwalters/use-autopkgtest-reboot
tests: Port to Debian autopkgtest reboot API
2020-06-19 14:37:19 -04:00
OpenShift Merge Robot
9067c1ca49
Merge pull request #2136 from cgwalters/doc-timestamp
core: Add documentation for ostree_commit_get_timestamp()
2020-06-19 13:57:33 -04:00
OpenShift Merge Robot
addda9689b
Merge pull request #2137 from cgwalters/fix-unexported-symbol
sysroot: Remove unimplemented ostree_sysroot_lock_with_mount_namespace
2020-06-19 13:12:07 -04:00
Colin Walters
5aa22e0b1f tests: Port to Debian autopkgtest reboot API
See https://github.com/coreos/coreos-assembler/pull/1528

I think we can drop the old cosa reboot APIs after this,
though I've already forgotten where else I might have written
tests using it.
2020-06-19 13:04:22 +00:00
Colin Walters
935f879542 sysroot: Remove unimplemented ostree_sysroot_lock_with_mount_namespace
This came in with 5af403be0c but
was never implemented.

I noticed this now because the Rust ostree bindings generate a
wrapper for it which the linker tries to use.
2020-06-18 21:19:43 +00:00
Colin Walters
c87a4c71bc core: Add documentation for ostree_commit_get_timestamp()
Working on some rpm-ostree bits and was going to pass
this to the `chrono` crate and I forgot the format, went
to look at the docs.  Oops.
2020-06-18 20:57:53 +00:00
OpenShift Merge Robot
7c2d45e7cb
Merge pull request #2133 from jlebon/pr/ci-commitmessage
ci: Import latest ci-commitmessage-submodules from rpm-ostree
2020-06-18 10:00:32 -04:00