Commit Graph

3255 Commits

Author SHA1 Message Date
Jonathan Lebon
14d51a6a9e .editorconfig: Adjust for CXX files 2021-01-23 17:12:09 +01:00
Kelvin Fan
9198e50c47 docs/HACKING: Update kola ext test instructions 2021-01-22 21:05:22 +01:00
Colin Walters
71496c0958 rust: Add a bit more info on cxx, move up StringMapping
Docs are good, and move up `StringMapping` since multiple things
are likely to use it.
2021-01-22 03:00:21 -05:00
Colin Walters
a4487578a7 Remove some uses of goto out
All of these cases are actually fine, but in general we
can't use `goto out` since we started using C++ exceptions
which will skip that control flow.
2021-01-21 21:01:45 -05:00
Jonathan Lebon
ee34a2d57f docs/HACKING: drop reference to make vmcheck HOSTS=...
This is no longer supported.

Keep and rework the bit about `make vmsync`.

Also drop the bit about FAHC and CAHC.
2021-01-21 18:55:22 -05:00
Kelvin Fan
f8efa6808d HACKING: Update vmcheck instructions
Following c7a9c3b1dd,
it is no longer necessary to provision your own VM.
2021-01-21 17:16:47 -05:00
Colin Walters
01e6c56415 rust/fedora_integration: Support export RPMOSTREE_KOJI_JSON_API_HOST=...
The service where this is hosted was intended to be temporary; support
overriding it so if it goes down in the future people can at
least use a systemd unit file override to change it easily.
2021-01-21 12:46:32 -05:00
Jonathan Lebon
91b48be098 ci: Set RPM_BUILD_NCPUS when building RPMs
Otherwise, it defaults to `_SC_NPROCESSORS_ONLN` (via `%make_build` ->
`%_smp_mflags` -> `%_smp_build_ncpus` -> `%{getncpus}` ->
48c0f28834/rpmio/macro.c (L583)).
And that's going to be wrong in Kubernetes because we're constrained via
cgroups.

The `%_smp_build_ncpus` macro allows overriding this logic via
`RPM_BUILD_NCPUS`.

See: https://github.com/coreos/coreos-ci/issues/23
See: https://github.com/coreos/coreos-assembler/pull/632
See: https://github.com/coreos/coreos-assembler/pull/1287
2021-01-21 11:57:13 -05:00
dependabot[bot]
9138739c49 build(deps): bump serde from 1.0.118 to 1.0.120
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.118 to 1.0.120.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.118...v1.0.120)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-21 08:03:12 -05:00
dependabot[bot]
a974d95288 build(deps): bump serde_json from 1.0.60 to 1.0.61
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.60 to 1.0.61.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.60...v1.0.61)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-21 06:06:56 -05:00
Colin Walters
a9a4e022e6 rust: Port history to cxx-rs
Just keeping up momentum on the porting, we're getting closer
to being able to drop cbindgen entirely.
2021-01-20 17:22:29 -05:00
Kelvin Fan
7586503ee2 vmcheck/test-misc-2: Fix $cursor variable
Minor cleanup.

Follow up from https://github.com/coreos/rpm-ostree/pull/2461/.
2021-01-20 15:20:41 -05:00
Colin Walters
402912e030 Make make check work again
Our CI isn't running the C unit tests because it goes via RPM,
and while we could potentially add `%check` there...I don't
quite want to do that right now since it also runs the Rust
tests which means we rebuild all the Rust code again in debug
mode etc.

Change the C unit tests to compile in C++ mode, which is
enough for local testing.

Longer term I think the C unit tests will go away in favor
of Rust tests.
2021-01-20 13:10:14 -05:00
dependabot[bot]
70cfb765f6 build(deps): bump libdnf from 45981d5 to 1ebad38
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `45981d5` to `1ebad38`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](45981d5f53...1ebad384f6)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-20 11:32:41 -05:00
Kelvin Fan
3f9fe06d24 daemon: Record agent's systemd service
In https://github.com/coreos/rpm-ostree/pull/2395/, agent's id is
recorded in the journal. Similarly, record the systemd service that
called the client, as well.

Related to https://github.com/coreos/rpm-ostree/issues/1747.
2021-01-20 10:13:02 -05:00
Luca BRUNO
8447fd7cfa github: update dependabot config
This updates dependabot config, dropping the "tooling" subdirectory
fixing the path to the top-level cargo manifest.
2021-01-20 09:32:14 -05:00
Luca BRUNO
21be64b3d6 libpriv/passwd: move RPM layering logic to Rust
This moves to Rust the RPM layering logic for users and groups
databases.
2021-01-19 19:32:27 -05:00
Jonathan Lebon
8c34dbe9d7 spec: Add BuildRequires: make
https://fedoraproject.org/wiki/Changes/Remove_make_from_BuildRoot

(Upstreamed from 6e99dd065d)

Really, libdnf's `cmake` BR does pull it into the buildroot anyway right
now. Though we do still need it as well.
2021-01-19 11:45:20 -05:00
Colin Walters
db76e56931 Update configure.ac
Co-authored-by: Jonathan Lebon <jonathan@jlebon.com>
2021-01-18 19:19:09 -05:00
Colin Walters
774923fce3 Release 2021.1 2021-01-18 19:19:09 -05:00
Colin Walters
1c573200dd scripts: Rework /var/lib/rpm-state creation, port to new style
Move the creation of the directory up into core.

Avoid the use of `goto out` since we can't really
do this anymore with C++ exceptions in play.
2021-01-18 12:54:35 -05:00
Colin Walters
f0122a761e scripts: Use bwrap --ro-bind-data rather than mutating target
This is part of avoiding `goto out`, but is also just better
because we're not mutating the target system.
2021-01-18 12:54:35 -05:00
Colin Walters
a8a1317748 rust: Port progress.rs to cxx-rs
Only slightly tricky thing here was double checking which
places pass "optional &str" (represented as empty strings)
and which don't.
2021-01-18 12:07:53 -05:00
Colin Walters
2f82733e65 apply-live: Extend /etc test case
- Further nested sub/sub directories
- symbolic links that point to various corner cases like `/`, are
  broken, to parents etc.
2021-01-18 05:51:05 -05:00
Jonathan Lebon
f9c0dc0828 app/compose: Drop rojig-related prototypes
Those functions don't actually exist (they're conditionally built under
the `ex` command now).
2021-01-15 19:03:42 -05:00
Jonathan Lebon
6a3e3d807d core: Allow overriding downloaded RPMs target dir
This will be useful for a follow-up patch which adds `rpm-ostree compose
extensions` where we'll want to download to a separate directory.
2021-01-15 19:03:42 -05:00
Jonathan Lebon
027f501f4a Makefile-rpm-ostree: Link to librt
For some reason, when building with `-g -Og`, I get a linker error for
a missing `lio_listio`. Adding `-lrt` fixes it. (We already link against
this transitively, so it's not actually a net new `DT_NEEDED`.)
2021-01-15 19:03:42 -05:00
Colin Walters
a3d6c85e06 rust: Fix a pub(crate) warning, add a few doc comments
Mainly went to fix the warning but decided to add some
comments while I was in here.
2021-01-15 18:25:27 -05:00
Colin Walters
becbf17fe3 rust: Update to openat-ext 0.1.10
We want the `remove_all()` fix and the new copy API.
2021-01-15 14:51:33 -05:00
Colin Walters
54ab9175ac rust: Add CxxResult
This is a workaround for the non-customizability of the cxx-rs
propagation of Rust result to C++ exception.  Right now we're
losing context.  Work around this by formatting on the Rust
side at exit points, explicitly converting an `anyhow::Error`
by printing it in "single line context".

Since we're likely to gain more things like this, unify
this with `cxx_bridge_gobject::` into a single `cxxrsutil::`.
2021-01-15 06:59:30 -05:00
Colin Walters
d0c6871d80 apply-live: Avoid clobbering changes in /etc
Gather the current diff of `/etc`, and filter out changes in
the tree which would overwrite it.

There is an OSTree API for diffs but it's a bit awkward, missing
some APIs in the Rust bindings and also `GFile` based unfortunately.
Doing this in Rust is nicer.  The dirdiff code obviously needs
a lot more testing, but I think it's right.
2021-01-15 05:41:53 -05:00
Colin Walters
1f76758513 testutils: Add script-shell, remove shell wrapper
Let's make it very convenient to reproduce the container
for our scripts even from inside a booted system.  Avoids
the need for a duplicate shell script implementation.
2021-01-14 17:55:40 -05:00
Colin Walters
d8076b1132 scripts: Pass script as a memfd
This avoids writing content to the target root, which is
good in general.

But more specifically this is prep for
`rpm-ostree testutils script-shell` which would operate
on the booted root (mounted read-only), in contrast
to the current default checkout that the compose path does.
2021-01-14 17:55:40 -05:00
Timothée Ravier
6650540c2f Remove now unused mkdocs.yml 2021-01-13 16:01:16 -05:00
Timothée Ravier
1b0f79906e Cargo.lock: Update for DNF Count Me support 2021-01-13 14:53:50 -05:00
Timothée Ravier
6cfdb1f585 countme: Add DNF Count Me support
Add support for the DNF Count Me feature [1,2,3] as a standalone
rpm-ostree subcommand called weekly via a systemd timer.

[1] https://fedoraproject.org/wiki/Changes/DNF_Better_Counting
[2] https://dnf.readthedocs.io/en/latest/conf_ref.html?highlight=countme#options-for-both-main-and-repo
[3] https://github.com/rpm-software-management/ci-dnf-stack/blob/master/dnf-behave-tests/features/countme.feature
2021-01-13 14:53:50 -05:00
Colin Walters
0e9941495e Add a Rust helper to create a sealed memfd, use in shlib backend
Prep for other code using memfds.
2021-01-12 22:07:18 -05:00
Colin Walters
241cce1ec4 testutils: Port to cxx-rs
Another easy port like cliwrap.
2021-01-12 19:35:59 -05:00
Luca BRUNO
bdf8269dfa libpriv/passwd: move passwd database to Rust
This moves to Rust the in-memory structure holding passwd entries
(users and groups).
2021-01-12 18:50:30 -05:00
Colin Walters
803e4db50c journal: Add filters for _UID=0
This strengthens our journal filtering to ensure that we won't
read entries from e.g. a malicious process logging with a
syslog identifier `ostree`.
2021-01-12 17:13:00 -05:00
Colin Walters
6c66bf1072 Rename livefs → apply-live in more places
Keep up the renaming momentum by using the new name in more
places.

Prep for further work.
2021-01-12 03:15:49 -05:00
Colin Walters
2247e7af62 configure: Update contact email
atomic-devel@ is dead.
2021-01-11 15:10:07 -05:00
Colin Walters
5b19195fe8 With vendored source, use cargo build --offline
This way we're sure it will build in e.g. Koji.  Right now
it's annoying to test that locally; one needs to explicitly
create a no-network container to do so strictly.  But
cargo has a convenient `--offline` flag, and nothing else
in our build stack should touch the network.
2021-01-11 15:10:07 -05:00
Colin Walters
9f590fa155 Also print Rust-side features/cfg in rpm-ostree --version
Let's make it obvious whether the `fedora-infrastructure` feature
is compiled in.
2021-01-11 13:03:04 -05:00
Colin Walters
29d051e895 Add fedora-integration: Support override replace https://bodhi/...
This adds support for e.g.:

```
$ rpm-ostree override replace https://bodhi.fedoraproject.org/updates/FEDORA-2020-2908628031
```

This will find the Koji builds from the listed update, download
all the RPMs (that aren't debuginfo) and pass them for overrides
in the same way we support `override replace http://somewebserver/foo.rpm`
now.

We also support directly linking a Koji build:
```
$ rpm-ostree override replace https://koji.fedoraproject.org/koji/buildinfo?buildID=1625029
```

Bodhi has a modern HTTP+JSON API, and the lack of a Koji equivalent
drove me to create https://github.com/cgwalters/koji-sane-json-api
and we currently depend on an instance set up in the OpenShift CI
cluster.

I hope it shouldn't take long to deploy this in Fedora Infra,
but I don't want to block on it.

Also notably this still downloads *all* the other RPMs even
ones that aren't installed.  Handling that truly correctly
would require moving this logic to the daemon and core.

All of this functionality is keyed off a `cfg(feature = "fedora-integration")`
that is detected by a Rust `build.rs` which parses the build environment's
`/etc/os-release` for now.
2021-01-11 13:03:04 -05:00
dependabot[bot]
1f408bd396 build(deps): bump libdnf from 2ee2a47 to 45981d5
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `2ee2a47` to `45981d5`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](2ee2a47e19...45981d5f53)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-11 03:31:29 -05:00
Colin Walters
9d080fc1fa Port Rust journal bits to cxx-rs
This is cleaner if we move the "catch and print errors" into
the Rust side.  We lose the "print errors as red" part but
eh we can readd that later Rust side.
2021-01-09 11:41:41 -05:00
Colin Walters
2b9b83d9ce tests: Fix rsync of base rpmdb with sqlite
Apparently small rpmdb changes can cause the size to stay the
same due to preallocation, and rsync defaults to skipping files
based on (name, size, mtime).  It's really ostree's mtime canonicalization
that's unfortunate here.

Anyways, we obviously don't care about performance here so use
`-I` to disable that rsync check.

(Also remove the `mkdir -p` since it's not necessary since a long time)

Closes: https://github.com/coreos/rpm-ostree/issues/2435
2021-01-09 10:13:20 -05:00
Colin Walters
4b233daca7 Port composepost to cxx-rs
This one was easy.
2021-01-07 20:50:43 -05:00
Colin Walters
52eaa6b6b6 Port cliwrap FFI to cxx-rs
The example of how the `cliwrap_entrypoint()` function
can just be directly bound with this is a great
example of the cleanup.
2021-01-07 17:05:41 -05:00