Commit Graph

523 Commits

Author SHA1 Message Date
Colin Walters
12cc4e5f5b composepost: Fix race condition in timestamp checking
`syncfs()` isn't going to do anything on e.g. `tmpfs` and even
if it did wouldn't fix any race conditions because that's about
synchronizing in memory changes to disk, but won't change
what system calls return.

Some investigation turned up
https://stackoverflow.com/questions/14392975/timestamp-accuracy-on-ext4-sub-millsecond
and `current_fs_time` is now:
https://www.kernel.org/doc/html/v5.12/core-api/timekeeping.html

Basically there's a "coarse" time that might only update once every 10ms
for example.

Let's just sleep 100ms for now.

I think we should be using the inode versions, but we can investigate
that separately.
2021-04-30 15:15:10 -04:00
Colin Walters
7b57f2b9d0 Add and use the camino library for UTF-8 paths
ostree hard requires UTF-8 paths (and really we should
never have any non-UTF-8 paths in the OS in general).  The
camino library has types that are both `Path` and `&str` and
has a convenient `try_into()` too to avoid us duplicating
the error handling.
2021-04-30 09:56:05 -04:00
Jonathan Lebon
ba7a72995b treefile: Add new repo-packages field for pinning packages to repos
This addresses the server compose side of
https://github.com/coreos/rpm-ostree/issues/2584.

One tricky bit is handling overrides across included treefiles (or
really, even within a single treefile): as usual, higher-level treefiles
should override lowel-level ones. Rust makes it pretty nice to handle.

For now this just supports a `repo` field, but one could imagine e.g.
`repos` (which takes an array of repoids instead), or e.g.
`exclude-repos`.

The actual core implementation otherwise is pretty straightforward.

This should help a lot in RHCOS where we currently use many `exclude=`
directives in repo files to get it to do what we want.

This is also kind of a requirement for modularity support because as
soon as rpm-ostree becomes modules-aware, modular filtering logic will
break composes which assume rpm-ostree treats modular and non-modular
packages the same.
2021-04-30 09:13:18 -04:00
Luca BRUNO
3b76a7eeef composepost: move rpmdb linking logic to Rust
This ports the rpmdb hardlinking logic which serves to maintain
a coherent single source of truth across tools.
2021-04-29 16:05:27 -04:00
Luca Bruno
30d5c79272
Merge pull request #2795 from coreos/dependabot/cargo/cxx-1.0.48
build(deps): bump cxx from 1.0.47 to 1.0.48
2021-04-29 15:17:47 +00:00
Luca BRUNO
f1138dd134 composepost: move SELinux workaround logic to Rust
This ports the timestamp updating logic which serves as a workaround
for SELinux leaking host details.
2021-04-29 10:47:13 -04:00
Jonathan Lebon
fdd6ff83cb rust/treefile: Fix minor comment typo 2021-04-29 09:52:02 -04:00
Jonathan Lebon
c095b97bf3 libpriv: Bridge print_treepkg_diff_from_sysroot_path()
This will be necessary for Rust-based client commands to be able to do
the regular "post-deployment package diff".
2021-04-29 09:52:02 -04:00
Jonathan Lebon
0474d40138 rust: Drop includes.rs
This dates from the cbindgen era and isn't needed anymore.
2021-04-29 09:52:02 -04:00
Jonathan Lebon
d465eeac68 rust/main: Pass CLI args to command entrypoints
The only entrypoint so far doesn't use it (`countme`), but a future new
entrypoint will.

Also mention that the commands should add themselves to the array in
libmain.cxx if applicable so it shows up in `--help`.
2021-04-29 09:52:02 -04:00
dependabot[bot]
06aad3adf1
build(deps): bump cxx from 1.0.47 to 1.0.48
Bumps [cxx](https://github.com/dtolnay/cxx) from 1.0.47 to 1.0.48.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.47...1.0.48)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-29 08:55:14 +00:00
Luca Bruno
4ad6ebe84b
Merge pull request #2784 from cgwalters/add-tracing
Add Rust tracing crate usage
2021-04-29 08:41:47 +00:00
Luca Bruno
bb971372ec
Merge pull request #2793 from coreos/dependabot/cargo/cxx-build-1.0.47
build(deps): bump cxx-build from 1.0.46 to 1.0.47
2021-04-28 23:22:32 +00:00
dependabot[bot]
bdd1a656db
build(deps): bump cxx-build from 1.0.46 to 1.0.47
Bumps [cxx-build](https://github.com/dtolnay/cxx) from 1.0.46 to 1.0.47.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.46...1.0.47)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-28 19:34:31 +00:00
dependabot[bot]
2509003b69
build(deps): bump cxx from 1.0.46 to 1.0.47
Bumps [cxx](https://github.com/dtolnay/cxx) from 1.0.46 to 1.0.47.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.46...1.0.47)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-28 19:34:23 +00:00
Colin Walters
b2c8c0fba6 Add Rust tracing crate usage
Related to https://github.com/ostreedev/ostree-rs-ext/pull/21
And inspired by seeing some discussion around the
https://crates.io/crates/tracing
crate.
2021-04-28 15:32:42 -04:00
Luca Bruno
178e2200e9
Merge pull request #2789 from lucab/ups/composepost-tmp-chmod
composepost: fix fchmod flags for tmpdir (glibc compat)
2021-04-28 19:30:46 +00:00
Colin Walters
e2bcf01ac1 Fix bwrap usage for mutate-os-release
Followup to https://pagure.io/fedora-infrastructure/issue/9909

In the refactor we were passing `unified_core: true` unconditionally which was wrong,
as that implies using fuse.  Anyways what we really want here is an immutable bwrap
and not `rofiles-fuse` annyways.  So refactor things to use that.

From https://kojipkgs.fedoraproject.org//work/tasks/7579/66867579/runroot.log
```
fuse: device not found, try 'modprobe fuse' first
fuse: device not found, try 'modprobe fuse' first
bwrap: execvp realpath: No such file or directory
fusermount: failed to unmount /tmp/rpmostree-rofiles-fuseAAphRY: Invalid argument
fusermount: failed to unmount /tmp/rpmostree-rofiles-fuseSCLs24: Invalid argument
error: Updating os-release with commit version: Running realpath: bwrap(realpath): Child process killed by signal 1
```
2021-04-28 13:27:40 -04:00
Luca BRUNO
6c75f57766
composepost: fix fchmod flags for tmpdir (glibc compat)
The `AT_SYMLINK_NOFOLLOW` flag for chmod was not implemented in glibc
until very recent releases (v2.32 in 2020-08).
Before that version, passing this flag will always result in `ENOTSUP`.

Refs:
 * https://github.com/bminor/glibc/blob/glibc-2.31/sysdeps/unix/sysv/linux/fchmodat.c#L36
 * https://github.com/bminor/glibc/blob/glibc-2.32/sysdeps/unix/sysv/linux/fchmodat.c
2021-04-28 13:38:40 +00:00
Luca BRUNO
c708b47fc9
composepost: expand unit test surface 2021-04-28 13:37:51 +00:00
Colin Walters
387db0c820 Require ostree v2021.2, use ostree-ext crate
This uses the ostree and ostree-ext crates from git as a demo/WIP
for https://mail.gnome.org/archives/ostree-list/2021-April/msg00000.html
2021-04-27 09:50:02 -04:00
Luca Bruno
bd6fe21575
Merge pull request #2779 from coreos/dependabot/cargo/cxx-build-1.0.46
build(deps): bump cxx-build from 1.0.45 to 1.0.46
2021-04-26 14:00:08 +00:00
dependabot[bot]
1a53403483
build(deps): bump cxx from 1.0.45 to 1.0.46
Bumps [cxx](https://github.com/dtolnay/cxx) from 1.0.45 to 1.0.46.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.45...1.0.46)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-26 10:23:55 +00:00
dependabot[bot]
5909d17081
build(deps): bump cxx-build from 1.0.45 to 1.0.46
Bumps [cxx-build](https://github.com/dtolnay/cxx) from 1.0.45 to 1.0.46.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.45...1.0.46)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-26 06:59:37 +00:00
Luca BRUNO
575ffb0ab6 composepost: optimize disk roundtrips
This reduces the amount of mkdir/rmdir performed, directly computing
the parent path instead.
2021-04-22 13:11:52 -04:00
Luca BRUNO
ad365df4b0 composepost: move rootfs symlinks creation to Rust
This ports the post-processing logic which creates symlinks for
several known state directories under /usr.
2021-04-22 13:11:52 -04:00
Jonathan Lebon
666f7d9df6
Merge pull request #2754 from cgwalters/origin-treefile-prep-2 2021-04-20 10:46:27 -04:00
Colin Walters
a798175a85 core: Fix tempetc guard to be no-op if /etc exists
This is further hardening to prevent a situation like
https://github.com/coreos/rpm-ostree/issues/2771
where we would crash on an ostree-based host that has both
`/etc` and `/usr/etc` as physical directories.

That shouldn't happen after the bwrap fix, but we might
as well be more correct here.
2021-04-20 08:12:45 -04:00
Colin Walters
8758b0faa2 bwrap: Fix selftest to be truly immutable
We should never have any effect on the host system, so let's
use the more direct APIs which allow us to use the immutable
flag, don't mount `/var` etc.

Crucially this also avoids us running through the tempetc
guard which would try to rename `usr/etc` which can trigger
on an ostree based host.

Closes: https://github.com/coreos/rpm-ostree/issues/2771
2021-04-20 08:09:57 -04:00
Colin Walters
a015fb345f Expose more from treefile via cxx-rs; adapt _install_langs to use it
We only honor `_install_langs` in the compose path, so move the
code to look at the treefile and move the string concatention to
Rust.
2021-04-19 19:59:06 -04:00
Jonathan Lebon
78b7f934e4 rust/passwd: Fix handling of previous mode
We need to handle the case where no previous commit exists. This is
expressed from the C side by passing the empty string.

We're currently not testing this, though... AFAIK no distro uses this
right now anyway and hopefully we simplify a lot of this when we move to
systemd-sysusers!

Fixes: #2580
Fixes: #2769
2021-04-19 19:56:08 -04:00
Jonathan Lebon
2ad8543b3e
Merge pull request #2758 from cgwalters/treefile-validate-repos
compose: Move repos/lockfile-repos validation to Rust
2021-04-19 11:43:21 -04:00
Luca Bruno
610815eb83
Merge pull request #2765 from coreos/dependabot/cargo/ostree-sys-0.7.3
build(deps): bump ostree-sys from 0.7.2 to 0.7.3
2021-04-19 12:35:19 +00:00
Luca BRUNO
0d79b25a52 postprocess: move /var tmpfiles.d translation logic to Rust
This ports to Rust the auto-tmfiles.d translation logic which is
meant to consume a populated `/var` and to produce a matching
`rpm-ostree-1-autovar.conf` instead.
It also adds a unit-test covering most codepaths.
2021-04-19 07:39:58 -04:00
dependabot[bot]
84e8cec344
build(deps): bump ostree-sys from 0.7.2 to 0.7.3
Bumps [ostree-sys](https://gitlab.com/fkrull/ostree-rs) from 0.7.2 to 0.7.3.
- [Release notes](https://gitlab.com/fkrull/ostree-rs/tags)
- [Commits](https://gitlab.com/fkrull/ostree-rs/commits/master)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-19 10:15:38 +00:00
Luca Bruno
acda0753e6
Merge pull request #2764 from coreos/dependabot/cargo/cxx-1.0.45
build(deps): bump cxx from 1.0.43 to 1.0.45
2021-04-19 08:38:23 +00:00
Luca Bruno
44dbfd1fcd
Merge pull request #2763 from coreos/dependabot/cargo/cxx-build-1.0.45
build(deps): bump cxx-build from 1.0.42 to 1.0.45
2021-04-19 08:37:54 +00:00
dependabot[bot]
c4b66a0e66
build(deps): bump cxx from 1.0.43 to 1.0.45
Bumps [cxx](https://github.com/dtolnay/cxx) from 1.0.43 to 1.0.45.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.43...1.0.45)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-19 06:29:58 +00:00
dependabot[bot]
36ef9b8ca2
build(deps): bump cxx-build from 1.0.42 to 1.0.45
Bumps [cxx-build](https://github.com/dtolnay/cxx) from 1.0.42 to 1.0.45.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.42...1.0.45)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-19 06:29:45 +00:00
Colin Walters
db227840f1 daemon: Always write staged JSON field
It's an important property, and it's more self-documenting if we
always emit the value rather than only when it's `true`.
2021-04-16 17:05:48 -04:00
Colin Walters
5845e3ab6a compose: Move repos/lockfile-repos validation to Rust
More prep for removing treespec.
2021-04-16 17:05:11 -04:00
Colin Walters
834328f963 Move ref parsing into core
We went through a lot of gyrations on this one.  It's only
relevant to server side composes, so having the core parse
the treefile for it directly just makes sense.
2021-04-14 18:24:42 -04:00
Colin Walters
53b4fc8c26 Read treefile directly for releasever
More draining of treespec to treefile.
2021-04-14 18:24:42 -04:00
Colin Walters
c792587645 Read treefile directly for selinux and recommends flags
Part of removing treespec in favor of treefile.
2021-04-14 18:24:42 -04:00
Colin Walters
69a79bc8e6 Read treefile directly for documentation flag
Part of removing treespec in favor of treefile.
2021-04-14 18:24:42 -04:00
Luca BRUNO
8e231b5aad composepost: uniform results and visibility for cxxbridge
This aligns the result types and visibility for all post-processing
functions that are bridged to C++.
2021-04-13 11:15:52 -04:00
Luca Bruno
358865675a
Merge pull request #2737 from cgwalters/treefile-client-prep
Treefile client prep
2021-04-12 16:04:07 +00:00
dependabot[bot]
89d4e2dcf1 build(deps): bump cxx from 1.0.41 to 1.0.43
Bumps [cxx](https://github.com/dtolnay/cxx) from 1.0.41 to 1.0.43.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.41...1.0.43)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-12 10:58:08 -04:00
Colin Walters
fb7514240f treefile: derive(Default)
So that it's easy to construct without deserializing.  Prep
for converting from origins.
2021-04-10 11:14:24 -04:00
Colin Walters
fab62ac9fc treefile: Make all members pub(crate)
We've been doing this piecemeal so far, let's do it all at once
now.  Prep for converting origin files to treefiles in `origin.rs`.
2021-04-10 11:12:48 -04:00