IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
The only part left that we will need to keep ~forever is
the treefile parsing `rojig:` because it's used by coreos-assembler.
But all we need is to propagate it into the JSON treefile.
The inevitable followup to https://github.com/coreos/rpm-ostree/pull/2278
that I was too cowardly to do at the time. But it's time to admit
the 2 months or so of work on this was wasted. We have too much
tech debt and this is a large chunk of C/C++ code that touches everything
in the codebase in a nontrivial way.
Bigger picture, I'm going to work on
https://github.com/coreos/fedora-coreos-tracker/issues/828
which will strongly orient rpm-ostree towards the container world instead.
We'll still obviously keep the rpm package world around, but only
as a secondary layer. What rojig was trying to do in putting "images"
inside an RPM was conflating layers. It would have had a lot of
benefits probably if we'd truly pushed it over the edge into completion,
but that didn't happen. Let's focus on containers instead.
There's still a lot more rojig code to delete but this first patch removes
the bulk of it. Touching everything that references e.g. `RPMOSTREE_REFSPEC_TYPE_ROJIG`
etc. can come as a 3rd phase.
We have fully transitioned to cxx-rs! This drops a lot of now
dead code; only one binding system to think about generating
source code. For example, a notable advantage of cxx-rs
is it doesn't scan the whole source code, so running `make`
doesn't spew errors from cbindgen not understanding bits.
Now that `ci/installdeps.sh` gracefully exits if run as non-root,
we can fold the cargo bits into the our build scripts and avoid
invoking both of them.
However, now we need to split test deps to separate file because
we won't have `cargo` in the main cosa pod. This also fixes a FIXME.
Steal the `grep` invocation from cosa and make it a declarative
text file so we can have comments per package etc.
This way we at least get unit test coverage (which...
our unit test coverage doesn't do much because our
main code paths require privileges or virt).
One main blocker to this is that rustc doesn't expose
first-class support for this yet:
https://github.com/rust-lang/rust/issues/39699
At a practical level this works when building in release
mode but fails with `cargo test` for some reason; linker
arguments being pruned? Not sure.
So I was able to use this when composing to find a bug,
but then for some other reason the client
side apparently infinite loops inside libsolv.
So we're not enabling this yet for those reasons, but
let's land the build infrastructure now.
```
(lldb) thread backtrace
* thread #4, name = 'pool-/usr/bin/r'
* frame #0: 0x00007fd61b97200f libc.so.6`__memcpy_sse2_unaligned_erms + 623
frame #1: 0x00007fd61cbc88e6 libasan.so.6`__asan::asan_realloc(void*, unsigned long, __sanitizer::BufferedStackTrace*) + 214
frame #2: 0x00007fd61cc4b725 libasan.so.6`__interceptor_realloc + 245
frame #3: 0x00007fd61baec43e libsolv.so.1`solv_realloc + 30
frame #4: 0x00007fd61baf0414 libsolv.so.1`repodata_add_dirstr + 276
frame #5: 0x00007fd61bb6f755 libsolvext.so.1`end_element + 53
frame #6: 0x00007fd61b05855d libxml2.so.2`xmlParseEndTag1.constprop.0 + 317
frame #7: 0x00007fd61b063548 libxml2.so.2`xmlParseTryOrFinish.isra.0 + 888
frame #8: 0x00007fd61af7ed20 libxml2.so.2`xmlParseChunk + 560
frame #9: 0x00007fd61bb727e7 libsolvext.so.1`solv_xmlparser_parse + 183
frame #10: 0x00007fd61bb5ea0e libsolvext.so.1`repo_add_rpmmd + 254
frame #11: 0x000055a4fce7a5f5 rpm-ostree`::load_filelists_cb(repo=<unavailable>, fp=<unavailable>) at dnf-sack.cpp:444:23
frame #12: 0x000055a4fce7cad6 rpm-ostree`load_ext(_DnfSack*, libdnf::Repo*, _hy_repo_repodata, char const*, char const*, int (*)(s_Repo*, _IO_FILE*), _GError**) at dnf-sack.cpp:430:13
frame #13: 0x000055a4fce7df60 rpm-ostree`dnf_sack_load_repo at dnf-sack.cpp:1789:26
frame #14: 0x000055a4fce7eee9 rpm-ostree`dnf_sack_add_repo at dnf-sack.cpp:2217:28
frame #15: 0x000055a4fce7f0fb rpm-ostree`dnf_sack_add_repos at dnf-sack.cpp:2271:32
frame #16: 0x000055a4fce870ee rpm-ostree`dnf_context_setup_sack_with_flags at dnf-context.cpp:1796:29
frame #17: 0x000055a4fcdf757f rpm-ostree`rpmostree_context_download_metadata at rpmostree-core.cxx:1206:44
frame #18: 0x000055a4fcdf95c3 rpm-ostree`rpmostree_context_prepare at rpmostree-core.cxx:2001:48
frame #19: 0x000055a4fce54ab7 rpm-ostree`rpmostree_sysroot_upgrader_prep_layering at rpmostree-sysroot-upgrader.cxx:1018:38
frame #20: 0x000055a4fcdcb143 rpm-ostree`deploy_transaction_execute(_RpmostreedTransaction*, _GCancellable*, _GError**) at rpmostreed-transaction-types.cxx:1445:49
frame #21: 0x000055a4fcdba4cd rpm-ostree`transaction_execute_thread(_GTask*, void*, void*, _GCancellable*) at rpmostreed-transaction.cxx:340:34
frame #22: 0x00007fd61c58f7e2 libgio-2.0.so.0`g_task_thread_pool_thread + 114
frame #23: 0x00007fd61c3d7e54 libglib-2.0.so.0`g_thread_pool_thread_proxy.lto_priv.0 + 116
frame #24: 0x00007fd61c3d52b2 libglib-2.0.so.0`g_thread_proxy + 82
frame #25: 0x00007fd61b8af3f9 libpthread.so.0`start_thread + 233
frame #26: 0x00007fd61b9c9903 libc.so.6`__clone + 67
(lldb)
```
So...at some point we somehow lost `-Wall` in our
default compiler flags which means we were missing some
potentially important warnings. And
we used to have `-Werror` on in CI which combined
with the above was strongly opinionated about not
landing warnings in git master.
Our default stance here remains the same; we have
an opinionated set of `-Werror=` that applies in
*all* configurations. However that set moves
into Automake - I don't think we need to do
compiler version detection anymore, we can assume
a modern compiler.
We also add back in `-Wall` by default now.
Further in CI, add `-Werror`. The implementation
here is in our buildsystem rather than
`export CXXFLAGS=-Werror` because unfortunately
we have to fix things in libdnf too, and I don't
want to block entirely on that.
First, the public shared library only depends on a few
things (not the libdnf dependencies) so let's ensure we
only link it to those libraries.
And then, I realized we don't actually need the libdnf
dependencies here - I think I only added those back here
when trying vainly to keep the C unit tests working. But
we don't have those anymore! So we can delete the duplication
and fully rely on Cargo taking care of libdnf.
Conceptually for a static library we don't "link" it against
anything in Automake, that happens at the final stage with
the Rust linker.
This is now further migration towards Cargo/Rust possible
because we switched our main binary. We've had an internal
`libdnf-sys` crate for a while, but now it can take over
the build of the underlying library too (like many `-sys`
crates support).
This itself is just an incremental step towards migrating
the main rpm-ostree build system to e.g. cmake too (or
perhaps directly with the `cc` crate, not sure yet) and
driving it via `cargo` too.
We now have bidirectional calling between Rust and C++,
but we are generating two static libraries that we then
link together with a tiny C++ `main.cxx`.
Let's make another huge leap towards oxdiation by
having Rust be the entrypoint. This way cargo natively
takes care of linking the internal Rust library, and
our C++ internals become the library.
In other words, we've now fully inverted from
"C app with internal Rust library"
to "Rust binary with internal C++ library".
In order to make this work though we have to finally
kill the C unit tests. But mostly everything covered
there is either being converted to Rust, or covered
elsewhere anyways.
Now as the doc comments in `main.rs` say...this is
a bit awkward because all the CLI code is still in C++.
Porting stuff to use e.g. `structopt` natively would
be a bit of a slog. For now, we basically rely on
the fact that the Rust-native CLIs are all hidden
commands.
Update submodule: libdnf
Prep for "Rust-as-main", where I want to build libdnf statically.
And this really completes the "library thinout" story because
now we avoid dragging our *private* `libdnf.so` into the caller's
address space, which can cause potential conflicts if they're
also linking the system one. (Which could easily occur with
something like gnome-software)
All we were using libdnf for (indirectly via libsolv) is comparing
version strings but librpm can already do that for us.
cxx.rs (aka cxxbridge) and cbindgen are
both generating source code. Since the last release
we've introduced the former, and we need to ensure
that the generated cxx.rs source ends up in release tarballs
the same way as the cbindgen code.
Rationalize and clean up the binding infrastructure.
Drop support for the vendored cbindgen which we
weren't actually using:
Closes: https://github.com/coreos/rpm-ostree/issues/2392
Move the cxx-rs and cbindgen bits into the same place,
and update our CoreOS CI build to use a separate `Makefile.bindings`
that just generates the code, so our CI still "works like"
a main Koji RPM build.
Rust has a nice crate for doing perfect hashing. Move that
code into Rust and drop the dependency on `gperf`. This also
helps move away from Autotools.
This is much better than bindgen because it's fully safe. It's
much more ergonomic too:
- Invoke Rust methods-on-structs just like C++ methods-on-structs
- Rust `Result<>` is translated automatically to exceptions
See https://cxx.rs/context.html for more.
We set it to the same value in both cases. There's no reason
to require it at build time anyways.
Part of thinning out the Autotools side in prep for using Cargo.
We aren't going to be shipping major new versions to RHEL7
ever again, so drop this compat definition.
Drop this as part of thinning out our Autotools build system
in preparation for moving to `build.rs`.
This was only needed for RHEL7; RHEL8 and above ships bwrap with
the OS.
Drop this as part of thinning out our Autotools build system
in preparation for moving to `build.rs`.
There won't be any support for writing to the bdb backend in f34, so
e.g. pkglayering won't work (and obviously even composes wouldn't work
once the buildroot moves to f34).
Instead of requiring the whole world to add an `rpmdb` key in their
manifests, let's just add a compile flag for it, and tweak the spec file
to use this flag on f34.
First, we now need the `vendor/` directory at the toplevel because
that's where `Cargo.toml` is.
Now this triggers another bug introduced in the build system
with how we're handling the `rpmostree-rust.h` header.
We ended up vendoring a pre-generated one in the tarball
mainly because RHEL doesn't include cbindgen.
Now probably in the future I'd like to fix that.
But let's clean this up - the tarball generation process copies
the file into `rpmostree-rust-prebuilt.h`, and build machinery
detects that and entirely skips looking for or trying to build
our internal cbindgen.
This is part of investigating using https://cxx.rs/
In order to make this really work, we need to convert some of our C
code to C++ so we can include cxx.rs-generated code.
This starts by converting just two files as a starting point.
I did the minimal porting; I didn't try to actually rewrite them
to resemble modern C++, just "C in C++ mode".
This command allows users to cheaply inject configuration files in the
initramfs stage without having to regenerate the whole initramfs (or
even a new OSTree commit). This will be useful for configuring services
involved in bringing up the root block device.
```
$ echo 'hello world' > /etc/foobar
$ rpm-ostree ex initramfs-etc --track /etc/foobar
Staging deployment... done
Run "systemctl reboot" to start a reboot
$ rpm-ostree status
State: idle
Deployments:
ostree://fedora:fedora/x86_64/coreos/testing-devel
Version: 32.20200716.dev.1 (2020-07-16T02:47:29Z)
Commit: 9a817d75bef81b955179be6e602d1e6ae350645b6323231a62ba2ee6e5b9644b
GPGSignature: (unsigned)
InitramfsEtc: /etc/foobar
● ostree://fedora:fedora/x86_64/coreos/testing-devel
Version: 32.20200716.dev.1 (2020-07-16T02:47:29Z)
Commit: 9a817d75bef81b955179be6e602d1e6ae350645b6323231a62ba2ee6e5b9644b
GPGSignature: (unsigned)
$ reboot
(boot into rd.break)
sh-5.0# cat /etc/foobar
hello world
```
See the libostree side of this at:
https://github.com/ostreedev/ostree/pull/2155
Lots more discussions in:
https://github.com/coreos/fedora-coreos-tracker/issues/94Closes: #1930
This avoids us leaking all of the Rust symbols as public
into our shared library.
Came up in team chat since one person tried to copy just `/usr/bin/rpm-ostree`
from the dev container into a target and got a linker crash; but
really we want to avoid all the duplicated symbols entirely.
(And we should investigate cross-language LTO because that's the
only way to get full savings)
We're seeing some CI failures that I think are a bug in rojig.
In the bigger picture...we never actually started using this,
and I think longer term shipping os updates via containers
probably makes more sense.
I put a *lot* of effort into this code and it's pretty cool
so it's hard to just delete it. And *maybe* someone out there
is using it (but I doubt it). So rather than just deleting
it entirely let's make it a build-time option.
I verified that it builds at least.
The instructions for doing a release were outdated. Take the opportunity
to just move it to a dedicated markdown file and breaking it out to make
it easier to follow.
This is one step short of using a release checklist like some of the
other CoreOS projects do. This is a good base to eventually do that if
we want.
Going to update rpm-ostree for RHEL 8.3, we did a huge bump
in libdnf which now defaults to enabling zchunk in its build
system. We added the infrastructure before to detect things,
so propagate that to libdnf.
The current `rpm-ostree-2020.1-1.fc31.x86_64` in Fedora
was [built with a truly ancient libostree](https://kojipkgs.fedoraproject.org//packages/rpm-ostree/2020.1/1.fc31/data/logs/x86_64/root.log)
because Fedora's build system is weird and only adds packages
released after "gold" into the buildroot via an override
that times out.
This actively breaks things because rpm-ostree isn't
detecting the read-only sysroot.
Let's bump our hard requirement.
