IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Our "CoreOS CI Jenkins" flow is great, but the reliability/debuggabilty
of the system isn't the best. This is an experient to *add* another CI
flow that can be run in Prow too.
I'd like to at least have a basic e2e (compose tree, run client tests)
flow that can operate in Prow so that if CoreOS CI Jenkins is down or
flaky, we at least have some confidence in our commits.
Longer term I think we can figure out how to better share CI flows
across these:
https://github.com/coreos/fedora-coreos-tracker/issues/263#issuecomment-772903759
When we started using cxxrs, most of the glib-rs objects like
`OstreeRepo`/`OstreeSysroot` were owned by C++ and passed
down into Rust. That motivated the addition of the special
bridging infrastructure to re-create a glib-rs wrapper
type from what cxxrs wants (a `Pin<&mut T>`).
But now that we're adding more in Rust, we have the need
to pass these objects back into C++. In fact this will
hopefully soon because the default case as more of the
binary entrypoint becomes Rust.
Add another trait with a method `gobj_rewrap()` that converts
in the other direction. This implementation took me a number
of tries before I finally settled on simply using `mem::transmute()`.
There are a *lot* of caveats listed on the docs for that function,
but I think it really is what we want here. See the link for pending work
on a Rust RFC to enable safe transmutes for some cases, and I believe
that would cover this use case:
https://internals.rust-lang.org/t/pre-rfc-v2-safe-transmute/11431
I've verified this works in a separate patch, but this commit
also adds a simple test case - this goes all the way from:
Rust glib-rs `ostree::Repo` (holding strong ref)
-> Rust `Pin<&mut ostree_sys::OstreeRepo>`
-> (internal cxx-rs C bridge)
-> C++ `OstreeRepo&` reference
-> C `OstreeRepo*` pointer
Which is quite the dance if you think about it!
clippy complains about `to_*` not taking `&self` - I think
we can simplify this more by using our `write_file_with()` API
that we're using in other places. It was explicitly designed
with serde in mind. It's just more efficient and nicer to
serialize to a `BufWriter` instead of to a string, then
writing the string.
Do not perform an upgrade if detected that an updates driver has
been registered.
Add --bypass-driver option to force an upgrade regardless of whether an
updates driver has been registered.
Now that `ci/installdeps.sh` gracefully exits if run as non-root,
we can fold the cargo bits into the our build scripts and avoid
invoking both of them.
However, now we need to split test deps to separate file because
we won't have `cargo` in the main cosa pod. This also fixes a FIXME.
Steal the `grep` invocation from cosa and make it a declarative
text file so we can have comments per package etc.
Doing the distro-sync costs ~3 minutes per execution, and it
happens multiple times. Let's just ensure our images are up
to date instead.
Also drop the continuous repo (for now) - we added this
to test bleeding edge ostree, but I think we need to reintroduce
"git master" builds as whole separate CI flow (layered container)
instead.
I didn't deep dive on this, just observed that the new output
matches what we laid down. There's ~2.5 years of changes
and ~200 commits between 2.28 and 2.48.
When working on a PR to add a sub-crate I hit the fact
that our `find` bit wasn't fully accurate and spent some
time debugging the fact that the code I got after `make`
wasn't up to date.
Since cargo is smart in general, let's stop trying to
second guess its dependencies and just run `cargo build`
every time `make` is run.
(I'm not sure why we didn't do this from the start)
This is actually a perfect example of the tradeoffs involved
in our team of C/Rust programmers trying to use C++ so we
can use cxx-rs =)
It turns out that "for (auto foo : bar)" is really yet another
one of those C++ sharp edges just waiting to cut. That
version does a copy of each value; one almost always wants to use
"for (auto & foo : bar)" which like Rust's `.iter()`.
In most of our code we were OK making copies, they were just
a performance hit, but in another place I was relying on the
"side effect" of `.c_str()` but that meant our value got
destroyed.
Print error and exit if cannot find `tests/vmcheck/image.qcow2` or
bad symlink causing failure to spawn a VM when calling the
`vm_kola_spawn()` function.
Specifically motivated by adding some docs in `treefile.rs` around
how to add a field, but I decided to just do a pass and
document everything at least a little.
View with e.g. `cargo doc --document-private-items`.
This way we at least get unit test coverage (which...
our unit test coverage doesn't do much because our
main code paths require privileges or virt).
One main blocker to this is that rustc doesn't expose
first-class support for this yet:
https://github.com/rust-lang/rust/issues/39699
At a practical level this works when building in release
mode but fails with `cargo test` for some reason; linker
arguments being pruned? Not sure.
So I was able to use this when composing to find a bug,
but then for some other reason the client
side apparently infinite loops inside libsolv.
So we're not enabling this yet for those reasons, but
let's land the build infrastructure now.
```
(lldb) thread backtrace
* thread #4, name = 'pool-/usr/bin/r'
* frame #0: 0x00007fd61b97200f libc.so.6`__memcpy_sse2_unaligned_erms + 623
frame #1: 0x00007fd61cbc88e6 libasan.so.6`__asan::asan_realloc(void*, unsigned long, __sanitizer::BufferedStackTrace*) + 214
frame #2: 0x00007fd61cc4b725 libasan.so.6`__interceptor_realloc + 245
frame #3: 0x00007fd61baec43e libsolv.so.1`solv_realloc + 30
frame #4: 0x00007fd61baf0414 libsolv.so.1`repodata_add_dirstr + 276
frame #5: 0x00007fd61bb6f755 libsolvext.so.1`end_element + 53
frame #6: 0x00007fd61b05855d libxml2.so.2`xmlParseEndTag1.constprop.0 + 317
frame #7: 0x00007fd61b063548 libxml2.so.2`xmlParseTryOrFinish.isra.0 + 888
frame #8: 0x00007fd61af7ed20 libxml2.so.2`xmlParseChunk + 560
frame #9: 0x00007fd61bb727e7 libsolvext.so.1`solv_xmlparser_parse + 183
frame #10: 0x00007fd61bb5ea0e libsolvext.so.1`repo_add_rpmmd + 254
frame #11: 0x000055a4fce7a5f5 rpm-ostree`::load_filelists_cb(repo=<unavailable>, fp=<unavailable>) at dnf-sack.cpp:444:23
frame #12: 0x000055a4fce7cad6 rpm-ostree`load_ext(_DnfSack*, libdnf::Repo*, _hy_repo_repodata, char const*, char const*, int (*)(s_Repo*, _IO_FILE*), _GError**) at dnf-sack.cpp:430:13
frame #13: 0x000055a4fce7df60 rpm-ostree`dnf_sack_load_repo at dnf-sack.cpp:1789:26
frame #14: 0x000055a4fce7eee9 rpm-ostree`dnf_sack_add_repo at dnf-sack.cpp:2217:28
frame #15: 0x000055a4fce7f0fb rpm-ostree`dnf_sack_add_repos at dnf-sack.cpp:2271:32
frame #16: 0x000055a4fce870ee rpm-ostree`dnf_context_setup_sack_with_flags at dnf-context.cpp:1796:29
frame #17: 0x000055a4fcdf757f rpm-ostree`rpmostree_context_download_metadata at rpmostree-core.cxx:1206:44
frame #18: 0x000055a4fcdf95c3 rpm-ostree`rpmostree_context_prepare at rpmostree-core.cxx:2001:48
frame #19: 0x000055a4fce54ab7 rpm-ostree`rpmostree_sysroot_upgrader_prep_layering at rpmostree-sysroot-upgrader.cxx:1018:38
frame #20: 0x000055a4fcdcb143 rpm-ostree`deploy_transaction_execute(_RpmostreedTransaction*, _GCancellable*, _GError**) at rpmostreed-transaction-types.cxx:1445:49
frame #21: 0x000055a4fcdba4cd rpm-ostree`transaction_execute_thread(_GTask*, void*, void*, _GCancellable*) at rpmostreed-transaction.cxx:340:34
frame #22: 0x00007fd61c58f7e2 libgio-2.0.so.0`g_task_thread_pool_thread + 114
frame #23: 0x00007fd61c3d7e54 libglib-2.0.so.0`g_thread_pool_thread_proxy.lto_priv.0 + 116
frame #24: 0x00007fd61c3d52b2 libglib-2.0.so.0`g_thread_proxy + 82
frame #25: 0x00007fd61b8af3f9 libpthread.so.0`start_thread + 233
frame #26: 0x00007fd61b9c9903 libc.so.6`__clone + 67
(lldb)
```
Forcing use of GCC breaks trying to use `clang -fsanitize=address`.
The real fix here anyways should have been avoiding passing
compiler flags to a compiler that doesn't understand them.
Pretty straightforward. Haven't moved `ror_lockfile_write` yet because
that's trickier to do and I'm still figuring out the most elegant way to
do this within cxx.rs' constraints.
Right now, we're using libdnf APIs from Rust via hand-crafted `extern C`
interfaces, which is extra dangerous because there is no signature
checking that happens at compile-time.
Until either we can automate libdnf bindings or use its C++ API directly
via cxx.rs, let's do some basic wrapping in C++ ourselves and use libdnf
through that API only instead. That gives us a lot more confidence and
makes the libdnf API feel more natural to use in Rust.
We don't need it and it won't work in the unprivileged path where we're
running this in a cosa supermin (of course, this is all a bit silly
because we don't actually need privileges to begin with for this, but
there's a lot of momentum in sticking with that workflow).
Update submodule: libglnx
Now that we inject the `%_dbpath /usr/share/rpm` macro, `rpm -q`
will start using it. But in RPM script invocation, we don't
want them to see any RPM database at all - trying to query it
should be a clean failure.
We lost this at some point during the CI re-shuffle. We need to
constrain cargo builds too to respect our CPU allocation.
This doesn't totally keep all jobs under 5 since e.g. we could have 5
make jobs and 5 cargo codegen builds going at once, but I think as long
as it's not something ridiculous like 40, it should be fine. Otherwise
we'll tighten it more.
Follow-up to previous commit: we had another path where we made a
temporary rootfs and symlinked `/var/lib/rpm` to the base rpmdb. That of
course broke now that we inject a macro to point the rpmdb to
`/usr/share/rpm`.
Rework this to use `/usr/share/rpm` since that's our canonical location
for now, but also add the compat symlinks so that this logic should keep
working even on trees without the injected macro yet.
We don't technically need this yet, but it mirrors how it's set up in
our composes so that if there's code that wants to use the new location
too, it'll just work.
We trigger a librpm macro file load in many of our paths. Since the
default value shipped by rpm's macro file sets `_dbpath` to
`/var/lib/rpm`, we have to explicitly set that back to `/usr/share/rpm`
in those paths.
This became more problematic recently with libsolv v0.7.17 which fully
keys off of `_dbpath` to find the rpmdb path to load:
04d4d036b2
And it's not technically wrong; we really should make that macro not
lie. This is what this patch does by injecting an RPM macro file in our
composes which sets it to /usr/share/rpm. So then e.g. the `rpm` CLI
doesn't actually need the `/var/lib/rpm` backcompat link anymore, though
there's no harm in leaving it.
In the future, we should be able to drop this once we move all of Fedora
to `/usr/lib/sysimage/rpm` (see
https://github.com/coreos/fedora-coreos-tracker/issues/639).
Closes: #2548
Originally the Rust apply-live code was exposed from Rust to C
via bindgen. But when working on that, I hit the problem
that our output infrastructure was C...and the "reverse direction"
binding stuff was just ugly.
This PR again IMO shows the value of the investment in cxx-rs
because we can now seamlessly call back from the Rust side
into a "C++-ish" progress API, which the C++ side is updated
to use.
The level of indirection here is obviously pretty silly
because the main thing on the C++ output side is basically
a function dispatcher, but...I didn't want to try to rework
that into Rust fully yet. (But, the moment we do this
whole area will get a *lot* cleaner)
Anyways, in the end this makes it easy for the apply-live
code to output progress to the user which was sorely
needed.
Our output system is very confusing in that we bridge over
DBus in some cases and not others. In preparation for allowing
Rust code to call into the C++ progress system which contains
that delegation layer, rename the Rust progress to `console_`
to clearly show that it should only be invoked by code that
knows it's writing to a tty.
Having our binary depend on the shared library, which in
turn depends on the binary (at runtime) is messy.
Instead, statically compile the shlib code into our binary.
This duplicates the text a bit, but it's not a lot of code.
The goal is to more easily in the future to e.g. move the
shared library out into a separate git repository entirely
that runs on a separate lifecycle - that would still build
using Automake for example while the main git repository
switches to purely cargo.
Another motivation is avoiding linker issues I had with other
patches due to this semi-cyclical dependency.
So...at some point we somehow lost `-Wall` in our
default compiler flags which means we were missing some
potentially important warnings. And
we used to have `-Werror` on in CI which combined
with the above was strongly opinionated about not
landing warnings in git master.
Our default stance here remains the same; we have
an opinionated set of `-Werror=` that applies in
*all* configurations. However that set moves
into Automake - I don't think we need to do
compiler version detection anymore, we can assume
a modern compiler.
We also add back in `-Wall` by default now.
Further in CI, add `-Werror`. The implementation
here is in our buildsystem rather than
`export CXXFLAGS=-Werror` because unfortunately
we have to fix things in libdnf too, and I don't
want to block entirely on that.
