IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
As rpm-ostree is transitioning to a primarily-Rust project,
our dependency on running `make` to generate `.gitignore` is a
problem.
For example, just opening the project in an IDE that runs rust-analyzer,
the latter will start a build and generate files in the `target/` directory,
but because r-a doesn't know to run `make` we won't have `.gitignore` yet.
I think further what we should do is try changing the C/Automake side
to write generated files into `target/` too.
This way we at least get unit test coverage (which...
our unit test coverage doesn't do much because our
main code paths require privileges or virt).
One main blocker to this is that rustc doesn't expose
first-class support for this yet:
https://github.com/rust-lang/rust/issues/39699
At a practical level this works when building in release
mode but fails with `cargo test` for some reason; linker
arguments being pruned? Not sure.
So I was able to use this when composing to find a bug,
but then for some other reason the client
side apparently infinite loops inside libsolv.
So we're not enabling this yet for those reasons, but
let's land the build infrastructure now.
```
(lldb) thread backtrace
* thread #4, name = 'pool-/usr/bin/r'
* frame #0: 0x00007fd61b97200f libc.so.6`__memcpy_sse2_unaligned_erms + 623
frame #1: 0x00007fd61cbc88e6 libasan.so.6`__asan::asan_realloc(void*, unsigned long, __sanitizer::BufferedStackTrace*) + 214
frame #2: 0x00007fd61cc4b725 libasan.so.6`__interceptor_realloc + 245
frame #3: 0x00007fd61baec43e libsolv.so.1`solv_realloc + 30
frame #4: 0x00007fd61baf0414 libsolv.so.1`repodata_add_dirstr + 276
frame #5: 0x00007fd61bb6f755 libsolvext.so.1`end_element + 53
frame #6: 0x00007fd61b05855d libxml2.so.2`xmlParseEndTag1.constprop.0 + 317
frame #7: 0x00007fd61b063548 libxml2.so.2`xmlParseTryOrFinish.isra.0 + 888
frame #8: 0x00007fd61af7ed20 libxml2.so.2`xmlParseChunk + 560
frame #9: 0x00007fd61bb727e7 libsolvext.so.1`solv_xmlparser_parse + 183
frame #10: 0x00007fd61bb5ea0e libsolvext.so.1`repo_add_rpmmd + 254
frame #11: 0x000055a4fce7a5f5 rpm-ostree`::load_filelists_cb(repo=<unavailable>, fp=<unavailable>) at dnf-sack.cpp:444:23
frame #12: 0x000055a4fce7cad6 rpm-ostree`load_ext(_DnfSack*, libdnf::Repo*, _hy_repo_repodata, char const*, char const*, int (*)(s_Repo*, _IO_FILE*), _GError**) at dnf-sack.cpp:430:13
frame #13: 0x000055a4fce7df60 rpm-ostree`dnf_sack_load_repo at dnf-sack.cpp:1789:26
frame #14: 0x000055a4fce7eee9 rpm-ostree`dnf_sack_add_repo at dnf-sack.cpp:2217:28
frame #15: 0x000055a4fce7f0fb rpm-ostree`dnf_sack_add_repos at dnf-sack.cpp:2271:32
frame #16: 0x000055a4fce870ee rpm-ostree`dnf_context_setup_sack_with_flags at dnf-context.cpp:1796:29
frame #17: 0x000055a4fcdf757f rpm-ostree`rpmostree_context_download_metadata at rpmostree-core.cxx:1206:44
frame #18: 0x000055a4fcdf95c3 rpm-ostree`rpmostree_context_prepare at rpmostree-core.cxx:2001:48
frame #19: 0x000055a4fce54ab7 rpm-ostree`rpmostree_sysroot_upgrader_prep_layering at rpmostree-sysroot-upgrader.cxx:1018:38
frame #20: 0x000055a4fcdcb143 rpm-ostree`deploy_transaction_execute(_RpmostreedTransaction*, _GCancellable*, _GError**) at rpmostreed-transaction-types.cxx:1445:49
frame #21: 0x000055a4fcdba4cd rpm-ostree`transaction_execute_thread(_GTask*, void*, void*, _GCancellable*) at rpmostreed-transaction.cxx:340:34
frame #22: 0x00007fd61c58f7e2 libgio-2.0.so.0`g_task_thread_pool_thread + 114
frame #23: 0x00007fd61c3d7e54 libglib-2.0.so.0`g_thread_pool_thread_proxy.lto_priv.0 + 116
frame #24: 0x00007fd61c3d52b2 libglib-2.0.so.0`g_thread_proxy + 82
frame #25: 0x00007fd61b8af3f9 libpthread.so.0`start_thread + 233
frame #26: 0x00007fd61b9c9903 libc.so.6`__clone + 67
(lldb)
```
So...at some point we somehow lost `-Wall` in our
default compiler flags which means we were missing some
potentially important warnings. And
we used to have `-Werror` on in CI which combined
with the above was strongly opinionated about not
landing warnings in git master.
Our default stance here remains the same; we have
an opinionated set of `-Werror=` that applies in
*all* configurations. However that set moves
into Automake - I don't think we need to do
compiler version detection anymore, we can assume
a modern compiler.
We also add back in `-Wall` by default now.
Further in CI, add `-Werror`. The implementation
here is in our buildsystem rather than
`export CXXFLAGS=-Werror` because unfortunately
we have to fix things in libdnf too, and I don't
want to block entirely on that.
This is now further migration towards Cargo/Rust possible
because we switched our main binary. We've had an internal
`libdnf-sys` crate for a while, but now it can take over
the build of the underlying library too (like many `-sys`
crates support).
This itself is just an incremental step towards migrating
the main rpm-ostree build system to e.g. cmake too (or
perhaps directly with the `cc` crate, not sure yet) and
driving it via `cargo` too.
We now have bidirectional calling between Rust and C++,
but we are generating two static libraries that we then
link together with a tiny C++ `main.cxx`.
Let's make another huge leap towards oxdiation by
having Rust be the entrypoint. This way cargo natively
takes care of linking the internal Rust library, and
our C++ internals become the library.
In other words, we've now fully inverted from
"C app with internal Rust library"
to "Rust binary with internal C++ library".
In order to make this work though we have to finally
kill the C unit tests. But mostly everything covered
there is either being converted to Rust, or covered
elsewhere anyways.
Now as the doc comments in `main.rs` say...this is
a bit awkward because all the CLI code is still in C++.
Porting stuff to use e.g. `structopt` natively would
be a bit of a slog. For now, we basically rely on
the fact that the Rust-native CLIs are all hidden
commands.
Update submodule: libdnf
Notably this includes [std::optional](https://en.cppreference.com/w/cpp/utility/optional)
which I'd like to use.
The dnf-5-devel branch uses this too, and C++-17 is now well
supported in current GCC, which is available in RHEL8 DTS too.
This is part of investigating using https://cxx.rs/
In order to make this really work, we need to convert some of our C
code to C++ so we can include cxx.rs-generated code.
This starts by converting just two files as a starting point.
I did the minimal porting; I didn't try to actually rewrite them
to resemble modern C++, just "C in C++ mode".
When g-ir-scanner runs, it uses a stub binary to help introspect the
target rpm-ostree library. That binary needs to link to the library, and
of course, transitively, to our bundled libdnf. Since we run the scanner
uninstalled, we use `LD_LIBRARY_PATH` to point the stub at the libdnf
from the build directory.
When we compile rpm-ostree, we use `-rpath` to point it at our libdnf,
which emits a `DT_RPATH` attribute. However, during dynamic linking at
runtime, it turns out that `LD_LIBRARY_PATH` has *lower* precedence than
`DT_RPATH`. This means that if there is already a libdnf at
`/usr/lib64/rpm-ostree`, it takes precedence.
This subtlety is mostly fine to ignore usually, because the stub doesn't
actually run any rpm-ostree business logic, so it doesn't really matter
that it runs against the "technically wrong" `libdnf.so.2`.
Where it becomes obvious something is off however is if we've just
built a new libdnf, and we have new code which references symbols from
the new libdnf that aren't in the stale libdnf at `$libdir/rpm-ostree`.
And this is exactly what was happening in CI (though it's of course
possible to reproduce this locally as well): we were using a new symbol,
`hy_goal_favor`, and building inside the cosa buildroot image, which
already has rpm-ostree installed, and so `ld.so` chose the bundled
libdnf of the installed rpm-ostree when loading the `g-ir-scanner` stub.
Thus why it failed with:
```
./tmp-introspectzh0n2cga/.libs/lt-RpmOstree-1.0: symbol lookup error: .libs/librpmostree-1.so.1: undefined symbol: hy_goal_favor
```
And of course, the root cause here has nothing to do with the stub in
itself. It's actually trivially easy to see the behaviour difference wrt
`LD_LIBRARY_PATH` on installed binaries. Using a shared object
from `python3-libdnf` which links to libdnf without using rpath, we can
see that `LD_LIBRARY_PATH` has an effect:
```
$ ldd /usr/lib64/python3.7/site-packages/libdnf/_transaction.so | grep libdnf
libdnf.so.2 => /lib64/libdnf.so.2 (0x00007fdde789e000)
$ LD_LIBRARY_PATH=./libdnf-build/libdnf ldd /usr/lib64/python3.7/site-packages/libdnf/_transaction.so | grep libdnf
libdnf.so.2 => ./libdnf-build/libdnf/libdnf.so.2 (0x00007fa615048000)
```
Whereas with rpm-ostree:
```
$ ldd /usr/bin/rpm-ostree | grep libdnf
libdnf.so.2 => /usr/lib64/rpm-ostree/libdnf.so.2 (0x00007f7da5271000)
$ LD_LIBRARY_PATH=./libdnf-build/libdnf ldd /usr/bin/rpm-ostree | grep libdnf
libdnf.so.2 => /usr/lib64/rpm-ostree/libdnf.so.2 (0x00007fc905dbd000)
```
And going further down the rabbit hole, `DT_RPATH` is in fact considered
deprecated for this reason; it makes it harder to run with uninstalled
libraries or whatever reason one has to want to override a library (one
can still use `LD_PRELOAD`, though the semantics are different, and it's
less commonly used than `LD_LIBRARY_PATH`). Instead, we should use
`DT_RUNPATH`, which does have lower precedence than `LD_LIBRARY_PATH`,
matching the usual behaviour.
To make the linker emit `DT_RUNPATH` attributes, we have to use the
`--enable-new-dtags` flag. This also then fixes the g-ir-scanner issue
described above.
This brings us back in sync with the latest libdnf git master. This
required a bunch of work both on the libdnf and rpm-ostree side to get
working. See e.g.
https://github.com/rpm-software-management/libdnf/issues/645.
A few things to adapt to:
- soname bump to `libdnf.so.2`
- `DnfAdvisory` is no longer a `GObject` (annoyingly it's not replaced
by something we can keep a ref on, so this requires some hacks to
steal from the `GPtrArray` -- could enhance libdnf for this later)
- disable SWDB history writing
- use new reldep public API
- update for latest `hy_subject_get_best_selector()` API
This now unlocks the possibility to add support for modules. (One can
see hints of this in the diff by the fact that `libdnf` links to
`libmodulemd1`.)
Update submodule: libdnf
Closes: #1404
Approved by: cgwalters
As something that manages your base operating system, we care
about reliability, predictability, as well as performance and
low-level access to native operating system facilities. The
C programming language is great for the latter two, but fails
at providing a truly memory-safe environment. Rust is fairly
unique in providing a language that doesn't carry a runtime,
so we can gradually "oxidize" and convert our C code without
imposing additional overhead. It's also got a lot of modern
design niceties, like not having a null pointer.
Let's pull the trigger here and hard require Rust. It's the
programming language I personally want to be primarily writing in for
years to come.
This is also in line with a recent trend of reducing our
experimental/optional matrix.
Closes: #1509
Approved by: jlebon
This ends up being different than what landed in librsvg (that
was imported into ostree) because in rpm-ostree we've basically
been using `git archive`-based tarball generation rather than `make dist`
for a long time. And supporting `make dist` looks like it'd get into
handling the `libdnf` bits and walking into `cmake` land so...yeah
let's not do that.
The canonical sources are in git (recursively via submodule),
except for the Rust sources, which cargo can download dynamically,
and with this patch we support glomming all of that together
into a tarball.
(And turn off `make dist` so people understand how we do it)
Tested by `make -f Makefile.dist-packaging dist-snapshot`, then
copying the resulting tarball into a container with `--net=none`
and building there.
Closes: #1391
Approved by: jlebon
Automake was warning about duplicate `clean-local` definitions, let's
do the nonrecursive automake dance.
And while I'm here, let's add some Rust unit tests that actually run
on `make check` too, since the whole unit testing bits of Rust are
awesome.
(I also tweaked the propagate bits to use the nicer `is_null()` method)
Closes: #1389
Approved by: jlebon
Let's modernize and start supporting YAML treefiles. I'll dare make the
sweeping generalization that most people would prefer reading and
writing YAML over JSON.
This takes bits from coreos-assembler[1] that know how to serialize a
YAML file and spit it back out as a JSON and makes it into a shared lib
that we can link against. We could use this eventually for JSON inputs
as well to force a validation check before composing.
If we go this route, we could then turn on `--enable-rust` in FAHC for
now and drop the duplicate code in coreos-assembler.
[1] https://github.com/cgwalters/coreos-assemblerCloses: #1377
Approved by: cgwalters
Making this a separate commit, since it's a first for rpm-ostree. We now
have a conf file complete with man page! No options yet though.
Interestingly, there was a function called `rpmostreed_reload_config`
which was declared but never defined. Didn't look too much into that.
We make sure that the config is part of the things we reload when users
call `rpm-ostree reload`.
Closes: #1204
Approved by: cgwalters
We previously added a projectatomic/libdnf fork. I just pushed an
`rpmostree-branch` to there with one PR on top. This is an unfortunate turn but
the C++ thing still needs to be worked out.
Obviously we don't want a permanent fork, but I find it hard to justify spending
a whole lot of time pitching in to help with the libdnf C++ conversion versus
all the other work I could be doing.
Update submodule: libdnf
Closes: #1114
Approved by: jlebon
This matches reality; I picked what's in CentOS 7 mainline today. More
importantly this also fixes a build error with `-Werror=undef` because we had a
trailing underscore `_` at the end and never noticed.
Closes: #1156
Approved by: jlebon
In git worktrees, `.git` is not a directory but a regular file
containing the path to the real git directory. This was causing us to
not use `git describe` in the versioning info, which is really useful to
sanity check that you're running the code you think you're running. Fix
this by just checking that a `.git` entry exists.
Closes: #1102
Approved by: cgwalters
The actual problem I am trying to fix with this is fallout from the
introduction of `/usr/libexec/rpm-ostreed`, which required a SELinux
policy change. Specifically for CentOS, the base policy is rev'd
slowly.
My hope was that by merging the daemon code back into `/usr/bin/rpm-ostree`
which is labeled `install_exec_t`, starting via systemd would do
the right thing. It turns out that doesn't happen.
Now later, I'm picking this patch back up because I want to do multprocessing in
the daemon (and in the core), and it makes sense to share code between them,
because multiprocessing will need to go through a re-exec path.
Another benefit is we avoid duplicated text (libglnx, internal helpers) between
the two binaries.
Closes: #292
Approved by: jlebon
Many projects do this, and it really helps debugging to know the
exact hash.
(Of course this is broken in traditional rpm builds from a tarball,
and rpmdistro-gitoverlay injects it into the Version field,
but it will help me for vmcheck debugging)
Closes: #584
Approved by: jlebon
My development environment is now using "pet" docker containers.
I use VMs for testing things that require that (like rpm-ostree).
This patch builds on work from @jlebon in
https://github.com/projectatomic/rpm-ostree/pull/509
to rework `vmcheck` such that it can work on any `ssh-config`. By
default we expect this to be Vagrant.
However, I go a lot farther and delete the `vmbuild` code that was
trying to do builds in a container on the target VM. I think this is
still worth pursuing at some point, but for now I think it's
reasonable to assume that the rpm-ostree developer audience uses Linux
as their host workstation and hence has containers.
(There's another important point here in that for developing lower
level things like rpm-ostree, there's a strong push to make the VM
disposable and not a pet)
Closes: #516
Approved by: jlebon
Mostly because when building with CLang, I get a spam of warnings
like:
```
/home/walters/src/github/projectatomic/rpm-ostree/libdnf/libdnf/dnf-transaction.h:37:27: warning: redefinition of typedef 'DnfTransaction' is a C11 feature [-Wtypedef-redefinition]
G_DECLARE_DERIVABLE_TYPE (DnfTransaction, dnf_transaction, DNF, TRANSACTION, GObject)
```
Closes: #501
Approved by: jlebon
See https://github.com/projectatomic/rpm-ostree/issues/405
This patch adds an (off by default) `--enable-new-name` build option
which currently defaults to `nts`. This is purely additive, and
the intention is that we'll support the rpm-ostree name in
perpetuity most likely.
At the moment, we add a new name for:
- /usr/bin/$name
- The systemd unit file
But we notably *don't* attempt to add a new name to the DBus API,
as it'd be a lot more invasive of a patch, and less payoff (it's
mostly just programs/scripts that interact with the DBus).
Closes: #497
Approved by: jlebon
So I was trying to hack on my host's copy of rpm-ostree inside a pet
docker container, but ran into a conflict with libhif since dnf uses
it. I think we basically need to *always* build the bundled path,
rather than what I'm doing with CAHC and FADC where it's built as a
regular RPM.
It's not really sustainable right now for us to have both bundled and
not-bundled build paths - and we need to support co-installation with
dnf.
Another major issue is that we want to version lock with libhif -
right now our CI and both CAHC/FADC track libhif master, but that
means everything breaks if libhif breaks and we don't immediately
port.
git submodules solve all of these problems - the same as we're doing
with libglnx.
libglnx is *designed* for use as a git submodule, where as libhif
needs to support being both bundled and not-bundled. So we end up
with some hacks on our side, but I think it's all not too bad. I've
marked build rules with `# bundled libhif` so we know where to find
them later when libhif is stable.
Closes: #357
Approved by: jlebon
During autoreconf, automake would emit many warnings regarding the
option 'subdir-objects' being disabled. We squash those warnings by
enabling the option.
We also fix Makefile.am so that it includes the patched libglnx Makefile
rather than the original one, which would cause libglnx output to be
placed in the literal dir './$(libglnx_srcpath)'.
During autoreconf, we would get a warning due to privdatadir being
defined in both Makefile-rpm-ostree.am and Makefile.am. Remove the
instance in Makefile.am.
On the plus side, we share some code between the library and the
binary now. On the downside, because `librpmostreepriv.la` is a
noinst library, its code text is duplicated between the shared library
and binary, at least until we either:
- Have the binary solely use the public shared library (like ostree does)
- Install `librpmostreepriv.so` to e.g. `/usr/lib64/rpm-ostree/librpmostreepriv.so`
without the headers being public
This will help build release engineering and other types of tools;
for example, rather than parsing the output of `db diff`, one
should be able to call an API.
Initially, this adds the generic infrastructure for a public shared
library, with a new function call to do the equivalent of `rpm -qa` on
a particular OSTree commit.
Closes: https://github.com/projectatomic/rpm-ostree/issues/117
Closes: https://github.com/projectatomic/rpm-ostree/pull/124
The high level goal is to deprecate libgsystem. I was trying to share
code between ostree/rpm-ostree, but it was too painful to commit to
forver frozen ABI for new utility APIs.
The git submodule approach will much more easily allow breaking
API/ABI, and iterate on APIs until they either land in GLib or not.
Note that libglnx will not use GFile*, so a full port to it will
involve also not using that. Thus, it will be necessarily
incremental; in the meantime we'll link to both libgsystem and
libglnx.
This will move to a separate repository. This allows a clearer
separation between the core tool (which is shipped on client systems
too), and the compose infrastructure.
Furthermore, I want to make the autobuilder a Docker container.
