IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
I'd like to embed structured metadata about the originating git
repository. See [this example](https://pagure.io/fedora-atomic-host-continuous/c/142b12020d7efe18b56d039304efea102a210790?branch=master). However, I think what we really
want here is a *single* value which has subkeys.
One thing in the back of my mind too is...we could use this to
enhance our "change detection". Right now we checksum the sack,
treefile, and treecompose-post. But down the line, I'd
like to support more sophisticated postprocessing, where the
script might reference external files or the like.
In that case, we could stop checksumming the post script, and rely on whether or
not the git repo changed. (This would conversely mean we would do a build even
if e.g. the repo's `README.md` changed, but we can address that with a
post-assemble content check).
Anyways though, for now, this gets us the ability to more easily drop more
structured metadata in the commit, whether it's input git repos, tests that
passed, etc.
Note a trap that bit me here: since the metadata we write here is *host* endian,
but `ostree show --raw` byteswaps (it needs to since the core ostree variant
is always big endian), we get inverted numbers if the host is little.
I think we should probably canonicalize our metadata to big endian; this should
be pretty backwards compatible since I doubt anyone has been adding raw numbers
so far.
Closes: #676
Approved by: jlebon
Should have no effect right now, since there's actually
no way we expose at the moment to write anything other
than strings.
But it will be useful when we add a way to write arbitrary metadata; see
https://github.com/projectatomic/rpm-ostree/pull/676Closes: #679
Approved by: jlebon
Preparatory work for a future patch; by having a hash it's easier to handle
duplicates. Note I started trying to use decl-after-stmt but hit the fact we're
still using `goto out` here.
Closes: #679
Approved by: jlebon
I just noticed this while reading the code. Right now we're actually following
links (since `AT_SYMLINK_NOFOLLOW` isn't implemented), which could definitely
lead to bad things if e.g. one has a symlink pointing to a file which shouldn't
be executable or world-readable etc.
(Also I noticed our hardlink-breaking-only-if-regular logic is now broken;
I added a TODO)
Closes: #677
Approved by: jlebon
I still need to query whether Fedora has anything like this; maybe `/boot`? But
things are not likely to work right now if we do, so let's ignore it.
Closes: #677
Approved by: jlebon
This gives me now:
```
error: While applying overrides for pkg mock: fstatat: etc/mock/custom-1-aarch64.cfg: No such file or directory
```
Which is definitely more useful for debugging 💣.
Closes: #677
Approved by: jlebon
Make use of the new support for parallel execution. In the best case,
this should bring down the execution time to about a third.
Closes: #675
Approved by: cgwalters
Allow the `make vmcheck` target to take a HOSTS var, which is simply a
space-separated list of hosts on which we can run testsuites. Add a
multitest.py script that takes care of monitoring and scheduling the
tests onto the nodes.
The script itself is "dumb": we don't know how long each test can take,
so we can't do any smart/heuristic scheduling that could save more time.
Closes: #675
Approved by: cgwalters
1. Don't require an ssh-config
In the case of redhat-ci, the VMs are already fully configured for the
system (injected in the hosts file, host key accepted, etc...). So
there's no need to have an ssh-config there. In general, it should be
acceptable to run the vmcheck suite against a resolvable host without
having to create an ssh-config for it.
2. Make the host name configurable
Rather than hardcoding "vmcheck" as the hostname, allow overridding it
by specifying a VM env var directly. We also prepare the various scripts
to make use of the $VM variable whenever host-specific dirs/files are
created so that parallel runs won't step on each other.
Closes: #675
Approved by: cgwalters
Allow users to directly specify an RPM file on the command-line. The
"packages_added" array of the PkgChange() method can now contain
absolute paths to RPM files.
Grow the origin format to have a new "requested-local" key. This is
similar to the "requested" key, except that the packages are always
installed from cache. The "requested-local" array values also embed the
SHA-256 of the header we expect.
There is now a new "LocalPackages" line in the status. These packages
are a subset of the "packages" element (which are printed as
"LayeredPackages") and represent the packages that are explicitly marked
for installing from cache.
Interesting design choices/notes:
- Just as before, even with foo-1.0-1.x86_64 installed from RPM, a
user can still request "/usr/bin/foo": it will be made dormant. As
soon as foo stops being explicitly layered from the RPM, it will try
to fulfill the request by going to the repos. This allows users to
"pin" a layered package to a certain RPM, and then unpin it.
- The strings/NEVRAs in "requested" and "requested-local" are strictly
distinct. This allows us to be able to tell what the user means
exactly when they do "rpm-ostree uninstall".
Closes: #657
Approved by: cgwalters
We start by adding support in the core for installing packages strictly
from the cache repo. We fool the libdnf stack by re-exporting the header
as an RPM, and explicitly marking it for install. The treefile format
supports specifying the expected SHA-256 of the metadata header, in case
the cache for a specific NEVRA changed.
Closes: #657
Approved by: cgwalters
This will fix rpm-ostree-in-mock-in-koji. The drawback is minor: post scripts
will have network access. But we're going to be testing the no-network case in
our Docker-based builds, so that's fine.
Closes: #672
Approved by: jlebon
Pull #646 introduced a subtle regression: we went from always including
a "packages" entry to only including it if there are packages present.
Albeit it's easy to guard against, though to be nice, let's make it
easier for consumers by always including it.
Reported-by: Micah Abbott <miabbott@redhat.com>
Closes: #670
Approved by: cgwalters
We'll keep a copy here for now, though the canonical
version should be viewed as the Fedora dist-git.
Updated-by: Colin Walters <walters@verbum.org>
Closes: #667
Approved by: jlebon
Down the line we should have a saner stringification, and also
log things like the invoking `loginuid` etc.
But this gets us something.
Closes: #660
Approved by: jlebon
Preparation for idle exit - the daemon knows when it should idle exit. It's
tempting to move all of the mainloop logic there, but it'd be more code churn
than I want to do right now.
Closes: #660
Approved by: jlebon
This is part of the idle exit work - the higher level daemon logic needs to
track running transaction state, so we can combine that with whether we have `> 0` clients.
(Yes, we need a better stringification of txns)
Closes: #660
Approved by: jlebon
Now `systemctl status rpm-ostreed` will show it. In future patches I plan to add
whether we have an active txn and our pending idle exit timeout.
Closes: #660
Approved by: jlebon
This is like what bluez does. With this, we have a stronger mechanism to avoid
races with future work to auto-exit on idle. Registered clients
hold a reference to the daemon effectively.
Note that calling `UnregisterClient` is optional if the calling process is going
to exit soon - as is the case for using the command line binary via e.g.
`rpm-ostree status`.
Closes: #660
Approved by: jlebon
More info here:
- https://lists.freedesktop.org/archives/dbus/2015-May/016671.html
- https://github.com/cgwalters/test-exit-on-idle
This is preparatory work for doing exit-on-idle, which we want to do because we
currently leak memory badly. Even if we didn't leak, it's still better to not
have services sit around resident.
While I had the patient open, I did some code cleanup such as switching to
direct `g_main_context_iteration()` and using `sd_journal_print()` instead of
`g_warning()` etc.
Closes: #660
Approved by: jlebon
The equivalent of https://github.com/ostreedev/ostree/pull/718
but for this codebase.
I just picked one example at random, there's plenty of others, but I don't want
to do any kind of tree-wide conversion since we have lots of outstanding
patches.
Closes: #664
Approved by: jlebon
It's useful to see how many new objects were added and how much storage they
use. Just part of a general trend where I'd like to make a lot more extensive
use of the journal, particularly with structured logging.
Closes: #661
Approved by: jlebon
This should help us if we ever find corrupt repodata; I thought
that might be happening in https://pagure.io/releng/issue/6602
Also, consolidate the disabled repos printing, since I think
we don't need to be verbose about those.
Closes: #613
Approved by: jlebon
This release includes a fix for [CVE-2017-2623](https://bugzilla.redhat.com/show_bug.cgi?id=1422157).
There are a few new features, such as `systemctl reload rpm-ostreed` now being
supported. Some bugfixes such as memory leak fixes. Besides that, there's a lot
of internal refactoring going on in preparation for work on local RPM
installation.
Closes: #663
Approved by: jlebon
These were leftovers from the very original code which did support exit-on-idle;
we're going to do it differently now.
Closes: #659
Approved by: jlebon
This won't actually help with much right now; just splitting out this change to
highlight it as part of future work for exit-on-idle.
Closes: #659
Approved by: jlebon
While reading a recent conversation about GPG checking at treecompose
time, I had a sudden thought - were we actually doing verification
client side? Turned out, we aren't. That happens as part of
`dnf_transaction_commit()` which we don't use.
That function verifies every package at one go, but for us I think it's better
to do it before "importing". We shouldn't have untrusted bits that we've
unpacked (they might have suid binaries, for one thing).
This is an embarassing problem, but it's worth emphasizing that everyone should
be retrieving repodata at a minimum over TLS, which sets a baseline. On RHEL, we
already do pinned TLS, and there are discussions about extending that elsewhere.
See: https://bugzilla.redhat.com/show_bug.cgi?id=1422157Closes: #656
Approved by: jlebon
In the case of client layering, we hackily use the treespec because
that's what the core understands (for now), but it really shouldn't be
part of the final commit, nor should we rely on it.
This patch starts the path towards moving us away from the treespec by
not embedding it in client layers, and instead directly inserting
layered packages under the "rpmostree.packages" key.
The SERVER_BASE case still embeds the treespec, since only the container
path uses it for now and it needs it.
Down the line, we'll want to make the treespec just one of the methods
by which we initialize RpmOstreeContext. But nothing stops us from
hiding that detail already.
Closes: #650
Approved by: cgwalters
We don't want to expose the host's `/tmp` since that means scripts could
potentially find things like the X11 socket or whatever.
To debug things better, add a quick bash script to run bwrap like the C code
does. Perhaps down the line we can add `rpm-ostree internals run-bwrap` or so.
Closes: #647
Approved by: jlebon
This is the culmination of the work paved by the previous commits. We no
longer handle package set mutations in the upgrader, but rather let
clients modify the origin as needed for that. As such, the upgrader is
now completely memoryless. We no longer try to error out if a package is
already in the base and we no longer drop out packages previously
layered packages that are now in the base. *Which* packages actually get
layered is determined during deployment and not coded in the origin.
At an API level, the deployment variant returned (and thus what gets
printed by `status --json`) now has both "packages" and
"requested-packages". The former retains its original semantic: it's the
set of packages that are *actually* layered. The latter contains
everything from the origin, which includes the former set + dormant
packages.
Accordingly, the output of `rpm-ostree status` also now distinguishes
between "RequestedPackages" and "LayeredPackages". Extra: we now quote
provides that contain spaces.
Closes: #646
Approved by: cgwalters
Ideally, we want to parse out everything in the origin on init and then
cache the values to make it even easier to inspect it. We make this more
explicit by renaming the struct members to "cache_*" and also caching
the initramfs args. This doesn't actually save any allocations right
now (though it might in the future), but we do the same for everything
else, and it's nice to be consistent.
Closes: #646
Approved by: cgwalters
There are multiple reasons for this. The first is that with the
introduction of "dormant" packages, it becomes impossible from simply
looking at a deployment's origin whether it was layered or not. We
*could* make the upgrader turn around and insert back the "real" final
packages that were layered. But we're really trying to reinforce the
concept that the origin is mutable and represents an input for the
upgrader. Derived data shouldn't live there.
Whether a deployment is actually layered or not instead is determined by
the deployment commit itself. The embedded treespec records the exact
packages that were layered, which we will later use to enhance status
output.
Closes: #646
Approved by: cgwalters
There are two main issues right now; first, we don't pick up manual changes to
`.origin` files, which occurs when one needs to sed it to remove `unconfigured`
for example. Second, we need to reload changes to the remotes.
Closes: #598
Approved by: jlebon
Clean up the output of vmcheck a bit by printing out ssh debug details
and rpm-ostree status output to the log file. This will help make it
easier to quickly see which tests failed (as well as helping satisfy my
OCD for a nice output).
Closes: #645
Approved by: cgwalters
On the first call to vmsync/vmoverlay, we do an immediate vm_ssh_wait()
to check if we have a live VM. However, we don't necessarily have a
working rpm-ostree in there if we're hacking on stuff. It gets annoying
to wait for the timeout there.
Let's just work around that by instead only calling status if we got
there through vm_reboot_cmd(), which is most likely when we're most
interested in the status output anyway.
Closes: #645
Approved by: cgwalters
