IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
For higher level tools driving rpm-ostree, the command line arguments
are mostly OK as inputs, but the addition of `--write-commitid-to`
shows that we really want structured data that these tools can parse.
JSON is a good enough interchange format, let's use that.
Most notably this does a pkgdiff in the JSON which several higher
level tools do too.
Closes: #1529
Approved by: jlebon
Prep for rojig-only compose support. We want to be able
to quickly detect whether or not there were changes in the
input, just like we do when targeting an OSTree repository.
Closes: #1528
Approved by: jlebon
I'm trying to have a more opinionated model where custom builds
use inheritance, and currently one can only have a single
`postprocess-script`.
Further, in YAML it's very convenient to use inline vs external
data.
Closes: #1527
Approved by: jlebon
It turns out we basically have to slap an `Option<T>` around
everything, (in particular `bool` etc.) we need to be able
to distinguish in (I believe) all the cases between
"value unspecified" and "value provided".
Concretely it didn't work to try to set `machineid-compat: false`
in an included yaml treefile becuase it was just defaulted to `true`
by the toplevel.
Down the line we should move all of the parsing into Rust
and have two different `struct` types for "YAML we load" versus
"verified treefile".
Closes: https://github.com/projectatomic/rpm-ostree/issues/1524Closes: #1525
Approved by: lucab
This is for: https://github.com/projectatomic/rpm-ostree/issues/718
But I'm not going to close that issue as this only does the server
side, and I think we should support it client side too.
Since I wrote that issue, we ended up skipping the `dnf_transaction_depsolve()`
API, and hence we don't need to block on a libdnf change. So
this was quite simple.
Closes: #1513
Approved by: jlebon
We should be expecting testpkg-1.1-1 here, not 1.0-1. This was passing
before because of the nondeterministic `find` output (fixed in the
previous commit) which could spit out the older rojig RPM.
Closes: #1491
Approved by: cgwalters
This test was relying on the order in which `find` reports matching path
names to find the right RPM. This was failing for me locally sometimes
because it matched the wrong RPM file. Fix this by just directly
referencing the full path name since we can.
Closes: #1491
Approved by: cgwalters
My fix to the testsuite in #1488 in which I made the `machineid-compat`
test part of `test-basic.sh` wasn't correct since the basic tests in
`libbasic-test.sh` also check that the default behaviour without the
`machineid-compat` option is to include it.
Let's just do this right and split out the `machineid-compat` test into
its own run.
Closes: #1491
Approved by: cgwalters
I noticed that the latest Fedora Atomic Host 28 and Silverblue did not
have an `OSTREE_VERSION` line in `/etc/os-release` even though both
specified `mutate-os-release` in their manifests. This turned out to be
due to the fact that `/usr/lib/os-release` is now a symlink to a
variant-specific file (e.g. `os-release-atomichost`), so we would
fallback to mutating `/usr/lib/os.release.d/os-release-fedora` instead.
Fix this by just taking the nuclear option of running `realpath` in the
rootfs directly. This is more maintainable than trying to keep up with
changes in variants/naming/etc. There's related discussions to this in
the original [PR](https://github.com/projectatomic/rpm-ostree/pull/410)
which introduced the feature re. resolving symlinks within the rootfs.
Closes: #1481
Approved by: jlebon
Rather than silently ignoring it.
In theory...we could write to /usr/lib/systemd instead of `/etc`
but eh...I feel like what we really want to do is make it convenient
to write a preset file from the YAML.
(We could have an `add-files` content that takes values literally
which would be nice in YAML and suck in JSON)
A general thread running through this is that for people making
*derivatives* of a CoreOS-like system, having to create their
own `exampleos-release` package is an annoying hurdle.
Anyways for now we're fixing the bug that we were silently ignoring
it.
Closes: #1488
Approved by: jlebon
The rojig spec is almost entirely rpm-ostree implementation details;
let's not have lots of people fork/duplicate it. Rather add the bits
of rojig to the treefile that people need to define (most notably
the name).
Prep for stabilizing rojig.
I had a few false starts with this PR; managing ownership/lifetimes
across C/Rust is just complicated. I got bit hard by the fact that
the workdir in `--unified-core` is really dfd-relative, and had to
do a dance to propagate the dfd into rust, as well as down into
the rojig builder.
Closes: #1484
Approved by: jlebon
Make logging work the same as it does for the vmcheck-STI work
(at some point I'll try to unify the 3 parallel+script implementions
we have). This fixes the problem that when the test times out,
the filename won't have `.txt` and S3 won't have the right MIME type.
Closes: #1479
Approved by: jlebon
In line with the recent trend of marking things stable, and in
preparation for stabilizing `rojig://` - Let's stabilize the `--unified-core`
option for `compose tree`.
I'm not sure we could make it the default anytime soon; today it trips
over bugs in the PAM package in RHEL7 for example. But it
works fine for Fedora, and I think the code/design are good enough to be stable.
Closes: #1465
Approved by: jlebon
Follow-up improvement after
https://github.com/openshift/os/pull/135
This should ensure it survives systemd's preset run on firstboot.
Although honestly...what we should *really* do is check whether
the `default.target` symlink target exists, and if not reset it
to `multi-user.target` so no one would have to care, but that'd
be conceptually separate from this, so I may do it later.
Closes: #1427
Approved by: jlebon
We actually want systemd's `ConditionFirstBoot` to fire. The
primary rationale here is that we're adopting Ignition for Fedora CoreOS,
and having `ConditionFirstBoot=` function will help a lot, as the idea
is it only runs once.
However, I discovered that this breaks the `units` directive for example,
as systemd blows away all the unit state in `/etc`. The correct thing
to do from the start is to use presets. We could add an implementation of
`units` which works with this on and instead writes a preset file but...eh.
My plan is to at some point introduce an "epoch" and flip various defaults,
this one, `tmp-is-dir`, the passwd file handling, etc.
See: https://github.com/dustymabe/bootengine/pull/11Closes: #1425
Approved by: jlebon
Let's modernize and start supporting YAML treefiles. I'll dare make the
sweeping generalization that most people would prefer reading and
writing YAML over JSON.
This takes bits from coreos-assembler[1] that know how to serialize a
YAML file and spit it back out as a JSON and makes it into a shared lib
that we can link against. We could use this eventually for JSON inputs
as well to force a validation check before composing.
If we go this route, we could then turn on `--enable-rust` in FAHC for
now and drop the duplicate code in coreos-assembler.
[1] https://github.com/cgwalters/coreos-assemblerCloses: #1377
Approved by: cgwalters
This renames the remaining C files, tests, etc. There are only
a few hits for `jigdo` left; changing them would be a format break,
so let's wait to do that until we need to.
Closes: #1279
Approved by: jlebon
We're continuing an incremental renaming process; previously we changed
the most user-visible strings. Now we're doing some internal variables,
and notably the cached refs and the origin files - the latter set is
things that end up on disk.
This leaves the biggest items; renaming APIs, files, and tests.
Closes: #1276
Approved by: jlebon
I noticed that Ubuntu also uses the original "jigdo", so let's start
pulling off the band-aid here and do a mass rename.
For this first pass I'm focusing on CLI entrypoints and docs, as that's what
people are going to see; renaming all of the internal C functions, structure
variables etc. can come later.
Closes: #1269
Approved by: jlebon
Changes in a server-side tree can cause the need for clients to import different
objects from packages. For example, turning on documentation. Another more
subtle case is where an object might "move" from package A to B by being deleted
from A - then the jigdo build process will pick the B version.
We need a "cache validation key"; a way for the server to tell the client that
the objects it should import from the package have changed. Initially I was
thinking of using the libostree "content hash" but that would be awkward as we'd
have to do an import on the server side too.
After more consideration I realized a simple *count* of the number of objects
actually works, because (as I note in a comment) changing a file in the tree
will result in it ending up in the jigdoRPM (and count as a deletion). And
obviously adding or removing objects changes the count too.
In fact we could have done this *without* breaking the format by just having the
client start recording the number of xattr entries, but this adds greater
flexibility down the line since we can in theory change how we do cache
invalidation if we *really* need to (but at the cost of triggering clients to
redownload packages).
Note the client logic got moved around as now we need to parse all the xattrs
before we decide what packages to download.
My test case here is turning on docs - I noticed this actually affects *every*
package which was surprising to me; I expected at least some packages wouldn't
have docs. I'll double check this.
It'd be good to have a "moving object" case too which I may look at.
Closes: https://github.com/projectatomic/rpm-ostree/issues/1197Closes: #1256
Approved by: jlebon
Since we changed things to have `jigdoSet = pkgSet`, we can just require exactly
`${npkgs}` here on import, which is what we found from `db list`.
Closes: #1256
Approved by: jlebon
The way we import packages in jigdo mode is different from package layering; we
may only import a subset of files for example. In general, we need to treat
jigdo differently.
Related: https://github.com/projectatomic/rpm-ostree/issues/1197Closes: #1238
Approved by: jlebon
I *think* this is why our tests started failing recently. It seems somehow very
unlikely to me though that we'd somehow managed to avoid `f` in the boot
checksums until now, but without doing some math...it seems plausible.
Closes: #1243
Approved by: jlebon
In #875 AKA b46fc35901 we
added support for the `releasever` option in treefiles. I am
pretty sure it worked at the time...but I didn't add tests.
Either it never worked or some refactoring broke it. The whole chain of
`GKeyFile` → `GVariant` is so confusing. Anyways fix it by copying the string.
Now let's use it by default in the compose tests, and while we're here bump
those to F27.
I'm doing this patch now as I was playing with doing a compose from
the `/usr/share/rpm-ostree/treefile.json` and wanted to use the stock
`.repo` files.
Closes: #1220
Approved by: jlebon
When I tried to use my WIP client patches to do:
`rpm-ostree rebase rojig://fahc:fedora-atomic-host`,
I got a missing file object which turned out to
be the client importing the i686 RPMs.
This was passing in the test suite because we don't mirror i686 of course, but
on the client side right now we end up using all enabled repos, and since Fedora
is multiarch, the behavior is going to be...not predictable.
Thinking a bit about on this problem I actually happened to recall
the RPM `%{_isa}` macro which is used in Fedora in various places;
for example to "arch bind" `-devel` packages to their base. See
for example [this case](33c7dc02bc/f/ostree.spec (_79)) in libostree.
As I noted at first, the core problem here is that the "final"
RPM architecture field is not symmetric in any way with the definition
of that `%{_isa}` macro. See:
d9d47e0114/installplatform (L25)
The *third* solution I ended up on here is to iterate over the
`Provides` on the server side and we take the first thing
that matches `Provides: %{name}(whatever)`.
I briefly thought about trying to somehow drive into libsolv the
logic to prefer the jigdoRPM's native architecture...IIRC yum did
something like that in the past but it was never done in libsolv?
Anyways the dependencies here are now more correct, so other tools
will also handle it.
Closes: #1213
Approved by: jlebon
Following up on `--ex-jigdo-output-rpm`, add support for writing the entire set
to an output directory. This is intended for use cases like FAHC, where we're
generating data outside of the upstream Fedora infrastructure. Further, we want
to support having our own history stored reliably, even if upstream prunes RPMs.
Now, this can be interesting even for upstreams like Fedora, as it naturally
captures just the subset of RPMs; doing full history support for that would
likely be a lot more palatable than for Everything.
Closes: #1165
Approved by: jlebon
Part of the goal of jigdo ♲📦 is to support organizations switching to *only*
providing RPMs. An intermediate step there is to "lock" the repo and jigdo
together; we don't want to update the ref if building the jigdoRPM fails.
Add an option to perform `rpm-ostree compose tree` and `rpm-ostree ex
commit2jigdo` together; notably we generate a commit, but only update the ref
once the jigdoRPM is built.
Closes: #1165
Approved by: jlebon
- Actually use separate `${test_tmpdir}` for test setup (closes a race)
- Merge stdout/stderr (more readable)
- Ensure logs are renamed to `.txt` even on failure
- Use `--progress` for some feedback
- Use `-j +1` so that even on unicore machines we get at least 2
jobs (and in general NCPUS+1)
Closes: #1188
Approved by: jlebon
Basically the `rpmostree_context_relabel()` call we had in the treecompose path
for unified core didn't actually have any effect as the core code did a relabel
and unset the array.
I think this may actually be a regression from: https://github.com/projectatomic/rpm-ostree/pull/1137
though I didn't verify.
Anyways looking at this, the code is a lot simpler if we change the API so that
the "normal" relabeling is folded into `rpmostree_context_assemble()`. Then we
change the public relabel API to be "force relabel" which we use in the unified
core 🌐 treecompose path.
This shrinks the jigdoRPM for FAH from 90MB to 68MB.
Closes: https://github.com/projectatomic/rpm-ostree/issues/1172Closes: #1173
Approved by: jlebon
This is used in critical paths like pungi, so let's be sure it works;
the semantics are a bit subtle as it overrides setting the ref.
Closes: #1161
Approved by: jlebon
The new `rpmdb.pkglist` metadata is a cheap way of retrieving the set of
packages in a commit. I'd like to make use of it as much as possible vs.
checking out the rpmdb and setting up a DnfSack.
Of course, in the case of layered commits, it doesn't matter *as* much,
because a layered commit being present in the repo should mean that a
deployment is currently using it, and we should learn to reuse the rpmdb
checkout of that deployment. Though keeping it consistent across both
server and client commits makes implementing `OstreeDeployment`-agnostic
things like `db diff` more efficient too. I also plan to use this in the
upcoming auto-update code.
Closes: #1158
Approved by: cgwalters
This fixes another thing broken with `compose --ex-unified-core`;
for e.g. `/usr/bin/ping` from `iputils`, the classic example of a filecaps
binary.
As I'm writing this commit message I realize it will actually also
take effect for package layering unnecessarily; we'll pointlessly
break the hardlink. But eh, it doesn't matter right now, we can
optimize that later.
Closes: #1151
Approved by: jlebon
Rather than entirely symlinking `systemctl` → `/bin/true`, in order
to e.g. have NetworkManager be enabled, we need to process presets.
This is one of the things that's breaking FAHC where I did a
`--ex-unified-core` deployment.
(Actually it's a bit tempting to run a mass preset pass at the end,
but for now let's do this)
Implementation note: this is our first use of GResources, which
is a handy way to embed data into our final binary.
Closes: https://github.com/projectatomic/rpm-ostree/issues/550Closes: #1148
Approved by: jlebon
This is prep for a rework of
https://github.com/projectatomic/rpm-ostree/pull/621
For a no-op `rpm-ostree upgrade` (i.e. no updates available), as long as
layering is enabled, we pay the cost of checking out the base tree, *mostly*
only to get the base rpmdb.
This is prep for fixing that down the line by knowing we always have the "base"
tree's rpmdb checked out. Then in the layering case we only modify
`/usr/share/rpm` (eventually that will point to `/usr/lib/sysimage/rpm`).
Teaching `rpmostree-core.c` about this can follow on later.
Closes: #1142
Approved by: jlebon
Having the "jigdo set" in repodata makes it so we can parallel download the
jigdo RPM with the set. However for now, I kept the jigdo set in the jigdoRPM,
since that way it'll be covered by the signature.
Also, this changes the way we inject metadata to use a magic comment string,
since trying to pass a gigantic macro to `rpmbuild` via its argv didn't work out
so well (it looks like rpmbuild eats newlines). This approach is more robust.
Closes: https://github.com/projectatomic/rpm-ostree/issues/1132Closes: #1140
Approved by: jlebon
With unified core 🌐 we are *relying* on the devino cache
for correctness when using `bare-user` repos. Otherwise lots
of bad things will happen as we won't hit the happy path from
[this libostree PR](https://github.com/ostreedev/ostree/pull/1297)
(I should probably add an assertion there that we aren't trying to commit
`user.ostreemeta`).
It looks like I had this working in some of the old unified core WIP patches,
but it was lost when rebasing 🏄.
We noticed this when I was trying to deploy jigdo in FAHC and the system
wouldn't boot as various things rely on those suid transitions.
Closes: #1139
Approved by: jlebon
We don't want to have to download all of `/usr/share/rpm` just to get
the list of packages used to compose the tree. This is fundamental
information that needs to be easier to discover. So let's stick it right
in the commit metadata. There's various use cases for this information,
including easily checking for and displaying updates and a pkglist-aware
version of `ostree log`.
Closes: #1134
Approved by: cgwalters
Let's "repo bind" the OIRPM by default; this makes the rpm-md repo feel a lot
more like an ostree remote, and IMO is just a really good idea in general to
increase predictabilty.
Closes: #1130
Approved by: jlebon
Tracking issue: https://github.com/projectatomic/rpm-ostree/issues/1081
To briefly recap: Let's experiment with doing ostree-in-RPM, basically the
"compose" process injects additional data (SELinux labels for example) in an
"ostree image" RPM, like `fedora-atomic-host-27.8-1.x86_64.rpm`. That "ostree
image" RPM will contain the OSTree commit+metadata, and tell us what RPMs we
need need to download. For updates, like `yum update` we only download changed
RPMs, plus the new "oirpm". But SELinux labeling, depsolving, etc. are still
done server side, and we still have a reliable OSTree commit checksum.
This is a lot like [Jigdo](http://atterer.org/jigdo/)
Here we fully demonstrate the concept working end-to-end; we use the
"traditional" `compose tree` to commit a bunch of RPMs to an OSTree repo, which
has a checksum, version etc. Then the new `ex commit2jigdo` generates the
"oirpm". This is the "server side" operation. Next simulating the client side,
`jigdo2commit` takes the OIRPM and uses it and downloads the "jigdo set" RPMs,
fully regenerating *bit for bit* the final OSTree commit.
If you want to play with this, I'd take a look at the `test-jigdo.sh`; from
there you can find other useful bits like the example `fedora-atomic-host.spec`
file (though the canonical copy of this will likely land in the
[fedora-atomic](http://pagure.io/fedora-atomic) manifest git repo.
Closes: #1103
Approved by: jlebon
When we added the `--ex-unified-core` option our caching story got
very messy because the non-unified core caches RPMs, but unified
does ostree repo caching.
For jigdo, we want the RPMs. Fix this by mirroring the RPMs using
`--download-only` and pointing the tests consistently at that.
Closes: #1122
Approved by: jlebon
We removed this in review, but I rediscovered why I added it. We
fail the `g_assert (sepolicy_matches)` if we already had packages
done with the final label when we go to reuse the cache.
(Basically, if we use the cache multiple times it's hard to avoid
relabeling all the time which is unfortunate...gets back a bit to
a way to annotate pkgcache commits as supporting multiple policies)
Closes: #1109
Approved by: jlebon
