Commit Graph

56 Commits

Author SHA1 Message Date
rfairley
a20d10e3d5 libpriv/util: Add date field in auto-versioning
This adds an optional date field to the prefix
passed by automatic_version_prefix. An example of specifying
the field is as follows:

10.<date:%Y>

And the fields progress like:

10.2018.0
10.2018.1
10.2018.2
10.2019.0

The date format creates a new "current date" string using
valid date directives passed into g_date_time_format().

If there is a problem reading the given date format,
an error is given and the next version is returned as NULL.

If no <date:...> tag is detected in the auto version prefix,
the same behavior as before (appending .1 and incrementing) occurs.

This may be helpful to avoid writing glue code to auto-update
the version if a date string in the commit version is desired.
Otherwise, --add-metadata-string=version= is an alternative for
complete customization.

Fixes: #1712

Closes: #1721
Approved by: jlebon
2019-01-14 18:49:41 +00:00
Ruixin Bao
9920094cfa tests: add test for sorting + conversion of sysuser
As title, added tests to check both conversion + sorting for sysuser
entries, and verifies its correctness.

Closes: #1519
Approved by: cgwalters
2018-08-30 17:37:27 +00:00
Ruixin Bao
4a942acdce passwd-util: adds conversion of group entries to sysusers
Similar to the passwd conversion, this commit
implements conversion of group entries into sysuser struct entries

Tests are also included in the commit

Closes: #1519
Approved by: cgwalters
2018-08-30 17:37:27 +00:00
Ruixin Bao
b4d301836d test: add unit test for passwd to sysuser conversion
Add initial implementation for unit test to test
converting passwd entries -> sysusers

Closes: #1519
Approved by: cgwalters
2018-08-30 17:37:27 +00:00
Jonathan Lebon
4249dc8c63 libpriv: Directly parse NEVRAs, don't use branches
When I initially added support for local RPM layering (#657), I was
operating under the assumption that there is no reliable way to parse a
NEVRA back to its constituent elements. So we worked around this by
doing lookups in the pkgcache and comparing against branches converted
back to their NEVRAs, which was pretty hacky.

Later on, I learned that you *can* reliably parse a NEVRA back to its
elements because RPM forbids all fields from having a colon `:` and all
fields other than the package name to have a dash `-`. In fact, there is
a helper from `libdnf` to do exactly this.

This patch swaps out the old way of decomposing NEVRAs and converting
NEVRAs to their cache branch notation with a simpler more direct
approach using the helper function from `libdnf`.

Closes: #1395
Approved by: cgwalters
2018-06-08 20:51:30 +00:00
Colin Walters
9df2428a1a tests: Move upgrade/rebase tests from unit to vmcheck/test-upgrades
Now we stop running rpm-ostreed as non-root, which is going to be
a maintenance pain going forward.  If we do introduce non-VM based
tests I think we should look to doing in-container testing.

Closes: #1339
Approved by: cgwalters
2018-04-25 20:28:56 +00:00
Colin Walters
4c3d9cdbce tests: Split unit test-basic.sh into introspection + upgrade-rebase
Following up to: https://github.com/projectatomic/rpm-ostree/pull/1336

It makes sense to keep the library tests as unit tests (although
we should also support doing them installed).

The upgrade-rebase tests will move into vmcheck/ soon.

Closes: #1338
Approved by: jlebon
2018-04-17 13:08:24 +00:00
Colin Walters
4f42b94da7 tests: Migrate some basic unit tests that start daemon to vmcheck
The unit tests run an rpm-ostree daemon as non-root, which worked
surprisingly well for quite a while.  But it started failing when
working on a patch which adds caching that writes to `/var`.

Since we have the vmcheck system now, let's switch over to that.
This PR moves the random "basic" tests we'd accumulated like
one to verify `StateRoot` is only in `status --verbose`, but not
the tests for the `rebase` command etc.

Closes: #1336
Approved by: jlebon
2018-04-16 17:53:20 +00:00
Jonathan Lebon
5173f579ca tests/vmcheck: add coverage for --check and --preview
We had no coverage for `--check` and `--preview`. Now that they use the
more efficient auto-updates API, it should be fine to document/recommend
to users. Let's add some coverage in the existing auto-updates tests to
make sure we don't regress on them e.g. only working when auto-update
"check" mode is on.

Closes: #1268
Approved by: cgwalters
2018-03-05 22:42:26 +00:00
Colin Walters
b881f33ba7 Add support for rebase rojig://
This is an initial drop of support for:
`rpm-ostree rebase rojig://fahc:fedora-atomic-host`.  We also
then support `rpm-ostree upgrade` from that.

There's a lot that could be improved here; the test coverage is relatively
minimal. A blocking issue there is having a realistic jigdo setup, and that's
going to require changing how we do testing. For now, this means that if we want
to e.g. change the format we'll have to temporarily disable this test, get the
format change in, update FAHC, then re-enable the test.

Closes: #1166
Approved by: jlebon
2018-02-02 20:18:58 +00:00
Colin Walters
2f04450b74 tests/basic: set -x after setup
So it's easier to debug when things fail.

Closes: #1166
Approved by: jlebon
2018-02-02 20:18:58 +00:00
Colin Walters
2c270a6644 rebase: Add support for providing ostree:// prefix
This fixes an asymmetry we introduced when rendering using the `ostree://`
prefix; we now support parsing it as well.  We reject the `rojig://` prefix
explicitly for now.

On the implementation side...things are messy. My thought is that we do our best
to canonicalize at the entrypoints for both the client side as well as the
server side. However, for backwards compatibility we can't go all the way to
writing out `ostree://` to disk in the origin files.

I'm actually uncertain if Cockpit will deal with this...need to test that.

Closes: #1210
Approved by: jlebon
2018-01-17 14:19:39 +00:00
Jonathan Lebon
b3f19cdc4e libpriv/rpm-util: insert pkglist metadata sorted
To make the new pkglist metadata even more usable, let's insert it
sorted. This ensures that we can bsearch the GVariant on the
client-side.

Enhance the bsearch utility function we have to deal with duplicate key
names. Although this is not the case today in rpm-ostree-managed streams
I know, some packages are allowed to have multiple versions installed,
so let's make sure we handle that deterministically by always returning
the first (earliest) version of the package.

Closes: #1158
Approved by: cgwalters
2017-12-20 13:10:36 +00:00
Colin Walters
1c0e354571 importer: Rework API
Now that the importer *only* imports into OSTree repos, let's
clean up the API so that the `OstreeRepo` and `OstreeSePolicy`
are passed as constructor args.

Also rework things so there's only one constructor API that
steals the fd.

This is prep for adding another async import API.

Closes: #1124
Approved by: jlebon
2017-12-07 19:44:19 +00:00
Colin Walters
d99175f664 status: Prefix ostree refspecs with ostree://
In preparation for jigdo, which would be like `jigdo://`.

Closes: #1136
Approved by: jlebon
2017-12-07 13:48:08 +00:00
Colin Walters
9c004e1833 Change unpacking to use a single ostree txn
Right now each ostree txn incurs a `syncfs()`; see
https://github.com/ostreedev/ostree/issues/1184
And before this patch, we were doing a txn per package import.

We can really do better in libostree - we'll fix that, but in the short term
let's use a bigger txn for every package. However, the obvious change here of
simply hoisting up the txn is that on failure for imports, we'd discard all
downloaded packages.  We fix that by changing the auto txn API to have
a `commit_on_failure` boolean, and use it in cases where we're doing
imports.

This is prep work for jigdo, where we'll be using the import path all the time.
My bigger plan is to do multithreaded imports.

Closes: #1116
Approved by: jlebon
2017-11-29 16:48:19 +00:00
Colin Walters
366826132a libpriv: Rename "unpacker" to "importer", split off "unpacker-core"
In the jigdo path we don't actually want to import the OIRPM literally
into ostree.  I considered adding jigdo logic into `rpmostree-unpacker.c`
but it'd be a mess as the functionality is quite logically separate
from importing.

So split off an `unpacker-core.c` file which has the bare libarchive+RPM
helpers, and rename `RpmOstreeUnpacker` to `RpmOstreeImporter`.

Closes: #1110
Approved by: jlebon
2017-11-20 20:57:33 +00:00
Ruixin Bao
5ca9c731d7 tests/check: Add unit tests for kargs options
Added unit tests for rpm-ostree ex kargs --delete,
--append and --replace.

Also exposed two getter functions for kargs table
and array so people can retrieve information from
kargs.

Also includes a minor fix for a bug caught by the unit
test.

Closes: #1013
Approved by: cgwalters
2017-11-06 18:49:36 +00:00
Colin Walters
e066dc93c8 tests: Skip running introspection tests if built with ASAN
This is the rpm-ostree equivalent of
<47b4dd1b38>

Unfortunately, introspection uses `dlopen(), which doesn't quite
work when the DSO is compiled with ASAN but the outer executable
isn't.

Prep for syncing PAPR config with ostree.

Closes: #1000
Approved by: jlebon
2017-09-24 13:32:00 +00:00
Jonathan Lebon
c08ca8f922 daemon/upgrader: make use of override-commit-ids
Currently, when setting the `override-commit` key in the origin, the
upgrader pulls that commit checksum directly and then updates the
refspec to point to it. This behaviour was inherited from its ostree
version; at the time it was implemented, the pull code didn't support
passing a specific commit for a given refspec. However, we now have
the override-commit-ids option, which will make libostree update the ref
for us.

We change the code here to make use of it and simplify the function.
This also fixes the corner case of local branches: we shouldn't change
the ref if we're on a local branch. This is actually what drove me to
this patch as I was debugging #981.

(Aside: I'm still not convinced updating the refspec is always the
correct thing to do even in the remote case, though it's a bit messy to
disentangle).

Closes: #984
Approved by: cgwalters
2017-09-12 17:23:25 +00:00
Colin Walters
8bb604ce67 tree-wide: Port to ostree_repo_{open,create}_at()
A lot of code gets nicer.

Closes: #922
Approved by: jlebon
2017-08-17 15:28:14 +00:00
Colin Walters
787c880b64 bin/rebase: Add -b and -m options
The rebase command syntax has confused people a lot.  Let's follow
git here and add a `-b/--branch` option and encourage people to use
that.  The case of switching remotes is `-m/--remote`; it's definitely
unfortunate that `-r` is already taken for `--reboot`.

One thing I'm a little bit unhappy about is how we're doing logic
on the client side here.  Changing the DBus API for this would
also be awkward though.

Closes: https://github.com/projectatomic/rpm-ostree/issues/886

Closes: #890
Approved by: jlebon
2017-08-08 16:02:15 +00:00
Colin Walters
34b5a004a8 daemon: Add a sanitycheck(/bin/true) before we deploy a tree
This is a followup to https://github.com/projectatomic/rpm-ostree/pull/888
but more comprehensive; in the layering case, the sanitycheck runs
after all the `%posttrans` scripts, so we'll get a consistent error message
for the `rm -rf /` test.

We also do the sanitycheck for the "pure ostree" case, as well as cases
where we didn't actually layer packages (including `ex override remove` as
well as simply regenerating an initrd).

There's obviously a lot more we could do in a sanitycheck; as I say in the
comment it's tempting to consider trying to boot systemd (in a fully volatile
config), but for now let's do this. In the end of course the admin has rollback
too.

Closes: #892
Approved by: jlebon
2017-07-27 17:58:58 +00:00
Colin Walters
2082b3f8fb lib: Add version macros and version checking function
The version checking function in particular is really useful for people doing
`from gi.repository import RpmOstree`, which we'd like at least some things like
Anaconda and Pungi to do.

Closes: #891
Approved by: jlebon
2017-07-21 20:35:26 +00:00
Colin Walters
fee6d06bf4 lib: Expose new API around basearch
For https://pagure.io/atomic-wg/issue/299 we need to make it
more convenient to substitute the architecture in an installation
context.  I plan to use this API inside `rpmostreepayload` in Anaconda,
so we can substitute the same value of `${basearch}` we use in treefiles
since https://github.com/projectatomic/rpm-ostree/pull/305

Now, you might wonder - why do we need an API wrapping libdnf? It's because
libdnf is not API stable yet. We're just exposing a tiny subset. In theory we
could use the Python dnf bindings in Anaconda, but things get slightly weird if
rpmostreepayload depends on dnf. Perhaps we'll do that down the road, but for
now this a small API surface to maintain (forever).

This change reworks the internal `varsubst` bits to take a pure `DnfContext`,
since we don't want to spin up a whole `RpmOstreeContext` just to do some
string substitutions.

Closes: #877
Approved by: jlebon
2017-07-21 16:02:41 +00:00
Jonathan Lebon
2b547c2d0a tests: transition to build_rpm
Nuke all the previous goop that was used to create RPMs at `make check`
time and transition all the tests to use the new `build_rpm` function.
It definitely feels cleaner to use. It's also really nice to have the
spec live in the same file as the test that uses it.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
47cc5f60ed tests/check: remove test-compose.sh
Nowadays, we have a much more in-depth and realistic compose test suite
that obsoletes this test.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
4c831e619c check/test-utils.c: assert no error first
Minor tweak; use g_assert_no_error() before using g_assert(ret) so that
we actually get a printout of the error the test fails.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
853737f667 unpacker: encode NEVRA info in commit metadata
It seems silly that to find out more detailed information about the
NEVRA of a cached pkg, we have to resort to write out the header to
disk, then reading it back in with librpm in order to tease out the info
we want. Let's just encode that information directly in the commit
metadata and provide a helper to fetch it.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:22 +00:00
Jonathan Lebon
44bcb9d60a tests: merge cache_branch_to_nevra into test-utils
I didn't realize at the time I wrote the cache_branch_to_nevra test that
the already existing test-utils.c would be the perfect place to add this
test. Merge the two together now.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:21 +00:00
Jonathan Lebon
752368eb5b rebase: allow rebasing to a local branch
This is a follow-up to commit 77acf62. There, we added support for
rebasing from a local branch to another local branch. But in testing,
it's also really useful to be able to rebase from a remote-based refspec
to a local branch. We allow this here by slightly expanding the syntax
of allowed refspecs.

Now, we can use rpm-ostree all the time rather than fallback to `ostree
admin deploy`, which isn't pkg-aware.

Closes: #764
Approved by: cgwalters
2017-05-05 21:01:26 +00:00
Colin Walters
e4b5e77e64 tests: Add unit tests for varsubst
Trying to debug https://bugzilla.redhat.com/show_bug.cgi?id=1381357

I don't see the problem yet, but let's exercise the code a bit.

Closes: #755
Approved by: jlebon
2017-04-27 18:57:10 +00:00
Colin Walters
a180a35a48 status: Do show Commit with --verbose, don't show StateRoot without it
`Commit` is useful in cases where one wants to e.g. run `rpm-ostree db diff`.
`StateRoot` however is going to be in the vast majority of cases identical
across deployments.

Closes: #743
Approved by: jlebon
2017-04-21 13:31:21 +00:00
Colin Walters
378216b5a1 tests/ucontainer: Also disable xattrs here
This actually only worked before because of
https://github.com/ostreedev/ostree/pull/775

Closes: #723
Approved by: jlebon
2017-04-04 17:48:26 +00:00
Colin Walters
3e833659b7 daemon: Add a separate OSExperimental interface
Just like `rpm-ostree ex`, for things like `ex livefs` that have DBus
interfaces, we should segregate these off so that people know they're unstable.
And conversely that they can test for the presence of the method on the main
interface for stability.

I initially tried having the same `RpmostreeOS` object implement both
but couldn't work out how to do that; see https://mail.gnome.org/archives/gtk-app-devel-list/2017-March/msg00161.html

Closes: #701
Approved by: jlebon
2017-03-23 19:24:41 +00:00
Colin Walters
710ea30612 main: Move "rpm-ostree container" → "rpm-ostree ex container"
Now all of our "experimental" bits are under one roof, so we get consistent
handling for them.

Closes: #683
Approved by: jlebon
2017-03-16 19:57:14 +00:00
Colin Walters
62a93c7850 postprocess: Handle f26 /etc/nsswitch.conf configuration
F26 put sss first, which broke our regexp. When we switch to sysusers, man it'll
be nice to dump ♲ this.

Closes: https://github.com/projectatomic/rpm-ostree/issues/685

Closes: #686
Approved by: jlebon
2017-03-16 18:39:38 +00:00
Jonathan Lebon
5c5ebc1b0a core: add support for installing from cache
We start by adding support in the core for installing packages strictly
from the cache repo. We fool the libdnf stack by re-exporting the header
as an RPM, and explicitly marking it for install. The treefile format
supports specifying the expected SHA-256 of the metadata header, in case
the cache for a specific NEVRA changed.

Closes: #657
Approved by: cgwalters
2017-03-10 18:35:11 +00:00
Colin Walters
14a9135ea0 main: Output YAML for --version
Equivalent of https://github.com/ostreedev/ostree/pull/691
Basically, it's nicer to parse.

Closes: #665
Approved by: jlebon
2017-03-09 14:23:34 +00:00
Colin Walters
b034381506 daemon: Implement "reload"
There are two main issues right now; first, we don't pick up manual changes to
`.origin` files, which occurs when one needs to sed it to remove `unconfigured`
for example. Second, we need to reload changes to the remotes.

Closes: #598
Approved by: jlebon
2017-02-24 21:36:45 +00:00
Colin Walters
775c7819b7 daemon: Avoid erroring out on startup/status with origin unconfigured-state
As part of an earlier cleanup of origin parsing, we started checking
the origin `unconfigured-state` even just starting the daemon, which
is kind of bad.

It's tempting to flip the default for the parser so that we *only* check
unconfigured state if we go to upgrade, but let's not do that in this patch.

Closes: #626
Approved by: jlebon
2017-02-14 14:00:09 +00:00
Jonathan Lebon
283b915ecf check: use jq for asserting json elements
Closes: #609
Approved by: cgwalters
2017-02-08 17:11:06 +00:00
Jonathan Lebon
cdac757434 daemon: delete overrides if not doing a deploy
We seem to have lost this little guy during the latest unification work.

Closes: #608
Approved by: cgwalters
2017-02-07 22:43:32 +00:00
Colin Walters
f02130b19e tests: Hard require jq
I don't see a reason not to require more stuff to execute tests,
and having it be optional reduces our coverage unnecessarily.

Closes: #580
Approved by: jlebon
2017-01-18 20:24:31 +00:00
Colin Walters
99bca8ad3f Suppress uid == 0 check in unit tests
Our `make check` runs an unprivileged rpm-ostreed if run as non-root; this is a
feature. We didn't notice in the CI tests since those run as "docker-uid0". It
does break my local workflow though.

Closes: #573
Approved by: jlebon
2017-01-16 19:03:02 +00:00
Jonathan Lebon
b68209b6d4 daemon: start with one commit only when resolving versions
During a deploy operation, we would fetch commit objects from the remote
to resolve the version string. If gpg-verify was turned on, we would
fail to pull them if some of the commits were not signed. This is
because we pulled them in batches. We partially address this by only
fetching the HEAD commit on the first pass. This allows `upgrade`
operations to work just as well as `deploy` operations.

Though there is still an issue if we have to traverse farther back than
when signed commits become unsigned (unless they happen to fall on a
batch boundary). We leave that unsolved for now, since that would likely
require a more complex solution and it's not clear whether it's a real
world issue (signers can just retroactively sign commits).

Copy the gpghome from ostree so that we can test GPG-related cases in
our suite.

Closes: #527

Closes: #557
Approved by: cgwalters
2016-12-24 12:28:48 +00:00
Jonathan Lebon
e10c97007f rebase: add support for rebasing to a specific rev
Expand the available options in the Rebase() D-Bus method to also have a
"revision" key. Its value has the same semantics as the "revision" key
in the Deploy() method (e.g. the "revision=" and "version=" prefixes are
also supported). Also expand the rebase CLI to allow for specifying the
revision as an additional argument.

This allows users to rebase to a specific version or checksum, rather
than only to the latest. Conceptually, this is the equivalent of doing a
rebase followed by a deploy. I.e. we specify an override-commit in the
origin and expect the same behaviours that apply after a deploy to also
apply here.

Closes: #212

Closes: #555
Approved by: cgwalters
2016-12-21 20:00:43 +00:00
Jonathan Lebon
2d07772aa3 test-basic.sh: add test for bad out-of-branch deploy
Closes: #495
Approved by: cgwalters
2016-10-20 14:06:58 +00:00
Colin Walters
cc2551dc06 Switch to glib cleanup macros
We don't have a lot of outstanding changes to the C code, so now seems
like a good time to do this.  I implemented this with some highly
sophisticated sed commands like:

```
find -name '*.c' | while read name; do sed -i -e 's,gs_unref_object \([A-Za-z]*\) \*,g_autoptr(\1),' ${name}; done
```

Part of dropping the dependency on libgsystem, same as what we're
doing in ostree.

Closes: #393
Approved by: jlebon
2016-07-15 13:08:58 +00:00
Jonathan Lebon
4f68f73b04 test-basic.sh: fix jq filter for new json output
The test-basic.sh test was failing on machines that had jq installed.
Will have to make sure to add it to the PR tester.

Closes: #360
Approved by: cgwalters
2016-06-29 19:17:38 +00:00