Commit Graph

988 Commits

Author SHA1 Message Date
Jonathan Lebon
52b325970d vmcheck: move vmbuild and vmoverlay to scripts
That way we can make use of libvm instead of relying on vagrant.

Closes: #394
Approved by: cgwalters
2016-07-15 19:14:19 +00:00
Jonathan Lebon
194c2bf5cd libvm.sh: add vm_setup and vm_rsync
In preparation for being vagrant independent, let's factor out some
things directly into the libvm.

Closes: #394
Approved by: cgwalters
2016-07-15 19:14:19 +00:00
Jonathan Lebon
8942268ecb test.sh: write test result to log for easier grepping
Closes: #394
Approved by: cgwalters
2016-07-15 19:14:19 +00:00
Jonathan Lebon
ab6fe314aa vmcheck/test.sh: support external ssh-config
Closes: #394
Approved by: cgwalters
2016-07-15 19:14:19 +00:00
Jonathan Lebon
ce2a434104 libtest.sh: output file in assert_file_has_content
This is the same patch as:
https://github.com/ostreedev/ostree/pull/264

Closes: #394
Approved by: cgwalters
2016-07-15 19:14:19 +00:00
Jonathan Lebon
2da585023d vmcheck: remove test-basic.sh
This is a useless test for now since we should be able to test on a VM
even if there are updates available.

I initially had it just as an example of the vmcheck test harness,
without actually thinking too much into what it was testing. But we do
need a nice test-basic.sh that will give rpm-ostree a good general
workout.

Closes: #394
Approved by: cgwalters
2016-07-15 19:14:19 +00:00
Colin Walters
cc2551dc06 Switch to glib cleanup macros
We don't have a lot of outstanding changes to the C code, so now seems
like a good time to do this.  I implemented this with some highly
sophisticated sed commands like:

```
find -name '*.c' | while read name; do sed -i -e 's,gs_unref_object \([A-Za-z]*\) \*,g_autoptr(\1),' ${name}; done
```

Part of dropping the dependency on libgsystem, same as what we're
doing in ostree.

Closes: #393
Approved by: jlebon
2016-07-15 13:08:58 +00:00
Colin Walters
7888b9d7db core: Require hardlinks for checkouts
Not fixing any known bug, just making use of the new option to ensure
checkout does what we want.

Closes: #388
Approved by: jlebon
2016-07-12 19:09:51 +00:00
Jonathan Lebon
7cebf962c1 docs: fix ostree and CONTRIBUTING.md links
- Update links to OSTree documentation.
- Fix CONTRIBUTING.md by making symlink in top dir.

Closes: #387
Approved by: cgwalters
2016-07-12 15:46:53 +00:00
Colin Walters
4abb3ebff7 build: Make bubblewrap path configurable
So that it's easier to build bubblewrap as `Source1` in an RPM
embedded (flatpak is using a git submodule, but for this I think
`Source1` is better).

Closes: #384
Approved by: jlebon
2016-07-11 19:20:09 +00:00
Colin Walters
e48ec22fc2 Release 2016.4
Closes: #383
Approved by: jlebon
2016-07-08 19:52:24 +00:00
Jonathan Lebon
8016319bcb postprocess: fix typo in error msg
Closes: #381
Approved by: cgwalters
2016-07-07 21:54:57 +00:00
Jonathan Lebon
4c392139b5 vagrant/Makefile: add some convenience targets
dockerwait: if we're running on a freshly booted VM, we want to make
sure that docker is running before trying to spawn the container.

check: runs the same target inside the container; this is useful for
coverage analysis, where we want the same binaries to run both the `make
check` and `make vmcheck` testsuites.

clean: for convenience.

Also make sure we pass CFLAGS to both the configure (to propagate to
libhif's cmake) and the make steps. This is also of use for coverage
analysis, where we want to compile with `--coverage`.

Closes: #378
Approved by: cgwalters
2016-07-07 11:55:09 +00:00
Colin Walters
1aa6e1d300 Adapt to const hif_package_get_nevra()
See https://github.com/rpm-software-management/libhif/pull/158

Closes: #366
Approved by: jlebon
2016-07-06 19:50:47 +00:00
Jonathan Lebon
f9ff54f4ab unpacker: make error prefix a proper prefix
Closes: #377
Approved by: cgwalters
2016-07-06 13:26:39 +00:00
Jonathan Lebon
cf07b45065 Makefile-hif.am: link by force and add clean hook
Add a clean hook so that `make clean` on the top dir also cleans the
libhif artifacts.

Add -f to `ln` -- Make marks the target as out-of-date if the symlink
breaks (which will happen if you just `make clean` in the libhif-build
dir), but the `ln` will cause the build to fail since it already exists.

Closes: #377
Approved by: cgwalters
2016-07-06 13:26:39 +00:00
Jonathan Lebon
4877887e49 vagrant: use sync after deploying
For some reason, the VM would sometimes take a very long time to reboot
after deploying the new commit. Adding a sync before rebooting fixes
this, though I'm not sure why the filesystem doesn't sync itself as fast
without it.

Closes: #371
Approved by: cgwalters
2016-07-05 15:26:45 +00:00
Jonathan Lebon
30881c3df0 Dockerfile.builder: update for building libhif
Now that libhif is a submodule, it means that we have to have the
buildreqs for it as well in the build img.

Closes: #371
Approved by: cgwalters
2016-07-05 15:26:45 +00:00
Jonathan Lebon
f8907b6d53 vmcheck: add more pkg layering tests
- Rename test-layering.sh to test-layering-basic.sh and make it test
  both pkg-add and pkg-remove.
- Add test-layering-relayer.sh, which verifies that pkgs are properly
  relayered during the creation of new deployments (e.g. upgrades,
  rebases, deploys).
- Add test-layering-rpmdb.sh, which verifies that packages respect the
  rpm requirements before being overlayed.

Closes: #371
Approved by: cgwalters
2016-07-05 15:26:45 +00:00
Jonathan Lebon
ba05972bbb vmcheck/test.sh: make more silent and add some bling
Print out a better summary that sums up the passes, failures, and skips.
Silence the restoring step at the end of the testsuite by only echo'ing
to the log file. Finally, add some colouring because colours are
awesome.

Closes: #371
Approved by: cgwalters
2016-07-05 15:26:45 +00:00
Jonathan Lebon
7611cb223d rebase: allow local refs
The `rpmostreed_refspec_parse_partial` method would only work with
refspecs that include a remote. This was causing rebases to fail when
specifying a local ref. However, it should be perfectly valid to do
this. Adapt the function.

Closes: #371
Approved by: cgwalters
2016-07-05 15:26:45 +00:00
Jonathan Lebon
77acf62274 daemon: make deploy work for local branches
We would unconditionally call ostree's pull() regardless of whether the
branch we were tracking was remote or not. We need this to work to make
it easier to write tests, though it's a worthy addition on its own.

Closes: #371
Approved by: cgwalters
2016-07-05 15:26:45 +00:00
Jonathan Lebon
ff5dee43fb configure.ac: exit if libhif's cmake fails
Closes: #371
Approved by: cgwalters
2016-07-05 15:26:45 +00:00
Jonathan Lebon
8c76381a97 daemon: only add base-checksum if pkgs layered
The origin might have an empty "requested" key in the "packages"
section, which is probably an indication that packages were once added
but were then removed.

Explicitly check the length of the pkg array in case it's actually
empty before adding the base-checksum to the deployment variant.

Closes: #371
Approved by: cgwalters
2016-07-05 15:26:45 +00:00
Gerard Braad
5fb4a04e85 Add links to customization resource
Closes: #375
Approved by: jlebon
2016-07-05 13:06:08 +00:00
Jonathan Lebon
0c4aaf25de docs: add section about package layering
Add some notes in the manual about package layering. Would be nice to
have `ostree admin unlock` also be part of the ostree manual for a
better contrast of the two.

Looking at the other snippets in the document, this makes me think we
probably should also add wrappers in `/usr/bin/atomic` for the new
commands. Or maybe we should wait until they're not in preview mode
anymore.

Closes: #374
Approved by: cgwalters
2016-07-05 00:41:33 +00:00
Jonathan Lebon
dba554a76b tests: use our self-built libhif library
Tests were failing because we were silently falling back to the system
libhif, which, as of PR #158, changed a string from non-const to const,
which meant we were double free-ing strings.

Closes: #373
Approved by: cgwalters
2016-07-04 22:18:06 +00:00
Colin Walters
3b08080052 core: Use hif_repo_download_packages()
Now that it's upstreamed.  Also use the new API to show the total size
of packages to download.

Closes: #363
Approved by: jlebon
2016-07-01 02:34:51 +00:00
Colin Walters
c68ea0a29a core: Add /var/lib/vagrant -> /usr/lib/vagrant
I'm thinking this list of things actually needs to live outside of the
source code and be more obvious (and we should file bugs to get the
packages fixed), but in the short term my demo today uses Vagrant and
it'd be nice if it was installable.

Closes: #364
Approved by: jlebon
2016-06-30 15:02:42 +00:00
Colin Walters
1489fc9bbe core: Do not attempt to upgrade (or remove) packages from base
I was hitting a strange segfault when trying to add a package,
and it ended up being that `krb5-libs` was in the transaction,
but its `rpmteKey()` was `NULL`.

It took me a while to realize that the reason this was happening is
`krb5-libs` was in the base, but there was a newer `krb5-workstation`
package wants a newer version.

We're going to encounter interesting issues with packages that have
hard version locking, where one half of the package is in the base and
the other half is layered.

It works for me to drop out `UPGRADE` etc. from the transaction.  In
this case, what will happen is libsolv seems to silently avoid
upgrading to the newer version of `krb5-workstation`.

In general, we're going to need `pkg-add` to be able to upgrade as
well at the same time, but that's for a later patch.

Closes: #362
Approved by: jlebon
2016-06-30 14:44:28 +00:00
Colin Walters
125c482b1d Switch to using libhif as a git submodule
So I was trying to hack on my host's copy of rpm-ostree inside a pet
docker container, but ran into a conflict with libhif since dnf uses
it.  I think we basically need to *always* build the bundled path,
rather than what I'm doing with CAHC and FADC where it's built as a
regular RPM.

It's not really sustainable right now for us to have both bundled and
not-bundled build paths - and we need to support co-installation with
dnf.

Another major issue is that we want to version lock with libhif -
right now our CI and both CAHC/FADC track libhif master, but that
means everything breaks if libhif breaks and we don't immediately
port.

git submodules solve all of these problems - the same as we're doing
with libglnx.

libglnx is *designed* for use as a git submodule, where as libhif
needs to support being both bundled and not-bundled.  So we end up
with some hacks on our side, but I think it's all not too bad.  I've
marked build rules with `# bundled libhif` so we know where to find
them later when libhif is stable.

Closes: #357
Approved by: jlebon
2016-06-30 14:27:55 +00:00
Jonathan Lebon
4f68f73b04 test-basic.sh: fix jq filter for new json output
The test-basic.sh test was failing on machines that had jq installed.
Will have to make sure to add it to the PR tester.

Closes: #360
Approved by: cgwalters
2016-06-29 19:17:38 +00:00
Jonathan Lebon
7697702aaa vmcheck: strengthen test harness
We now make the test harness handle restoring the VM to the original
state. The wonderful thing about ostree here is that it's a perfect
shoo-in for this. We make a 'backup' of the current ref, and just have
to make sure that the VM is back on that ref after running each test.

This will allow us to write tests without worrying as much about
cleaning up in the event of an error.

Closes: #360
Approved by: cgwalters
2016-06-29 19:17:38 +00:00
Jonathan Lebon
0974a22834 vagrant: use new 'booted' key to get current csum
Closes: #360
Approved by: cgwalters
2016-06-29 19:17:38 +00:00
Jonathan Lebon
4fc6dd8b97 hacking: add make vmoverlay for faster hacking
For the regular hack/build/test cycles, creating a new deployment each
time becomes overkill. We add a new `make vmoverlay` target which
directly overlays the newly built binaries on an unlocked system.

We also make rsync smarter so that we can afford to not `make clean`
everytime while still allowing the host's repo to be fully configured.

Closes: #360
Approved by: cgwalters
2016-06-29 19:17:38 +00:00
Jonathan Lebon
d718b820ef Makefile-tests.am: more cleanups and add pkg bar
The new package bar will be used to test conflicting packages.

Closes: #360
Approved by: cgwalters
2016-06-29 19:17:38 +00:00
Jonathan Lebon
65f5678ecb vmcheck: add timeout option for vm_ssh_wait
Closes: #360
Approved by: cgwalters
2016-06-29 19:17:38 +00:00
Jonathan Lebon
55e1cfeb99 buildimg: add new prereqs
Closes: #360
Approved by: cgwalters
2016-06-29 19:17:38 +00:00
Colin Walters
9d473df11e build: Remove --enable-usrbin-atomic
We haven't used this in a long time now that the atomic command
exists.

Closes: #361
Approved by: jlebon
2016-06-29 16:44:08 +00:00
Colin Walters
462645e71c core: Fix up more nevra leaks
Really do need to make it `const` but let's do that separately.

Closes: #359
Approved by: jlebon
2016-06-29 15:17:18 +00:00
Colin Walters
974d725f30 core: Store pkg directly rather than leaking nevras
I noticed that actually `hif_package_get_nevra()` mallocs, but a bunch
of this core code seems to assume it's const.

Actually, I suspect there's no good reason for it to strdup, so I'll
fix that too.

But in the meantime, we actually don't need to indirect through the
nevra, we can just store the `HifPackage` that we wanted anyways as
the key.

Now, the *real* reason I wrote this is right now rpm-ostreed is
segfaulting due to failing to look up the key, and I can't figure
out why...this doesn't fix it, but it makes the code better.

Closes: #359
Approved by: jlebon
2016-06-29 15:17:18 +00:00
Colin Walters
5ff77217f3 scripts: Ignore glibc-headers.prein and vagrant*.prein
The first is pure legacy, the second is an adduser that we should
eventually handle, but not critical right now.

Closes: #358
Approved by: jlebon
2016-06-29 13:53:42 +00:00
Colin Walters
5b6bf4576f main: Distinguish "preview" and "experimental" commands
Let's say package layering is "preview" - we show it in help, but
marked `(preview)`.

Whereas the `container` and `internals` builtins this patch calls
"experimental" - we don't show them in help even.

Both get a warning message when executed.

Closes: #354
Approved by: jlebon
2016-06-28 14:56:14 +00:00
Colin Walters
000b0675bd core: Always refresh cache, rather than never
I was trying to upgrade my desktop today and hit errors due
the fact I was apparently only upgrading the base, and not
layered packages.  (Due to a newer shared library in the base
needing to be version locked with a layered package).

It turns out we were keeping the cache forever, which is really quite
the opposite of what we want here.

I haven't looked if librepo is doing If-Modified-Since etc. updates or
not, but if it isn't we need to fix that.

Closes: #355
Approved by: jlebon
2016-06-28 14:38:57 +00:00
Jonathan Lebon
8be7d4968a daemon: don't try to resolve rev
During the initialization of the daemon, it would try to load the
deployments and cache a few things. One of these steps wanted to look up
the rev of the deployment. However, if the rev no longer exists (e.g. we
just did a rebase, purged the ref, and rebooted), this would error out.

Replace the lookup by simply directly using the csum from the
deployment.

Closes: #353
Approved by: cgwalters
2016-06-27 17:29:26 +00:00
Jonathan Lebon
b41e39045b sysroot-upgrader: remove csum override on rebase
Rebasing on a new branch should not in any way care about which commit
we're currently overridden on.

Also fix a typo which would cause override-commit keys to not be
deleted from the origin.

Closes: #353
Approved by: cgwalters
2016-06-27 17:29:26 +00:00
Gerard Braad
34c504055b Links for more information
Closes: #351
Approved by: cgwalters
2016-06-26 21:18:58 +00:00
Gerard Braad
350718ffd5 Correct location of document link
Closes: #352
Approved by: giuseppe
2016-06-26 12:07:47 +00:00
Jonathan Lebon
225be81c40 RPMOSTreeSysroot: add "booted" entry to deployment variant
Now that the `status` command learned a `--json` option, we can pretty
much avoid parsing human-readable output. The only piece of information
that is missing from the JSON output compared to the output for humans
is *which* deployment we're currently booted in.

This patch fixes that shortcoming by adding a "booted" boolean variant
to the deployment variant.

Closes: #350
Approved by: cgwalters
2016-06-25 12:23:04 +00:00
Colin Walters
f35f5cf468 core: Initial implementation of %posttrans using bwrap+rofiles-fuse
In order to make many things work, we need to run scripts.  Short version:
For now, we:

 - Run `%posttrans`
 - Treat most `%post` as the same as `%posttrans`
 - Ignore `%preun` and such since we never uninstall

Most importantly though, we start to build up an "override" list
for script handling.  Currently it's just a blacklist of scripts
we don't need.

Significant work here would be needed to run Lua scripts, so far I've
been able to just skip them.

Closes: #338
Approved by: jlebon
2016-06-24 16:08:08 +00:00