Commit Graph

93 Commits

Author SHA1 Message Date
Colin Walters
4abb3ebff7 build: Make bubblewrap path configurable
So that it's easier to build bubblewrap as `Source1` in an RPM
embedded (flatpak is using a git submodule, but for this I think
`Source1` is better).

Closes: #384
Approved by: jlebon
2016-07-11 19:20:09 +00:00
Jonathan Lebon
8016319bcb postprocess: fix typo in error msg
Closes: #381
Approved by: cgwalters
2016-07-07 21:54:57 +00:00
Colin Walters
1aa6e1d300 Adapt to const hif_package_get_nevra()
See https://github.com/rpm-software-management/libhif/pull/158

Closes: #366
Approved by: jlebon
2016-07-06 19:50:47 +00:00
Jonathan Lebon
f9ff54f4ab unpacker: make error prefix a proper prefix
Closes: #377
Approved by: cgwalters
2016-07-06 13:26:39 +00:00
Colin Walters
3b08080052 core: Use hif_repo_download_packages()
Now that it's upstreamed.  Also use the new API to show the total size
of packages to download.

Closes: #363
Approved by: jlebon
2016-07-01 02:34:51 +00:00
Colin Walters
c68ea0a29a core: Add /var/lib/vagrant -> /usr/lib/vagrant
I'm thinking this list of things actually needs to live outside of the
source code and be more obvious (and we should file bugs to get the
packages fixed), but in the short term my demo today uses Vagrant and
it'd be nice if it was installable.

Closes: #364
Approved by: jlebon
2016-06-30 15:02:42 +00:00
Colin Walters
1489fc9bbe core: Do not attempt to upgrade (or remove) packages from base
I was hitting a strange segfault when trying to add a package,
and it ended up being that `krb5-libs` was in the transaction,
but its `rpmteKey()` was `NULL`.

It took me a while to realize that the reason this was happening is
`krb5-libs` was in the base, but there was a newer `krb5-workstation`
package wants a newer version.

We're going to encounter interesting issues with packages that have
hard version locking, where one half of the package is in the base and
the other half is layered.

It works for me to drop out `UPGRADE` etc. from the transaction.  In
this case, what will happen is libsolv seems to silently avoid
upgrading to the newer version of `krb5-workstation`.

In general, we're going to need `pkg-add` to be able to upgrade as
well at the same time, but that's for a later patch.

Closes: #362
Approved by: jlebon
2016-06-30 14:44:28 +00:00
Colin Walters
462645e71c core: Fix up more nevra leaks
Really do need to make it `const` but let's do that separately.

Closes: #359
Approved by: jlebon
2016-06-29 15:17:18 +00:00
Colin Walters
974d725f30 core: Store pkg directly rather than leaking nevras
I noticed that actually `hif_package_get_nevra()` mallocs, but a bunch
of this core code seems to assume it's const.

Actually, I suspect there's no good reason for it to strdup, so I'll
fix that too.

But in the meantime, we actually don't need to indirect through the
nevra, we can just store the `HifPackage` that we wanted anyways as
the key.

Now, the *real* reason I wrote this is right now rpm-ostreed is
segfaulting due to failing to look up the key, and I can't figure
out why...this doesn't fix it, but it makes the code better.

Closes: #359
Approved by: jlebon
2016-06-29 15:17:18 +00:00
Colin Walters
5ff77217f3 scripts: Ignore glibc-headers.prein and vagrant*.prein
The first is pure legacy, the second is an adduser that we should
eventually handle, but not critical right now.

Closes: #358
Approved by: jlebon
2016-06-29 13:53:42 +00:00
Colin Walters
000b0675bd core: Always refresh cache, rather than never
I was trying to upgrade my desktop today and hit errors due
the fact I was apparently only upgrading the base, and not
layered packages.  (Due to a newer shared library in the base
needing to be version locked with a layered package).

It turns out we were keeping the cache forever, which is really quite
the opposite of what we want here.

I haven't looked if librepo is doing If-Modified-Since etc. updates or
not, but if it isn't we need to fix that.

Closes: #355
Approved by: jlebon
2016-06-28 14:38:57 +00:00
Colin Walters
f35f5cf468 core: Initial implementation of %posttrans using bwrap+rofiles-fuse
In order to make many things work, we need to run scripts.  Short version:
For now, we:

 - Run `%posttrans`
 - Treat most `%post` as the same as `%posttrans`
 - Ignore `%preun` and such since we never uninstall

Most importantly though, we start to build up an "override" list
for script handling.  Currently it's just a blacklist of scripts
we don't need.

Significant work here would be needed to run Lua scripts, so far I've
been able to just skip them.

Closes: #338
Approved by: jlebon
2016-06-24 16:08:08 +00:00
Colin Walters
4738c6eb60 core: Remove a stray _percent_progress_end()
See https://github.com/projectatomic/rpm-ostree/pull/338

Closes: #348
Approved by: jlebon
2016-06-23 19:53:13 +00:00
Jonathan Lebon
2c7013e86f rpmostree-core.c: squash -Wunused-function
Closes: #336
Approved by: cgwalters
2016-06-20 19:57:37 +00:00
Colin Walters
6efc0453c9 unpacker: Clean up directory perms override
Link to the Fedora bug that introduced this, and also change things so
it's also used for the "compose" case because:

 - Again it doesn't add security
 - Tools that operate on "compose" repos have to work around this
   when doing checkouts, see e.g. https://lists.freedesktop.org/archives/xdg-app/2016-June/000241.html

Closes: #335
Approved by: jlebon
2016-06-20 19:13:34 +00:00
Colin Walters
3c77c36999 core: Make unprivileged case ignore ownership, add "_compose" context
I was in the process of trying to support `%post` scripts, and I
wanted to use `rpm-ostree container` for convenient and safe testing.
However the recent package layering changes broke it to error out
on perms like `filesystem`'s `root:mail` on `/var/mail`.

I decided to introduce a new `rpmostree_context_new_compose` which had
the current behavior, switch `compose tree` to use it, and then change
`_new_unprivileged` to *really* be unprivileged.  Specifically we
ignore file ownership (and fix dir owners) because we assume we'll be
operating with `bare-user` repos.

Closes: #327
Approved by: jlebon
2016-06-20 15:10:44 +00:00
Colin Walters
e53ccda425 core: Go back to not verifying %post
Otherwise `nxs upgrade` fails.  We'll do this better later; allow
people to use the current state.

Closes: #330
Approved by: jlebon
2016-06-17 20:21:56 +00:00
Jonathan Lebon
3073a42868 tmpfiles.d: rename and re-order
According to tmpfiles.d(5), files should follow the convention
<package>.conf or <package>-<part>.conf. So we rename
tmpfiles-ostree-integration.conf to rpm-ostree-0-integration.conf.

The 0 index is so that the autovar conf created by postprocess is
sourced *after* this one, so that `integration.conf` has higher
precedence if there are duplicate entries.

Closes: #325
Approved by: cgwalters
2016-06-15 20:41:45 +00:00
Colin Walters
a86826eff4 Add --noscripts concept for pkg-add/delete
Right now, while one can `pkg-add strace`, there are a lot of
packages with `%post`.  Since some current developers want
to use package layering as it is today, let's add the concept.

Even after we have a whitelist of scripts and have cleaned up Fedora
to use them, we'll still have potential issues with 3rd party RPMs
etc. for a long time, so allow people this out to stumble forward for
those and potentially run them by hand if necessary.

Closes: #311
Approved by: jlebon
2016-06-15 01:11:01 +00:00
Colin Walters
1a1e08fbf7 core: Verify no %posts for imported packages
This is bringing forward an old PR for libhif:
https://github.com/rpm-software-management/libhif/pull/39

Right now, we aren't running `%post` or any of the other variants.  A
lot of packages will work if we just ignore `%post`, others won't.
Let's be conservative until we start running them, and don't imply we
support things we don't yet.

Closes: #311
Approved by: jlebon
2016-06-15 01:11:01 +00:00
Colin Walters
0578e13de8 core: Set a useragent
See https://github.com/rpm-software-management/libhif/pull/140

I just noticed this while debugging HTTP requests.

Closes: #318
Approved by: giuseppe
2016-06-12 16:20:33 +00:00
Colin Walters
20e69a9692 status: Import systemd bits to use UTF-8 circle
It's slightly prettier, but this is just laying some
groundwork/precedent for importing more systemd code and using it for
our formatting.

Closes: #295
Approved by: jlebon
2016-06-09 19:33:19 +00:00
Jonathan Lebon
d99df468c7 libhif: always prefix include directives
This is in response to:
https://github.com/rpm-software-management/libhif/pull/138

Closes: #310
Approved by: cgwalters
2016-06-09 12:27:53 +00:00
Jonathan Lebon
6b07755d68 add RPMOSTREE_UNINSTALLED_PKGLIBDIR
Since we now run everything uninstalled, we can't expect the tmpfiles
conf file to be installed. We add an env var that will allow us to tell
rpm-ostree to look elsewhere. This is then used in test-compose.sh.

Closes: #304
Approved by: cgwalters
2016-06-06 21:14:07 +00:00
Colin Walters
3a93fc162f core: Checksum package checksums, not just NEVRA for change detection
GPG signing an RPM doesn't change its NEVRA, and we need to support
detecting the case when RPMs change from unsigned to signed (or vice
versa).

It's also quite common for local developers to rev RPMs without
bumping the release or whatever, so this will fix that too.

Closes: #291

Closes: #296
Approved by: jlebon
2016-06-02 18:40:32 +00:00
Colin Walters
93e4039f60 core: Dedup hardlink/tempfile code
I wanted to avoid yet another copy of the "generate tempfile name"
code, so moved it to libglnx:

  https://github.com/GNOME/libglnx/pull/14

This also closes the TODO about deduping the "break one hardlink" code
with the "break all links in one dir" code.  The core observation here
is that it's simpler to copy to a tempfile and rename over the
existing, rather than rename, create, unlink.

Closes: #293
Approved by: jlebon
2016-06-01 15:15:11 +00:00
Jonathan Lebon
ead1ecdd23 package layering: major rework
- Move the package layering logic away from pkg-add and into the
  upgrader
- Add pkg-delete
- Add dry-run option

Closes: #289
Approved by: cgwalters
2016-05-27 22:18:44 +00:00
Jonathan Lebon
a25cdde25c core: major rework
This patch prepares RpmOstreeContext for supporting package layering. A
relabel operation is added as well to support relabeling imported
packages if the sepolicy of the rootfs we're overlaying onto is
different from during import.

Closes: #289
Approved by: cgwalters
2016-05-27 22:18:44 +00:00
Jonathan Lebon
1ccc560c26 output: support printf type usage
Closes: #289
Approved by: cgwalters
2016-05-27 22:18:44 +00:00
Jonathan Lebon
ca162dece2 unpacker: major rework
- Delete unpack_to_dfd path
- Get rid of copynpaste stuff and use the newly reworked ostree
  libarchive API which now supports the callbacks we need

Closes: #289
Approved by: cgwalters
2016-05-27 22:18:44 +00:00
Jonathan Lebon
2a036cf8b4 package-layering fixups
This patch fixes up errors in the parent commit which added pkg-add.
Some of them are due to the rebase on top of the unprivileged
infrastructure.

- fix compile errors (due to libhif changes after rebase)
- delete duplicate prototype for rpmostree_sysroot_upgrader_deploy
- include allow-older in flags type
- fix change_upgrader_refspec to use g_strdup() (this was causing the
  wrong old refspec to be registered)
- in builtin-status.c, check for NULL before joining the packages array
- sysroot-upgrader: fix gtype function names
- roc_context_prepare_for_root(): delete unused param
- assemble_commit(): delete unused param
- RpmOstreeSysrootUpgraderFlags: fix docs
- Fix sysroot property name and add reboot opt

Closes: #289
Approved by: cgwalters
2016-05-27 22:18:44 +00:00
Colin Walters
beb026f701 pkg-add: New builtin to layer additional packages
This builds upon the earlier prototype in
https://github.com/cgwalters/atomic-pkglayer

The `.origin` file says for a replicated installation:

    [origin]
    refspec=local:rhel-atomic-host/7/x86_64/standard

If you then run `rpm-ostree pkg-add strace`, it will result in a new tree with:

    [origin]
    baserefspec=local:rhel-atomic-host/7/x86_64/standard

    [packages]
    requested=strace;

Work still remaining here is to teach `rpm-ostree status` and
`rpm-ostree upgrade` about this.

Closes: #289
Approved by: cgwalters
2016-05-27 22:18:44 +00:00
Jonathan Lebon
d9e8535bcf treecompose: fix crash when "remove-from-packages" used
This works around a potential issue with libsolv if we go down the
rpmostree_get_pkglist_for_root() path. Though rpm has been using the
/usr/share/rpm location (since the RpmOstreeContext set the _dbpath
macro), the /var/lib/rpm directory will still exist, but be empty.
libsolv gets confused because it sees the /var/lib/rpm dir and doesn't
even try the /usr/share/rpm location, and eventually dies when it tries
to load the data.

So we set the symlink now. This is also what we do on boot anyway for
compatibility reasons using tmpfiles.

This also means we don't have to do the /var/lib/rpm --> /usr/share/rpm
transition during the rootfs postprocess (but we still have to clean up
db and lock files).

Also get rid of the unused pkglist variable.

NB: I used the GFile & gs APIs to mesh with the surrounding code.

Closes: #290
Approved by: cgwalters
2016-05-27 17:06:08 +00:00
Colin Walters
2cab8216b4 core: Adapt to libhif API query change
We'll track git master.

Closes: #286
Approved by: jlebon
2016-05-19 17:44:34 +00:00
Colin Walters
cbadc9bdaa libpriv: Print repository name for each package too
I'm trying to debug why I'm getting an older version of `docker`,
and it's useful to see the repository name we're getting something
from.  Yum does this by default.

(Though we should probably consider column formatted output too
 on a tty)

Closes: #282
Approved by: jlebon
2016-05-11 16:07:50 +00:00
Jonathan Lebon
582650e9c7 daemon: add TaskBegin, TaskEnd, and PercentProgress
Closes: #260
Approved by: cgwalters
2016-05-11 15:03:38 +00:00
Colin Walters
b718e28d79 postprocess: Add --add ostree to dracut invocation
Right now the `ostree.rpm` package always configures dracut to inject
the ostree setup via a conf file.  But it's actually simpler and
cleaner to just have callers specify it explicitly.

https://bugzilla.redhat.com/show_bug.cgi?id=1331369

Closes: #276
Approved by: jlebon
2016-05-06 14:50:42 +00:00
Colin Walters
4fc25d74a3 compose: Introduce variable substitution for ${basearch}
This comes from the tradition of yum repo files.  It's significantly
easier for rpm-ostree users building multiple architectures if some
core templating for the treefiles is built in.

Otherwise, everyone needs to learn about a new wrapper tool for
rpm-ostree, and said tool would need to re-do the same "basearch"
evaluation that is already occuring inside libhif.

This commit also paves the way for introducing `${releasever}`
substitution.

NOTE: This depends on pending changes to libhif git.

Closes: #274
Approved by: jlebon
2016-04-28 15:38:53 +00:00
Colin Walters
e2e754e94c compose: Reuse previous commit lookup in passwd checking
A future commit is going to change our parsing of the "ref" member in
treefiles, so ensure we only load it once early on in compose-tree.
We already looked up the previous commit there, so just pass it down
rather than reloading the ref.

Closes: #274
Approved by: jlebon
2016-04-28 15:38:53 +00:00
Giuseppe Scrivano
94a2804b0a compose: support adding external files
This will allow to copy arbitrary files into the rootfs, specifying something like:

"add-files": [["service.template", "/exports/service.template"],
              ["config.json.template", "/exports/config.json.template"]]

It is quite useful when building a container image.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #253
Approved by: cgwalters
2016-04-27 20:21:12 +00:00
Colin Walters
63761e5127 core: Skip over local packages for downloads
Because `hif_source_get_location()` is actually a reference to the
upstream repo, we shouldn't create a cache directory there.

This is just a two line fix, but I changed some things so that we keep
track of the number of local packages, in order to make the "Need to
download" print accurate.

We still need to add them (confusingly) to the `packages_to_download`
because that's also used for the container path which imports them.

Closes: #255

Closes: #256
Approved by: jlebon
2016-03-31 17:56:19 +00:00
Colin Walters
3350c0e062 postprocess: Fix "compose tree" regression with /etc
The fact that we didn't notice this is a testament to the state of
testing =(

I think I want to change the container build path to also only
grab /usr and /etc rather than having the paths be different,
but for now - the quick fix.

Pull request: #243
Approved by: jlebon
2016-03-22 18:35:33 +00:00
Colin Walters
ed7fc02b04 postprocess: Ensure we error out if selinux is enabled, but no policy found
I was tracking another regression where we seem to have lost
`/usr/etc` contents which manifested as `Labeling with... (null)`
which was clearly wrong.

Now this change actually impacts the test suite - we now (again IMO
correctly) error out if `selinux: true`.  The `no-selinux-tag` test
no longer makes sense, so delete it.

We do need more "real" tests that use selinux on and off.

Pull request: #243
Approved by: jlebon
2016-03-22 18:35:33 +00:00
Jonathan Lebon
f8b87efc1f RpmOstreeContext: add set_repo()
The new context methods recently added for unprivileged infrastructure
would set the ostreerepo member when calling new_unprivileged(). Since
here we will be using the sysroot repo, we need the ability to set the
repo. Just add a new set function for now, maybe move it into
new_system() later.

Also, make sure we dereference the repo on finalize.

Pull request: #240
Approved by: cgwalters
2016-03-21 19:57:06 +00:00
Jonathan Lebon
224e4b31d8 RpmOstreeTreespec: allow missing ref element
There will be cases where we don't actually want to tag a ref on the
final assembled commit. This will be true for package layering.

Pull request: #240
Approved by: cgwalters
2016-03-21 19:57:06 +00:00
Jonathan Lebon
bf44928f40 RpmOstreeContext: remove unused params
Pull request: #240
Approved by: cgwalters
2016-03-21 19:57:06 +00:00
Jonathan Lebon
8594931c67 RpmOstreeTreespec: allow missing repos element
We allow the "repos" key to be missing, which just means that we let
libhif do its job, i.e. automatically use the repos in repodir that are
enabled.

We still do a check in rpmostree_context_setup() to make sure that at
least one repo is functional.

Pull request: #240
Approved by: cgwalters
2016-03-21 19:57:06 +00:00
Colin Walters
f46302c2ab unpacker: Fix a 32 bit format string error 2016-02-23 13:44:14 -05:00
Colin Walters
397fdd8d33 Use a dummy install root for unprivileged composes
Sadly, libhif keeps trying to auto-create it.  It'll need patching
there, and possibly in librpm.
2016-02-22 14:27:46 -05:00
Colin Walters
f577279fac libpriv: Unify /usr/local, /etc -> /usr/etc, rpmdb handling
Right now we're doing the /etc -> /usr/etc inside the RPM import, but
we might as well do the /usr/local bits in both.  Also, use
/usr/share/rpm by default for treecompose too so that is unified.

Other things like systemd unit files and kernel handling are only
going to be used for host side composes.
2016-02-22 14:27:46 -05:00