Commit Graph

1742 Commits

Author SHA1 Message Date
Colin Walters
0515f59d70 bin/compose: Use autocleanups for context structure
Prep for more code cleanup here and porting to new style.  We
had a mix of owner pointers.  Fix this by consistently having
the `self` struct own the data.

Closes: #903
Approved by: jlebon
2017-08-02 17:41:43 +00:00
Colin Walters
08e3f5de7f lib: Log task output for pre/post scripts
Some of the scripts can be expensive (`shared-mime-info` for example), and we
appear to be hanging for at least a few seconds even on fast hardware. I'd like
to have a lot more logging here...potentially something like showing
individual package names live in the terminal, but this is a start.

Closes: #902
Approved by: jlebon
2017-08-02 17:16:28 +00:00
Ruixin
53c39632d4 Deployment: add endoflife
When commit metadata contains ostree.endoflife attribute,
its information will be added to the deployment Variant,
which will later be shown as a red & bold message when
'rpm-ostree status' command is called.

A test is added for future regression

Closes: #889
Approved by: cgwalters
2017-08-02 13:53:10 +00:00
Colin Walters
0d4d6be94f Implement file triggers (%transfiletriggerin) for layered pkgs
File triggers are a post-RHEL7 thing; more information at
http://rpm.org/user_doc/file_triggers.html

There are two notable users I've been testing this with;
`glib2` and `vagrant`.  The `vagrant` one is more immediately urgent,
since it makes `vagrant-libvirt` work, which I currently rely on
for my workstation dev.

I've tested things successfully with `vagrant`, and I did verify that we run the
`glib2` ones when doing `rpm-ostree ex container`.

Long term, more transaction file triggers are likely to live in
"base" packages like `glib2`.  We don't implement those yet, but
extending this to do that shouldn't be too hard.

There was *significant* what I'd call reverse engineering of the
implementation in librpm.  The file triggers code there is spread out
and abstracted in a few different places in the code.  I found
trying to understand what header values were involved to be quite
tricky.

There are some corner cases like multiple patterns that I *think*
this does correctly, but could use more validation.  The main
question I had was - is it required that the patterns for e.g.
`%transfiletriggerin` and `%transfiletriggerun` be identical?

Closes: https://github.com/projectatomic/rpm-ostree/issues/648

Closes: #869
Approved by: jlebon
2017-07-27 20:58:09 +00:00
Jonathan Lebon
d4effe8f47 dbus: don't register if not active
Follow-up tweak to #894. Make the client smarter so we only register
when we know we can. We could be more sophisticated here and e.g.
introduce the concept of "read-only" clients in the daemon to only allow
access to non-mutating methods, though let's delay that discussion at
least until the daemon learns to auto-exit.

Closes: #898

Closes: #900
Approved by: cgwalters
2017-07-27 20:34:22 +00:00
Colin Walters
34b5a004a8 daemon: Add a sanitycheck(/bin/true) before we deploy a tree
This is a followup to https://github.com/projectatomic/rpm-ostree/pull/888
but more comprehensive; in the layering case, the sanitycheck runs
after all the `%posttrans` scripts, so we'll get a consistent error message
for the `rm -rf /` test.

We also do the sanitycheck for the "pure ostree" case, as well as cases
where we didn't actually layer packages (including `ex override remove` as
well as simply regenerating an initrd).

There's obviously a lot more we could do in a sanitycheck; as I say in the
comment it's tempting to consider trying to boot systemd (in a fully volatile
config), but for now let's do this. In the end of course the admin has rollback
too.

Closes: #892
Approved by: jlebon
2017-07-27 17:58:58 +00:00
Jonathan Lebon
ddc0f40355 overlay.sh: also overlay /etc to /usr/etc
This is needed to test changes to the D-Bus config file.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
95bc93205e vmcheck/test-basic.sh: adjust for new polkit policy
Now that we require clients to have an active session to RegisterClient,
we can't use runuser to check for non-root functionality. Add a new
vm_cmd_as() function to allow connecting as a different user. While
we're there, do some minor cleanups to consistently use `local` when
possible.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
f6c422a6d5 vmcheck/test-basic.sh: strengthen pkg-add test
This test would actually fail even if the bin user were allowed to
install a package because there are no enabled repos to install. Fix it
so that we know we have foo there and explicitly check that the error
message is what we expect.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
577aef98e2 app: spawn polkit authentication agent
Spawn pkttyagent when trying to call a method that may require
authentication to give users a chance to provide auth right from the
terminal.

Since we're now relying on polkit for authorizing most of the OS
interface methods, let's drop the root check on those.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
21109e18ae dbus: drop root check on RegisterClient()
Drop the root check for RegisterClient() so that any UID can use it to
then safely invoke methods. UIDs which already have an active session do
not need to authenticate for RegisterClient().

In order to implement this, we move the polkit authority setup into the
RpmostreedSysroot object and enable polkit on the interface.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
3ade73dedd daemon: factor out uid fetching
Make a method specialized in fetching the UID of a D-Bus client.

Prep for polkit tty auth agent.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
d66917df7c libglnx: bump for testsuite fix
ea6df95f22

Otherwise, clang complains about it.

Update submodule: libglnx

Closes: #896
Approved by: cgwalters
2017-07-26 19:22:51 +00:00
Jonathan Lebon
53978f6f03 sync.sh: also copy out /etc files
We do install one thing there: the dbus config file.

Closes: #896
Approved by: cgwalters
2017-07-26 19:22:51 +00:00
Jonathan Lebon
a2878de382 ci: no longer install polkit-devel
This should get pulled in automatically now.

Closes: #896
Approved by: cgwalters
2017-07-26 19:22:51 +00:00
Colin Walters
5763027b5f Update libglnx, do some porting to new APIs
In particular I wanted to start using `GLNX_AUTO_PREFIX_ERROR`.

Update submodule: libglnx

Closes: #893
Approved by: jlebon
2017-07-21 22:37:17 +00:00
Colin Walters
2082b3f8fb lib: Add version macros and version checking function
The version checking function in particular is really useful for people doing
`from gi.repository import RpmOstree`, which we'd like at least some things like
Anaconda and Pungi to do.

Closes: #891
Approved by: jlebon
2017-07-21 20:35:26 +00:00
Colin Walters
fee6d06bf4 lib: Expose new API around basearch
For https://pagure.io/atomic-wg/issue/299 we need to make it
more convenient to substitute the architecture in an installation
context.  I plan to use this API inside `rpmostreepayload` in Anaconda,
so we can substitute the same value of `${basearch}` we use in treefiles
since https://github.com/projectatomic/rpm-ostree/pull/305

Now, you might wonder - why do we need an API wrapping libdnf? It's because
libdnf is not API stable yet. We're just exposing a tiny subset. In theory we
could use the Python dnf bindings in Anaconda, but things get slightly weird if
rpmostreepayload depends on dnf. Perhaps we'll do that down the road, but for
now this a small API surface to maintain (forever).

This change reworks the internal `varsubst` bits to take a pure `DnfContext`,
since we don't want to spin up a whole `RpmOstreeContext` just to do some
string substitutions.

Closes: #877
Approved by: jlebon
2017-07-21 16:02:41 +00:00
Colin Walters
b46fc35901 Add releasever opt, avoid opening up host's rpmdb in treecompose
Closes: https://github.com/projectatomic/rpm-ostree/issues/546

Previously, we'd open up the host's rpmdb for both `compose tree`
and `ex container`.  In the first case, because we require root, we'd
succeed.  For `ex container`, we'd spew an error.

Fixing this was trickier than I thought.  First because there was
*also* a libdnf bug here: https://github.com/rpm-software-management/libdnf/pull/307

Second, there's a compatibility hazard here for anyone using `.repo` files that
reference `$releasever`. This actually happened to me with `ex container` as I'd
just done a `ln -s /etc/yum.repos.d/fedora.repo rpmmd.repos.d`.  I fixed
that first by doing a `sed -i -e 's,$releasever,26,' rpmmd.repos.d/*.repo`.

As far as I can see today, none of Fedora Atomic or CentOS AH rely on this. But
in order to enhance compatibility, let's add a "releasever" option. This makes
it easier again to reuse stock `.repo` files if we wanted to do so.

(Also, I realized we can just use `/usr/share/empty` as *the* canonical immutable
 empty directory)

Closes: #875
Approved by: jlebon
2017-07-21 15:02:59 +00:00
Colin Walters
b52263506f Update libdnf
This pulls in a few fixes, like `metadata_expire`; the specific one for this PR
is the keyring+installroot fix.

Update submodule: libdnf

Closes: #875
Approved by: jlebon
2017-07-21 15:02:59 +00:00
Colin Walters
2f3ca1bc8e tests: Add a test case for a %post that does rm -rf /
I was thinking today about our script handling, and I realized
an excellent way to showcase the advancement rpm-ostree makes
over traditional package managers is the fact that we survive a
`%post` script that does `rm -rf /`!

See e.g. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=858521

It's been this way ever since we first implemented package layering;
the fact that we construct a new root and use bubblewrap to sandbox
makes us very resilient to this type of thing.

But, let's add a test case for this to be sure we preserve this behavior; for
example, if in the future we for some reason we decide to leak some host state
into the scripts.

Closes: #888
Approved by: jlebon
2017-07-21 14:45:30 +00:00
Colin Walters
1f3ebba982 scripts: Honor the -e flag for scripts
This is required for glibc-all-langpacks at least:
https://bugzilla.redhat.com/show_bug.cgi?id=1367585

Otherwise, its usage is...extraordinarily rare. In fact looking at a snapshot of
`rpm-specs-20170518.tar.xz` from Fedora, the only other use is in
`postfix.spec`, and it appears bogus (the value is already expanded at build
time).

But the glibc case is special, as the value of `install_langs` is indeed
potentially dynamic per system.

Closes: #873
Approved by: jlebon
2017-07-18 19:21:15 +00:00
Colin Walters
4222407657 core/scripts: Cleanup refactor
No functional changes; the main thing is to change the "ignore" case to an early
return, and also introduce a "medium level" function that does argument parsing.

Prep for file trigger work.

Closes: #873
Approved by: jlebon
2017-07-18 19:21:15 +00:00
Colin Walters
325ee354e9 core/scripts: Drop support for ignore-scripts
This was a transient hack that I don't think we need any longer.
Since then we've now committed to e.g. ignoring `%pretrans` etc.,
and we also have the hardcoded blacklist.

It never really worked in a complete sense since one would have to re-specify
the ignore on every operation, which was not sustainable.

Closes: #873
Approved by: jlebon
2017-07-18 19:21:15 +00:00
Colin Walters
7a1dd234a3 docs: Update HACKING.md
Closes: https://github.com/projectatomic/rpm-ostree/issues/878

Closes: #880
Approved by: cgwalters
2017-07-18 18:31:50 +00:00
Jonathan Lebon
1806259cc1 ci: stop working around updates-source mirrors
I initially did this because mirrors weren't stable before the F26
release. But now, the canonical source itself is unstable, so let's try
our luck again with mirrors. Might have better luck now that it's
released.

Closes: #881
Approved by: mbarnes
2017-07-18 17:38:43 +00:00
Colin Walters
f38bb61f91 Update libglnx
And port the passwd util.

Update submodule: libglnx

Closes: #867
Approved by: jlebon
2017-07-18 16:43:40 +00:00
Jonathan Lebon
f25444554d ci: no longer run gnome-desktop-testing
We don't currently install any installed tests usable by
gnome-desktop-testing. This conveniently works around not having it
readily packaged on CentOS, though we could always build from source if
the need comes.

Closes: #871
Approved by: cgwalters
2017-07-18 13:58:38 +00:00
Jonathan Lebon
cf19d83502 ci: unite testsuites and run vmcheck on centos
We can be a bit less wasteful here by merging the check and vmcheck
suites into a single suite. The check suite today takes a negligible
amount of time to run, so we're not gaining much by parallelizing them.
It's more of a sanity check at this point before we start vmcheck.

Also start running vmcheck on CentOS 7. We adapt the ci scripts to
accomodate both Fedora and CentOS target machines.

This commit also switches to Fedora 26 as the primary test base.

Closes: #871
Approved by: cgwalters
2017-07-18 13:58:38 +00:00
Jonathan Lebon
114d53a02d vmcheck: fixes for CentOS 7
The `install` command in CentOS 7 is too old to understand that
`-Dt foo/bar` means creating both `foo` and `bar`, which is useful so
that we avoid an explicit `mkdir` before. But we can't do that here.

Closes: #871
Approved by: cgwalters
2017-07-18 13:58:38 +00:00
Matthew Barnes
2934799741 tests: Support older versions of rsync
Older versions of rsync (< 3.1) lack an --ignore-missing-args option.
Use this workaround [1] to allow "make vmsync" to work on CentOS 7.

[1] https://stackoverflow.com/questions/43391493/rsync-simulate-ignore-missing-args-on-old-server-version

Closes: #870
Approved by: cgwalters
2017-07-13 21:35:47 +00:00
Matthew Barnes
c7024d69ae daemon: Avoid "Updating from: (null)" message
Check for a NULL refspec before emitting message in
package_diff_transaction_execute().

Closes: #876
Approved by: cgwalters
2017-07-13 20:14:31 +00:00
Colin Walters
a64b222c66 build-sys: Update release instructions
I got confused since ostree changed to post-release bumps.  Let's
document the current process here.

Closes: #866
Approved by: jlebon
2017-07-11 06:37:40 +00:00
Colin Walters
37a2de7f05 Release 2017.7 2017-07-10 17:03:16 -04:00
Jonathan Lebon
50ad253ebe codebase: partial porting to new style
Just some minor porting of the less glamorous parts of the codebase.

Closes: #865
Approved by: cgwalters
2017-07-07 20:03:28 +00:00
Jonathan Lebon
f1437028cf core: only update repos enabled for packages
We don't care about repos enabled only for METADATA. This works around
an interesting libdnf behaviour in which all repos in redhat.repo are
enabled for metadata, even if disabled.

Downstream RHBZ: #1468302

Closes: #863
Approved by: cgwalters
2017-07-06 18:59:16 +00:00
Jonathan Lebon
760110596b ci: start testing on F26
Now that PAPR has support for pre-release images of Fedora Atomic Host
26, let's start testing there. We mark it as not required for the time
being.

Closes: #860
Approved by: cgwalters
2017-07-06 18:24:47 +00:00
Jonathan Lebon
87b978c088 ci: make compose test required
This has been around for a while now and has found regressions in the
past, so let's make it required. Also rename the contexts to make it
clear that what version of Fedora they're running on.

Closes: #860
Approved by: cgwalters
2017-07-06 18:24:47 +00:00
Colin Walters
f599f4bc7c libpriv: Some style porting
Scattered bits, just keeping up some momentum.

Closes: #861
Approved by: jlebon
2017-07-06 14:29:38 +00:00
Jonathan Lebon
5b541f4859 app: support ex override replace for local RPMs
Add experimental support for replacing packages from the base layer with
local RPMs. This is useful for example, to cherry pick a fixed package,
or to roll back to a previous package version. Like with pkg removals,
only files in /usr are actually replaced.

This patch also contains a few usability improvements as well, e.g.
showing the full NEVRA of removed packages rather than just their names,
and support for resetting overrides using either the pkgname or NEVRA.

Closes: #852
Approved by: cgwalters
2017-07-05 20:59:35 +00:00
Jonathan Lebon
7b4134c8e6 core: make cachebranch finder function public
Prep for replacement overrides.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Jonathan Lebon
63405d655b core: plug leak from ignore_scripts hash table
Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Jonathan Lebon
2da68cb510 sysroot-core: minor style updates
This function was already mostly in the new style. Just moving the
declarations closer to where they are used. No functional changes. The
logic will be tweaked in an upcoming commit.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Jonathan Lebon
3b0b9fb98a vmcheck: handle livefs reruns stronger
Make sure that we wipe out any leftover configuration files from a
previous run before layering the test pkg, or we'll get false positive.
Also make sure to correctly clean up the VM in the case the livefs test
errors out.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Jonathan Lebon
970a20974b build_rpm: print out nevra rather than just name
It's more precise and makes it possible to find out if e.g. pkg
upgrades made it to the tree.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Colin Walters
14431f7f09 lib/scripts: Handle script interpreters
Seen in the wild with `vagrant`'s use of `%post -p /usr/bin/ruby`. This was a
very easy fix, and actually makes the code a little bit nicer, as we no longer
need to explicitly make the script executable, since we now pass it as
`argv[1]`, the same way librpm does. That in turn would make it possible to fix
the TODO and use `bwrap --file`, but that can come later.

Closes: https://github.com/projectatomic/rpm-ostree/issues/856

Closes: #858
Approved by: jlebon
2017-07-04 15:51:20 +00:00
Colin Walters
1f33f42c7a scripts: Port some to new style, minor cleanup
Prep for further work.

Closes: #858
Approved by: jlebon
2017-07-04 15:51:20 +00:00
Jonathan Lebon
2b547c2d0a tests: transition to build_rpm
Nuke all the previous goop that was used to create RPMs at `make check`
time and transition all the tests to use the new `build_rpm` function.
It definitely feels cleaner to use. It's also really nice to have the
spec live in the same file as the test that uses it.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
5cb1fa5460 libtest: add build_rpm function
Having to prepare RPMs in advance separately from the tests that use
them severely limits our ability to test various cases and to iterate
quickly when creating tests.

Add a new `build_rpm` function which can basically build the RPM on the
fly and update the yum repo afterwards. It makes it trivial to test
things like package updates:

    build_rpm foo 1.0 1
    <stuff>
    build_rpm foo 1.0 2
    <stuff>

The RPMs are all created inside the temporary test directory and thus
cleaned up on exit.

I'm doing this in a separate commit because it's the most important diff
of the transition and might be easy to lose in the larger diff where we
move all the tests to make use of this.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
47cc5f60ed tests/check: remove test-compose.sh
Nowadays, we have a much more in-depth and realistic compose test suite
that obsoletes this test.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00