Commit Graph

1526 Commits

Author SHA1 Message Date
Colin Walters
2082b3f8fb lib: Add version macros and version checking function
The version checking function in particular is really useful for people doing
`from gi.repository import RpmOstree`, which we'd like at least some things like
Anaconda and Pungi to do.

Closes: #891
Approved by: jlebon
2017-07-21 20:35:26 +00:00
Colin Walters
fee6d06bf4 lib: Expose new API around basearch
For https://pagure.io/atomic-wg/issue/299 we need to make it
more convenient to substitute the architecture in an installation
context.  I plan to use this API inside `rpmostreepayload` in Anaconda,
so we can substitute the same value of `${basearch}` we use in treefiles
since https://github.com/projectatomic/rpm-ostree/pull/305

Now, you might wonder - why do we need an API wrapping libdnf? It's because
libdnf is not API stable yet. We're just exposing a tiny subset. In theory we
could use the Python dnf bindings in Anaconda, but things get slightly weird if
rpmostreepayload depends on dnf. Perhaps we'll do that down the road, but for
now this a small API surface to maintain (forever).

This change reworks the internal `varsubst` bits to take a pure `DnfContext`,
since we don't want to spin up a whole `RpmOstreeContext` just to do some
string substitutions.

Closes: #877
Approved by: jlebon
2017-07-21 16:02:41 +00:00
Colin Walters
b46fc35901 Add releasever opt, avoid opening up host's rpmdb in treecompose
Closes: https://github.com/projectatomic/rpm-ostree/issues/546

Previously, we'd open up the host's rpmdb for both `compose tree`
and `ex container`.  In the first case, because we require root, we'd
succeed.  For `ex container`, we'd spew an error.

Fixing this was trickier than I thought.  First because there was
*also* a libdnf bug here: https://github.com/rpm-software-management/libdnf/pull/307

Second, there's a compatibility hazard here for anyone using `.repo` files that
reference `$releasever`. This actually happened to me with `ex container` as I'd
just done a `ln -s /etc/yum.repos.d/fedora.repo rpmmd.repos.d`.  I fixed
that first by doing a `sed -i -e 's,$releasever,26,' rpmmd.repos.d/*.repo`.

As far as I can see today, none of Fedora Atomic or CentOS AH rely on this. But
in order to enhance compatibility, let's add a "releasever" option. This makes
it easier again to reuse stock `.repo` files if we wanted to do so.

(Also, I realized we can just use `/usr/share/empty` as *the* canonical immutable
 empty directory)

Closes: #875
Approved by: jlebon
2017-07-21 15:02:59 +00:00
Colin Walters
b52263506f Update libdnf
This pulls in a few fixes, like `metadata_expire`; the specific one for this PR
is the keyring+installroot fix.

Update submodule: libdnf

Closes: #875
Approved by: jlebon
2017-07-21 15:02:59 +00:00
Colin Walters
2f3ca1bc8e tests: Add a test case for a %post that does rm -rf /
I was thinking today about our script handling, and I realized
an excellent way to showcase the advancement rpm-ostree makes
over traditional package managers is the fact that we survive a
`%post` script that does `rm -rf /`!

See e.g. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=858521

It's been this way ever since we first implemented package layering;
the fact that we construct a new root and use bubblewrap to sandbox
makes us very resilient to this type of thing.

But, let's add a test case for this to be sure we preserve this behavior; for
example, if in the future we for some reason we decide to leak some host state
into the scripts.

Closes: #888
Approved by: jlebon
2017-07-21 14:45:30 +00:00
Colin Walters
1f3ebba982 scripts: Honor the -e flag for scripts
This is required for glibc-all-langpacks at least:
https://bugzilla.redhat.com/show_bug.cgi?id=1367585

Otherwise, its usage is...extraordinarily rare. In fact looking at a snapshot of
`rpm-specs-20170518.tar.xz` from Fedora, the only other use is in
`postfix.spec`, and it appears bogus (the value is already expanded at build
time).

But the glibc case is special, as the value of `install_langs` is indeed
potentially dynamic per system.

Closes: #873
Approved by: jlebon
2017-07-18 19:21:15 +00:00
Colin Walters
4222407657 core/scripts: Cleanup refactor
No functional changes; the main thing is to change the "ignore" case to an early
return, and also introduce a "medium level" function that does argument parsing.

Prep for file trigger work.

Closes: #873
Approved by: jlebon
2017-07-18 19:21:15 +00:00
Colin Walters
325ee354e9 core/scripts: Drop support for ignore-scripts
This was a transient hack that I don't think we need any longer.
Since then we've now committed to e.g. ignoring `%pretrans` etc.,
and we also have the hardcoded blacklist.

It never really worked in a complete sense since one would have to re-specify
the ignore on every operation, which was not sustainable.

Closes: #873
Approved by: jlebon
2017-07-18 19:21:15 +00:00
Colin Walters
7a1dd234a3 docs: Update HACKING.md
Closes: https://github.com/projectatomic/rpm-ostree/issues/878

Closes: #880
Approved by: cgwalters
2017-07-18 18:31:50 +00:00
Jonathan Lebon
1806259cc1 ci: stop working around updates-source mirrors
I initially did this because mirrors weren't stable before the F26
release. But now, the canonical source itself is unstable, so let's try
our luck again with mirrors. Might have better luck now that it's
released.

Closes: #881
Approved by: mbarnes
2017-07-18 17:38:43 +00:00
Colin Walters
f38bb61f91 Update libglnx
And port the passwd util.

Update submodule: libglnx

Closes: #867
Approved by: jlebon
2017-07-18 16:43:40 +00:00
Jonathan Lebon
f25444554d ci: no longer run gnome-desktop-testing
We don't currently install any installed tests usable by
gnome-desktop-testing. This conveniently works around not having it
readily packaged on CentOS, though we could always build from source if
the need comes.

Closes: #871
Approved by: cgwalters
2017-07-18 13:58:38 +00:00
Jonathan Lebon
cf19d83502 ci: unite testsuites and run vmcheck on centos
We can be a bit less wasteful here by merging the check and vmcheck
suites into a single suite. The check suite today takes a negligible
amount of time to run, so we're not gaining much by parallelizing them.
It's more of a sanity check at this point before we start vmcheck.

Also start running vmcheck on CentOS 7. We adapt the ci scripts to
accomodate both Fedora and CentOS target machines.

This commit also switches to Fedora 26 as the primary test base.

Closes: #871
Approved by: cgwalters
2017-07-18 13:58:38 +00:00
Jonathan Lebon
114d53a02d vmcheck: fixes for CentOS 7
The `install` command in CentOS 7 is too old to understand that
`-Dt foo/bar` means creating both `foo` and `bar`, which is useful so
that we avoid an explicit `mkdir` before. But we can't do that here.

Closes: #871
Approved by: cgwalters
2017-07-18 13:58:38 +00:00
Matthew Barnes
2934799741 tests: Support older versions of rsync
Older versions of rsync (< 3.1) lack an --ignore-missing-args option.
Use this workaround [1] to allow "make vmsync" to work on CentOS 7.

[1] https://stackoverflow.com/questions/43391493/rsync-simulate-ignore-missing-args-on-old-server-version

Closes: #870
Approved by: cgwalters
2017-07-13 21:35:47 +00:00
Matthew Barnes
c7024d69ae daemon: Avoid "Updating from: (null)" message
Check for a NULL refspec before emitting message in
package_diff_transaction_execute().

Closes: #876
Approved by: cgwalters
2017-07-13 20:14:31 +00:00
Colin Walters
a64b222c66 build-sys: Update release instructions
I got confused since ostree changed to post-release bumps.  Let's
document the current process here.

Closes: #866
Approved by: jlebon
2017-07-11 06:37:40 +00:00
Colin Walters
37a2de7f05 Release 2017.7 2017-07-10 17:03:16 -04:00
Jonathan Lebon
50ad253ebe codebase: partial porting to new style
Just some minor porting of the less glamorous parts of the codebase.

Closes: #865
Approved by: cgwalters
2017-07-07 20:03:28 +00:00
Jonathan Lebon
f1437028cf core: only update repos enabled for packages
We don't care about repos enabled only for METADATA. This works around
an interesting libdnf behaviour in which all repos in redhat.repo are
enabled for metadata, even if disabled.

Downstream RHBZ: #1468302

Closes: #863
Approved by: cgwalters
2017-07-06 18:59:16 +00:00
Jonathan Lebon
760110596b ci: start testing on F26
Now that PAPR has support for pre-release images of Fedora Atomic Host
26, let's start testing there. We mark it as not required for the time
being.

Closes: #860
Approved by: cgwalters
2017-07-06 18:24:47 +00:00
Jonathan Lebon
87b978c088 ci: make compose test required
This has been around for a while now and has found regressions in the
past, so let's make it required. Also rename the contexts to make it
clear that what version of Fedora they're running on.

Closes: #860
Approved by: cgwalters
2017-07-06 18:24:47 +00:00
Colin Walters
f599f4bc7c libpriv: Some style porting
Scattered bits, just keeping up some momentum.

Closes: #861
Approved by: jlebon
2017-07-06 14:29:38 +00:00
Jonathan Lebon
5b541f4859 app: support ex override replace for local RPMs
Add experimental support for replacing packages from the base layer with
local RPMs. This is useful for example, to cherry pick a fixed package,
or to roll back to a previous package version. Like with pkg removals,
only files in /usr are actually replaced.

This patch also contains a few usability improvements as well, e.g.
showing the full NEVRA of removed packages rather than just their names,
and support for resetting overrides using either the pkgname or NEVRA.

Closes: #852
Approved by: cgwalters
2017-07-05 20:59:35 +00:00
Jonathan Lebon
7b4134c8e6 core: make cachebranch finder function public
Prep for replacement overrides.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Jonathan Lebon
63405d655b core: plug leak from ignore_scripts hash table
Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Jonathan Lebon
2da68cb510 sysroot-core: minor style updates
This function was already mostly in the new style. Just moving the
declarations closer to where they are used. No functional changes. The
logic will be tweaked in an upcoming commit.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Jonathan Lebon
3b0b9fb98a vmcheck: handle livefs reruns stronger
Make sure that we wipe out any leftover configuration files from a
previous run before layering the test pkg, or we'll get false positive.
Also make sure to correctly clean up the VM in the case the livefs test
errors out.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Jonathan Lebon
970a20974b build_rpm: print out nevra rather than just name
It's more precise and makes it possible to find out if e.g. pkg
upgrades made it to the tree.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Colin Walters
14431f7f09 lib/scripts: Handle script interpreters
Seen in the wild with `vagrant`'s use of `%post -p /usr/bin/ruby`. This was a
very easy fix, and actually makes the code a little bit nicer, as we no longer
need to explicitly make the script executable, since we now pass it as
`argv[1]`, the same way librpm does. That in turn would make it possible to fix
the TODO and use `bwrap --file`, but that can come later.

Closes: https://github.com/projectatomic/rpm-ostree/issues/856

Closes: #858
Approved by: jlebon
2017-07-04 15:51:20 +00:00
Colin Walters
1f33f42c7a scripts: Port some to new style, minor cleanup
Prep for further work.

Closes: #858
Approved by: jlebon
2017-07-04 15:51:20 +00:00
Jonathan Lebon
2b547c2d0a tests: transition to build_rpm
Nuke all the previous goop that was used to create RPMs at `make check`
time and transition all the tests to use the new `build_rpm` function.
It definitely feels cleaner to use. It's also really nice to have the
spec live in the same file as the test that uses it.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
5cb1fa5460 libtest: add build_rpm function
Having to prepare RPMs in advance separately from the tests that use
them severely limits our ability to test various cases and to iterate
quickly when creating tests.

Add a new `build_rpm` function which can basically build the RPM on the
fly and update the yum repo afterwards. It makes it trivial to test
things like package updates:

    build_rpm foo 1.0 1
    <stuff>
    build_rpm foo 1.0 2
    <stuff>

The RPMs are all created inside the temporary test directory and thus
cleaned up on exit.

I'm doing this in a separate commit because it's the most important diff
of the transition and might be easy to lose in the larger diff where we
move all the tests to make use of this.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
47cc5f60ed tests/check: remove test-compose.sh
Nowadays, we have a much more in-depth and realistic compose test suite
that obsoletes this test.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
4445df2c01 tests: add libtest.c for binary tests
Import libostreetest.c from ostreedev/ostree as libtest.c. This is just
a really useful and outrageous way of using libtest.sh from C.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
4c831e619c check/test-utils.c: assert no error first
Minor tweak; use g_assert_no_error() before using g_assert(ret) so that
we actually get a printout of the error the test fails.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
1c17e93029 tap-test: create tmpdir in /var/tmp
So that we can use xattrs.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
0fafd8ea18 tap-test: fix typo and actually create .test marker
We have this chunk in tap-test:

    if test -f ${test_tmpdir}/.test; then
      rm "${tempdir}" -rf
    fi

But (1) we were never actually touching `.test` and (2), we were
checking in `test_tmpdir` instead of `tmpdir`. This meant that the test
dirs were never actually cleaned up.

Create the marker file and make sure we rm the right dir. Also re-indent
to remove tabs.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Colin Walters
137271efee libpriv/passwd: Port to new style
All straightforward. Just chipping away at the porting.

Closes: #851
Approved by: jlebon
2017-06-26 18:03:12 +00:00
Colin Walters
c7c89e8735 status: Use more concise GPGSignature without --verbose
Since it takes up a lot of room and should be something that's just always
working.

Closes: https://github.com/projectatomic/rpm-ostree/issues/842

Closes: #848
Approved by: jlebon
2017-06-26 15:51:46 +00:00
Colin Walters
4d72f8d787 cmdline: Stop printing GPG results during pulls
It's really verbose for deploy, and in general, let's follow
the Unix philosophy of silence = success.

Closes: https://github.com/projectatomic/rpm-ostree/issues/843

Closes: #850
Approved by: jlebon
2017-06-26 14:36:41 +00:00
Kalev Lember
417a3d8d4e daemon: Install dbus introspection files
This makes it possible to use the dbus introspection files in other apps
without having to bundle them.

Closes: #841
Approved by: cgwalters
2017-06-23 19:15:42 +00:00
Jonathan Lebon
478b22db10 libdnf: bump for stale cache fix
As mentioned in

  https://github.com/rpm-software-management/libdnf/pull/297,

this affects our compose path.

Update submodule: libdnf

Closes: #840
Approved by: cgwalters
2017-06-23 18:33:51 +00:00
Jonathan Lebon
453e9ca570 rpm-util: factor out get_by_pkgname
This will be needed multiple times in the future.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:22 +00:00
Jonathan Lebon
3ae90ce5fc upgrader: make metadata tmp dir part of upgrader
I initially planned to use the metadata tmp dir for another reason than
just in finalize_overlays(), but I will not anymore. Regardless, this
seems like a nice patch which also allows us to complete the style
conversion in this file.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:22 +00:00
Jonathan Lebon
853737f667 unpacker: encode NEVRA info in commit metadata
It seems silly that to find out more detailed information about the
NEVRA of a cached pkg, we have to resort to write out the header to
disk, then reading it back in with librpm in order to tease out the info
we want. Let's just encode that information directly in the commit
metadata and provide a helper to fetch it.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:22 +00:00
Jonathan Lebon
4507c581cf core: factor out pkgcache nevra to branch lookup
This will be useful in the future for another function.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:22 +00:00
Jonathan Lebon
44bcb9d60a tests: merge cache_branch_to_nevra into test-utils
I didn't realize at the time I wrote the cache_branch_to_nevra test that
the already existing test-utils.c would be the perfect place to add this
test. Merge the two together now.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:21 +00:00
Jonathan Lebon
7b6237c14f postprocess: prefix more errors
This is a follow-up to #845. Judging from the output of the erroring
compose[1], the error seems to happen after committing all the objects.
Add more detailed prefixes there.

[1] https://lists.fedoraproject.org/archives/list/cloud@lists.fedoraproject.org/thread/2MZ5KFEVITKWJZYOTSAYCZ7XYAJCWZCT/

Closes: #846
Approved by: cgwalters
2017-06-23 15:48:05 +00:00
Colin Walters
64b049b8eb postprocess: Style porting
Prep for debugging: https://lists.fedoraproject.org/archives/list/cloud@lists.fedoraproject.org/thread/2MZ5KFEVITKWJZYOTSAYCZ7XYAJCWZCT/
Almost everything is converted in this file now.

Closes: #845
Approved by: jlebon
2017-06-23 13:50:57 +00:00