keremet/rpm-ostree
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Remove large chunks of rojig code

Browse Source 
The inevitable followup to https://github.com/coreos/rpm-ostree/pull/2278
that I was too cowardly to do at the time.  But it's time to admit
the 2 months or so of work on this was wasted.  We have too much
tech debt and this is a large chunk of C/C++ code that touches everything
in the codebase in a nontrivial way.

Bigger picture, I'm going to work on
https://github.com/coreos/fedora-coreos-tracker/issues/828
which will strongly orient rpm-ostree towards the container world instead.
We'll still obviously keep the rpm package world around, but only
as a secondary layer.  What rojig was trying to do in putting "images"
inside an RPM was conflating layers.  It would have had a lot of
benefits probably if we'd truly pushed it over the edge into completion,
but that didn't happen.  Let's focus on containers instead.

There's still a lot more rojig code to delete but this first patch removes
the bulk of it.  Touching everything that references e.g. `RPMOSTREE_REFSPEC_TYPE_ROJIG`
etc. can come as a 3rd phase.
This commit is contained in:
Colin Walters 2021-05-18 16:04:51 -04:00
parent ba6a2acb4d
commit 562e03f7c1
28 changed files with 2 additions and 3926 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
Makefile-libpriv.am
View File

@ -54,14 +54,3 @@ librpmostreepriv_sources = \
src/libpriv/rpmostree-libarchive-input-stream.cxx \
src/libpriv/rpmostree-libarchive-input-stream.h \
$(NULL)
if BUILDOPT_ROJIG
librpmostreepriv_sources += \
src/libpriv/rpmostree-rojig-build.cxx \
src/libpriv/rpmostree-rojig-build.h \
src/libpriv/rpmostree-rojig-assembler.cxx \
src/libpriv/rpmostree-rojig-assembler.h \
src/libpriv/rpmostree-rojig-core.h \
src/libpriv/rpmostree-rojig-client.cxx \
$(NULL)
endif

8
Makefile-rpm-ostree.am
View File

@ -66,14 +66,6 @@ librpmostreeinternals_la_SOURCES = \
$(librpmostree_1_la_SOURCES) \
$(NULL)
if BUILDOPT_ROJIG
librpmostreeinternals_la_SOURCES += \
src/app/rpmostree-ex-builtin-commit2rojig.cxx \
src/app/rpmostree-ex-builtin-rojig2commit.cxx \
src/app/rpmostree-compose-builtin-rojig.cxx \
$(NULL)
endif
nodist_librpmostreeinternals_la_SOURCES = $(dbus_built_sources) $(nodist_librpmostreepriv_sources)
rpmostree_common_cflags = -I$(srcdir)/src/app -I$(srcdir)/src/daemon \

9
configure.ac
View File

@ -125,15 +125,6 @@ AM_CONDITIONAL(RUST_DEBUG, [test "x$rust_debug_release" = "xdebug"])
dnl Unconditional now.
RPM_OSTREE_FEATURES="$RPM_OSTREE_FEATURES rust"
AC_ARG_ENABLE(rojig,
AC_HELP_STRING([--enable-rojig],
[Support for shipping ostree commits via RPMs [default=no]]))
AM_CONDITIONAL([BUILDOPT_ROJIG], test x$enable_rojig = xyes)
AM_COND_IF([BUILDOPT_ROJIG], [
RPM_OSTREE_FEATURES="$RPM_OSTREE_FEATURES rojig"
AC_DEFINE([BUILDOPT_ROJIG], 1, [Define if rojig is enabled])
])
AC_CONFIG_FILES([
Makefile
api-doc/Makefile

96
design/rojig.md
View File

@ -1,96 +0,0 @@
Introducing rpm-ostree rojig
--------
In the rpm-ostree project, we're blending an image system (libostree)
with a package system (libdnf). The goal is to gain the
advantages of both. However, the dual nature also brings overhead;
this proposal aims to reduce some of that by adding a new "rojig"
model to rpm-ostree that makes more operations use the libdnf side.
To do this, we're reviving an old idea: The [http://atterer.org/jigdo/](Jigdo)
approach to reassembling large "images" by downloading component packages. (We're
not using the code, just the idea).
In this approach, we're still maintaining the "image" model of libostree. When
one deploys an OSTree commit, it will reliably be bit-for-bit identical. It will
have a checksum and a version number. There will be *no* dependency resolution
on the client by default, etc.
The change is that we always use libdnf to download RPM packages as they exist
today, storing any additional data inside a new "ostree-image" RPM. In this
proposal, rather than using ostree branches, the system tracks an "ostree-image"
RPM that behaves a bit like a "metapackage".
Why?
----
The "dual" nature of the system appears in many ways; users and administrators
effectively need to understand and manage both systems.
An example is when one needs to mirror content. While libostree does support
mirroring, and projects like Pulp make use of it, support is not as widespread
as mirroring for RPM. And mirroring is absolutely critical for many
organizations that don't want to depend on Internet availability.
Related to this is the mapping of libostree "branches" and rpm-md repos. In
Fedora we offer multiple branches for Atomic Host, such as
`fedora/27/x86_64/atomic-host` as well as
`fedora/27/x86_64/testing/atomic-host`, where the latter is equivalent to `yum
--enablerepo=updates-testing update`. In many ways, I believe the way we're
exposing as OSTree branches is actually nicer - it's very clear when you're on
the testing branch.
However, it's also very *different* from the yum/dnf model. Once package
layering is involved (and for a lot of small scale use cases it will be,
particularly for desktop systems), the libostree side is something that many
users and administrators have to learn *in addition* to their previous "mental model"
of how the libdnf/yum/rpm side works with `/etc/yum.repos.d` etc.
Finally, for network efficiency; on the wire, libostree has two formats, and the
intention is that most updates hit the network-efficient static delta path, but
there are various cases where this doesn't happen, such as if one is skipping a
few updates, or today when rebasing between branches. In practice, as soon as
one involves libdnf, the repodata is already large enough that it's not worth
trying to optimize fetching content over simply redownloading changed RPMs.
(Aside: people doing custom systems tend to like the network efficiency of "pure
ostree" where one doesn't pay the "repodata cost" and we will continue to
support that.)
How?
----
We've already stated that a primary design goal is to preserve the "image"
functionality by default. Further, let's assume that we have an OSTree commit,
and we want to point it at a set of RPMs to use as the rojig source. The source
OSTree commit can have modified, added to, or removed data from the RPM set, and
we will support that. Examples of additional data are the initramfs and RPM
database.
We're hence treating the RPM set as just data blobs; again, no dependency
resolution, `%post` scripts or the like will be executed on the client. Or again
to state this more strongly, installation will still result in an OSTree commit
with checksum that is bit-for-bit identical.
A simple approach is to scan over the set of files in the RPMs, then the set
of files in the OSTree commit, and add RPMs which contain files in the OSTree
commit to our "rojig set".
However, a major complication is SELinux labeling. It turns out that in a lot of
cases, doing SELinux labeling is expensive; there are thousands of regular
expressions involved. However, RPM packages themselves don't contain labels;
instead the labeling is stored in the `selinux-policy-targeted` package, and
further complicating things is that there are also other packages that add
labeling configuration such as `container-selinux`. In other words there's a
circular dependency: packages have labels, but labels are contained in packages.
We go to great lengths to handle this in rpm-ostree for package layering, and we
need to do the same for rojig.
We can address this by having our OIRPM contain a mapping of (package, file
path) to a set of extended attributes (including the key `security.selinux`
one).
At this point, if we add in the new objects such as the metadata objects from
the OSTree commit and all new content objects that aren't part of a package,
we'll have our rojigRPM. (There is some [further complexity](https://pagure.io/fedora-atomic/issue/94) around
handling the initramfs and SELinux labeling that we'll omit for now).

3
docs/treefile.md
View File

@ -319,9 +319,6 @@ It supports the following parameters:
All options listed here are subject to change or removal in a future
version of `rpm-ostree`.
* `rojig`: Object, optional. Sub-keys are `name`, `summary`, `license`,
and `description`. Of those, `name` and `license` are mandatory.
* `lockfile-repos`: array of strings, optional: Semantically similar to
`repo`, but these repos will only be used to fetch packages locked
via lockfiles. This is useful when locked packages are kept

5
src/app/rpmostree-builtin-compose.cxx
View File

@ -44,11 +44,6 @@ static RpmOstreeCommand compose_subcommands[] = {
{ "extensions", RPM_OSTREE_BUILTIN_FLAG_LOCAL_CMD,
"Download RPM packages guaranteed to depsolve with a base OSTree",
rpmostree_compose_builtin_extensions },
#ifdef BUILDOPT_ROJIG
{ "rojig", (RpmOstreeBuiltinFlags)(RPM_OSTREE_BUILTIN_FLAG_LOCAL_CMD | RPM_OSTREE_BUILTIN_FLAG_HIDDEN),
"EXPERIMENTAL: Build a rojig RPM from a treefile, output to a local rpm-md repo",
rpmostree_compose_builtin_rojig },
#endif
{ NULL, (RpmOstreeBuiltinFlags)0, NULL, NULL }
};

6
src/app/rpmostree-builtin-ex.cxx
View File

@ -29,12 +29,6 @@ static RpmOstreeCommand ex_subcommands[] = {
{ "apply-live", (RpmOstreeBuiltinFlags)0,
"Apply pending deployment changes to booted deployment",
rpmostree_ex_builtin_apply_live },
#ifdef BUILDOPT_ROJIG
{ "commit2rojig", (RpmOstreeBuiltinFlags)RPM_OSTREE_BUILTIN_FLAG_LOCAL_CMD,
"Convert an OSTree commit into an rpm-ostree rojig", rpmostree_ex_builtin_commit2rojig },
{ "rojig2commit", (RpmOstreeBuiltinFlags)RPM_OSTREE_BUILTIN_FLAG_LOCAL_CMD,
"Convert an rpm-ostree rojig into an OSTree commit", rpmostree_ex_builtin_rojig2commit },
#endif
{ "history", (RpmOstreeBuiltinFlags)RPM_OSTREE_BUILTIN_FLAG_LOCAL_CMD,
"Inspect rpm-ostree history of the system", rpmostree_ex_builtin_history },
{ "initramfs-etc", (RpmOstreeBuiltinFlags)0,

26
src/app/rpmostree-builtin-status.cxx
View File

@ -653,31 +653,7 @@ print_one_deployment (RPMOSTreeSysroot *sysroot_proxy,
}
break;
case RPMOSTREE_REFSPEC_TYPE_ROJIG:
{
g_autoptr(GVariant) rojig_description = NULL;
g_variant_dict_lookup (dict, "rojig-description", "@a{sv}", &rojig_description);
if (rojig_description)
{
g_autoptr(GVariantDict) dict = g_variant_dict_new (rojig_description);
const char *repo = NULL;
g_variant_dict_lookup (dict, "repo", "&s", &repo);
const char *name = NULL;
g_variant_dict_lookup (dict, "name", "&s", &name);
const char *evr = NULL;
g_variant_dict_lookup (dict, "evr", "&s", &evr);
const char *arch = NULL;
g_variant_dict_lookup (dict, "arch", "&s", &arch);
g_assert (repo && name);
g_print ("%s:%s", repo, name);
if (evr && arch)
g_print ("-%s.%s", evr, arch);
}
else
{
g_print ("%s", canonrefspec);
}
}
break;
g_assert_not_reached ();
}
}
else

607
src/app/rpmostree-compose-builtin-rojig.cxx
View File

@ -1,607 +0,0 @@
/* -*- mode: C; c-file-style: "gnu"; indent-tabs-mode: nil; -*-
*
* Copyright (C) 2013,2014 Colin Walters <walters@verbum.org>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published
* by the Free Software Foundation; either version 2 of the licence or (at
* your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General
* Public License along with this library; if not, write to the
* Free Software Foundation, Inc., 59 Temple Place, Suite 330,
* Boston, MA 02111-1307, USA.
*/
#include "config.h"
#include <string.h>
#include <glib-unix.h>
#include <json-glib/json-glib.h>
#include <gio/gunixinputstream.h>
#include <gio/gunixoutputstream.h>
#include <libdnf/libdnf.h>
#include <libdnf/dnf-repo.h>
#include <sys/mount.h>
#include <stdio.h>
#include <linux/magic.h>
#include <sys/statvfs.h>
#include <sys/vfs.h>
#include <libglnx.h>
#include <rpm/rpmmacro.h>
#include <optional>
#include "rpmostree-compose-builtins.h"
#include "rpmostree-util.h"
#include "rpmostree-core.h"
#include "rpmostree-json-parsing.h"
#include "rpmostree-rojig-build.h"
#include "rpmostree-postprocess.h"
#include "rpmostree-package-variants.h"
#include "rpmostree-libbuiltin.h"
#include "rpmostree-rpm-util.h"
#include "rpmostree-composeutil.h"
#include "libglnx.h"
static gboolean opt_force_commit;
static gboolean opt_cache_only;
static char *opt_cachedir;
static gboolean opt_download_only;
static gboolean opt_dry_run;
static char *opt_metadata_json;
static char *opt_write_composejson_to;
static GOptionEntry rojig_option_entries[] = {
{ "force-commit", 0, 0, G_OPTION_ARG_NONE, &opt_force_commit, "Always create a new rojig RPM, even if nothing appears to have changed", NULL },
{ "cache-only", 0, 0, G_OPTION_ARG_NONE, &opt_cache_only, "Assume cache is present, do not attempt to update it", NULL },
{ "cachedir", 0, 0, G_OPTION_ARG_STRING, &opt_cachedir, "Cached state", "CACHEDIR" },
{ "download-only", 0, 0, G_OPTION_ARG_NONE, &opt_download_only, "Like --dry-run, but download RPMs as well; requires --cachedir", NULL },
{ "dry-run", 0, 0, G_OPTION_ARG_NONE, &opt_dry_run, "Just print the transaction and exit", NULL },
{ "add-metadata-from-json", 0, 0, G_OPTION_ARG_STRING, &opt_metadata_json, "Parse the given JSON file as object, convert to GVariant, append to OSTree commit", "JSON" },
{ "write-composejson-to", 0, 0, G_OPTION_ARG_STRING, &opt_write_composejson_to, "Write JSON to FILE containing information about the compose run", "FILE" },
{ NULL }
};
typedef struct {
RpmOstreeContext *corectx;
GHashTable *metadata;
GLnxTmpDir workdir_tmp;
int rootfs_dfd;
int workdir_dfd; /* Note: may be an alias for workdir_tmp */
int cachedir_dfd; /* Note: may be an alias for workdir_tmp */
OstreeRepo *repo;
OstreeRepo *pkgcache_repo;
OstreeRepoDevInoCache *devino_cache;
char *rojig_spec;
char *previous_version;
char *previous_inputhash;
std::optional<rust::Box<rpmostreecxx::Treefile>> treefile_rs;
JsonParser *treefile_parser;
JsonNode *treefile_rootval; /* Unowned */
JsonObject *treefile; /* Unowned */
RpmOstreeTreespec *treespec;
} RpmOstreeRojigCompose;
static void
rpm_ostree_rojig_compose_free (RpmOstreeRojigCompose *ctx)
{
g_clear_object (&ctx->repo);
g_clear_object (&ctx->pkgcache_repo);
g_clear_object (&ctx->corectx);
g_clear_pointer (&ctx->metadata, g_hash_table_unref);
glnx_close_fd (&ctx->rootfs_dfd);
if (ctx->workdir_dfd != ctx->workdir_tmp.fd)
glnx_close_fd (&ctx->workdir_dfd);
if (ctx->cachedir_dfd != ctx->workdir_tmp.fd)
glnx_close_fd (&ctx->cachedir_dfd);
if (g_getenv ("RPMOSTREE_PRESERVE_TMPDIR"))
g_printerr ("Preserved workdir: %s\n", ctx->workdir_tmp.path);
else
(void)glnx_tmpdir_delete (&ctx->workdir_tmp, NULL, NULL);
g_clear_pointer (&ctx->devino_cache, (GDestroyNotify)ostree_repo_devino_cache_unref);
g_free (ctx->rojig_spec);
g_free (ctx->previous_version);
g_free (ctx->previous_inputhash);
ctx->treefile_rs = nullptr;
g_clear_object (&ctx->treefile_parser);
g_clear_object (&ctx->treespec);
g_free (ctx);
}
G_DEFINE_AUTOPTR_CLEANUP_FUNC(RpmOstreeRojigCompose, rpm_ostree_rojig_compose_free)
static gboolean
install_packages (RpmOstreeRojigCompose *self,
gboolean *out_unmodified,
char **out_new_inputhash,
GCancellable *cancellable,
GError **error)
{
DnfContext *dnfctx = rpmostree_context_get_dnf (self->corectx);
{ int tf_dfd = self->treefile_rs->get_workdir();
g_autofree char *abs_tf_path = glnx_fdrel_abspath (tf_dfd, ".");
dnf_context_set_repo_dir (dnfctx, abs_tf_path);
}
/* For compose, always try to refresh metadata; we're used in build servers
* where fetching should be cheap. Otherwise, if --cache-only is set, it's
* likely an offline developer laptop case, so never refresh.
*/
if (!opt_cache_only)
dnf_context_set_cache_age (dnfctx, 0);
else
dnf_context_set_cache_age (dnfctx, G_MAXUINT);
rpmostree_context_set_treespec (self->corectx, self->treespec);
{ g_autofree char *tmprootfs_abspath = glnx_fdrel_abspath (self->rootfs_dfd, ".");
if (!rpmostree_context_setup (self->corectx, tmprootfs_abspath, NULL,
cancellable, error))
return FALSE;
}
/* For unified core, we have a pkgcache repo. This may be auto-created under
* the workdir, or live explicitly in the dir for --cache.
*/
self->pkgcache_repo = ostree_repo_create_at (self->cachedir_dfd, "pkgcache-repo",
OSTREE_REPO_MODE_BARE_USER, NULL,
cancellable, error);
if (!self->pkgcache_repo)
return FALSE;
rpmostree_context_set_repos (self->corectx, self->repo, self->pkgcache_repo);
self->devino_cache = ostree_repo_devino_cache_new ();
rpmostree_context_set_devino_cache (self->corectx, self->devino_cache);
if (!rpmostree_context_prepare (self->corectx, cancellable, error))
return FALSE;
rpmostree_print_transaction (dnfctx);
g_autofree char *ret_new_inputhash = NULL;
if (!rpmostree_composeutil_checksum (dnf_context_get_goal (dnfctx),
self->repo,
self->treefile_rs, self->treefile,
&ret_new_inputhash, error))
return FALSE;
g_print ("Input state hash: %s\n", ret_new_inputhash);
/* Only look for previous checksum if caller has passed *out_unmodified */
if (self->previous_inputhash && out_unmodified != NULL)
{
if (strcmp (self->previous_inputhash, ret_new_inputhash) == 0)
{
*out_unmodified = TRUE;
return TRUE; /* NB: early return */
}
}
if (opt_dry_run)
return TRUE; /* NB: early return */
self->treefile_rs->sanitycheck_externals();
/* --- Downloading packages --- */
if (!rpmostree_context_download (self->corectx, cancellable, error))
return FALSE;
if (!rpmostree_context_import (self->corectx, cancellable, error))
return FALSE;
if (opt_download_only)
return TRUE; /* 🔚 Early return */
rpmostreecxx::passwd_compose_prep(self->rootfs_dfd, self->treefile_rs);
rpmostree_context_set_tmprootfs_dfd (self->corectx, self->rootfs_dfd);
if (!rpmostree_context_assemble (self->corectx, cancellable, error))
return FALSE;
/* Now reload the policy from the tmproot, and relabel the pkgcache - this
* is the same thing done in rpmostree_context_commit(). But here we want
* to ensure our pkgcache labels are accurate, since that will
* be important for the ostree-rojig work.
*/
{ g_autoptr(OstreeSePolicy) sepolicy = ostree_sepolicy_new_at (self->rootfs_dfd, cancellable, error);
rpmostree_context_set_sepolicy (self->corectx, sepolicy);
if (!rpmostree_context_force_relabel (self->corectx, cancellable, error))
return FALSE;
}
if (out_unmodified)
*out_unmodified = FALSE;
*out_new_inputhash = util::move_nullify (ret_new_inputhash);
return TRUE;
}
/* Prepare a context - this does some generic pre-compose initialization from
* the arguments such as loading the treefile and any specified metadata.
*/
static gboolean
rpm_ostree_rojig_compose_new (const char *treefile_path,
RpmOstreeRojigCompose **out_context,
GCancellable *cancellable,
GError **error)
{
g_autoptr(RpmOstreeRojigCompose) self = g_new0 (RpmOstreeRojigCompose, 1);
/* Init fds to -1 */
self->workdir_dfd = self->rootfs_dfd = self->cachedir_dfd = -1;
/* Test whether or not bwrap is going to work - we will fail inside e.g. a Docker
* container without --privileged or userns exposed.
*/
rpmostreecxx::bubblewrap_selftest();
if (opt_cachedir)
{
/* Put the workdir under the cachedir, so it's all on one filesystem;
* this will let us do hardlinks.
*/
if (!glnx_opendirat (AT_FDCWD, opt_cachedir, TRUE, &self->cachedir_dfd, error))
return glnx_prefix_error (error, "Opening cachedir '%s'", opt_cachedir);
if (!glnx_shutil_rm_rf_at (self->cachedir_dfd, "work", cancellable, error))
return FALSE;
if (!glnx_shutil_mkdir_p_at (self->cachedir_dfd, "work", 0755, cancellable, error))
return FALSE;
if (!glnx_opendirat (self->cachedir_dfd, "work", TRUE, &self->workdir_dfd, error))
return FALSE;
}
else
{
/* No cache? Then allocate a temporary workdir, and put the cachedir
* under it.
*/
g_autofree char *tmpdir = g_build_filename (g_getenv ("TMPDIR") ?: "/var/tmp", "rpm-ostree.XXXXXX", NULL);
if (!glnx_mkdtempat (AT_FDCWD, tmpdir, 0700, &self->workdir_tmp, error))
return FALSE;
self->cachedir_dfd = self->workdir_dfd = self->workdir_tmp.fd;
}
/* In rojig mode, we have a temporary repo */
self->repo = ostree_repo_create_at (self->workdir_dfd, "repo-build",
OSTREE_REPO_MODE_BARE_USER, NULL,
cancellable, error);
if (!self->repo)
return glnx_prefix_error (error, "Creating repo-build");
self->metadata = g_hash_table_new_full (g_str_hash, g_str_equal, g_free,
(GDestroyNotify)g_variant_unref);
if (opt_metadata_json)
{
if (!rpmostree_composeutil_read_json_metadata_from_file (opt_metadata_json,
self->metadata, error))
return FALSE;
}
self->corectx = rpmostree_context_new_compose (self->cachedir_dfd, self->repo);
const char *arch = dnf_context_get_base_arch (rpmostree_context_get_dnf (self->corectx));
self->treefile_rs = rpmostreecxx::treefile_new (treefile_path, arch, self->workdir_dfd);
self->treefile_parser = json_parser_new ();
auto json = self->treefile_rs->get_json_string()
if (!json_parser_load_from_data (self->treefile_parser, json.data(), json.length(), error))
return FALSE;
self->treefile_rootval = json_parser_get_root (self->treefile_parser);
if (!JSON_NODE_HOLDS_OBJECT (self->treefile_rootval))
return glnx_throw (error, "Treefile root is not an object");
self->treefile = json_node_get_object (self->treefile_rootval);
self->treespec = rpmostree_composeutil_get_treespec (self->corectx,
self->treefile_rs,
self->treefile,
TRUE,
error);
*out_context = util::move_nullify (self);
return TRUE;
}
static gboolean
impl_rojig_build (RpmOstreeRojigCompose *self,
const char *outdir,
gboolean *out_changed,
GCancellable *cancellable,
GError **error)
{
*out_changed = FALSE;
if (getuid () != 0)
{
g_printerr ("NOTICE: Running this command as non-root is currently known not to work completely.\n");
g_printerr ("NOTICE: Proceeding anyways.\n");
g_usleep (3 * G_USEC_PER_SEC);
}
const char *rojig_name = ror_treefile_get_rojig_name (self->treefile_rs);
if (!rojig_name)
return glnx_throw (error, "No `rojig` entry in manifest");
g_autoptr(GKeyFile) tsk = g_key_file_new ();
const char *rojig_output_repo_id = "rpmostree-rojig-output";
g_autofree char *rojig_spec = g_strconcat (rojig_output_repo_id, ":", rojig_name, NULL);
g_key_file_set_string (tsk, "tree", "rojig", rojig_spec);
{ const char *repos[] = { rojig_output_repo_id, };
g_key_file_set_string_list (tsk, "tree", "repos", (const char*const*)repos, 1);
}
g_autoptr(RpmOstreeTreespec) treespec = rpmostree_treespec_new_from_keyfile (tsk, error);
if (!treespec)
return FALSE;
g_autoptr(RpmOstreeContext) corectx = rpmostree_context_new_compose (self->cachedir_dfd, self->repo);
DnfContext *dnfctx = rpmostree_context_get_dnf (corectx);
if (!glnx_shutil_mkdir_p_at (self->workdir_dfd, "rojig-repos", 0755, cancellable, error))
return FALSE;
{ g_autofree char *repopath = g_strconcat ("rojig-repos/", rojig_output_repo_id, ".repo", NULL);
g_autofree char *repo_contents = g_strdup_printf ("[%s]\n"
"baseurl=file://%s\n"
"gpgcheck=0\n",
rojig_output_repo_id,
outdir);
if (!glnx_file_replace_contents_at (self->workdir_dfd, repopath,
(guint8*)repo_contents, -1, (GLnxFileReplaceFlags)0,
cancellable, error))
return FALSE;
}
g_autofree char *reposdir_abspath = glnx_fdrel_abspath (self->workdir_dfd, "rojig-repos");
dnf_context_set_repo_dir (dnfctx, reposdir_abspath);
rpmostree_context_set_treespec (corectx, treespec);
if (!rpmostree_context_setup (corectx, NULL, NULL, cancellable, error))
return FALSE;
if (!rpmostree_context_prepare_rojig (corectx, TRUE, cancellable, error))
return FALSE;
DnfPackage *rojig_pkg = rpmostree_context_get_rojig_pkg (corectx);
if (rojig_pkg)
{
g_print ("Previous rojig: %s\n", dnf_package_get_nevra (rojig_pkg));
self->previous_version = g_strdup (dnf_package_get_version (rojig_pkg));
self->previous_inputhash = g_strdup (rpmostree_context_get_rojig_inputhash (corectx));
}
else
{
g_print ("No previous rojig package found: %s\n", rojig_name);
}
/* Set this early here, so we only have to set it one more time in the
* complete exit path too.
*/
const char rootfs_name[] = "rootfs.tmp";
if (!glnx_shutil_rm_rf_at (self->workdir_dfd, rootfs_name, cancellable, error))
return FALSE;
if (mkdirat (self->workdir_dfd, rootfs_name, 0755) < 0)
return glnx_throw_errno_prefix (error, "mkdirat(%s)", rootfs_name);
if (!glnx_opendirat (self->workdir_dfd, rootfs_name, TRUE,
&self->rootfs_dfd, error))
return FALSE;
g_autofree char *next_version = NULL;
if (json_object_has_member (self->treefile, "automatic-version-prefix") &&
/* let --add-metadata-string=version=... take precedence */
!g_hash_table_contains (self->metadata, OSTREE_COMMIT_META_KEY_VERSION))
{
const char *ver_prefix =
_rpmostree_jsonutil_object_require_string_member (self->treefile, "automatic-version-prefix", error);
if (!ver_prefix)
return FALSE;
const char *ver_suffix = NULL;
if (!_rpmostree_jsonutil_object_get_optional_string_member (self->treefile, "automatic-version-suffix", &ver_suffix, error))
return FALSE;
next_version = _rpmostree_util_next_version (ver_prefix, ver_suffix, self->previous_version, error);
if (!next_version)
return FALSE;
g_hash_table_insert (self->metadata, g_strdup (OSTREE_COMMIT_META_KEY_VERSION),
g_variant_ref_sink (g_variant_new_string (next_version)));
}
else
{
auto v = (GVariant *)g_hash_table_lookup (self->metadata, OSTREE_COMMIT_META_KEY_VERSION);
if (v)
{
g_assert (g_variant_is_of_type (v, G_VARIANT_TYPE_STRING));
next_version = g_variant_dup_string (v, NULL);
}
}
/* Download rpm-md repos, packages, do install */
g_autofree char *new_inputhash = NULL;
{ gboolean unmodified = FALSE;
if (!install_packages (self, opt_force_commit ? NULL : &unmodified,
&new_inputhash, cancellable, error))
return FALSE;
gboolean is_dry_run = opt_dry_run || opt_download_only;
if (unmodified)
{
const char *force_nocache_msg = "; use --force-commit to override";
g_print ("No apparent changes since previous commit%s\n",
is_dry_run ? "." : force_nocache_msg);
/* Note early return */
return TRUE;
}
else if (is_dry_run)
{
g_print ("--dry-run complete");
g_print ("; exiting\n");
/* Note early return */
return TRUE;
}
}
/* Bind metadata from the libdnf context */
if (!g_hash_table_contains (self->metadata, "rpmostree.rpmmd-repos"))
{
g_hash_table_insert (self->metadata, g_strdup ("rpmostree.rpmmd-repos"),
rpmostree_context_get_rpmmd_repo_commit_metadata (self->corectx));
}
/* Destroy this now so the libdnf stack won't have any references
* into the filesystem before we manipulate it.
*/
g_clear_object (&self->corectx);
if (g_strcmp0 (g_getenv ("RPM_OSTREE_BREAK"), "post-yum") == 0)
return FALSE;
/* Start postprocessing */
if (!rpmostree_treefile_postprocessing (self->rootfs_dfd, self->treefile_rs, self->treefile,
next_version, TRUE,
cancellable, error))
return glnx_prefix_error (error, "Postprocessing");
/* Until here, we targeted "rootfs.tmp" in the working directory. Most
* user-configured postprocessing has run. Now, we need to perform required
* conversions like handling /boot. We generate a new directory "rootfs" that
* has just what we want using "rootfs.tmp", as a source. This implicitly
* discards anything else that happens to be in rootfs.tmp, like the `/dev`
* nodes we create for example.
*/
const char final_rootfs_name[] = "rootfs";
if (!glnx_shutil_rm_rf_at (self->workdir_dfd, final_rootfs_name, cancellable, error))
return FALSE;
if (!glnx_ensure_dir (self->workdir_dfd, final_rootfs_name, 0755, error))
return FALSE;
{ glnx_autofd int target_rootfs_dfd = -1;
if (!glnx_opendirat (self->workdir_dfd, final_rootfs_name, TRUE,
&target_rootfs_dfd, error))
return FALSE;
if (!rpmostree_prepare_rootfs_for_commit (self->rootfs_dfd, target_rootfs_dfd,
self->treefile,
cancellable, error))
return glnx_prefix_error (error, "Preparing rootfs for commit");
glnx_close_fd (&self->rootfs_dfd);
/* Remove the old root, then retarget rootfs_dfd to the final one */
if (!glnx_shutil_rm_rf_at (self->workdir_dfd, rootfs_name, cancellable, error))
return FALSE;
self->rootfs_dfd = glnx_steal_fd (&target_rootfs_dfd);
}
/* Insert our input hash */
g_hash_table_replace (self->metadata, g_strdup ("rpmostree.inputhash"),
g_variant_ref_sink (g_variant_new_string (new_inputhash)));
*out_changed = TRUE;
return TRUE;
}
/* Perform required postprocessing, and invoke rpmostree_compose_commit(). */
static gboolean
impl_write_rojig (RpmOstreeRojigCompose *self,
const char *outdir,
GCancellable *cancellable,
GError **error)
{
g_auto(GVariantBuilder) composemeta_builder;
g_variant_builder_init (&composemeta_builder, G_VARIANT_TYPE ("a{sv}"));
gboolean selinux = TRUE;
if (!_rpmostree_jsonutil_object_get_optional_boolean_member (self->treefile, "selinux", &selinux, error))
return FALSE;
/* Convert metadata hash to GVariant */
g_autoptr(GVariant) metadata = rpmostree_composeutil_finalize_metadata (self->metadata, self->rootfs_dfd, error);
if (!metadata)
return FALSE;
if (!rpmostree_rootfs_postprocess_common (self->rootfs_dfd, cancellable, error))
return FALSE;
if (!rpmostree_postprocess_final (self->rootfs_dfd, self->treefile, TRUE,
cancellable, error))
return FALSE;
if (self->treefile)
{
rpmostreecxx::check_passwd_group_entries (self->repo, self->rootfs_dfd, self->treefile_rs, "");
}
if (!ostree_repo_prepare_transaction (self->repo, NULL, cancellable, error))
return FALSE;
/* The penultimate step, just basically `ostree commit` */
g_autofree char *new_revision = NULL;
if (!rpmostree_compose_commit (self->rootfs_dfd, self->repo, NULL,
metadata, NULL, selinux, self->devino_cache,
&new_revision, cancellable, error))
return FALSE;
const char *rojig_spec_path = ror_treefile_get_rojig_spec_path (self->treefile_rs);
if (!rojig_spec_path)
return glnx_throw (error, "treefile has no rojig section");
if (!rpmostree_commit2rojig (self->repo, self->pkgcache_repo, new_revision,
self->workdir_dfd, rojig_spec_path,
outdir, cancellable, error))
return FALSE;
g_autoptr(GVariant) new_commit = NULL;
if (!ostree_repo_load_commit (self->repo, new_revision, &new_commit,
NULL, error))
return FALSE;
OstreeRepoTransactionStats stats = { 0, };
if (!ostree_repo_commit_transaction (self->repo, &stats, cancellable, error))
return glnx_prefix_error (error, "Commit");
if (!rpmostree_composeutil_write_composejson (self->repo,
opt_write_composejson_to, &stats,
new_revision, new_commit, &composemeta_builder,
cancellable, error))
return FALSE;
return TRUE;
}
gboolean
rpmostree_compose_builtin_rojig (int argc,
char **argv,
RpmOstreeCommandInvocation *invocation,
GCancellable *cancellable,
GError **error)
{
g_autoptr(GOptionContext) context = g_option_context_new ("TREEFILE OUTDIR");
if (!rpmostree_option_context_parse (context,
rojig_option_entries,
&argc, &argv,
invocation,
cancellable,
NULL, NULL, NULL, NULL, NULL,
error))
return FALSE;
if (argc < 3)
{
rpmostree_usage_error (context, "TREEFILE and OUTDIR must be specified", error);
return FALSE;
}
const char *treefile_path = argv[1];
const char *outdir = argv[2];
g_autoptr(RpmOstreeRojigCompose) self = NULL;
if (!rpm_ostree_rojig_compose_new (treefile_path, &self, cancellable, error))
return FALSE;
g_assert (self); /* Pacify static analysis */
gboolean changed;
if (!impl_rojig_build (self, outdir, &changed, cancellable, error))
return FALSE;
if (changed)
{
/* Do the ostree commit, then generate rojig RPM */
if (!impl_write_rojig (self, outdir, cancellable, error))
return FALSE;
}
return TRUE;
}

1
src/app/rpmostree-compose-builtin-tree.cxx
View File

@ -42,7 +42,6 @@
#include "rpmostree-composeutil.h"
#include "rpmostree-core.h"
#include "rpmostree-json-parsing.h"
#include "rpmostree-rojig-build.h"
#include "rpmostree-postprocess.h"
#include "rpmostree-package-variants.h"
#include "rpmostree-libbuiltin.h"

1
src/app/rpmostree-compose-builtins.h
View File

@ -27,7 +27,6 @@
G_BEGIN_DECLS
gboolean rpmostree_compose_builtin_tree (int argc, char **argv, RpmOstreeCommandInvocation *invocation, GCancellable *cancellable, GError **error);
gboolean rpmostree_compose_builtin_rojig (int argc, char **argv, RpmOstreeCommandInvocation *invocation, GCancellable *cancellable, GError **error);
gboolean rpmostree_compose_builtin_install (int argc, char **argv, RpmOstreeCommandInvocation *invocation, GCancellable *cancellable, GError **error);
gboolean rpmostree_compose_builtin_postprocess (int argc, char **argv, RpmOstreeCommandInvocation *invocation, GCancellable *cancellable, GError **error);
gboolean rpmostree_compose_builtin_commit (int argc, char **argv, RpmOstreeCommandInvocation *invocation, GCancellable *cancellable, GError **error);

1
src/app/rpmostree-composeutil.cxx
View File

@ -39,7 +39,6 @@
#include "rpmostree-util.h"
#include "rpmostree-core.h"
#include "rpmostree-json-parsing.h"
#include "rpmostree-rojig-build.h"
#include "rpmostree-package-variants.h"
#include "rpmostree-libbuiltin.h"
#include "rpmostree-rpm-util.h"

110
src/app/rpmostree-ex-builtin-commit2rojig.cxx
View File

@ -1,110 +0,0 @@
/* -*- mode: C; c-file-style: "gnu"; indent-tabs-mode: nil; -*-
*
* Copyright (C) 2017 Colin Walters <walters@verbum.org>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published
* by the Free Software Foundation; either version 2 of the licence or (at
* your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General
* Public License along with this library; if not, write to the
* Free Software Foundation, Inc., 59 Temple Place, Suite 330,
* Boston, MA 02111-1307, USA.
*/
#include "config.h"
#include <string.h>
#include <glib-unix.h>
#include <gio/gunixoutputstream.h>
#include <libdnf/libdnf.h>
#include <sys/mount.h>
#include <stdio.h>
#include <libglnx.h>
#include <rpm/rpmmacro.h>
#include "rpmostree-ex-builtins.h"
#include "rpmostree-util.h"
#include "rpmostree-core.h"
#include "rpmostree-rojig-core.h"
#include "rpmostree-rojig-build.h"
#include "rpmostree-postprocess.h"
#include "rpmostree-libbuiltin.h"
#include "rpmostree-rpm-util.h"
#include "libglnx.h"
static char *opt_repo;
static char *opt_pkgcache_repo;
static gboolean opt_only_contentdir;
static GOptionEntry commit2rojig_option_entries[] = {
{ "repo", 0, 0, G_OPTION_ARG_STRING, &opt_repo, "OSTree repo", "REPO" },
{ "pkgcache-repo", 0, 0, G_OPTION_ARG_STRING, &opt_pkgcache_repo, "Pkgcache OSTree repo", "REPO" },
{ "only-contentdir", 0, 0, G_OPTION_ARG_NONE, &opt_only_contentdir, "Do not generate RPM, only output content directory", NULL },
{ NULL }
};
gboolean
rpmostree_ex_builtin_commit2rojig (int argc,
char **argv,
RpmOstreeCommandInvocation *invocation,
GCancellable *cancellable,
GError **error)
{
g_autoptr(GOptionContext) context = g_option_context_new ("REV TREEFILE OUTPUTDIR");
if (!rpmostree_option_context_parse (context,
commit2rojig_option_entries,
&argc, &argv,
invocation,
cancellable,
NULL, NULL, NULL, NULL, NULL,
error))
return FALSE;
if (argc != 4)
{
rpmostree_usage_error (context, "REV OIRPM-SPEC OUTPUTDIR are required", error);
return FALSE;
}
if (!(opt_repo && opt_pkgcache_repo))
{
rpmostree_usage_error (context, "--repo and --pkgcache-repo must be specified", error);
return FALSE;
}
const char *rev = argv[1];
const char *treefile = argv[2];
const char *outputdir = argv[3];
if (!g_str_has_prefix (outputdir, "/"))
return glnx_throw (error, "outputdir must be absolute");
g_autoptr(OstreeRepo) repo = ostree_repo_open_at (AT_FDCWD, opt_repo, cancellable, error);
if (!repo)
return FALSE;
g_autoptr(OstreeRepo) pkgcache_repo = ostree_repo_open_at (AT_FDCWD, opt_pkgcache_repo, cancellable, error);
if (!pkgcache_repo)
return FALSE;
g_auto(GLnxTmpDir) tmpd = { 0, };
if (!glnx_mkdtemp ("rpmostree-commit2rojig-XXXXXX", 0700, &tmpd, error))
return FALSE;
g_autoptr(RORTreefile) treefile_rs = ror_treefile_new (treefile, NULL, tmpd.fd, error);
const char *rojig_spec_path = ror_treefile_get_rojig_spec_path (treefile_rs);
if (!rojig_spec_path)
return glnx_throw (error, "treefile has no rojig section");
if (!rpmostree_commit2rojig (repo, pkgcache_repo, rev, tmpd.fd, rojig_spec_path, outputdir,
cancellable, error))
return FALSE;
return TRUE;
}

173
src/app/rpmostree-ex-builtin-rojig2commit.cxx
View File

@ -1,173 +0,0 @@
/* -*- mode: C; c-file-style: "gnu"; indent-tabs-mode: nil; -*-
*
* Copyright (C) 2017 Colin Walters <walters@verbum.org>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published
* by the Free Software Foundation; either version 2 of the licence or (at
* your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General
* Public License along with this library; if not, write to the
* Free Software Foundation, Inc., 59 Temple Place, Suite 330,
* Boston, MA 02111-1307, USA.
*/
#include "config.h"
#include <string.h>
#include <glib-unix.h>
#include <gio/gunixoutputstream.h>
#include <libdnf/libdnf.h>
#include <sys/mount.h>
#include <stdio.h>
#include <libglnx.h>
#include <rpm/rpmmacro.h>
#include "rpmostree-ex-builtins.h"
#include "rpmostree-util.h"
#include "rpmostree-core.h"
#include "rpmostree-rojig-assembler.h"
#include "rpmostree-postprocess.h"
#include "rpmostree-libbuiltin.h"
#include "rpmostree-rpm-util.h"
#include "libglnx.h"
static char *opt_repo;
static char *opt_rpmmd_reposdir;
static char *opt_releasever;
static char **opt_enable_rpmmdrepo;
static char *opt_oirpm_version;
static GOptionEntry rojig2commit_option_entries[] = {
{ "repo", 0, 0, G_OPTION_ARG_STRING, &opt_repo, "OSTree repo", "REPO" },
{ "rpmmd-reposd", 'd', 0, G_OPTION_ARG_STRING, &opt_rpmmd_reposdir, "Path to yum.repos.d (rpmmd) config directory", "PATH" },
{ "enablerepo", 'e', 0, G_OPTION_ARG_STRING_ARRAY, &opt_enable_rpmmdrepo, "Enable rpm-md repo with id ID", "ID" },
{ "releasever", 0, 0, G_OPTION_ARG_STRING, &opt_releasever, "Value for $releasever", "RELEASEVER" },
{ "oirpm-version", 'V', 0, G_OPTION_ARG_STRING, &opt_oirpm_version, "Use this specific version of OIRPM", "VERSION" },
{ NULL }
};
typedef struct {
OstreeRepo *repo;
GLnxTmpDir tmpd;
RpmOstreeContext *ctx;
} RpmOstreeRojig2CommitContext;
static void
rpm_ostree_rojig2commit_context_free (RpmOstreeRojig2CommitContext *ctx)
{
g_clear_object (&ctx->repo);
(void) glnx_tmpdir_delete (&ctx->tmpd, NULL, NULL);
g_free (ctx);
}
G_DEFINE_AUTOPTR_CLEANUP_FUNC(RpmOstreeRojig2CommitContext, rpm_ostree_rojig2commit_context_free)
/* Initialize a context for converting a rojig to a commit.
*/
static gboolean
rpm_ostree_rojig2commit_context_new (RpmOstreeRojig2CommitContext **out_context,
GCancellable *cancellable,
GError **error)
{
g_autoptr(RpmOstreeRojig2CommitContext) self = g_new0 (RpmOstreeRojig2CommitContext, 1);
self->repo = ostree_repo_open_at (AT_FDCWD, opt_repo, cancellable, error);
if (!self->repo)
return FALSE;
/* Our workdir lives in the repo for command line testing */
if (!glnx_mkdtempat (ostree_repo_get_dfd (self->repo),
"tmp/rpmostree-rojig-XXXXXX", 0700, &self->tmpd, error))
return FALSE;
self->ctx = rpmostree_context_new_compose (self->tmpd.fd, self->repo);
DnfContext *dnfctx = rpmostree_context_get_dnf (self->ctx);
if (opt_rpmmd_reposdir)
dnf_context_set_repo_dir (dnfctx, opt_rpmmd_reposdir);
*out_context = util::move_nullify (self);
return TRUE;
}
static gboolean
impl_rojig2commit (RpmOstreeRojig2CommitContext *self,
const char *rojig_id,
GCancellable *cancellable,
GError **error)
{
g_autoptr(GKeyFile) tsk = g_key_file_new ();
g_key_file_set_string (tsk, "tree", "rojig", rojig_id);
if (opt_oirpm_version)
g_key_file_set_string (tsk, "tree", "rojig-version", opt_oirpm_version);
if (opt_releasever)
g_key_file_set_string (tsk, "tree", "releasever", opt_releasever);
if (opt_enable_rpmmdrepo)
g_key_file_set_string_list (tsk, "tree", "repos",
(const char *const*)opt_enable_rpmmdrepo,
g_strv_length (opt_enable_rpmmdrepo));
g_autoptr(RpmOstreeTreespec) treespec = rpmostree_treespec_new_from_keyfile (tsk, error);
if (!treespec)
return FALSE;
/* We're also "pure" rojig - this adds assertions that we don't depsolve for example */
rpmostree_context_set_treespec (self->ctx, treespec);
if (!rpmostree_context_setup (self->ctx, NULL, NULL, cancellable, error))
return FALSE;
if (!rpmostree_context_prepare_rojig (self->ctx, FALSE, cancellable, error))
return FALSE;
gboolean rojig_changed;
if (!rpmostree_context_execute_rojig (self->ctx, &rojig_changed, cancellable, error))
return FALSE;
return TRUE;
}
gboolean
rpmostree_ex_builtin_rojig2commit (int argc,
char **argv,
RpmOstreeCommandInvocation *invocation,
GCancellable *cancellable,
GError **error)
{
g_autoptr(GOptionContext) context = g_option_context_new ("REPOID:OIRPM-NAME");
if (!rpmostree_option_context_parse (context,
rojig2commit_option_entries,
&argc, &argv,
invocation,
cancellable,
NULL, NULL, NULL, NULL, NULL,
error))
return FALSE;
if (argc != 2)
{
rpmostree_usage_error (context, "REPOID:OIRPM-NAME is required", error);
return FALSE;
}
if (!opt_repo)
{
rpmostree_usage_error (context, "--repo must be specified", error);
return FALSE;
}
const char *oirpm = argv[1];
g_autoptr(RpmOstreeRojig2CommitContext) self = NULL;
if (!rpm_ostree_rojig2commit_context_new (&self, cancellable, error))
return FALSE;
if (!impl_rojig2commit (self, oirpm, cancellable, error))
return FALSE;
return TRUE;
}

4
src/app/rpmostree-ex-builtins.h
View File

@ -32,10 +32,6 @@ G_BEGIN_DECLS
BUILTINPROTO(unpack);
BUILTINPROTO(apply_live);
#ifdef BUILDOPT_ROJIG
BUILTINPROTO(commit2rojig);
BUILTINPROTO(rojig2commit);
#endif
BUILTINPROTO(history);
BUILTINPROTO(initramfs_etc);

44
src/daemon/rpmostree-sysroot-upgrader.cxx
View File

@ -482,51 +482,7 @@ rpmostree_sysroot_upgrader_pull_base (RpmOstreeSysrootUpgrader *self,
break;
case RPMOSTREE_REFSPEC_TYPE_ROJIG:
{
#ifdef BUILDOPT_ROJIG
// Not implemented yet, though we could do a query for the provides
if (override_commit)
return glnx_throw (error, "Specifying commit overrides for rojig:// is not implemented yet");
g_autoptr(GKeyFile) tsk = g_key_file_new ();
g_key_file_set_string (tsk, "tree", "rojig", refspec);
const char *rojig_version = rpmostree_origin_get_rojig_version (self->origin);
if (rojig_version)
g_key_file_set_string (tsk, "tree", "rojig-version", rojig_version);
g_autoptr(RpmOstreeTreespec) treespec = rpmostree_treespec_new_from_keyfile (tsk, error);
if (!treespec)
return FALSE;
/* This context is currently different from one that may be created later
* for e.g. package layering. I can't think why we couldn't unify them,
* but for now it seems a lot simpler to keep the symmetry that
* rojig == ostree pull.
*/
g_autoptr(RpmOstreeContext) ctx =
rpmostree_context_new_client (self->repo);
/* We use / as a source root mostly so we get $releasever from it so
* things work out of the box. That said this is kind of wrong and we'll
* really need a way for users to configure a different releasever when
* e.g. rebasing across majors.
*/
rpmostree_context_set_treespec (ctx, treespec);
if (!rpmostree_context_setup (ctx, NULL, "/", cancellable, error))
return FALSE;
/* We're also "pure" rojig - this adds assertions that we don't depsolve for example */
if (!rpmostree_context_prepare_rojig (ctx, FALSE, cancellable, error))
return FALSE;
DnfPackage *rojig_pkg = rpmostree_context_get_rojig_pkg (ctx);
new_base_rev = g_strdup (rpmostree_context_get_rojig_checksum (ctx));
gboolean rojig_changed; /* Currently unused */
if (!rpmostree_context_execute_rojig (ctx, &rojig_changed, cancellable, error))
return FALSE;
if (rojig_changed)
rpmostree_origin_set_rojig_description (self->origin, rojig_pkg);
#else
return glnx_throw (error, "rojig is not supported in this build of rpm-ostree");
#endif
}
}

1
src/libpriv/rpmostree-core-private.h
View File

@ -23,7 +23,6 @@
#include <optional>
#include "libglnx.h"
#include "rpmostree-rojig-core.h"
#include "rpmostree-core.h"
#include "rpmostree-output.h"
#include "rpmostree-cxxrs.h"

105
src/libpriv/rpmostree-core.cxx
View File

@ -35,7 +35,6 @@
#include <set>
#include "rpmostree-core-private.h"
#include "rpmostree-rojig-core.h"
#include "rpmostree-postprocess.h"
#include "rpmostree-rpm-util.h"
#include "rpmostree-scripts.h"
@ -1782,104 +1781,6 @@ add_remaining_pkgcache_pkgs (RpmOstreeContext *self,
return TRUE;
}
static char *
parse_provided_checksum (const char *provide_data,
GError **error)
{
if (*provide_data != '(')
return (char*)glnx_null_throw (error, "Expected '('");
provide_data++;
const char *closeparen = strchr (provide_data, ')');
if (!closeparen)
return (char*)glnx_null_throw (error, "Expected ')'");
g_autofree char *ret = g_strndup (provide_data, closeparen - provide_data);
if (strlen (ret) != OSTREE_SHA256_STRING_LEN)
return (char*)glnx_null_throw (error, "Expected %u characters", OSTREE_SHA256_STRING_LEN);
return util::move_nullify (ret);
}
static gboolean
setup_rojig_state (RpmOstreeContext *self,
GError **error)
{
g_assert (self->rojig_spec);
g_assert (!self->rojig_pkg);
g_assert (!self->rojig_checksum);
g_autofree char *rojig_repoid = NULL;
g_autofree char *rojig_name = NULL;
{ const char *colon = strchr (self->rojig_spec, ':');
if (!colon)
return glnx_throw (error, "Invalid rojig spec '%s', expected repoid:name", self->rojig_spec);
rojig_repoid = g_strndup (self->rojig_spec, colon - self->rojig_spec);
rojig_name = g_strdup (colon + 1);
}
const char *rojig_version = NULL;
g_variant_dict_lookup (self->spec->dict, "rojig-version", "&s", &rojig_version);
hy_autoquery HyQuery query = hy_query_create (dnf_context_get_sack (self->dnfctx));
hy_query_filter (query, HY_PKG_REPONAME, HY_EQ, rojig_repoid);
hy_query_filter (query, HY_PKG_NAME, HY_EQ, rojig_name);
if (rojig_version)
hy_query_filter (query, HY_PKG_VERSION, HY_EQ, rojig_version);
g_autoptr(GPtrArray) pkglist = hy_query_run (query);
if (pkglist->len == 0)
{
if (!self->rojig_allow_not_found)
return glnx_throw (error, "Failed to find rojig package '%s'", self->rojig_spec);
else
{
/* Here we leave rojig_pkg NULL */
return TRUE;
}
}
g_ptr_array_sort (pkglist, compare_pkgs);
/* We use the last package in the array which should be newest */
self->rojig_pkg = static_cast<DnfPackage*>(g_object_ref (pkglist->pdata[pkglist->len-1]));
/* Iterate over provides directly to provide a nicer error on mismatch */
gboolean found_vprovide = FALSE;
g_autoptr(DnfReldepList) provides = dnf_package_get_provides (self->rojig_pkg);
const gint n_provides = dnf_reldep_list_count (provides);
for (int i = 0; i < n_provides; i++)
{
DnfReldep *provide = dnf_reldep_list_index (provides, i);
const char *provide_str = dnf_reldep_to_string (provide);
if (g_str_equal (provide_str, RPMOSTREE_ROJIG_PROVIDE_V5))
{
found_vprovide = TRUE;
}
else if (g_str_has_prefix (provide_str, RPMOSTREE_ROJIG_PROVIDE_COMMIT))
{
const char *rest = provide_str + strlen (RPMOSTREE_ROJIG_PROVIDE_COMMIT);
self->rojig_checksum = parse_provided_checksum (rest, error);
if (!self->rojig_checksum)
return glnx_prefix_error (error, "Invalid %s", provide_str);
}
else if (g_str_has_prefix (provide_str, RPMOSTREE_ROJIG_PROVIDE_INPUTHASH))
{
const char *rest = provide_str + strlen (RPMOSTREE_ROJIG_PROVIDE_INPUTHASH);
self->rojig_inputhash = parse_provided_checksum (rest, error);
if (!self->rojig_inputhash)
return glnx_prefix_error (error, "Invalid %s", provide_str);
}
}
if (!found_vprovide)
return glnx_throw (error, "Package '%s' does not have Provides: %s",
dnf_package_get_nevra (self->rojig_pkg), RPMOSTREE_ROJIG_PROVIDE_V5);
if (!self->rojig_checksum)
return glnx_throw (error, "Package '%s' does not have Provides: %s",
dnf_package_get_nevra (self->rojig_pkg), RPMOSTREE_ROJIG_PROVIDE_COMMIT);
return TRUE;
}
/* Return all the packages that match lockfile constraints. Multiple packages may be
* returned per NEVRA so that libsolv can respect e.g. repo costs. */
static GPtrArray*
@ -2272,11 +2173,7 @@ rpmostree_context_prepare (RpmOstreeContext *self,
}
}
if (self->rojig_spec)
{
if (!setup_rojig_state (self, error))
return FALSE;
}
g_assert (!self->rojig_spec);
if (!self->rojig_pure)
{

67
src/libpriv/rpmostree-importer.cxx
View File

@ -35,7 +35,6 @@
#include "rpmostree-unpacker-core.h"
#include "rpmostree-importer.h"
#include "rpmostree-core.h"
#include "rpmostree-rojig-assembler.h"
#include "rpmostree-rpm-util.h"
#include "rpmostree-util.h"
#include <rpm/rpmlib.h>
@ -744,62 +743,6 @@ unprivileged_filter_cb (OstreeRepo *repo,
return OSTREE_REPO_COMMIT_FILTER_ALLOW;
}
#ifdef BUILDOPT_ROJIG
static OstreeRepoCommitFilterResult
rojig_filter_cb (OstreeRepo *repo,
const char *path,
GFileInfo *file_info,
gpointer user_data)
{
RpmOstreeImporter *self = ((cb_data*)user_data)->self;
GError **error = ((cb_data*)user_data)->error;
const gboolean error_was_set = (error && *error != NULL);
if (error_was_set)
return OSTREE_REPO_COMMIT_FILTER_SKIP;
/* Sanity check that path is absolute */
g_assert (path != NULL);
g_assert (*path == '/');
/* HACK: special-case rpm's `/var/lib/rpm`, otherwise libsolv can get confused.
* See https://github.com/projectatomic/rpm-ostree/pull/290
*/
if (g_str_has_prefix (path, "/var/lib/rpm"))
return OSTREE_REPO_COMMIT_FILTER_SKIP;
self->n_rojig_total++;
if (g_file_info_get_file_type (file_info) != G_FILE_TYPE_DIRECTORY)
{
self->rojig_next_xattrs = NULL;
if (!rpmostree_rojig_assembler_xattr_lookup (self->rojig_xattr_table, path,
self->rojig_xattrs,
&self->rojig_next_xattrs,
error))
return OSTREE_REPO_COMMIT_FILTER_SKIP;
/* No xattrs means we don't need to import it */
if (!self->rojig_next_xattrs)
{
self->n_rojig_skipped++;
return OSTREE_REPO_COMMIT_FILTER_SKIP;
}
}
return OSTREE_REPO_COMMIT_FILTER_ALLOW;
}
static GVariant*
rojig_xattr_cb (OstreeRepo *repo,
const char *path,
GFileInfo *file_info,
gpointer user_data)
{
auto self = static_cast<RpmOstreeImporter *>(user_data);
return util::move_nullify (self->rojig_next_xattrs);
}
#endif
static GVariant*
xattr_cb (OstreeRepo *repo,
const char *path,
@ -864,11 +807,7 @@ import_rpm_to_repo (RpmOstreeImporter *self,
const gboolean unprivileged = ostree_repo_get_mode (repo) == OSTREE_REPO_MODE_BARE_USER_ONLY;
if (self->rojig_mode)
{
#ifdef BUILDOPT_ROJIG
filter = rojig_filter_cb;
#else
g_assert_not_reached ();
#endif
}
else if (unprivileged)
filter = unprivileged_filter_cb;
@ -884,13 +823,7 @@ import_rpm_to_repo (RpmOstreeImporter *self,
ostree_repo_commit_modifier_new (static_cast<OstreeRepoCommitModifierFlags>(modifier_flags), filter, &fdata, NULL);
if (self->rojig_mode)
{
#ifdef BUILDOPT_ROJIG
ostree_repo_commit_modifier_set_xattr_callback (modifier, rojig_xattr_cb,
NULL, self);
g_assert (self->sepolicy == NULL);
#else
g_assert_not_reached ();
#endif
}
else
{

630
src/libpriv/rpmostree-rojig-assembler.cxx
View File

@ -1,630 +0,0 @@
/* -*- mode: C; c-file-style: "gnu"; indent-tabs-mode: nil; -*-
*
* Copyright (C) 2017 Red Hat, Inc.
*
* Licensed under the GNU Lesser General Public License Version 2.1
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
#include "config.h"
#include <gio/gunixinputstream.h>
#include "rpmostree-libarchive-input-stream.h"
#include "rpmostree-unpacker-core.h"
#include "rpmostree-rojig-assembler.h"
#include "rpmostree-core.h"
#include "rpmostree-rpm-util.h"
#include <rpm/rpmlib.h>
#include <rpm/rpmlog.h>
#include <rpm/rpmfi.h>
#include <rpm/rpmts.h>
#include <archive.h>
#include <archive_entry.h>
#include <string.h>
#include <stdlib.h>
typedef enum {
STATE_COMMIT,
STATE_DIRMETA,
STATE_DIRTREE,
STATE_NEW_CONTENTIDENT,
STATE_NEW,
STATE_XATTRS_TABLE,
STATE_XATTRS_PKG,
} RojigAssemblerState;
static gboolean
throw_libarchive_error (GError **error,
struct archive *a)
{
return glnx_throw (error, "%s", archive_error_string (a));
}
typedef GObjectClass RpmOstreeRojigAssemblerClass;
struct RpmOstreeRojigAssembler
{
GObject parent_instance;
RojigAssemblerState state;
DnfPackage *pkg;
GVariant *commit;
GVariant *meta;
char *checksum;
GVariant *xattrs_table;
struct archive *archive;
struct archive_entry *next_entry;
int fd;
};
G_DEFINE_TYPE(RpmOstreeRojigAssembler, rpmostree_rojig_assembler, G_TYPE_OBJECT)
static void
rpmostree_rojig_assembler_finalize (GObject *object)
{
RpmOstreeRojigAssembler *self = (RpmOstreeRojigAssembler*)object;
if (self->archive)
archive_read_free (self->archive);
g_clear_pointer (&self->commit, (GDestroyNotify)g_variant_unref);
g_clear_pointer (&self->meta, (GDestroyNotify)g_variant_unref);
g_free (self->checksum);
g_clear_object (&self->pkg);
g_clear_pointer (&self->xattrs_table, (GDestroyNotify)g_variant_unref);
glnx_close_fd (&self->fd);
G_OBJECT_CLASS (rpmostree_rojig_assembler_parent_class)->finalize (object);
}
static void
rpmostree_rojig_assembler_class_init (RpmOstreeRojigAssemblerClass *klass)
{
GObjectClass *gobject_class = G_OBJECT_CLASS (klass);
gobject_class->finalize = rpmostree_rojig_assembler_finalize;
}
static void
rpmostree_rojig_assembler_init (RpmOstreeRojigAssembler *self)
{
self->fd = -1;
}
/*
* rpmostree_rojig_assembler_new_take_fd:
* @fd: Fd, ownership is taken
* @pkg: (optional): Package reference, used for metadata
* @error: error
*
* Create a new unpacker instance. The @pkg argument, if
* specified, will be inspected and metadata such as the
* origin repo will be added to the final commit.
*/
RpmOstreeRojigAssembler *
rpmostree_rojig_assembler_new_take_fd (int *fd,
DnfPackage *pkg,
GError **error)
{
glnx_fd_close int owned_fd = glnx_steal_fd (fd);
struct archive *archive = rpmostree_unpack_rpm2cpio (owned_fd, error);
if (archive == NULL)
return NULL;
auto ret = (RpmOstreeRojigAssembler *)g_object_new (RPMOSTREE_TYPE_ROJIG_ASSEMBLER, NULL);
ret->archive = util::move_nullify (archive);
ret->pkg = (DnfPackage*)(pkg ? g_object_ref (pkg) : NULL);
ret->fd = glnx_steal_fd (&owned_fd);
return util::move_nullify (ret);
}
static GVariant *
rojig_read_variant (const GVariantType *vtype,
struct archive *a,
struct archive_entry *entry,
GCancellable *cancellable,
GError **error)
{
const char *path = archive_entry_pathname (entry);
const struct stat *stbuf = archive_entry_stat (entry);
if (!S_ISREG (stbuf->st_mode))
return (GVariant*)glnx_null_throw (error, "Expected regular file for entry: %s", path);
if (!rpmostree_check_size_within_limit (stbuf->st_size, OSTREE_MAX_METADATA_SIZE,
path, error))
return NULL;
g_assert_cmpint (stbuf->st_size, >=, 0);
const size_t total = stbuf->st_size;
g_autofree guint8* buf = (guint8*)g_malloc (total);
size_t bytes_read = 0;
while (bytes_read < total)
{
ssize_t r = archive_read_data (a, buf + bytes_read, total - bytes_read);
if (r < 0)
return throw_libarchive_error (error, a), nullptr;
if (r == 0)
break;
bytes_read += r;
}
g_assert_cmpint (bytes_read, ==, total)
;
/* Need to take ownership once now, then pass it as a parameter twice */
guint8* buf_owned = util::move_nullify (buf);
return g_variant_new_from_data (vtype, buf_owned, bytes_read, FALSE, g_free, buf_owned);
}
/* Remove leading prefix */
static const char *
peel_entry_pathname (struct archive_entry *entry,
GError **error)
{
const char *pathname = archive_entry_pathname (entry);
static const char prefix[] = "./usr/lib/ostree-jigdo/";
if (!g_str_has_prefix (pathname, prefix))
return (const char*)glnx_null_throw (error, "Entry does not have prefix '%s': %s", prefix, pathname);
pathname += strlen (prefix);
const char *nextslash = strchr (pathname, '/');
if (!nextslash)
return (const char*)glnx_null_throw (error, "Missing subdir in %s", pathname);
return nextslash+1;
}
static gboolean
rojig_next_entry (RpmOstreeRojigAssembler *self,
gboolean *out_eof,
struct archive_entry **out_entry,
GCancellable *cancellable,
GError **error)
{
if (g_cancellable_set_error_if_cancelled (cancellable, error))
return FALSE;
*out_eof = FALSE;
if (self->next_entry)
{
*out_entry = util::move_nullify (self->next_entry);
return TRUE; /* 🔚 Early return */
}
/* Loop, skipping non-regular files */
struct archive_entry *entry = NULL;
while (TRUE)
{
int r = archive_read_next_header (self->archive, &entry);
if (r == ARCHIVE_EOF)
{
*out_eof = TRUE;
return TRUE; /* 🔚 Early return */
}
if (r != ARCHIVE_OK)
return throw_libarchive_error (error, self->archive);
const struct stat *stbuf = archive_entry_stat (entry);
/* We only have regular files, ignore intermediate dirs */
if (!S_ISREG (stbuf->st_mode))
continue;
/* Otherwise we're done */
break;
}
*out_eof = FALSE;
*out_entry = entry; /* Owned by archive */
return TRUE;
}
static struct archive_entry *
rojig_require_next_entry (RpmOstreeRojigAssembler *self,
GCancellable *cancellable,
GError **error)
{
gboolean eof = FALSE;
struct archive_entry *entry = NULL;
if (!rojig_next_entry (self, &eof, &entry, cancellable, error))
return FALSE;
if (eof)
return (decltype(entry))glnx_null_throw (error, "Unexpected end of archive");
return entry;
}
static char *
parse_checksum_from_pathname (const char *pathname,
GError **error)
{
/* We have an extra / */
if (strlen (pathname) != OSTREE_SHA256_STRING_LEN + 1)
return (char*)glnx_null_throw (error, "Invalid checksum path: %s", pathname);
g_autoptr(GString) buf = g_string_new ("");
g_string_append_len (buf, pathname, 2);
g_string_append (buf, pathname+3);
return g_string_free (util::move_nullify (buf), FALSE);
}
/* First step: read metadata: the commit object and its metadata, suitable for
* GPG verification.
*/
gboolean
rpmostree_rojig_assembler_read_meta (RpmOstreeRojigAssembler *self,
char **out_checksum,
GVariant **out_commit,
GVariant **out_detached_meta,
GCancellable *cancellable,
GError **error)
{
g_assert_cmpint (self->state, ==, STATE_COMMIT);
struct archive_entry *entry = rojig_require_next_entry (self, cancellable, error);
if (!entry)
return FALSE;
const char *entry_path = peel_entry_pathname (entry, error);
if (!entry_path)
return FALSE;
if (!g_str_has_prefix (entry_path, RPMOSTREE_ROJIG_COMMIT_DIR "/"))
return glnx_throw (error, "Unexpected entry: %s", entry_path);
entry_path += strlen (RPMOSTREE_ROJIG_COMMIT_DIR "/");
g_autofree char *checksum = parse_checksum_from_pathname (entry_path, error);
if (!checksum)
return FALSE;
g_autoptr(GVariant) commit = rojig_read_variant (OSTREE_COMMIT_GVARIANT_FORMAT,
self->archive, entry, cancellable, error);
g_autoptr(GVariant) meta = NULL;
g_autoptr(GChecksum) hasher = g_checksum_new (G_CHECKSUM_SHA256);
g_checksum_update (hasher, (const guint8*)g_variant_get_data (commit), g_variant_get_size (commit));
const char *actual_checksum = g_checksum_get_string (hasher);
if (!g_str_equal (checksum, actual_checksum))
return glnx_throw (error, "Checksum mismatch; described='%s' actual='%s'",
checksum, actual_checksum);
entry = rojig_require_next_entry (self, cancellable, error);
entry_path = peel_entry_pathname (entry, error);
if (!entry_path)
return FALSE;
if (g_str_equal (entry_path, RPMOSTREE_ROJIG_COMMIT_DIR "/meta"))
{
meta = rojig_read_variant (G_VARIANT_TYPE ("a{sv}"), self->archive, entry,
cancellable, error);
if (!meta)
return FALSE;
}
else
{
self->next_entry = entry; /* Stash for next call */
}
self->state = STATE_DIRMETA;
self->checksum = g_strdup (actual_checksum);
self->commit = g_variant_ref (commit);
self->meta = meta ? g_variant_ref (meta) : NULL;
*out_checksum = util::move_nullify (checksum);
*out_commit = util::move_nullify (commit);
*out_detached_meta = util::move_nullify (meta);
return TRUE;
}
static gboolean
process_contentident (RpmOstreeRojigAssembler *self,
OstreeRepo *repo,
struct archive_entry *entry,
const char *meta_pathname,
GCancellable *cancellable,
GError **error)
{
GLNX_AUTO_PREFIX_ERROR ("Processing content-identical", error);
/* Read the metadata variant, which has an array of checksum, metadata
* for regfile objects that have identical content.
*/
if (!g_str_has_suffix (meta_pathname, "/01meta"))
return glnx_throw (error, "Malformed contentident: %s", meta_pathname);
const char *contentident_id_start = meta_pathname + strlen (RPMOSTREE_ROJIG_NEW_CONTENTIDENT_DIR "/");
const char *slash = strchr (contentident_id_start, '/');
if (!slash)
return glnx_throw (error, "Malformed contentident: %s", meta_pathname);
// g_autofree char *contentident_id_str = g_strndup (contentident_id_start, slash - contentident_id_start);
g_autoptr(GVariant) meta = rojig_read_variant (RPMOSTREE_ROJIG_NEW_CONTENTIDENT_VARIANT_FORMAT,
self->archive, entry,
cancellable, error);
/* Read the content */
// FIXME match contentident_id
entry = rojig_require_next_entry (self, cancellable, error);
if (!entry)
return FALSE;
const char *content_pathname = peel_entry_pathname (entry, error);
if (!content_pathname)
return FALSE;
if (!g_str_has_suffix (content_pathname, "/05content"))
return glnx_throw (error, "Malformed contentident: %s", content_pathname);
const struct stat *stbuf = archive_entry_stat (entry);
/* Copy the data to a temporary file; a better optimization would be to write
* the data to the first object, then clone it, but that requires some
* more libostree API. As far as I can see, one can't reliably seek with
* libarchive; only some formats support it, and cpio isn't one of them.
*/
g_auto(GLnxTmpfile) tmpf = { 0, };
if (!glnx_open_anonymous_tmpfile (O_RDWR | O_CLOEXEC, &tmpf, error))
return FALSE;
const size_t total = stbuf->st_size;
const size_t bufsize = MIN (128*1024, total);
g_autofree guint8* buf = (guint8*)g_malloc (bufsize);
size_t bytes_read = 0;
while (bytes_read < total)
{
ssize_t r = archive_read_data (self->archive, buf, MIN (bufsize, total - bytes_read));
if (r < 0)
return throw_libarchive_error (error, self->archive);
if (r == 0)
break;
if (glnx_loop_write (tmpf.fd, buf, r) < 0)
return glnx_throw_errno_prefix (error, "write");
bytes_read += r;
}
g_assert_cmpint (bytes_read, ==, total);
g_clear_pointer (&buf, g_free);
const guint n = g_variant_n_children (meta);
for (guint i = 0; i < n; i++)
{
const char *checksum;
guint32 uid,gid,mode;
g_autoptr(GVariant) xattrs = NULL;
g_variant_get_child (meta, i, "(&suuu@a(ayay))", &checksum, &uid, &gid, &mode, &xattrs);
/* See if we already have this object */
gboolean has_object;
if (!ostree_repo_has_object (repo, OSTREE_OBJECT_TYPE_FILE, checksum,
&has_object, cancellable, error))
return FALSE;
if (has_object)
continue;
uid = GUINT32_FROM_BE (uid);
gid = GUINT32_FROM_BE (gid);
mode = GUINT32_FROM_BE (mode);
if (lseek (tmpf.fd, 0, SEEK_SET) < 0)
return glnx_throw_errno_prefix (error, "lseek");
g_autoptr(GInputStream) istream = g_unix_input_stream_new (tmpf.fd, FALSE);
/* Like _ostree_stbuf_to_gfileinfo() - TODO make that public with a
* better content writing API.
*/
g_autoptr(GFileInfo) finfo = g_file_info_new ();
g_file_info_set_attribute_uint32 (finfo, "standard::type", G_FILE_TYPE_REGULAR);
g_file_info_set_attribute_boolean (finfo, "standard::is-symlink", FALSE);
g_file_info_set_attribute_uint32 (finfo, "unix::uid", uid);
g_file_info_set_attribute_uint32 (finfo, "unix::gid", gid);
g_file_info_set_attribute_uint32 (finfo, "unix::mode", mode);
g_file_info_set_attribute_uint64 (finfo, "standard::size", total);
g_autoptr(GInputStream) objstream = NULL;
guint64 objlen;
if (!ostree_raw_file_to_content_stream (istream, finfo, xattrs, &objstream,
&objlen, cancellable, error))
return FALSE;
g_autofree guchar *csum = NULL;
if (!ostree_repo_write_content (repo, checksum, objstream, objlen, &csum,
cancellable, error))
return FALSE;
}
return TRUE;
}
static gboolean
state_transition (RpmOstreeRojigAssembler *self,
const char *pathname,
RojigAssemblerState new_state,
GError **error)
{
if (self->state > new_state)
return glnx_throw (error, "Unexpected state for path: %s", pathname);
self->state = new_state;
return TRUE;
}
/* Process new objects included in the rojigRPM */
gboolean
rpmostree_rojig_assembler_write_new_objects (RpmOstreeRojigAssembler *self,
OstreeRepo *repo,
GCancellable *cancellable,
GError **error)
{
GLNX_AUTO_PREFIX_ERROR ("Writing new objects", error);
g_assert_cmpint (self->state, ==, STATE_DIRMETA);
/* TODO sort objects in order for importing, verify we're not
* importing an unknown object.
*/
while (TRUE)
{
gboolean eof = FALSE;
struct archive_entry *entry = NULL;
if (!rojig_next_entry (self, &eof, &entry, cancellable, error))
return FALSE;
if (eof)
break;
g_assert (entry); /* Pacify static analysis */
const char *pathname = peel_entry_pathname (entry, error);
if (!pathname)
return FALSE;
if (g_str_has_prefix (pathname, RPMOSTREE_ROJIG_DIRMETA_DIR "/"))
{
if (!state_transition (self, pathname, STATE_DIRMETA, error))
return FALSE;
g_autofree char *checksum =
parse_checksum_from_pathname (pathname + strlen (RPMOSTREE_ROJIG_DIRMETA_DIR "/"), error);
if (!checksum)
return FALSE;
g_autoptr(GVariant) dirmeta = rojig_read_variant (OSTREE_DIRMETA_GVARIANT_FORMAT,
self->archive, entry,
cancellable, error);
g_autofree guint8*csum = NULL;
if (!ostree_repo_write_metadata (repo, OSTREE_OBJECT_TYPE_DIR_META,
checksum, dirmeta, &csum, cancellable, error))
return FALSE;
}
else if (g_str_has_prefix (pathname, RPMOSTREE_ROJIG_DIRTREE_DIR "/"))
{
if (!state_transition (self, pathname, STATE_DIRTREE, error))
return FALSE;
g_autofree char *checksum =
parse_checksum_from_pathname (pathname + strlen (RPMOSTREE_ROJIG_DIRTREE_DIR "/"), error);
if (!checksum)
return FALSE;
g_autoptr(GVariant) dirtree = rojig_read_variant (OSTREE_TREE_GVARIANT_FORMAT,
self->archive, entry,
cancellable, error);
g_autofree guint8*csum = NULL;
if (!ostree_repo_write_metadata (repo, OSTREE_OBJECT_TYPE_DIR_TREE,
checksum, dirtree, &csum, cancellable, error))
return FALSE;
}
else if (g_str_has_prefix (pathname, RPMOSTREE_ROJIG_NEW_CONTENTIDENT_DIR "/"))
{
if (!state_transition (self, pathname, STATE_NEW_CONTENTIDENT, error))
return FALSE;
if (!process_contentident (self, repo, entry, pathname, cancellable, error))
return FALSE;
}
else if (g_str_has_prefix (pathname, RPMOSTREE_ROJIG_NEW_DIR "/"))
{
if (!state_transition (self, pathname, STATE_NEW, error))
return FALSE;
g_autofree char *checksum =
parse_checksum_from_pathname (pathname + strlen (RPMOSTREE_ROJIG_NEW_DIR "/"), error);
if (!checksum)
return FALSE;
const struct stat *stbuf = archive_entry_stat (entry);
g_assert_cmpint (stbuf->st_size, >=, 0);
g_autoptr(GInputStream) archive_stream = _rpm_ostree_libarchive_input_stream_new (self->archive);
g_autofree guint8*csum = NULL;
if (!ostree_repo_write_content (repo, checksum, archive_stream,
stbuf->st_size, &csum, cancellable, error))
return FALSE;
}
else if (g_str_has_prefix (pathname, RPMOSTREE_ROJIG_XATTRS_DIR "/"))
{
self->next_entry = util::move_nullify (entry); /* Stash for next call */
break;
}
else
return glnx_throw (error, "Unexpected entry: %s", pathname);
}
return TRUE;
}
GVariant *
rpmostree_rojig_assembler_get_xattr_table (RpmOstreeRojigAssembler *self)
{
g_assert (self->xattrs_table);
return g_variant_ref (self->xattrs_table);
}
/* Loop over each package, returning its xattr set (as indexes into the xattr table) */
gboolean
rpmostree_rojig_assembler_next_xattrs (RpmOstreeRojigAssembler *self,
GVariant **out_objid_to_xattrs,
GCancellable *cancellable,
GError **error)
{
/* If we haven't loaded the xattr string table, do so */
if (self->state < STATE_XATTRS_TABLE)
{
struct archive_entry *entry = rojig_require_next_entry (self, cancellable, error);
if (!entry)
return FALSE;
const char *pathname = peel_entry_pathname (entry, error);
if (!pathname)
return FALSE;
if (!g_str_has_prefix (pathname, RPMOSTREE_ROJIG_XATTRS_TABLE))
return glnx_throw (error, "Unexpected entry: %s", pathname);
g_autoptr(GVariant) xattrs_table = rojig_read_variant (RPMOSTREE_ROJIG_XATTRS_TABLE_VARIANT_FORMAT,
self->archive, entry, cancellable, error);
if (!xattrs_table)
return FALSE;
g_assert (!self->xattrs_table);
self->xattrs_table = util::move_nullify (xattrs_table);
self->state = STATE_XATTRS_TABLE;
}
/* Init output variable for EOF state now */
*out_objid_to_xattrs = NULL;
/* Look for an xattr entry */
gboolean eof = FALSE;
struct archive_entry *entry = NULL;
if (!rojig_next_entry (self, &eof, &entry, cancellable, error))
return FALSE;
if (eof)
return TRUE; /* 🔚 Early return */
g_assert (entry); /* Pacify static analysis */
const char *pathname = peel_entry_pathname (entry, error);
if (!pathname)
return FALSE;
/* At this point there's nothing left besides xattrs, so throw if it doesn't
* match that filename pattern.
*/
if (!g_str_has_prefix (pathname, RPMOSTREE_ROJIG_XATTRS_PKG_DIR "/"))
return glnx_throw (error, "Unexpected entry: %s", pathname);
// const char *nevra = pathname + strlen (RPMOSTREE_ROJIG_XATTRS_PKG_DIR "/");
*out_objid_to_xattrs = rojig_read_variant (RPMOSTREE_ROJIG_XATTRS_PKG_VARIANT_FORMAT,
self->archive, entry, cancellable, error);
return TRUE;
}
/* Client side lookup for xattrs */
gboolean
rpmostree_rojig_assembler_xattr_lookup (GVariant *xattr_table,
const char *path,
GVariant *xattrs,
GVariant **out_xattrs,
GError **error)
{
int pos;
if (!rpmostree_variant_bsearch_str (xattrs, path, &pos))
{
const char *bn = glnx_basename (path);
if (!rpmostree_variant_bsearch_str (xattrs, bn, &pos))
{
// TODO add an "objects to skip" map; currently not found means
// "don't import"
*out_xattrs = NULL;
return TRUE;
/* rojigdata->caught_error = TRUE; */
/* return glnx_null_throw (&rojigdata->error, "Failed to find rojig xattrs for path '%s'", path); */
}
}
guint xattr_idx;
g_variant_get_child (xattrs, pos, "(&su)", NULL, &xattr_idx);
if (xattr_idx >= g_variant_n_children (xattr_table))
return glnx_throw (error, "Out of range rojig xattr index %u for path '%s'", xattr_idx, path);
*out_xattrs = g_variant_get_child_value (xattr_table, xattr_idx);
return TRUE;
}

73
src/libpriv/rpmostree-rojig-assembler.h
View File

@ -1,73 +0,0 @@
/* -*- mode: C; c-file-style: "gnu"; indent-tabs-mode: nil; -*-
*
* Copyright (C) 2015 Colin Walters <walters@verbum.org>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published
* by the Free Software Foundation; either version 2 of the licence or (at
* your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General
* Public License along with this library; if not, write to the
* Free Software Foundation, Inc., 59 Temple Place, Suite 330,
* Boston, MA 02111-1307, USA.
*/
#pragma once
#include "libglnx.h"
#include "rpmostree-rojig-core.h"
G_BEGIN_DECLS
typedef struct RpmOstreeRojigAssembler RpmOstreeRojigAssembler;
#define RPMOSTREE_TYPE_ROJIG_ASSEMBLER (rpmostree_rojig_assembler_get_type ())
#define RPMOSTREE_ROJIG_ASSEMBLER(inst) (G_TYPE_CHECK_INSTANCE_CAST ((inst), RPMOSTREE_TYPE_ROJIG_ASSEMBLER, RpmOstreeRojigAssembler))
#define RPMOSTREE_IS_ROJIG_ASSEMBLER(inst) (G_TYPE_CHECK_INSTANCE_TYPE ((inst), RPMOSTREE_TYPE_ROJIG_ASSEMBLER))
GType rpmostree_rojig_assembler_get_type (void);
G_DEFINE_AUTOPTR_CLEANUP_FUNC (RpmOstreeRojigAssembler, g_object_unref)
RpmOstreeRojigAssembler*
rpmostree_rojig_assembler_new_take_fd (int *fd,
DnfPackage *pkg, /* for metadata */
GError **error);
gboolean
rpmostree_rojig_assembler_read_meta (RpmOstreeRojigAssembler *rojig,
char **out_checksum,
GVariant **commit,
GVariant **detached_meta,
GCancellable *cancellable,
GError **error);
gboolean
rpmostree_rojig_assembler_write_new_objects (RpmOstreeRojigAssembler *rojig,
OstreeRepo *repo,
GCancellable *cancellable,
GError **error);
GVariant * rpmostree_rojig_assembler_get_xattr_table (RpmOstreeRojigAssembler *self);
gboolean
rpmostree_rojig_assembler_next_xattrs (RpmOstreeRojigAssembler *self,
GVariant **out_objid_to_xattrs,
GCancellable *cancellable,
GError **error);
gboolean
rpmostree_rojig_assembler_xattr_lookup (GVariant *xattr_table,
const char *path,
GVariant *xattrs,
GVariant **out_xattrs,
GError **error);
G_END_DECLS

1270
src/libpriv/rpmostree-rojig-build.cxx
View File

File diff suppressed because it is too large Load Diff

38
src/libpriv/rpmostree-rojig-build.h
View File

@ -1,38 +0,0 @@
/* -*- mode: C; c-file-style: "gnu"; indent-tabs-mode: nil; -*-
*
* Copyright (C) 2017 Red Hat, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published
* by the Free Software Foundation; either version 2 of the licence or (at
* your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General
* Public License along with this library; if not, write to the
* Free Software Foundation, Inc., 59 Temple Place, Suite 330,
* Boston, MA 02111-1307, USA.
*/
#pragma once
#include "libglnx.h"
#include "rpmostree-rojig-core.h"
G_BEGIN_DECLS
gboolean
rpmostree_commit2rojig (OstreeRepo *repo,
OstreeRepo *pkgcache_repo,
const char *commit,
int spec_dfd,
const char *spec,
const char *outputdir,
GCancellable *cancellable,
GError **error);
G_END_DECLS

321
src/libpriv/rpmostree-rojig-client.cxx
View File

@ -1,321 +0,0 @@
/* -*- mode: C; c-file-style: "gnu"; indent-tabs-mode: nil; -*-
*
* Copyright (C) 2017 Red Hat, Inc.
*
* Licensed under the GNU Lesser General Public License Version 2.1
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
/* This file contains the client-side portions of rojig that are "private"
* implementation detials of RpmOstreeContext. A better model down the line
* might be to have RpmOstreeRojigContext or so.
*/
#include "config.h"
#include <gio/gunixinputstream.h>
#include <gio/gunixoutputstream.h>
#include "rpmostree-rojig-assembler.h"
#include "rpmostree-core-private.h"
#include "rpmostree-rpm-util.h"
#include "rpmostree-output.h"
// For the rojig Requires parsing
#include <libdnf/dnf-reldep-list.h>
#include <string.h>
#include <stdlib.h>
static DnfPackage *
query_rojig_pkg (DnfContext *dnfctx,
const char *name_arch,
const char *evr,
GError **error)
{
hy_autoquery HyQuery query = hy_query_create (dnf_context_get_sack (dnfctx));
/* This changed in v4, we now go through the Provides: name(arch) */
const char *arch_start = strchr (name_arch, '(');
g_autofree char *name_owned = NULL;
const char *name;
if (arch_start)
{
name = name_owned = g_strndup (name_arch, arch_start - name_arch);
hy_query_filter (query, HY_PKG_PROVIDES, HY_EQ, name_arch);
}
else
name = name_arch;
hy_query_filter (query, HY_PKG_NAME, HY_EQ, name);
hy_query_filter (query, HY_PKG_EVR, HY_EQ, evr);
g_autoptr(GPtrArray) pkglist = hy_query_run (query);
if (pkglist->len == 0)
return (DnfPackage*)glnx_null_throw (error, "Failed to find package %s = %s", name_arch, evr);
return (DnfPackage*)g_object_ref (pkglist->pdata[0]);
}
static int
compare_pkgs (gconstpointer ap,
gconstpointer bp)
{
DnfPackage **a = (DnfPackage**)ap;
DnfPackage **b = (DnfPackage**)bp;
return dnf_package_cmp (*a, *b);
}
/* Walk over the list of cacheids for each package; if we have
* a cached rojig pkg with a different cacheid, invalidate it.
*/
static gboolean
invalidate_changed_cacheids (RpmOstreeContext *self,
DnfPackage *pkg,
GVariant *pkg_objid_to_xattrs,
guint *out_n_invalidated,
GCancellable *cancellable,
GError **error)
{
GLNX_AUTO_PREFIX_ERROR ("During rojig pkgcache invalidation", error);
OstreeRepo *pkgcache_repo = self->pkgcache_repo ?: self->ostreerepo;
const char *cacheid;
g_variant_get (pkg_objid_to_xattrs, "(&s@a(su))", &cacheid, NULL);
/* See if we have it cached */
g_autofree char *rojig_branch = rpmostree_get_rojig_branch_pkg (pkg);
g_autofree char *cached_rev = NULL;
if (!ostree_repo_resolve_rev (pkgcache_repo, rojig_branch, TRUE,
&cached_rev, error))
return FALSE;
/* Not cached? That's fine, on to the next */
if (!cached_rev)
return TRUE; /* Early return */
g_autoptr(GVariant) commit = NULL;
if (!ostree_repo_load_commit (pkgcache_repo, cached_rev, &commit, NULL, error))
return FALSE;
g_autoptr(GVariant) metadata = g_variant_get_child_value (commit, 0);
g_autoptr(GVariantDict) metadata_dict = g_variant_dict_new (metadata);
const char *current_cacheid = NULL;
g_variant_dict_lookup (metadata_dict, "rpmostree.rojig_cacheid", "&s", &current_cacheid);
if (g_strcmp0 (current_cacheid, cacheid))
{
if (!ostree_repo_set_ref_immediate (pkgcache_repo, NULL, rojig_branch, NULL,
cancellable, error))
return FALSE;
(*out_n_invalidated)++;
}
return TRUE;
}
/* Core logic for performing a rojig assembly client side. The high level flow is:
*
* - Download rpm-md
* - query for rojigRPM
* - query for rojigSet (dependencies of above)
* - download and parse rojigRPM
* - download and import rojigSet
* - commit all data to ostree
*/
gboolean
rpmostree_context_execute_rojig (RpmOstreeContext *self,
gboolean *out_changed,
GCancellable *cancellable,
GError **error)
{
OstreeRepo *repo = self->ostreerepo;
DnfPackage* oirpm_pkg = rpmostree_context_get_rojig_pkg (self);
const char *provided_commit = rpmostree_context_get_rojig_checksum (self);
DnfContext *dnfctx = rpmostree_context_get_dnf (self);
{ OstreeRepoCommitState commitstate;
gboolean has_commit;
if (!ostree_repo_has_object (repo, OSTREE_OBJECT_TYPE_COMMIT, provided_commit,
&has_commit, cancellable, error))
return FALSE;
if (has_commit)
{
if (!ostree_repo_load_commit (repo, provided_commit, NULL,
&commitstate, error))
return FALSE;
if (!(commitstate & OSTREE_REPO_COMMIT_STATE_PARTIAL))
{
*out_changed = FALSE;
return TRUE; /* 🔚 Early return */
}
}
}
rpmostree_output_message ("Updating to: %s:%s", dnf_package_get_reponame (oirpm_pkg), dnf_package_get_nevra (oirpm_pkg));
g_autoptr(GPtrArray) pkgs_required = g_ptr_array_new_with_free_func (g_object_unref);
/* Look at the Requires of the rojigRPM. Note that we don't want to do
* dependency resolution here - that's part of the whole idea, we're doing
* deterministic imaging.
*/
g_autoptr(DnfReldepList) requires = dnf_package_get_requires (oirpm_pkg);
const gint n_requires = dnf_reldep_list_count (requires);
Pool *pool = dnf_sack_get_pool (dnf_context_get_sack (dnfctx));
for (int i = 0; i < n_requires; i++)
{
DnfReldep *req = dnf_reldep_list_index (requires, i);
Id reqid = dnf_reldep_get_id (req);
if (!ISRELDEP (reqid))
continue;
Reldep *rdep = GETRELDEP (pool, reqid);
/* This is the core hack; we're searching for Requires that have exact '='
* versions. This assumes that the rpmbuild process won't inject such
* requirements.
*/
if (!(rdep->flags & REL_EQ))
continue;
/* Since v4 the server uses "Provides: name(arch) for archful */
const char *name_arch = pool_id2str (pool, rdep->name);
const char *evr = pool_id2str (pool, rdep->evr);
DnfPackage *pkg = query_rojig_pkg (dnfctx, name_arch, evr, error);
// FIXME: Possibly we shouldn't require a package to be in the repos if we
// already have it imported? This would help support downgrades if the
// repo owner has pruned.
if (!pkg)
return FALSE;
g_ptr_array_add (pkgs_required, g_object_ref (pkg));
}
g_ptr_array_sort (pkgs_required, compare_pkgs);
/* For now we first serially download the oirpm, but down the line we can do
* this async. Doing so will require putting more of the rojig logic into the
* core, so it knows not to import the rojigRPM.
*/
{ g_autoptr(GPtrArray) oirpm_singleton_pkglist = g_ptr_array_new ();
g_ptr_array_add (oirpm_singleton_pkglist, oirpm_pkg);
if (!rpmostree_context_set_packages (self, oirpm_singleton_pkglist, cancellable, error))
return FALSE;
}
if (!rpmostree_context_download (self, cancellable, error))
return FALSE;
glnx_fd_close int oirpm_fd = -1;
if (!rpmostree_context_consume_package (self, oirpm_pkg, &oirpm_fd, error))
return FALSE;
g_autoptr(RpmOstreeRojigAssembler) rojig = rpmostree_rojig_assembler_new_take_fd (&oirpm_fd, oirpm_pkg, error);
if (!rojig)
return FALSE;
g_autofree char *checksum = NULL;
g_autoptr(GVariant) commit = NULL;
g_autoptr(GVariant) commit_meta = NULL;
if (!rpmostree_rojig_assembler_read_meta (rojig, &checksum, &commit, &commit_meta,
cancellable, error))
return FALSE;
if (!g_str_equal (checksum, provided_commit))
return glnx_throw (error, "Package '%s' commit mismatch; Provides=%s, actual=%s",
dnf_package_get_nevra (oirpm_pkg), provided_commit, checksum);
g_printerr ("TODO implement GPG verification\n");
g_auto(RpmOstreeRepoAutoTransaction) txn = { 0, };
if (!rpmostree_repo_auto_transaction_start (&txn, repo, FALSE, cancellable, error))
return FALSE;
if (!ostree_repo_write_commit_detached_metadata (repo, checksum, commit_meta,
cancellable, error))
return FALSE;
/* Mark as partial until we're done */
if (!ostree_repo_mark_commit_partial (repo, checksum, TRUE, error))
return FALSE;
{ g_autofree guint8*csum = NULL;
if (!ostree_repo_write_metadata (repo, OSTREE_OBJECT_TYPE_COMMIT,
checksum, commit, &csum,
cancellable, error))
return FALSE;
}
if (!rpmostree_rojig_assembler_write_new_objects (rojig, repo, cancellable, error))
return FALSE;
if (!ostree_repo_commit_transaction (repo, NULL, cancellable, error))
return FALSE;
txn.initialized = FALSE;
/* Process the xattrs, including the cacheids before we compute what we need
* to download.
*/
g_autoptr(GHashTable) pkg_to_xattrs = g_hash_table_new_full (NULL, NULL,
(GDestroyNotify)g_object_unref,
(GDestroyNotify)g_variant_unref);
guint n_invalidated = 0;
for (guint i = 0; i < pkgs_required->len; i++)
{
auto pkg = static_cast<DnfPackage *>(pkgs_required->pdata[i]);
g_autoptr(GVariant) objid_to_xattrs = NULL;
if (!rpmostree_rojig_assembler_next_xattrs (rojig, &objid_to_xattrs, cancellable, error))
return FALSE;
if (!objid_to_xattrs)
return glnx_throw (error, "missing xattr entry: %s", dnf_package_get_name (pkg));
if (!invalidate_changed_cacheids (self, pkg, objid_to_xattrs,
&n_invalidated, cancellable, error))
return FALSE;
g_hash_table_insert (pkg_to_xattrs, g_object_ref (pkg), util::move_nullify (objid_to_xattrs));
}
/* And now, process the rojig set */
if (!rpmostree_context_set_packages (self, pkgs_required, cancellable, error))
return FALSE;
/* See what packages we need to import, print their size. TODO clarify between
* download/import.
*/
g_autoptr(GHashTable) pkgset_to_import = g_hash_table_new (NULL, NULL);
{ g_autoptr(GPtrArray) pkgs_to_import = rpmostree_context_get_packages_to_import (self);
guint64 dlsize = 0;
for (guint i = 0; i < pkgs_to_import->len; i++)
{
auto pkg = static_cast<DnfPackage *>(pkgs_to_import->pdata[i]);
dlsize += dnf_package_get_size (pkg);
g_hash_table_add (pkgset_to_import, pkg);
}
g_autofree char *dlsize_fmt = g_format_size (dlsize);
if (n_invalidated > 0)
rpmostree_output_message ("%u/%u packages to import (%u changed), download size: %s",
pkgs_to_import->len, n_requires, n_invalidated, dlsize_fmt);
else
rpmostree_output_message ("%u/%u packages to import, download size: %s",
pkgs_to_import->len, n_requires, dlsize_fmt);
}
/* Start the download and import, using the xattr data from the rojigRPM */
if (!rpmostree_context_download (self, cancellable, error))
return FALSE;
g_autoptr(GVariant) xattr_table = rpmostree_rojig_assembler_get_xattr_table (rojig);
if (!rpmostree_context_import_rojig (self, xattr_table, pkg_to_xattrs,
cancellable, error))
return FALSE;
/* Last thing is to delete the partial marker, just like
* ostree_repo_pull_with_options().
*/
if (!ostree_repo_mark_commit_partial (repo, checksum, FALSE, error))
return FALSE;
*out_changed = TRUE;
return TRUE;
}

86
src/libpriv/rpmostree-rojig-core.h
View File

@ -1,86 +0,0 @@
/* -*- mode: C; c-file-style: "gnu"; indent-tabs-mode: nil; -*-
*
* Copyright (C) 2017 Colin Walters <walters@verbum.org>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published
* by the Free Software Foundation; either version 2 of the licence or (at
* your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General
* Public License along with this library; if not, write to the
* Free Software Foundation, Inc., 59 Temple Place, Suite 330,
* Boston, MA 02111-1307, USA.
*/
#pragma once
#include <ostree.h>
#include "libglnx.h"
#include <rpm/rpmlib.h>
#include <libdnf/libdnf.h>
G_BEGIN_DECLS
/* A rojigRPM is structured as an ordered set of files/directories; we use numeric
* prefixes to ensure ordering. Most of the files are in GVariant format.
*
* The first entry in a rojigRPM is the OSTree commit and its detached metadata,
* so that can be GPG verified first - if that fails, we can then cleanly
* abort.
*
* The dirmeta/dirtree objects that are referenced by the commit follow.
*
* A special optimization is made for "content-identical" new objects,
* such as the initramfs right now which unfortunately has separate
* SELinux labels and hence different object checksum.
*
* The pure added content objects follow - content objects which won't be
* generated when we import the packages. One interesting detail is right now
* this includes the /usr/lib/tmpfiles.d/pkg-foo.conf objects that we generate
* server side, because we don't generate that client side in rojig mode.
*
* Finally, we have the xattr data, which is mostly in support of SELinux
* labeling (note this is done on the server side still). In order to
* dedup content, we have an xattr "string table" which is just an array
* of xattrs; then there is a GVariant for each package which contains
* a mapping of "objid" to an unsigned integer index into the xattr table.
* The "objid" can either be a full path, or a basename if that basename is
* unique inside a particular package. Since v5, there is also a "cacheid"
* which is used to invalidate client-side caching:
* https://github.com/projectatomic/rpm-ostree/issues/1197
*/
/* Use a numeric prefix to ensure predictable ordering */
#define RPMOSTREE_ROJIG_COMMIT_DIR "00commit"
#define RPMOSTREE_ROJIG_DIRMETA_DIR "02dirmeta"
#define RPMOSTREE_ROJIG_DIRTREE_DIR "03dirtree"
//#define RPMOSTREE_ROJIG_NEW_PKGIDENT "04new-pkgident"
//#define RPMOSTREE_ROJIG_NEW_PKGIDENT_VARIANT_FORMAT (G_VARIANT_TYPE ("a{ua{s(sa(uuua(ayay)))}}")) // Map<pkgid,Map<path,Set<(checksum,uid,gid,mode,xattrs)>)>>
#define RPMOSTREE_ROJIG_NEW_CONTENTIDENT_DIR "04new-contentident"
#define RPMOSTREE_ROJIG_NEW_CONTENTIDENT_VARIANT_FORMAT (G_VARIANT_TYPE ("a(suuua(ayay))")) // checksum,uid,gid,mode,xattrs
#define RPMOSTREE_ROJIG_NEW_DIR "05new"
#define RPMOSTREE_ROJIG_XATTRS_DIR "06xattrs"
#define RPMOSTREE_ROJIG_XATTRS_TABLE "06xattrs/00table"
#define RPMOSTREE_ROJIG_XATTRS_PKG_DIR "06xattrs/pkg"
/* Array of xattr (name, value) pairs */
#define RPMOSTREE_ROJIG_XATTRS_TABLE_VARIANT_FORMAT (G_VARIANT_TYPE ("aa(ayay)"))
/* cacheid + map of objid to index into table ↑ */
#define RPMOSTREE_ROJIG_XATTRS_PKG_VARIANT_FORMAT (G_VARIANT_TYPE ("(sa(su))"))
/* TODO: rename this from jigdo for the next major version */
#define RPMOSTREE_ROJIG_PROVIDE_V5 "rpmostree-jigdo(v5)"
#define RPMOSTREE_ROJIG_PROVIDE_COMMIT "rpmostree-jigdo-commit"
#define RPMOSTREE_ROJIG_PROVIDE_INPUTHASH "rpmostree-rojig-inputhash"
/* This one goes in the spec file to use as our replacement */
#define RPMOSTREE_ROJIG_SPEC_META_MAGIC "#@@@rpmostree_rojig_meta@@@"
G_END_DECLS

123
tests/compose/test-rojig-e2e.sh
View File

@ -1,123 +0,0 @@
#!/bin/bash
set -xeuo pipefail
dn=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=libcomposetest.sh
. "${dn}/libcomposetest.sh"
rpm-ostree --version > version.txt
if ! grep -q rojig version.txt; then
echo "ok skip no rojig support"
exit 0
fi
# Add a local rpm-md repo so we can mutate local test packages
treefile_append "repos" '["test-repo"]'
build_rpm test-pkg \
files "/usr/bin/test-pkg" \
install "mkdir -p %{buildroot}/usr/bin && echo localpkg data > %{buildroot}/usr/bin/test-pkg"
# The test suite writes to pwd, but we need repos in config
# Also we need to disable gpgcheck
echo gpgcheck=0 >> yumrepo.repo
ln yumrepo.repo config/test-repo.repo
treefile_append "packages" '["test-pkg"]'
treefile_set "documentation" 'False'
runcompose --add-metadata-string version=42.0
npkgs=$(rpm-ostree --repo=${repo} db list ${treeref} |grep -v '^ostree commit' | wc -l)
echo "npkgs=${npkgs}"
rpm-ostree --repo=${repo} db list ${treeref} test-pkg >test-pkg-list.txt
assert_file_has_content test-pkg-list.txt 'test-pkg-1.0-1.x86_64'
rev=$(ostree --repo=${repo} rev-parse ${treeref})
mkdir rojig-output
do_commit2rojig() {
targetrev=$1
echo "$(date): starting commit2rojig"
runasroot rpm-ostree ex commit2rojig --repo=${repo} --pkgcache-repo cache/pkgcache-repo ${targetrev} ${treefile} $(pwd)/rojig-output
(cd rojig-output && createrepo_c .)
echo "$(date): finished commit2rojig"
}
do_commit2rojig ${rev}
test -f rojig-output/x86_64/fedora-coreos-42.0-1.fc*.x86_64.rpm
ostree --repo=rojig-unpack-repo init --mode=bare-user
echo 'fsync=false' >> rojig-unpack-repo/config
# Technically this isn't part of config/ but eh
cat > config/rojig-test.repo <<eof
[rojig-test]
baseurl=file://$(pwd)/rojig-output
enabled=1
gpgcheck=0
eof
do_rojig2commit() {
echo "$(date): starting rojig2commit"
rpm-ostree ex rojig2commit -d $(pwd)/config -e cache -e test-repo -e rojig-test --repo=rojig-unpack-repo rojig-test:fedora-coreos | tee rojig2commit-out.txt
echo "$(date): finished rojig2commit"
}
do_rojig2commit
# there will generally be pkgs not in the rojig set, but let's at least assert it's > 0
assert_file_has_content rojig2commit-out.txt ${npkgs}/${npkgs}' packages to import'
ostree --repo=rojig-unpack-repo rev-parse ${rev}
echo "$(date): starting fsck"
ostree --repo=rojig-unpack-repo fsck
echo "$(date): finished fsck"
ostree --repo=rojig-unpack-repo refs > rojig-refs.txt
assert_file_has_content rojig-refs.txt 'rpmostree/rojig/test-pkg/1.0-1.x86__64'
echo "ok rojig ♲📦 fresh assembly"
origrev=${rev}
unset rev
# Update test-pkg
build_rpm test-pkg \
version 1.1 \
files "/usr/bin/test-pkg" \
install "mkdir -p %{buildroot}/usr/bin && echo localpkg data 1.1 > %{buildroot}/usr/bin/test-pkg"
# Also add an entirely new package
build_rpm test-newpkg \
files "/usr/bin/test-newpkg" \
install "mkdir -p %{buildroot}/usr/bin && echo new localpkg data > %{buildroot}/usr/bin/test-newpkg"
treefile_append "packages" '["test-newpkg"]'
runcompose --add-metadata-string version=42.1
newrev=$(ostree --repo=${repo} rev-parse ${treeref})
rpm-ostree --repo=${repo} db list ${treeref} test-newpkg >test-newpkg-list.txt
assert_file_has_content test-newpkg-list.txt 'test-newpkg-1.0-1.x86_64'
# Rojig version 42.1
do_commit2rojig ${newrev}
path=rojig-output/x86_64/fedora-coreos-42.1-1.fc*.x86_64.rpm
rpm -qp --requires ${path} > requires.txt
assert_file_has_content requires.txt 'glibc(.*) = '
assert_file_has_content requires.txt 'systemd(.*) = '
assert_file_has_content requires.txt 'test-pkg(.*) = 1.1-1'
# And pull it; we should download the newer version by default
do_rojig2commit
# Now we should only download 2 packages
assert_file_has_content rojig2commit-out.txt '2/[1-9][0-9]* packages to import'
for x in ${origrev} ${newrev}; do
ostree --repo=rojig-unpack-repo rev-parse ${x}
done
ostree --repo=rojig-unpack-repo fsck
ostree --repo=rojig-unpack-repo refs > rojig-refs.txt
# We should have both refs; GC will be handled by the sysroot upgrader
# via deployments, same way it is for pkg layering.
assert_file_has_content rojig-refs.txt 'rpmostree/rojig/test-pkg/1.0-1.x86__64'
assert_file_has_content rojig-refs.txt 'rpmostree/rojig/test-pkg/1.1-1.x86__64'
echo "ok rojig ♲📦 update!"
# Add all docs to test https://github.com/projectatomic/rpm-ostree/issues/1197
treefile_set "documentation" 'True'
runcompose --add-metadata-string version=42.2
newrev=$(ostree --repo=${repo} rev-parse ${treeref})
do_commit2rojig ${newrev}
find rojig-output -name '*.rpm' | tee rpms.txt
assert_file_has_content rpms.txt 'fedora-coreos-42.2.*x86_64'
do_rojig2commit
# Not every package has docs, but there are going to need to be changes
assert_file_has_content rojig2commit-out.txt '[1-9][0-9]*/[1-9][0-9]* packages to import ([1-9][0-9]* changed)'
ostree --repo=rojig-unpack-repo ls -R ${newrev} >/dev/null
echo "ok rojig ♲📦 updated docs"

37
tests/compose/test-rojig-pure.sh
View File

@ -1,37 +0,0 @@
#!/bin/bash
set -xeuo pipefail
dn=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=libcomposetest.sh
. "${dn}/libcomposetest.sh"
rpm-ostree --version > version.txt
if ! grep -q rojig version.txt; then
echo "ok skip no rojig support"
exit 0
fi
treefile_set "automatic-version-prefix" '"42"'
treefile_set "documentation" 'True'
mkdir rojig-repo
runcompose() {
(cd rojig-repo && createrepo_c .) && \
rm -f treecompose.json && \
runasroot rpm-ostree compose rojig --write-composejson-to $(pwd)/treecompose.json --cachedir=$(pwd)/cache ${treefile} $(pwd)/rojig-repo "$@" && \
(cd rojig-repo && createrepo_c .)
}
runcompose
test -f treecompose.json
test -f rojig-repo/x86_64/fedora-coreos-42-1.fc*.x86_64.rpm
echo "ok rojig ♲📦 initial"
runcompose
test '!' -f treecompose.json
echo "ok rojig no changes"
treefile_set "documentation" 'False'
runcompose
test -f treecompose.json
test -f rojig-repo/x86_64/fedora-coreos-42.1-1.fc*.x86_64.rpm
echo "ok rojig dropped docs"

72
tests/vmcheck/test-rojig-client.sh
View File

@ -1,72 +0,0 @@
#!/bin/bash
#
# Copyright (C) 2018 Red Hat, Inc.
#
# This library is free software; you can redistribute it and/or
# modify it under the terms of the GNU Lesser General Public
# License as published by the Free Software Foundation; either
# version 2 of the License, or (at your option) any later version.
#
# This library is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this library; if not, write to the
# Free Software Foundation, Inc., 59 Temple Place - Suite 330,
# Boston, MA 02111-1307, USA.
set -euo pipefail
. ${commondir}/libtest.sh
. ${commondir}/libvm.sh
set -x
osid=$(vm_cmd grep -E '^ID=' /etc/os-release)
if test "${osid}" != 'ID=fedora'; then
echo "ok skip on OS ID=${osid}"
exit 0
fi
vm_rpmostree --version > version.txt
if ! grep -q rojig version.txt; then
echo "ok skip no rojig support"
exit 0
fi
# Test rebasing to https://pagure.io/fedora-atomic-host-continuous
# in rojig:// mode.
vm_cmd 'cat >/etc/yum.repos.d/fahc.repo' <<EOF
[fahc]
baseurl=http://artifacts.ci.centos.org/sig-atomic/fahc/jigdo
gpgcheck=0
EOF
if vm_rpmostree rebase rojig://fahc:fedora-atomic-host 2>err.txt; then
fatal "Did rojig rebase without --experimental"
fi
assert_file_has_content_literal err.txt 'rojig:// refspec requires --experimental'
vm_rpmostree rebase --experimental rojig://fahc:fedora-atomic-host
vm_assert_status_jq '.deployments[0].origin|startswith("rojig://fahc:fedora-atomic-host")'
vm_cmd ostree refs > refs.txt
assert_file_has_content refs.txt '^rpmostree/rojig/kernel-core/'
echo "ok rojig client rebase "
version=$(vm_get_deployment_info 0 version)
version_major=$(echo ${version} | cut -f 1 -d '.')
version_minor=$(echo ${version} | cut -f 2 -d '.')
prev_version=${version_major}.$((${version_minor} - 1))
assert_not_streq "${version}" "${prev_version}"
vm_rpmostree deploy ${prev_version}
vm_assert_status_jq '.deployments[0].origin|startswith("rojig://fahc:fedora-atomic-host")' \
'.deployments[0].version == "'${prev_version}'"'
echo "ok rojig client deploy"
vm_cmd rpm-ostree status > status.txt
assert_file_has_content_literal status.txt 'fahc:fedora-atomic-host-'${prev_version}'-1'
echo "ok rojig status"