sysroot: Stabilize deployment finalization, add API and CLI

It's about time we do this; deployment finalization locking
is a useful feature.  An absolutely key thing here is that
we've slowly been moving towards the deployments as the primary
"source of truth".

Specifically in bootc for example, we will GC container images
not referenced by a deployment.

This is then neecessary to support a "pull but don't apply automatically" model.

This stabilizes the existing `ostree admin deploy --lock-finalization`
CLI, and adds a new `ostree admin unlock-finalization`.

We still check the old lock file path, but there's a new boolean
value as part of the staged deployment data which is intended
to be the source of truth in the future.  At some point then we
can drop the rpm-ostree lockfile handling.

Closes: https://github.com/ostreedev/ostree/issues/3025
This commit is contained in:
Colin Walters 2023-11-08 17:19:19 -05:00
parent 1ca4f02a69
commit 28cc761806
19 changed files with 337 additions and 12 deletions

View File

@ -30,6 +30,7 @@ ostree-admin-init-fs.1 ostree-admin-instutil.1 ostree-admin-stateroot-init.1 ost
ostree-admin-status.1 ostree-admin-set-origin.1 ostree-admin-switch.1 \
ostree-admin-undeploy.1 ostree-admin-upgrade.1 ostree-admin-unlock.1 \
ostree-admin-pin.1 ostree-admin-post-copy.1 ostree-admin-set-default.1 \
ostree-admin-lock-finalization.1 \
ostree-admin.1 ostree-cat.1 ostree-checkout.1 ostree-checksum.1 \
ostree-commit.1 ostree-create-usb.1 ostree-export.1 \
ostree-config.1 ostree-diff.1 ostree-find-remotes.1 ostree-fsck.1 \

View File

@ -70,6 +70,7 @@ ostree_SOURCES += \
src/ostree/ot-admin-builtin-diff.c \
src/ostree/ot-admin-builtin-deploy.c \
src/ostree/ot-admin-builtin-finalize-staged.c \
src/ostree/ot-admin-builtin-lock-finalization.c \
src/ostree/ot-admin-builtin-boot-complete.c \
src/ostree/ot-admin-builtin-undeploy.c \
src/ostree/ot-admin-builtin-set-default.c \

View File

@ -191,6 +191,7 @@ ostree_deployment_get_origin_relpath
ostree_deployment_get_unlocked
ostree_deployment_is_pinned
ostree_deployment_is_staged
ostree_deployment_is_finalization_locked
ostree_deployment_set_index
ostree_deployment_set_bootserial
ostree_deployment_set_bootconfig
@ -591,6 +592,7 @@ ostree_sysroot_write_origin_file
ostree_sysroot_stage_tree
ostree_sysroot_stage_tree_with_options
ostree_sysroot_stage_overlay_initrd
ostree_sysroot_change_finalization
ostree_sysroot_deploy_tree
ostree_sysroot_deploy_tree_with_options
ostree_sysroot_get_merge_deployment

View File

@ -121,6 +121,16 @@ License along with this library. If not, see <https://www.gnu.org/licenses/>.
</para></listitem>
</varlistentry>
<varlistentry>
<term><option>--lock-finalization</option></term>
<listitem><para>
The deployment will not be "finalized" by default on shutdown; to later
queue it, use <literal>ostree admin unlock-finalization</literal>.
</para></listitem>
</varlistentry>
<varlistentry>
<term><option>--karg-proc-cmdline</option></term>

View File

@ -0,0 +1,92 @@
<?xml version='1.0'?> <!--*-nxml-*-->
<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
<!--
Copyright 2023 Red Hat, Inc.
SPDX-License-Identifier: LGPL-2.0+
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library. If not, see <https://www.gnu.org/licenses/>.
-->
<refentry id="ostree">
<refentryinfo>
<title>ostree admin lock-finalization</title>
<productname>OSTree</productname>
<authorgroup>
<author>
<contrib>Developer</contrib>
<firstname>Colin</firstname>
<surname>Walters</surname>
<email>walters@verbum.org</email>
</author>
</authorgroup>
</refentryinfo>
<refmeta>
<refentrytitle>ostree admin lock-finalization</refentrytitle>
<manvolnum>1</manvolnum>
</refmeta>
<refnamediv>
<refname>ostree-admin-lock-finalization</refname>
<refpurpose>Change whether staged deployment will be queued for next boot</refpurpose>
</refnamediv>
<refsynopsisdiv>
<cmdsynopsis>
<command>ostree admin lock-finalization</command> <arg choice="opt" rep="repeat">OPTIONS</arg>
</cmdsynopsis>
</refsynopsisdiv>
<refsect1>
<title>Description</title>
<para>
This command requires a staged deployment. By default, this command ensures the deployment
will be set into a "finalization locked" state, which means it will not be queued for the next boot by default.
</para>
<para>
This is the same as the <literal>--lock-finalization</literal> argument for <literal>ostree admin deploy</literal>.
</para>
<para>
However more commonly, one will use the <literal>--unlock</literal> argument for this command to later unlock
a deployment which was finalization locked.
</para>
</refsect1>
<refsect1>
<title>Options</title>
<variablelist>
<varlistentry>
<term><option>--sysroot</option>="PATH"</term>
<listitem><para>
Path to the system to use rather than the current one.
</para></listitem>
</varlistentry>
<varlistentry>
<term><option>--unlock</option>,<option>-u</option></term>
<listitem><para>
Unlock the deployment finalization state.
</para></listitem>
</varlistentry>
</variablelist>
</refsect1>
</refentry>

View File

@ -23,6 +23,8 @@
LIBOSTREE_2023.8 {
global:
ostree_sysroot_update_post_copy;
ostree_deployment_is_finalization_locked;
ostree_sysroot_change_finalization;
} LIBOSTREE_2023.4;
/* Stub section for the stable release *after* this development one; don't

View File

@ -51,6 +51,7 @@ struct _OstreeDeployment
GKeyFile *origin;
OstreeDeploymentUnlockedState unlocked;
gboolean staged;
gboolean finalization_locked;
char **overlay_initrds;
char *overlay_initrds_id;
};

View File

@ -461,3 +461,18 @@ ostree_deployment_is_staged (OstreeDeployment *self)
{
return self->staged;
}
/**
* ostree_deployment_is_finalization_locked:
* @self: Deployment
*
* Returns: `TRUE` if deployment is queued to be "finalized" at shutdown time, but requires
* additional action.
*
* Since: 2023.8
*/
gboolean
ostree_deployment_is_finalization_locked (OstreeDeployment *self)
{
return self->finalization_locked;
}

View File

@ -71,6 +71,8 @@ GKeyFile *ostree_deployment_get_origin (OstreeDeployment *self);
_OSTREE_PUBLIC
gboolean ostree_deployment_is_staged (OstreeDeployment *self);
_OSTREE_PUBLIC
gboolean ostree_deployment_is_finalization_locked (OstreeDeployment *self);
_OSTREE_PUBLIC
gboolean ostree_deployment_is_pinned (OstreeDeployment *self);
_OSTREE_PUBLIC

View File

@ -3657,6 +3657,10 @@ ostree_sysroot_stage_tree_with_options (OstreeSysroot *self, const char *osname,
g_autoptr (GVariantBuilder) builder = g_variant_builder_new ((GVariantType *)"a{sv}");
g_variant_builder_add (builder, "{sv}", "target", serialize_deployment_to_variant (deployment));
if (opts->locked)
g_variant_builder_add (builder, "{sv}", _OSTREE_SYSROOT_STAGED_KEY_LOCKED,
g_variant_new_boolean (TRUE));
if (merge_deployment)
g_variant_builder_add (builder, "{sv}", "merge-deployment",
serialize_deployment_to_variant (merge_deployment));
@ -3706,6 +3710,73 @@ ostree_sysroot_stage_tree_with_options (OstreeSysroot *self, const char *osname,
return TRUE;
}
/**
* ostree_sysroot_change_finalization:
* @self: Sysroot
* @deployment: Deployment which must be staged
* @error: Error
*
* Given the target deployment (which must be the staged deployment) this API
* will toggle its "finalization locking" state. If it is currently locked,
* it will be unlocked (and hence queued to apply on shutdown).
*
* Since: 2023.8
*/
_OSTREE_PUBLIC
gboolean
ostree_sysroot_change_finalization (OstreeSysroot *self, OstreeDeployment *deployment,
GError **error)
{
GCancellable *cancellable = NULL;
g_assert (ostree_deployment_is_staged (deployment));
gboolean new_locked_state = !ostree_deployment_is_finalization_locked (deployment);
/* Read the staged state from disk */
glnx_autofd int fd = -1;
if (!glnx_openat_rdonly (AT_FDCWD, _OSTREE_SYSROOT_RUNSTATE_STAGED, TRUE, &fd, error))
return FALSE;
g_autoptr (GBytes) contents = ot_fd_readall_or_mmap (fd, 0, error);
if (!contents)
return FALSE;
g_autoptr (GVariant) staged_deployment_data
= g_variant_new_from_bytes ((GVariantType *)"a{sv}", contents, TRUE);
g_autoptr (GVariantDict) staged_deployment_dict = g_variant_dict_new (staged_deployment_data);
g_variant_dict_insert (staged_deployment_dict, _OSTREE_SYSROOT_STAGED_KEY_LOCKED, "b",
new_locked_state);
g_autoptr (GVariant) new_staged_deployment_data = g_variant_dict_end (staged_deployment_dict);
if (!glnx_file_replace_contents_at (fd, _OSTREE_SYSROOT_RUNSTATE_STAGED,
g_variant_get_data (new_staged_deployment_data),
g_variant_get_size (new_staged_deployment_data),
GLNX_FILE_REPLACE_NODATASYNC, cancellable, error))
return FALSE;
if (!new_locked_state)
{
/* Delete the legacy lock if there was any. */
if (!ot_ensure_unlinked_at (AT_FDCWD, _OSTREE_SYSROOT_RUNSTATE_STAGED_LOCKED, error))
return FALSE;
}
else
{
/* Create the legacy lockfile; see also the code in ot-admin-builtin-deploy.c */
if (!glnx_shutil_mkdir_p_at (AT_FDCWD,
dirname (strdupa (_OSTREE_SYSROOT_RUNSTATE_STAGED_LOCKED)), 0755,
cancellable, error))
return FALSE;
glnx_autofd int lockfd = open (_OSTREE_SYSROOT_RUNSTATE_STAGED_LOCKED,
O_CREAT | O_WRONLY | O_NOCTTY | O_CLOEXEC, 0640);
if (lockfd == -1)
return glnx_throw_errno_prefix (error, "touch(%s)", _OSTREE_SYSROOT_RUNSTATE_STAGED_LOCKED);
}
return TRUE;
}
/* Invoked at shutdown time by ostree-finalize-staged.service */
static gboolean
_ostree_sysroot_finalize_staged_inner (OstreeSysroot *self, GCancellable *cancellable,
@ -3722,11 +3793,22 @@ _ostree_sysroot_finalize_staged_inner (OstreeSysroot *self, GCancellable *cancel
}
/* Check if finalization is locked. */
if (!glnx_fstatat_allow_noent (AT_FDCWD, _OSTREE_SYSROOT_RUNSTATE_STAGED_LOCKED, NULL, 0, error))
gboolean locked = false;
(void)g_variant_lookup (self->staged_deployment_data, _OSTREE_SYSROOT_STAGED_KEY_LOCKED, "b",
&locked);
if (locked)
g_debug ("staged is locked via metadata");
else
{
if (!glnx_fstatat_allow_noent (AT_FDCWD, _OSTREE_SYSROOT_RUNSTATE_STAGED_LOCKED, NULL, 0,
error))
return FALSE;
if (errno == 0)
locked = TRUE;
}
if (locked)
{
ot_journal_print (LOG_INFO, "Not finalizing; found " _OSTREE_SYSROOT_RUNSTATE_STAGED_LOCKED);
ot_journal_print (LOG_INFO, "Not finalizing; deployment is locked for finalization");
return TRUE;
}

View File

@ -93,6 +93,9 @@ struct OstreeSysroot
OstreeSysrootDebugFlags debug_flags;
};
/* Key in staged deployment variant for finalization locking */
#define _OSTREE_SYSROOT_STAGED_KEY_LOCKED "locked"
#define OSTREE_SYSROOT_LOCKFILE "ostree/lock"
/* We keep some transient state in /run */
#define _OSTREE_SYSROOT_RUNSTATE_STAGED "/run/ostree/staged-deployment"

View File

@ -1106,6 +1106,8 @@ _ostree_sysroot_reload_staged (OstreeSysroot *self, GError **error)
* canonical "staged_deployment" reference.
*/
self->staged_deployment->staged = TRUE;
(void)g_variant_dict_lookup (staged_deployment_dict, _OSTREE_SYSROOT_STAGED_KEY_LOCKED,
"b", &self->staged_deployment->finalization_locked);
}
}

View File

@ -178,7 +178,10 @@ gboolean ostree_sysroot_stage_overlay_initrd (OstreeSysroot *self, int fd, char
typedef struct
{
gboolean unused_bools[8];
/* If set to true, then this deployment will be staged but "locked" and not automatically applied
* on reboot. */
gboolean locked;
gboolean unused_bools[7];
int unused_ints[8];
char **override_kernel_argv;
char **overlay_initrds;
@ -215,6 +218,10 @@ gboolean ostree_sysroot_stage_tree_with_options (OstreeSysroot *self, const char
OstreeDeployment **out_new_deployment,
GCancellable *cancellable, GError **error);
_OSTREE_PUBLIC
gboolean ostree_sysroot_change_finalization (OstreeSysroot *self, OstreeDeployment *deployment,
GError **error);
_OSTREE_PUBLIC
gboolean ostree_sysroot_deployment_set_mutable (OstreeSysroot *self, OstreeDeployment *deployment,
gboolean is_mutable, GCancellable *cancellable,

View File

@ -60,7 +60,7 @@ static GOptionEntry options[] = {
"Do not apply configuration (/etc and kernel arguments) from booted deployment", NULL },
{ "retain", 0, 0, G_OPTION_ARG_NONE, &opt_retain, "Do not delete previous deployments", NULL },
{ "stage", 0, 0, G_OPTION_ARG_NONE, &opt_stage, "Complete deployment at OS shutdown", NULL },
{ "lock-finalization", 0, G_OPTION_FLAG_HIDDEN, G_OPTION_ARG_NONE, &opt_lock_finalization,
{ "lock-finalization", 0, 0, G_OPTION_ARG_NONE, &opt_lock_finalization,
"Prevent automatic deployment finalization on shutdown", NULL },
{ "retain-pending", 0, 0, G_OPTION_ARG_NONE, &opt_retain_pending,
"Do not delete pending deployments", NULL },
@ -123,6 +123,10 @@ ot_admin_builtin_deploy (int argc, char **argv, OstreeCommandInvocation *invocat
return FALSE;
}
// Locking implies staging
if (opt_lock_finalization)
opt_stage = TRUE;
const char *refspec = argv[1];
OstreeRepo *repo = ostree_sysroot_repo (sysroot);
@ -236,6 +240,7 @@ ot_admin_builtin_deploy (int argc, char **argv, OstreeCommandInvocation *invocat
g_auto (GStrv) kargs_strv = kargs ? ostree_kernel_args_to_strv (kargs) : NULL;
OstreeSysrootDeployTreeOpts opts = {
.locked = opt_lock_finalization,
.override_kernel_argv = kargs_strv,
.overlay_initrds = overlay_initrd_chksums ? (char **)overlay_initrd_chksums->pdata : NULL,
};
@ -247,9 +252,11 @@ ot_admin_builtin_deploy (int argc, char **argv, OstreeCommandInvocation *invocat
return glnx_throw (error, "--stage cannot currently be combined with --retain arguments");
if (opt_not_as_default)
return glnx_throw (error, "--stage cannot currently be combined with --not-as-default");
/* touch file *before* we stage to avoid races */
/* For compatibility with older versions of ostree, also write this legacy file.
* This can likely be safely deleted in the middle of 2024 say. */
if (opt_lock_finalization)
{
g_debug ("Writing legacy finalization lockfile");
if (!glnx_shutil_mkdir_p_at (AT_FDCWD,
dirname (strdupa (_OSTREE_SYSROOT_RUNSTATE_STAGED_LOCKED)),
0755, cancellable, error))
@ -262,7 +269,7 @@ ot_admin_builtin_deploy (int argc, char **argv, OstreeCommandInvocation *invocat
_OSTREE_SYSROOT_RUNSTATE_STAGED_LOCKED);
}
/* use old API if we can to exercise it in CI */
if (!overlay_initrd_chksums)
if (!(overlay_initrd_chksums || opt_lock_finalization))
{
if (!ostree_sysroot_stage_tree (sysroot, opt_osname, revision, origin, merge_deployment,
kargs_strv, &new_deployment, cancellable, error))

View File

@ -0,0 +1,78 @@
/*
* Copyright (C) 2023 Red Hat, Inc.
*
* SPDX-License-Identifier: LGPL-2.0+
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library. If not, see <https://www.gnu.org/licenses/>.
*/
#include "config.h"
#include "ostree-sysroot-private.h"
#include "ostree.h"
#include "ot-admin-builtins.h"
#include "ot-admin-functions.h"
#include "ot-main.h"
#include "otutil.h"
#include <glib/gi18n.h>
static gboolean opt_unlock;
static GOptionEntry options[]
= { { "unlock", 'u', 0, G_OPTION_ARG_NONE, &opt_unlock, "Unlock finalization", NULL },
{ NULL } };
gboolean
ot_admin_builtin_lock_finalization (int argc, char **argv, OstreeCommandInvocation *invocation,
GCancellable *cancellable, GError **error)
{
g_autoptr (GOptionContext) context = g_option_context_new ("");
g_autoptr (OstreeSysroot) sysroot = NULL;
if (!ostree_admin_option_context_parse (context, options, &argc, &argv,
OSTREE_ADMIN_BUILTIN_FLAG_SUPERUSER, invocation, &sysroot,
cancellable, error))
return FALSE;
OstreeDeployment *staged = ostree_sysroot_get_staged_deployment (sysroot);
if (!staged)
return glnx_throw (error, "No staged deployment");
const gboolean is_locked = ostree_deployment_is_finalization_locked (staged);
if (opt_unlock && !is_locked)
{
g_print ("Staged deployment is already prepared for finalization\n");
return TRUE;
}
else if (!opt_unlock && is_locked)
{
g_print ("Staged deployment is already finalization locked\n");
return TRUE;
}
if (!ostree_sysroot_change_finalization (sysroot, staged, error))
return FALSE;
if (opt_unlock)
{
g_print ("Staged deployment is now queued to apply on shutdown\n");
}
else
{
g_print ("Staged deployment is now finalization locked\n");
}
return TRUE;
}

View File

@ -98,7 +98,9 @@ deployment_print_status (OstreeSysroot *sysroot, OstreeRepo *repo, OstreeDeploym
GKeyFile *origin = ostree_deployment_get_origin (deployment);
const char *deployment_status = "";
if (ostree_deployment_is_staged (deployment))
if (ostree_deployment_is_finalization_locked (deployment))
deployment_status = " (finalization locked)";
else if (ostree_deployment_is_staged (deployment))
deployment_status = " (staged)";
else if (is_pending)
deployment_status = " (pending)";

View File

@ -49,6 +49,7 @@ BUILTINPROTO (switch);
BUILTINPROTO (upgrade);
BUILTINPROTO (kargs);
BUILTINPROTO (post_copy);
BUILTINPROTO (lock_finalization);
#undef BUILTINPROTO

View File

@ -39,6 +39,8 @@ static OstreeCommand admin_subcommands[] = {
"Checkout revision REFSPEC as the new default deployment" },
{ "finalize-staged", OSTREE_BUILTIN_FLAG_NO_REPO | OSTREE_BUILTIN_FLAG_HIDDEN,
ot_admin_builtin_finalize_staged, "Internal command to run at shutdown time" },
{ "lock-finalization", OSTREE_BUILTIN_FLAG_NO_REPO, ot_admin_builtin_lock_finalization,
"Change the finalization locking state of the staged deployment" },
{ "boot-complete", OSTREE_BUILTIN_FLAG_NO_REPO | OSTREE_BUILTIN_FLAG_HIDDEN,
ot_admin_builtin_boot_complete, "Internal command to run at boot after an update was applied" },
{ "init-fs", OSTREE_BUILTIN_FLAG_NO_REPO, ot_admin_builtin_init_fs,

View File

@ -86,13 +86,15 @@ EOF
test '!' -f /run/ostree/staged-deployment
test '!' -f /run/ostree/staged-deployment
ostree admin deploy --stage staged-deploy --lock-finalization
ostree admin status > status.txt
assert_not_file_has_content status.txt 'finalization locked'
ostree admin deploy staged-deploy --lock-finalization
ostree admin status > status.txt
assert_file_has_content status.txt 'finalization locked'
test -f /run/ostree/staged-deployment
test -f /run/ostree/staged-deployment-locked
# check that we can cleanup the staged deployment
ostree admin undeploy 0
test ! -f /run/ostree/staged-deployment
test ! -f /run/ostree/staged-deployment-locked
echo "ok cleanup staged"
# And verify that re-staging cleans the previous lock
@ -130,6 +132,19 @@ EOF
test '!' -f /run/ostree/staged-deployment
echo "ok unstage"
ostree admin deploy staged-deploy --lock-finalization
ostree admin status > status.txt
assert_file_has_content status.txt 'finalization locked'
ostree admin lock-finalization > out.txt
assert_file_has_content_literal out.txt 'already finalization locked'
ostree admin status > status.txt
assert_file_has_content status.txt 'finalization locked'
ostree admin lock-finalization -u > out.txt
assert_file_has_content_literal out.txt 'now queued to apply'
ostree admin status > status.txt
assert_not_file_has_content status.txt 'finalization locked'
echo "ok finalization locking toggle"
# Staged should be overwritten by non-staged as first
commit=$(rpmostree_query_json '.deployments[0].checksum')
ostree admin deploy --stage staged-deploy