From 65df0ce39a1990e64eb74087270ac823ba0bd665 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Zbigniew=20J=C4=99drzejewski-Szmek?= Date: Tue, 24 May 2022 10:20:23 +0200 Subject: [PATCH 1/7] Minor wording fixes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Some NEWS entries are tweaked a bit to address complaints about readability from users. "udev" is pronounced as /ˈjuːdɛv/, like in "user", hence "a" not "an". --- NEWS | 35 +++++++++++++++---------------- src/login/logind-session-device.c | 2 +- 2 files changed, 18 insertions(+), 19 deletions(-) diff --git a/NEWS b/NEWS index 8ca049e5486..9ff9292311b 100644 --- a/NEWS +++ b/NEWS @@ -29,19 +29,19 @@ CHANGES WITH 251: and backward compatibility broken instead on the assumption that nobody can be affected given the current state of this interface. - * All kernels supported by systemd mix RDRAND (or similar) into the - entropy pool at early boot. This means that on those systems, even if - /dev/urandom is not yet initialized, it still returns bytes that - are at least as high quality as RDRAND. For that reason, we no longer - have reason to invoke RDRAND from systemd itself, which has - historically been a source of bugs. Furthermore, kernels ≥5.6 provide - the getrandom(GRND_INSECURE) interface for returning random bytes - before the entropy pool is initialized without warning into kmsg, - which is what we attempt to use if available. systemd's direct usage - of RDRAND has been removed. x86 systems ≥Broadwell that are running - an older kernel may experience kmsg warnings that were not seen with - 250. For newer kernels, non-x86 systems, or older x86 systems, there - should be no visible changes. + * All kernels supported by systemd mix bytes returned by RDRAND (or + similar) into the entropy pool at early boot. This means that on + those systems, even if /dev/urandom is not yet initialized, it still + returns bytes that are of at least RDRAND quality. For that reason, + we no longer have reason to invoke RDRAND from systemd itself, which + has historically been a source of bugs. Furthermore, kernels ≥5.6 + provide the getrandom(GRND_INSECURE) interface for returning random + bytes before the entropy pool is initialized without warning into + kmsg, which is what we attempt to use if available. systemd's direct + usage of RDRAND has been removed. x86 systems ≥Broadwell that are + running an older kernel may experience kmsg warnings that were not + seen with 250. For newer kernels, non-x86 systems, or older x86 + systems, there should be no visible changes. * sd-boot will now measure the kernel command line into TPM PCR 12 rather than PCR 8. This improves usefulness of the measurements on @@ -59,11 +59,10 @@ CHANGES WITH 251: * busctl capture now writes output in the newer pcapng format instead of pcap. - * A udev rule that imported hwdb matches for USB devices with - lowercase hexadecimal vendor/product ID digits was added in systemd - 250. This has been reverted, since uppercase hexadecimal digits are - supposed to be used, and we already had a rule for that with the - appropriate match. + * A udev rule that imported hwdb matches for USB devices with lowercase + hexadecimal vendor/product ID digits was added in systemd 250. This + has been reverted, since uppercase hexadecimal digits are supposed to + be used, and we already had a rule with the appropriate match. Users might need to adjust their local hwdb entries. diff --git a/src/login/logind-session-device.c b/src/login/logind-session-device.c index 19e00b996e5..003dbc0a959 100644 --- a/src/login/logind-session-device.c +++ b/src/login/logind-session-device.c @@ -120,7 +120,7 @@ static int session_device_open(SessionDevice *sd, bool active) { assert(sd->type != DEVICE_TYPE_UNKNOWN); assert(sd->node); - /* open device and try to get an udev_device from it */ + /* open device and try to get a udev_device from it */ fd = open(sd->node, O_RDWR|O_CLOEXEC|O_NOCTTY|O_NONBLOCK); if (fd < 0) return -errno; From 7bb36d2d7af2daa4171f114e1dd384927f89b2b8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Zbigniew=20J=C4=99drzejewski-Szmek?= Date: Mon, 23 May 2022 14:22:40 +0200 Subject: [PATCH 2/7] docs: reworder/rewrite BLS to read more like a specification MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit I tried not to introduce any semantic changes, but to reorder the whole text to be more usable as a reference specification: more sections are created and the discussion and justifications are moved to the end. Also, "BIOS" is changed to "firmware" in various places, and other parts of the text that made sense when this was originally written are now dated are adjusted. I separated and extended the examples a bit. The abstract at the top ("TL;DR: Currently there’s no common boot scheme…") is dropped. It didn't seem to fit anywhere. --- docs/BOOT_LOADER_SPECIFICATION.md | 503 ++++++++++++++++-------------- 1 file changed, 276 insertions(+), 227 deletions(-) diff --git a/docs/BOOT_LOADER_SPECIFICATION.md b/docs/BOOT_LOADER_SPECIFICATION.md index 40ec2f152b3..5905fc59009 100644 --- a/docs/BOOT_LOADER_SPECIFICATION.md +++ b/docs/BOOT_LOADER_SPECIFICATION.md @@ -7,120 +7,63 @@ SPDX-License-Identifier: LGPL-2.1-or-later # The Boot Loader Specification -_TL;DR: Currently there's no common boot scheme across architectures and -platforms for open-source operating systems. There's also little cooperation -between multiple distributions in dual-boot (or triple, … multi-boot) -setups. We'd like to improve this situation by getting everybody to commit to a -single boot configuration format that is based on drop-in files, and thus is -robust, simple, works without rewriting configuration files and is free of -namespace clashes._ +This document defines a set of file formats and naming conventions that allow +the boot loader configuration to be shared between multiple operating systems +and boot loaders installed on one device. -The Boot Loader Specification defines a scheme how different operating systems -can cooperatively manage a boot loader configuration directory, that accepts -drop-in files for boot menu items that are defined in a format that is shared -between various boot loader implementations, operating systems, and userspace -programs. The same scheme can be used to prepare OS media for cases where the -firmware includes a boot loader. The target audience for this specification is: +Operating systems cooperatively manage a boot loader configuration directory +that contains drop-in files, making multi-boot scenarios easy to support. Boot +menu items are defined via a simple format that can be understood by different +boot loader implementations, operating systems, and userspace programs. The +same scheme can be used to prepare OS media for cases where the firmware +includes a boot loader. + +## Target audience + +The target audience for this specification is: * Boot loader developers, to write a boot loader that directly reads its - configuration at runtime from these drop-in snippets + configuration from these files * Firmware developers, to add generic boot loading support directly to the firmware itself -* Distribution and Core OS developers, in order to create these snippets at - OS/kernel package installation time -* UI developers, for implementing a user interface that discovers the available - boot options -* OS Installer developers, to prepare their installation media and for setting - up the initial drop-in directory +* OS installer developers, to create appropriate partitions and set up the + initial drop-in directory +* Distribution developers, to create appropriate configuration snippets when + installing or updating kernel packages +* UI developers, to implement user interfaces that list and select among the + available boot options -## Why is there a need for this specification? +## The boot partition -Of course, without this specification things already work mostly fine. But here's why we think this specification is needed: - -* To make the boot more robust, as no explicit rewriting of configuration files - is required any more -* To allow an out of the box boot experience on any platform without the need - of traditional firmware mechanisms (e.g. BIOS calls, UEFI Boot Services) -* To improve dual-boot scenarios. Currently, multiple Linux installations tend - to fight over which boot loader becomes the primary one in possession of the - MBR, and only that one installation can then update the boot loader - configuration of it freely. Other Linux installs have to be manually - configured to never touch the MBR and instead install a chain-loaded boot - loader in their own partition headers. In this new scheme as all - installations share a loader directory no manual configuration has to take - place, and all participants implicitly cooperate due to removal of name - collisions and can install/remove their own boot menu entries at free will, - without interfering with the entries of other installed operating systems. -* Drop-in directories are otherwise now pretty ubiquitous on Linux as an easy - way to extend configuration without having to edit, regenerate or manipulate - configuration files. For the sake of uniformity, we should do the same for - extending the boot menu. -* Userspace code can sanely parse boot loader configuration which is essential - with modern BIOSes which do not necessarily initialize USB keyboards anymore - during boot, which makes boot menus hard to reach for the user. If userspace - code can parse the boot loader configuration, too, this allows for UIs that - can select a boot menu item to boot into, before rebooting the machine, thus - not requiring interactivity during early boot. -* To unify and thus simplify configuration of the various boot loaders around, - which makes configuration of the boot loading process easier for users, - administrators and developers alike. -* For boot loaders with configuration _scripts_ such as grub2, adopting this - spec allows for mostly static scripts that are generated only once at first - installation, but then do not need to be updated anymore as that is done via - drop-in files exclusively. - -## Why not simply rely on the EFI boot menu logic? - -EFI is not ubiquitous, especially not in embedded systems. If you have an EFI -system, it provides a boot options logic that can offer similar -functionality. Here's why we think that it is not enough for our uses: - -* The various EFI implementations implement the boot order/boot item logic to - different levels. Some firmware implementations do not offer a boot menu at - all and instead unconditionally follow the EFI boot order, booting the first - item that is working. -* If the firmware setup is used to reset all data usually all EFI boot entries - are lost, making the system entirely unbootable, as the firmware setups - generally do not offer a UI to define additional boot items. By placing the - menu item information on disk, it is always available, regardless if the BIOS - setup data is lost. -* Harddisk images should be movable between machines and be bootable without - requiring explicit EFI variables to be set. This also requires that the list - of boot options is defined on disk, and not in EFI variables alone. -* EFI is not universal yet (especially on non-x86 platforms), this - specification is useful both for EFI and non-EFI boot loaders. -* Many EFI systems disable USB support during early boot to optimize boot - times, thus making keyboard input unavailable in the EFI menu. It is thus - useful if the OS UI has a standardized way to discover available boot options - which can be booted to. - -## Technical Details - -Everything described below is located on a placeholder file system `$BOOT`. The -installer program should pick `$BOOT` according to the following rules: +Everything described below is located on a file system that will be called +`$BOOT`. The boot loader or user-space programs reading the boot loader +configuration should pick `$BOOT` according to the following rules: * On disks with an MBR partition table: - * If the OS is installed on a disk with an MBR partition table, and a - partition with the type id of 0xEA already exists it should be used as - `$BOOT`. - * Otherwise, if the OS is installed on a disk with an MBR partition table, a - new partition with type id of 0xEA shall be created, of a suitable size - (let's say 500MB), and it should be used as `$BOOT`. + + * A partition with the type ID of 0xEA shall be used as `$BOOT`. + * On disks with GPT (GUID Partition Table) - * If the OS is installed on a disk with GPT, and an Extended Boot Loader - Partition (or XBOOTLDR partition for short), i.e. a partition with GPT type - GUID of `bc13c2ff-59e6-4262-a352-b275fd6f7172`, already exists, it should + + * If an Extended Boot Loader Partition (XBOOTLDR partition for short) — a + partition with GPT type GUID of `bc13c2ff-59e6-4262-a352-b275fd6f7172` — + is found, it should be used as `$BOOT`. + + * Otherwise, the EFI System Partition (ESP for short) — a partition with + GPT type GUID of `c12a7328-f81f-11d2-ba4b-00a0c93ec93b` — should be used as `$BOOT`. - * Otherwise, if the OS is installed on a disk with GPT, and an EFI System - Partition (or ESP for short), i.e. a partition with GPT type UID of - `c12a7328-f81f-11d2-ba4b-00a0c93ec93b` already exists and is large enough - (let's say 250MB) and otherwise qualifies, it should be used as `$BOOT`. - * Otherwise, if the OS is installed on a disk with GPT, and if the ESP - already exists but is too small, a new suitably sized (let's say 500MB) - XBOOTLDR partition shall be created and used as `$BOOT`. - * Otherwise, if the OS is installed on a disk with GPT, and no ESP exists - yet, a new suitably sized (let's say 500MB) ESP should be created and used - as `$BOOT`. + +An installer for the operating system should use this logic when selecting or +creating partitions: + + * If `$BOOT` is not found, a new suitably sized partition (let's say 500MB) + should be created and used as `$BOOT`, matching the characteristics + described above. On disks with GPT, just an ESP partition without a + XBOOTLDR partition should be created. + + * If the OS is installed on a disk with GPT and the ESP partition is found + but is too small, a new suitably sized (let's say 500MB) XBOOTLDR partition + shall be created and used as `$BOOT`. This placeholder file system shall be determined during _installation time_, and an fstab entry may be created. It should be mounted to either `/boot/` or @@ -129,20 +72,22 @@ file system) might be supported by implementations. This is not recommended because the mounting of `$BOOT` is then dependent on and requires the mounting of the intermediate file system. -**Note:** _`$BOOT` should be considered **shared** among all OS installations -of a system. Instead of maintaining one `$BOOT` per installed OS (as `/boot/` -was traditionally handled), all installed OS share the same place to drop in -their boot-time configuration._ +**Note:** _`$BOOT` is **shared** among all OS installations of a system. +Instead of maintaining one `$BOOT` per installed OS (as `/boot/` was +traditionally handled), all installed OSes use the same place for boot-time +configuration._ For systems where the firmware is able to read file systems directly, `$BOOT` must be a file system readable by the firmware. For other systems and generic -installation and live media, `$BOOT` must be a VFAT (16 or 32) file -system. Applications accessing `$BOOT` should hence not assume that fancier -file system features such as symlinks, hardlinks, access control or case -sensitivity are supported. +installation and live media, `$BOOT` must be a VFAT (16 or 32) file system. +Applications accessing `$BOOT` should hence not assume that fancier file system +features such as symlinks, hardlinks, access control or case sensitivity are +supported. + +## Boot loader entries This specification defines two types of boot loader entries. The first type is -text based, very simple and suitable for a variety of firmware, architecture +text based, very simple, and suitable for a variety of firmware, architecture and image types ("Type #1"). The second type is specific to EFI, but allows single-file images that embed all metadata in the kernel binary itself, which is useful to cryptographically sign them as one file for the purpose of @@ -161,10 +106,10 @@ Note that the `$BOOT` partition is not supposed to be exclusive territory of this specification. This specification only defines semantics of the `/loader/` directory inside the file system (see below), but it doesn't intend to define ownership of the whole file system exclusively. Boot loaders, firmware, and -other software implementing this specification may choose to place other -files and directories in the same file system. For example, boot loaders that -implement this specification might install their own boot code into the `$BOOT` -partition. On systems where `$BOOT` is the ESP this is a particularly common +other software implementing this specification may choose to place other files +and directories in the same file system. For example, boot loaders that +implement this specification might install their own boot code on the same +partition; on systems where `$BOOT` is the ESP this is a particularly common setup. Implementations of this specification must be able to operate correctly if files or directories other than `/loader/` are found in the top level directory. Implementations that add their own files or directories to the file @@ -173,7 +118,7 @@ multiple users of the file system unlikely. ### Type #1 Boot Loader Specification Entries -We define two directories below `$BOOT`: +This specification defines two directories below `$BOOT`: * `$BOOT/loader/` is the directory containing all files needed for Type #1 entries @@ -187,102 +132,129 @@ the root of the file system. Specifically, if `$BOOT` is the ESP, then ESP, and not in the `/EFI/` subdirectory._ Inside the `$BOOT/loader/entries/` directory each OS vendor may drop one or -more configuration snippets with the suffix ".conf", one for each boot menu +more configuration snippets with the suffix `.conf`, one for each boot menu item. The file name of the file is used for identification of the boot item but shall never be presented to the user in the UI. The file name may be chosen -freely but should be unique enough to avoid clashes between OS -installations. More specifically it is suggested to include the machine ID -(`/etc/machine-id` or the D-Bus machine ID for OSes that lack -`/etc/machine-id`), the kernel version (as returned by `uname -r`) and an OS -identifier (The ID field of `/etc/os-release`). Example: -`$BOOT/loader/entries/6a9857a393724b7a981ebb5b8495b9ea-3.8.0-2.fc19.x86_64.conf`. +freely but should be unique enough to avoid clashes between OS installations. +More specifically, it is suggested to include the machine ID (`/etc/machine-id` +or the D-Bus machine ID for OSes that lack `/etc/machine-id`), the kernel +version (as returned by `uname -r`) and an OS identifier (the `ID=` field of +`/etc/os-release`). + +Example: `$BOOT/loader/entries/6a9857a393724b7a981ebb5b8495b9ea-3.8.0-2.fc19.x86_64.conf`. In order to maximize compatibility with file system implementations and restricted boot loader environments, and to minimize conflicting character use with other programs, file names shall be chosen from a restricted character -set: ASCII upper and lower case characters, digits, "+", "-", "_" and -".". Also, the file names should have a length of at least one and at most 255 -characters (including file name suffix). +set: ASCII upper and lower case characters, digits, "+", "-", "_" and ".". +Also, the file names should have a length of at least one and at most 255 +characters (including the file name suffix). -These configuration snippets shall be Unix-style text files (i.e. line -separation with a single newline character), in the UTF-8 encoding. The -configuration snippets are loosely inspired on Grub1's configuration -syntax. Lines beginning with '#' shall be ignored and used for commenting. The -first word of a line is used as key and shall be separated by one or more -spaces from its value. The following keys are known: +These configuration snippets shall be UNIX-style text files (i.e. lines +separated by a single newline character), in the UTF-8 encoding. The +configuration snippets are loosely inspired by Grub1's configuration syntax. +Lines beginning with "#" are used for comments and shall be ignored. The first +word of a line is used as key and is separated by one or more spaces from the +value. -* `title` shall contain a human readable title string for this menu item. This - will be displayed in the boot menu for the item. It is a good idea to - initialize this from the `PRETTY_NAME` of `/etc/os-release`. This name should - be descriptive and does not have to be unique. If a boot loader discovers two - entries with the same title it is a good idea to show more than just the raw - title in the UI, for example by appending the `version` field. This field is - optional. Example: "Fedora 18 (Spherical Cow)". -* `version` shall contain a human readable version string for this menu - item. This is usually the kernel version and is intended for use by OSes to - install multiple kernel versions at the same time with the same `title` - field. This field shall be in a syntax that is useful for Debian-style - version sorts, so that the boot loader UI can determine the newest version - easily and show it first or preselect it automatically. This field is - optional. Example: `3.7.2-201.fc18.x86_64`. -* `machine-id` shall contain the machine ID of the OS `/etc/machine-id`. This - is useful for boot loaders and applications to filter out boot entries, for - example to show only a single newest kernel per OS, or to group items by OS, - or to maybe filter out the currently booted OS in UIs that want to show only - other installed operating systems. This ID shall be formatted as 32 lower - case hexadecimal characters (i.e. without any UUID formatting). This key is - optional. Example: `4098b3f648d74c13b1f04ccfba7798e8`. -* `sort-key` shall contain a short string used for sorting entries on - display. This can be defined freely though should typically be initialized - from `IMAGE_ID=` or `ID=` from `/etc/os-release` of the relevant entry, - possibly suffixed. This field is optional. If set, it is used as primary - sorting key for the entries on display (lexicographically increasing). It - does not have to be unique (and usually is not). If non-unique the the - `machine-id` (lexicographically increasing) and `version` (lexicographically - decreasing, i.e. newest version first) fields described above are used as - secondary/ternary sorting keys. If this field is not set entries are - typically sorted by the `.conf` file name of the entry. -* `linux` refers to the Linux kernel to spawn and shall be a path relative to - `$BOOT`. It is recommended that every distribution creates a machine id and - version specific subdirectory below `$BOOT` and places its kernels and - initial RAM disk images there. Example: - `/6a9857a393724b7a981ebb5b8495b9ea/3.8.0-2.fc19.x86_64/linux`. -* `initrd` refers to the initrd to use when executing the kernel. This also +#### Type #1 Boot Loader Entry Keys + +The following keys are recognized: + +* `title` is a human-readable title for this menu item to be displayed in the + boot menu. It is a good idea to initialize this from the `PRETTY_NAME=` of + `/etc/os-release`. This name should be descriptive and does not have to be + unique. If a boot loader discovers two entries with the same title it should + show more than just the raw title in the UI, for example by appending the + `version` field. This field is optional. + + Example: `title Fedora 18 (Spherical Cow)` + +* `version` is a human-readable version for this menu item. This is usually the + kernel version and is intended for use by OSes to install multiple kernel + versions with the same `title` field. This field shall be in a syntax that is + useful for Debian-style version sorts, so that the boot loader UI can + determine the newest version easily and show it first or preselect it + automatically. This field is optional. + + Example: `version 3.7.2-201.fc18.x86_64` + +* `machine-id` is the machine ID of the OS. This can be used by boot loaders + and applications to filter out boot entries, for example to show only a + single newest kernel per OS, to group items by OS, or to filter out the + currently booted OS when showing only other installed operating systems. + This ID shall be formatted as 32 lower case hexadecimal characters + (i.e. without any UUID formatting). This key is optional. + + Example: `machine-id 4098b3f648d74c13b1f04ccfba7798e8` + +* `sort-key` is a short string used for sorting entries on display. This should + typically be initialized from the `IMAGE_ID=` or `ID=` fields of + `/etc/os-release`, possibly with an additional suffix. This field is + optional. If set, it is used as primary sorting key for the entries on + display (lexicographically increasing). It does not have to be unique (and + usually is not). If non-unique the the `machine-id` (lexicographically + increasing) and `version` (lexicographically decreasing, i.e. newest version + first) fields described above are used as secondary/ternary sorting keys. If + this field is not set entries are typically sorted by the `.conf` file name + of the entry. + + Example: `sort-key fedora` + +* `linux` is the Linux kernel to spawn and as a path relative to `$BOOT`. It + is recommended that every distribution creates a machine id and version + specific subdirectory below `$BOOT` and places its kernels and initial RAM + disk images there. + + Example: `linux /6a9857a393724b7a981ebb5b8495b9ea/3.8.0-2.fc19.x86_64/linux` + +* `initrd` is the initrd to use when executing the kernel. This also shall be a path relative to `$BOOT`. This key is optional. This key may appear more than once in which case all specified images are used, in the - order they are listed. Example: - `6a9857a393724b7a981ebb5b8495b9ea/3.8.0-2.fc19.x86_64/initrd`. + order they are listed. + + Example: `initrd 6a9857a393724b7a981ebb5b8495b9ea/3.8.0-2.fc19.x86_64/initrd` + * `efi` refers to an arbitrary EFI program. This also takes a path relative to `$BOOT`. If this key is set, and the system is not an EFI system this entry should be hidden. + * `options` shall contain kernel parameters to pass to the Linux kernel to spawn. This key is optional and may appear more than once in which case all specified parameters are used in the order they are listed. + + Example: `options root=UUID=6d3376e4-fc93-4509-95ec-a21d68011da2 quiet` + * `devicetree` refers to the binary device tree to use when executing the kernel. This also shall be a path relative to `$BOOT`. This key is - optional. Example: - `6a9857a393724b7a981ebb5b8495b9ea/3.8.0-2.fc19.armv7hl/tegra20-paz00.dtb`. + optional. + + Example: `devicetree 6a9857a393724b7a981ebb5b8495b9ea/3.8.0-2.fc19.armv7hl/tegra20-paz00.dtb` + * `devicetree-overlay` refers to a list of device tree overlays that should be applied by the boot loader. Multiple overlays are separated by spaces and applied in the same order as they are listed. This key is optional but - depends on the `devicetree` key. Example: - `/6a9857a393724b7a981ebb5b8495b9ea/overlays/overlay_A.dtbo - /6a9857a393724b7a981ebb5b8495b9ea/overlays/overlay_B.dtbo` -* `architecture` refers to the architecture this entry is defined for. The - argument should be an architecture identifier, using the architecture - vocabulary defined by the EFI specification (i.e. `IA32`, `x64`, `IA64`, - `ARM`, `AA64`, …). If specified and this does not match (case insensitively) - the local system architecture this entry should be hidden. + depends on the `devicetree` key. + + Example: `devicetree-overlay /6a9857a393724b7a981ebb5b8495b9ea/overlays/overlay_A.dtbo /6a9857a393724b7a981ebb5b8495b9ea/overlays/overlay_B.dtbo` + +* `architecture` refers to the architecture this entry is for. The argument + should be an architecture identifier, using the architecture vocabulary + defined by the EFI specification (i.e. `IA32`, `x64`, `IA64`, `ARM`, `AA64`, + …). If specified and it does not match the local system architecture this + entry should be hidden. The comparison should be done case-insensitively. + + Example: `architecture aa64` Each configuration drop-in snippet must include at least a `linux` or an `efi` -key and is otherwise not valid. Here's an example for a complete drop-in file: +key. Here is an example for a complete drop-in file: # /boot/loader/entries/6a9857a393724b7a981ebb5b8495b9ea-3.8.0-2.fc19.x86_64.conf title Fedora 19 (Rawhide) sort-key fedora machine-id 6a9857a393724b7a981ebb5b8495b9ea version 3.8.0-2.fc19.x86_64 - options root=UUID=6d3376e4-fc93-4509-95ec-a21d68011da2 + options root=UUID=6d3376e4-fc93-4509-95ec-a21d68011da2 quiet architecture x64 linux /6a9857a393724b7a981ebb5b8495b9ea/3.8.0-2.fc19.x86_64/linux initrd /6a9857a393724b7a981ebb5b8495b9ea/3.8.0-2.fc19.x86_64/initrd @@ -309,17 +281,19 @@ focus for this specification. More specifically, on non-EFI systems configuration snippets following this specification cannot be used to spawn other operating systems (such as Windows). +### Standard-conformance Marker File + Unfortunately, there are implementations of boot loading infrastructure that -are also using the /loader/entries/ directory, but place files in them that are -not valid by this specification. In order to minimize confusion a boot loader -implementation may place a file /loader/entries.srel next to the -/loader/entries/ directory containing the ASCII string "type1" (suffixed -with a UNIX newline). Tools that need to determine whether an existing -directory implements the semantics described here may check for this file and -contents: if it exists and contains the mentioned string, it shall assume a -standards compliant implementation is in place. If it exists but contains a -different string it shall assume non-standard semantics are implemented. If the -file does not exist no assumptions should be made. +are also using the `/loader/entries/` directory, but installing files that do +not follow this specification. In order to minimize confusion, a boot loader +implementation may place the file `/loader/entries.srel` next to the +`/loader/entries/` directory containing the ASCII string `type1` (followed by a +UNIX newline). Tools that need to determine whether an existing directory +implements the semantics described here may check for this file and contents: +if it exists and contains the mentioned string, it shall assume a +standards-compliant implementation is in place. If it exists but contains a +different string it shall assume other semantics are implemented. If the file +does not exist, no assumptions should be made. ### Type #2 EFI Unified Kernel Images @@ -327,31 +301,31 @@ A unified kernel image is a single EFI PE executable combining an EFI stub loader, a kernel image, an initramfs image, and the kernel command line. See the description of the `--uefi` option in [dracut(8)](http://man7.org/linux/man-pages/man8/dracut.8.html). Such unified -images will be searched for under `$BOOT/EFI/Linux/` and must have the +images are installed in the`$BOOT/EFI/Linux/` directory and must have the extension `.efi`. Support for images of this type is of course specific to systems with EFI firmware. Ignore this section if you work on systems not supporting EFI. Type #2 file names should be chosen from the same restricted character set as -Type #1 described above (but use a different file name suffix of `.efi` instead -of `.conf`). +Type #1 described above (but with the file name suffix of `.efi` instead of +`.conf`). Images of this type have the advantage that all metadata and payload that makes -up the boot entry is monopolized in a single PE file that can be signed +up the boot entry is contained in a single PE file that can be signed cryptographically as one for the purpose of EFI SecureBoot. A valid unified kernel image must contain two PE sections: -* `.cmdline` section with the kernel command line +* `.cmdline` section with the kernel command line, * `.osrel` section with an embedded copy of the [os-release](https://www.freedesktop.org/software/systemd/man/os-release.html) - file describing the image + file describing the image. -The `PRETTY_NAME=` and `VERSION_ID=` fields in the embedded os-release file are -used the same as `title` and `version` in the "boot loader specification" -entries. The `.cmdline` section is used instead of the `options` field. `linux` -and `initrd` fields are not necessary, and there is no counterpart for the -`machine-id` field. +The `PRETTY_NAME=` and `VERSION_ID=` fields in the embedded `os-release` file +are used the same as `title` and `version` in the Type #1 entries. The +`.cmdline` section is used instead of the `options` field. `linux` and `initrd` +fields are not necessary, and there is no counterpart for the `machine-id` +field. On EFI, any such images shall be added to the list of valid boot entries. @@ -374,43 +348,116 @@ path separator. This needs to be converted to an EFI-style "\\" separator in EFI boot loaders. -## Logic +## Locating boot entries -A _boot loader_ needs a file system driver to discover and read `$BOOT`, then -simply reads all files `$BOOT/loader/entries/*.conf`, and populates its boot -menu with this. On EFI, it then extends this with any unified kernel images -found in `$BOOT/EFI/Linux/*.efi`. It may also add additional entries, for -example a "Reboot into firmware" option. Optionally it may sort the menu based -on the `sort-key`, `machine-id` and `version` fields, and possibly others. It -uses the file name to identify specific items, for example in case it supports -storing away default entry information somewhere. A boot loader should -generally not modify these files. +A _boot loader_ locates `$BOOT`, then simply reads all files +`$BOOT/loader/entries/*.conf`, and populates its boot menu with this. On EFI, +it then extends this with any unified kernel images found in +`$BOOT/EFI/Linux/*.efi`. It may also add additional entries, for example a +"Reboot into firmware" option. Optionally it may sort the menu based on the +`sort-key`, `machine-id` and `version` fields, and possibly others. It uses the +file name to identify specific items, for example in case it supports storing +away default entry information somewhere. A boot loader should generally not +modify these files. For "Boot Loader Specification Entries" (Type #1), the _kernel package installer_ installs the kernel and initrd images to `$BOOT` (it is recommended to place these files in a vendor and OS and installation specific directory) and then generates a configuration snippet for it, placing this in -`$BOOT/loader/entries/xyz.conf`, with xyz as concatenation of machine id and -version information (see above). The files created by a kernel package are -private property of the kernel package and should be removed along with it. +`$BOOT/loader/entries/xyz.conf`, with "xyz" as concatenation of machine id and +version information (see above). The files created by a kernel package are tied +to the kernel package and should be removed along with it. For "EFI Unified Kernel Images" (Type #2), the vendor or kernel package -installer creates the combined image and drops it into `$BOOT/EFI/Linux/`. This -file is also private property of the kernel package and should be removed along -with it. +installer should create the combined image and drop it into +`$BOOT/EFI/Linux/`. This file is also tied to the kernel package and should be +removed along with it. A _UI application_ intended to show available boot options shall operate -similar to a boot loader, but might apply additional filters, for example by -filtering out the booted OS via the machine ID, or by suppressing all but the +similarly to a boot loader, but might apply additional filters, for example by +filtering the booted OS via the machine ID, or by suppressing all but the newest kernel versions. An _OS installer_ picks the right place for `$BOOT` as defined above (possibly -creating a partition and file system for it) and pre-creates the -`/loader/entries/` directory in it. It then installs an appropriate boot loader -that can read these snippets. Finally, it installs one or more kernel packages. +creating a partition and file system for it) and creates the `/loader/entries/` +directory in it. It then installs an appropriate boot loader that can read +these snippets. Finally, it installs one or more kernel packages. -## Out of Focus +## Additional discussion + +### Why is there a need for this specification? + +This specification brings the following advantages: + +* Installation of new boot entries is more robust, as no explicit rewriting of + configuration files is required. + +* It allows an out-of-the-box boot experience on any platform without the need + of traditional firmware mechanisms (e.g. BIOS calls, UEFI Boot Services). + +* It improves dual-boot scenarios. Without cooperation, multiple Linux + installations tend to fight over which boot loader becomes the primary one in + possession of the MBR or the boot partition, and only that one installation + can then update the boot loader configuration. Other Linux installs have to + be manually configured to never touch the MBR and instead install a + chain-loaded boot loader in their own partition headers. In this new scheme + all installations share a loader directory and no manual configuration has to + take place. All participants implicitly cooperate due to removal of name + collisions and can install/remove their own boot menu entries without + interfering with the entries of other installed operating systems. + +* Drop-in directories are now pretty ubiquitous on Linux as an easy way to + extend configuration without having to edit, regenerate or manipulate + configuration files. For the sake of uniformity, we should do the same for + the boot menu. + +* Userspace code can sanely parse boot loader configuration which is essential + with modern firmware which does not necessarily initialize USB keyboards + during boot, which makes boot menus hard to reach for the user. If userspace + code can parse the boot loader configuration too, UI can be written that + select a boot menu item to boot into before rebooting the machine, thus not + requiring interactivity during early boot. + +* To unify and thus simplify configuration of the various boot loaders, which + makes configuration of the boot loading process easier for users, + administrators, and developers alike. + +* For boot loaders with configuration _scripts_ such as grub2, adopting this + spec allows for mostly static scripts that are generated only once at first + installation, but then do not need to be updated anymore as that is done via + drop-in files exclusively. + +### Why not simply rely on the EFI boot menu logic? + +EFI is not ubiquitous, especially not in embedded systems. But even on systems +with EFI, which provides a boot options logic that can offer similar +functionality, this specfication is still needed for the following reasons: + +* The various EFI implementations implement the boot order/boot item logic to + different levels. Some firmware implementations do not offer a boot menu at + all and instead unconditionally follow the EFI boot order, booting the first + item that is working. + +* If the firmware setup is used to reset data, usually all EFI boot entries + are lost, making the system entirely unbootable, as the firmware setups + generally do not offer a UI to define additional boot items. By placing the + menu item information on disk, it is always available, even if the firmware + configuration is lost. + +* Harddisk images should be movable between machines and be bootable without + requiring firmare configuration. This also requires that the list + of boot options is defined on disk, and not in EFI variables alone. + +* EFI is not universal yet (especially on non-x86 platforms), this + specification is useful both for EFI and non-EFI boot loaders. + +* Many EFI systems disable USB support during early boot to optimize boot + times, thus making keyboard input unavailable in the EFI menu. It is thus + useful if the OS UI has a standardized way to discover available boot options + which can be booted to. + +### Out of Focus There are a couple of items that are out of focus for this specification: @@ -419,6 +466,7 @@ There are a couple of items that are out of focus for this specification: communicate to the boot loader the default boot loader entry temporarily or persistently. Defining a common scheme for this is certainly a good idea, but out of focus for this specification. + * This specification is just about "Free" Operating systems. Hooking in other operating systems (like Windows and macOS) into the boot menu is a different story and should probably happen outside of this specification. For example, @@ -426,6 +474,7 @@ There are a couple of items that are out of focus for this specification: runtime without explicit configuration (like `systemd-boot` does it), or via native configuration (for example via explicit Grub2 configuration generated once at installation). + * This specification leaves undefined what to do about systems which are upgraded from an OS that does not implement this specification. As the previous boot loader logic was largely handled by in distribution-specific From 14bacc744f0145a2de9e5db0eeaa94fe3d0401be Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Zbigniew=20J=C4=99drzejewski-Szmek?= Date: Mon, 23 May 2022 14:27:44 +0200 Subject: [PATCH 3/7] =?UTF-8?q?docs:=20/etc/os-release=20=E2=86=92=20os-re?= =?UTF-8?q?lease(3)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit It's /usr/lib/os-release now mostly. But let's link to the man page. --- docs/BOOT_LOADER_SPECIFICATION.md | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/docs/BOOT_LOADER_SPECIFICATION.md b/docs/BOOT_LOADER_SPECIFICATION.md index 5905fc59009..e02fb09d653 100644 --- a/docs/BOOT_LOADER_SPECIFICATION.md +++ b/docs/BOOT_LOADER_SPECIFICATION.md @@ -139,7 +139,7 @@ freely but should be unique enough to avoid clashes between OS installations. More specifically, it is suggested to include the machine ID (`/etc/machine-id` or the D-Bus machine ID for OSes that lack `/etc/machine-id`), the kernel version (as returned by `uname -r`) and an OS identifier (the `ID=` field of -`/etc/os-release`). +[os-release](https://www.freedesktop.org/software/systemd/man/os-release.html)). Example: `$BOOT/loader/entries/6a9857a393724b7a981ebb5b8495b9ea-3.8.0-2.fc19.x86_64.conf`. @@ -163,10 +163,11 @@ The following keys are recognized: * `title` is a human-readable title for this menu item to be displayed in the boot menu. It is a good idea to initialize this from the `PRETTY_NAME=` of - `/etc/os-release`. This name should be descriptive and does not have to be - unique. If a boot loader discovers two entries with the same title it should - show more than just the raw title in the UI, for example by appending the - `version` field. This field is optional. + [os-release](https://www.freedesktop.org/software/systemd/man/os-release.html). + This name should be descriptive and does not have to be unique. If a boot + loader discovers two entries with the same title it should show more than + just the raw title in the UI, for example by appending the `version` + field. This field is optional. Example: `title Fedora 18 (Spherical Cow)` @@ -190,14 +191,14 @@ The following keys are recognized: * `sort-key` is a short string used for sorting entries on display. This should typically be initialized from the `IMAGE_ID=` or `ID=` fields of - `/etc/os-release`, possibly with an additional suffix. This field is - optional. If set, it is used as primary sorting key for the entries on - display (lexicographically increasing). It does not have to be unique (and - usually is not). If non-unique the the `machine-id` (lexicographically - increasing) and `version` (lexicographically decreasing, i.e. newest version - first) fields described above are used as secondary/ternary sorting keys. If - this field is not set entries are typically sorted by the `.conf` file name - of the entry. + [os-release](https://www.freedesktop.org/software/systemd/man/os-release.html), + possibly with an additional suffix. This field is optional. If set, it is + used as primary sorting key for the entries on display (lexicographically + increasing). It does not have to be unique (and usually is not). If + non-unique the the `machine-id` (lexicographically increasing) and `version` + (lexicographically decreasing, i.e. newest version first) fields described + above are used as secondary/ternary sorting keys. If this field is not set + entries are typically sorted by the `.conf` file name of the entry. Example: `sort-key fedora` From 0f5a416c81917fb85ed6d8043041660c310cc8d9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Zbigniew=20J=C4=99drzejewski-Szmek?= Date: Mon, 23 May 2022 20:49:54 +0200 Subject: [PATCH 4/7] docs/BLS: rework the description of directory layout We said "`$BOOT/loader/` is the directory containing all files needed for Type #1 entries" which is blatantly wrong. And also saying that we define two directories, /loader and /loader/entries, but only ever defining the second one was not very consistent. Instead, let's say that /loader/ is for "boot loader configuration", and /loader/entries has the snippets. A new section about /// is added. This is described as the "recommended layout for additional files". Also, we said that ID= should be used in the file name, but in fact it wasn't in the example that was given, and afaik, nobody ever did that. So this part is reduced to say "kernel version (as returned by `uname -r`, including the OS identifier)". AFAIK, all distros include some form of OS identifier in the version, so this should be good enough. Since we now don't depend on autodetection (e.g. with entry-token and layout configured), the installed doesn't need to always create /loader/entries and things will still work. So don't say that the installer needs to create it. Part of the discussion is moved to the Discussion section. Overall, this brings the specification more in line with actual practice. --- docs/BOOT_LOADER_SPECIFICATION.md | 70 +++++++++++++++++++------------ 1 file changed, 43 insertions(+), 27 deletions(-) diff --git a/docs/BOOT_LOADER_SPECIFICATION.md b/docs/BOOT_LOADER_SPECIFICATION.md index e02fb09d653..4ab66b6f6ed 100644 --- a/docs/BOOT_LOADER_SPECIFICATION.md +++ b/docs/BOOT_LOADER_SPECIFICATION.md @@ -27,7 +27,7 @@ The target audience for this specification is: * Firmware developers, to add generic boot loading support directly to the firmware itself * OS installer developers, to create appropriate partitions and set up the - initial drop-in directory + initial boot loader configuration * Distribution developers, to create appropriate configuration snippets when installing or updating kernel packages * UI developers, to implement user interfaces that list and select among the @@ -118,28 +118,28 @@ multiple users of the file system unlikely. ### Type #1 Boot Loader Specification Entries -This specification defines two directories below `$BOOT`: - -* `$BOOT/loader/` is the directory containing all files needed for Type #1 - entries - -* `$BOOT/loader/entries/` is the directory containing the drop-in - snippets. This directory contains one `.conf` file for each boot menu item. +`$BOOT/loader/` is the main directory containing the configuration for the boot +loader. **Note:** _In all cases the `/loader/` directory should be located directly in the root of the file system. Specifically, if `$BOOT` is the ESP, then `/loader/` directory should be located directly in the root directory of the ESP, and not in the `/EFI/` subdirectory._ -Inside the `$BOOT/loader/entries/` directory each OS vendor may drop one or -more configuration snippets with the suffix `.conf`, one for each boot menu -item. The file name of the file is used for identification of the boot item but -shall never be presented to the user in the UI. The file name may be chosen -freely but should be unique enough to avoid clashes between OS installations. -More specifically, it is suggested to include the machine ID (`/etc/machine-id` -or the D-Bus machine ID for OSes that lack `/etc/machine-id`), the kernel -version (as returned by `uname -r`) and an OS identifier (the `ID=` field of -[os-release](https://www.freedesktop.org/software/systemd/man/os-release.html)). +`$BOOT/loader/entries/` is the directory containing the drop-in snippets +defining boot entries, one `.conf` file for each boot menu item. Each OS may +provide one or more such entries. + +The file name is used for identification of the boot item but shall never be +presented to the user in the UI. The file name may be chosen freely but should +be unique enough to avoid clashes between OS installations. More specifically, +it is suggested to include the `entry-token` (see +[kernel-install](https://www.freedesktop.org/software/systemd/man/kernel-install.html)) +or machine ID (see +[/etc/machine-id](https://www.freedesktop.org/software/systemd/man/machine-id.html)), +and the kernel version (as returned by `uname -r`, including the OS +identifier), so that the whole filename is +`$BOOT/loader/entries/-.conf`. Example: `$BOOT/loader/entries/6a9857a393724b7a981ebb5b8495b9ea-3.8.0-2.fc19.x86_64.conf`. @@ -271,16 +271,25 @@ i.e. it is a good idea that both images shipped as UEFI PE images and those which are not don't make unnecessary assumption on the underlying firmware, i.e. don't hard depend on legacy BIOS calls or UEFI boot services. -Note that these configuration snippets may only reference kernels (and EFI -programs) that reside on the same file system as the configuration snippets, -i.e. everything referenced must be contained in the same file system. This is -by design, as referencing other partitions or devices would require a -non-trivial language for denoting device paths. If kernels/initrds are to be -read from other partitions/disks the boot loader can do this in its own native -configuration, using its own specific device path language, and this is out of -focus for this specification. More specifically, on non-EFI systems -configuration snippets following this specification cannot be used to spawn -other operating systems (such as Windows). +When Type #1 configuration snippets refer to other files (for `linux`, +`initrd`, `efi`, `devicetree`, and `devicetree-overlay`), those files must be +located on the same partition. The naming of those files can be chosen by +the installer. A recommended scheme is described in the next section. + +### Recommended Directory Layout for Additional Files + +It is recommened to place the kernel and other other files comprising a single +boot loader entry in a separate directory: +`///`. This naming scheme uses the same +elements as the boot loader configuration snippet, providing the same level of +uniqueness. + +Example: `$BOOT/6a9857a393724b7a981ebb5b8495b9ea/3.8.0-2.fc19.x86_64/linux` + `$BOOT/6a9857a393724b7a981ebb5b8495b9ea/3.8.0-2.fc19.x86_64/initrd` + +Other naming schemes are possible. In particular, traditionally a flat naming +scheme with files in the root directory was used. This is not recommended +because it is hard to avoid conflicts in a multi-boot installation. ### Standard-conformance Marker File @@ -484,6 +493,13 @@ There are a couple of items that are out of focus for this specification: with the old scheme for old installations and use this new scheme only for new installations. +* Referencing kernels or initrds on other partitions other than the partition + containing the Type #1 boot loader entry. This is by design, as specifying + other partitions or devices would require a non-trivial language for denoting + device paths. In particular this means that on non-EFI systems configuration + snippets following this specification cannot be used to spawn other operating + systems (such as Windows). + ## Links From 46083ab321826d920b32de85c7328a5f39b7fff4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Zbigniew=20J=C4=99drzejewski-Szmek?= Date: Tue, 24 May 2022 21:20:36 +0200 Subject: [PATCH 5/7] basic/string-util: tweak strverscmp_improved() for some corner cases So far we had the rule that '' == '', '0_' == '0', but '_' > ''. This means that the general rule that strings are compared iteratively, and each segment that compares equal can be dropped and the comparison resumes at the following characters wasn't true in such cases. Similarly, '0~' < '0', but after dropping the common segment, '~' > ''. The special handling of empty strings is dropped, and '_' == '' and '~' < ''. --- src/fundamental/string-util-fundamental.c | 21 +++++++++++-------- src/test/test-string-util.c | 25 ++++++++++++++++++++++- 2 files changed, 37 insertions(+), 9 deletions(-) diff --git a/src/fundamental/string-util-fundamental.c b/src/fundamental/string-util-fundamental.c index feccb822ff7..73abc2f8c85 100644 --- a/src/fundamental/string-util-fundamental.c +++ b/src/fundamental/string-util-fundamental.c @@ -124,8 +124,8 @@ sd_int strverscmp_improved(const sd_char *a, const sd_char *b) { * (newer) 124-1 */ - if (isempty(a) || isempty(b)) - return CMP(strcmp_ptr(a, b), 0); + a = strempty(a); + b = strempty(b); for (;;) { const sd_char *aa, *bb; @@ -187,12 +187,6 @@ sd_int strverscmp_improved(const sd_char *a, const sd_char *b) { } if (is_digit(*a) || is_digit(*b)) { - /* Skip leading '0', to make 00123 equivalent to 123. */ - while (*a == '0') - a++; - while (*b == '0') - b++; - /* Find the leading numeric segments. One may be an empty string. So, * numeric segments are always newer than alpha segments. */ for (aa = a; is_digit(*aa); aa++) @@ -200,6 +194,17 @@ sd_int strverscmp_improved(const sd_char *a, const sd_char *b) { for (bb = b; is_digit(*bb); bb++) ; + /* Check if one of the strings was empty, but the other not. */ + r = CMP(a != aa, b != bb); + if (r != 0) + return r; + + /* Skip leading '0', to make 00123 equivalent to 123. */ + while (*a == '0') + a++; + while (*b == '0') + b++; + /* To compare numeric segments without parsing their values, first compare the * lengths of the segments. Eg. 12345 vs 123, longer is newer. */ r = CMP(aa - a, bb - b); diff --git a/src/test/test-string-util.c b/src/test/test-string-util.c index 1054f9ea31d..2faceca4e9e 100644 --- a/src/test/test-string-util.c +++ b/src/test/test-string-util.c @@ -852,8 +852,8 @@ static void test_strverscmp_improved_newer(const char *older, const char *newer) TEST(strverscmp_improved) { static const char * const versions[] = { - "", "~1", + "", "ab", "abb", "abc", @@ -917,6 +917,29 @@ TEST(strverscmp_improved) { /* invalid characters */ assert_se(strverscmp_improved("123_aa2-67.89", "123aa+2-67.89") == 0); + /* some corner cases */ + assert_se(strverscmp_improved("123.", "123") > 0); /* One more version segment */ + assert_se(strverscmp_improved("12_3", "123") < 0); /* 12 < 123 */ + assert_se(strverscmp_improved("12_3", "12") > 0); /* 3 > '' */ + assert_se(strverscmp_improved("12_3", "12.3") > 0); /* 3 > '' */ + assert_se(strverscmp_improved("123.0", "123") > 0); /* 0 > '' */ + assert_se(strverscmp_improved("123_0", "123") > 0); /* 0 > '' */ + assert_se(strverscmp_improved("123..0", "123.0") < 0); /* '' < 0 */ + + /* empty strings or strings with ignored characters only */ + assert_se(strverscmp_improved("", NULL) == 0); + assert_se(strverscmp_improved(NULL, "") == 0); + assert_se(strverscmp_improved("0_", "0") == 0); + assert_se(strverscmp_improved("_0_", "0") == 0); + assert_se(strverscmp_improved("_0", "0") == 0); + assert_se(strverscmp_improved("0", "0___") == 0); + assert_se(strverscmp_improved("", "_") == 0); + assert_se(strverscmp_improved("_", "") == 0); + assert_se(strverscmp_improved("_", "_") == 0); + assert_se(strverscmp_improved("", "~") > 0); + assert_se(strverscmp_improved("~", "") < 0); + assert_se(strverscmp_improved("~", "~") == 0); + /* non-ASCII digits */ (void) setlocale(LC_NUMERIC, "ar_YE.utf8"); assert_se(strverscmp_improved("1٠١٢٣٤٥٦٧٨٩", "1") == 0); From 2367bdcfc90198620ea2b905f450fc514fd93d93 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Zbigniew=20J=C4=99drzejewski-Szmek?= Date: Tue, 24 May 2022 16:25:58 +0200 Subject: [PATCH 6/7] docs/BLS: describe version comparisons Fixes #23346. --- docs/BOOT_LOADER_SPECIFICATION.md | 124 ++++++++++++++++++++-- src/fundamental/string-util-fundamental.c | 4 +- 2 files changed, 116 insertions(+), 12 deletions(-) diff --git a/docs/BOOT_LOADER_SPECIFICATION.md b/docs/BOOT_LOADER_SPECIFICATION.md index 4ab66b6f6ed..d489208542c 100644 --- a/docs/BOOT_LOADER_SPECIFICATION.md +++ b/docs/BOOT_LOADER_SPECIFICATION.md @@ -173,11 +173,12 @@ The following keys are recognized: * `version` is a human-readable version for this menu item. This is usually the kernel version and is intended for use by OSes to install multiple kernel - versions with the same `title` field. This field shall be in a syntax that is - useful for Debian-style version sorts, so that the boot loader UI can - determine the newest version easily and show it first or preselect it + versions with the same `title` field. This field is used for sorting entries, + so that the boot loader can order entries by age or select the newest one automatically. This field is optional. + See [Sorting](#sorting) below. + Example: `version 3.7.2-201.fc18.x86_64` * `machine-id` is the machine ID of the OS. This can be used by boot loaders @@ -192,13 +193,7 @@ The following keys are recognized: * `sort-key` is a short string used for sorting entries on display. This should typically be initialized from the `IMAGE_ID=` or `ID=` fields of [os-release](https://www.freedesktop.org/software/systemd/man/os-release.html), - possibly with an additional suffix. This field is optional. If set, it is - used as primary sorting key for the entries on display (lexicographically - increasing). It does not have to be unique (and usually is not). If - non-unique the the `machine-id` (lexicographically increasing) and `version` - (lexicographically decreasing, i.e. newest version first) fields described - above are used as secondary/ternary sorting keys. If this field is not set - entries are typically sorted by the `.conf` file name of the entry. + possibly with an additional suffix. This field is optional. Example: `sort-key fedora` @@ -393,6 +388,100 @@ creating a partition and file system for it) and creates the `/loader/entries/` directory in it. It then installs an appropriate boot loader that can read these snippets. Finally, it installs one or more kernel packages. +## Sorting + +The boot loader menu should generally show entries in some order meaningful to +the user. The `title` key is free-form and not suitable to be used as the +primary sorting key. Instead, the boot loader should use the following rules: +if `sort-key` is set on both entries, use in order of priority, +the `sort-key` (A-Z, increasing [alphanumerical order](#alphanumerical-order)), +`machine-id` (A-Z, increasing alphanumerical order), +and `version` keys (decreasing [version order](#version-order)). +If `sort-key` is set on one entry, it sorts earlier. +At the end, if necessary, when `sort-key` is not set or those fields are not +set or are all equal, the boot loader should sort using the file name of the +entry (decreasing version sort), with the suffix removed. + +**Note:** _This description assumes that the boot loader shows entries in a +traditional menu, with newest and "best" entries at the top, thus entries with +a higher version number are sorter *earlier*. The boot loader is free to +use a different direction (or none at all) during display._ + +### Alphanumerical order + +Free-form strings and machine IDs should be compared using a method equivalent +to [strcmp(3)](https://man7.org/linux/man-pages/man3/strcmp.3.html) on their +UTF-8 represenations. If just one of the strings is unspecified or empty, it +compares lower. If both strings are unspecified or empty, they compare equal. + +### Version order + +The following method should be used to compare version strings. The algorithm +is based on rpm's `rpmvercmp()`, but not identical. + +ASCII letters (`a-z`, `A-Z`) and digits (`0-9`) form alphanumerical components of the version. +Minus (`-`) separates the version and release parts. +Dot (`.`) separates parts of version or release. +Tilde (`~`) is a prefix that always compares lower. +Caret (`^`) is a prefix that always compares higher. + +Both strings are compared from the beginning until the end, or until the +strings are found to compare as different. In a loop: +1. Any characters which are outside of the set of listed above (`a-z`, `A-Z`, `0-9`, `-`, `.`, `~`, `^`) + are skipped in both strings. In particular, this means that non-ASCII characters + that are Unicode digits or letters are skipped too. +2. If one of the strings has ended: if the other string hasn't, the string that + has remaining characters compares higher. Otherwise, the strings compare + equal. +3. If the remaining part of one of strings starts with `~`: + if other remaining part does not start with `~`, + the string with `~` compares lower. Otherwise, both tilde characters are skipped. +4. The check from point 2. is repeated here. +5. If the remaining part of one of strings starts with `-`: + if the other remaining part does not start with `-`, + the string with `-` compares lower. Otherwise, both minus characters are skipped. +6. If the remaining part of one of strings starts with `^`: + if the other remaining part does not start with `^`, + the string with `^` compares higher. Otherwise, both caret characters are skipped. +6. If the remaining part of one of strings starts with `.`: + if the other remaining part does not start with `.`, + the string with `.` compares lower. Otherwise, both dot characters are skipped. +7. If either of the remaining parts starts with a digit, numerical prefixes are + compared numerically. Any leading zeroes are skipped. + The numerical prefixes (until the first non-digit character) are evaluated as numbers. + If one of the prefixes is empty, it evaluates as 0. + If the numbers are different, the string with the bigger number compares higher. + Otherwise, the comparison continues at the following characters at point 1. +8. Leading alphabetical prefixes are compared alphabetically. + The substrings are compared letter-by-letter. + If both letters are the same, the comparison continues with the next letter. + Capital letters compare lower than lower-case letters (`A < a`). + When the end of one substring has been reached (a non-letter character or the end + of the whole string), if the other substring has remaining letters, it compares higher. + Otherwise, the comparison continues at the following characters at point 1. + +Examples (with '' meaning the empty string): + +* `11 == 11` +* `systemd-123 == systemd-123` +* `bar-123 < foo-123` +* `123a > 123` +* `123.a > 123` +* `123.a < 123.b` +* `123a > 123.a` +* `11α == 11β` +* `A < a` +* '' < `0` +* `0.` > `0` +* `0.0` > `0` +* `0` < `~` +* '' < `~` + +Note: [systemd-analyze](https://www.freedesktop.org/software/systemd/man/systemd-analyze.html) +implements this version comparison algorithm as +``` +systemd-analyze compare-versions +``` ## Additional discussion @@ -467,6 +556,21 @@ functionality, this specfication is still needed for the following reasons: useful if the OS UI has a standardized way to discover available boot options which can be booted to. +### Why is the version comparsion logic so complicated? + +The `sort-key` allows us to group entries by "operating system", e.g. all +versions of Fedora together, no matter if they identify themselves as "Fedora +Workstation" or "Fedora Rawhide (prerelease)". The `sort-key` was introduced +only recently, so we need to provide a meaningful order for entries both with +and without it. Since it is a new concept, it is assumed that entries with +`sort-key` are newer. + +In a traditional menu with entries displayed vertically, we want names to be +sorter alpabetically (CentOS, Debian, Fedora, OpenSUSE, …), it would be strange +to have them in reverse order. But when multiple kernels are available for the +same installation, we want to display the latest kernel with highest priority, +i.e. earlier in the list. + ### Out of Focus There are a couple of items that are out of focus for this specification: diff --git a/src/fundamental/string-util-fundamental.c b/src/fundamental/string-util-fundamental.c index 73abc2f8c85..169568e2446 100644 --- a/src/fundamental/string-util-fundamental.c +++ b/src/fundamental/string-util-fundamental.c @@ -150,7 +150,7 @@ sd_int strverscmp_improved(const sd_char *a, const sd_char *b) { } /* If at least one string reaches the end, then longer is newer. - * Note that except for '~' prefixed segments, a string has more segments is newer. + * Note that except for '~' prefixed segments, a string which has more segments is newer. * So, this check must be after the '~' check. */ if (*a == '\0' || *b == '\0') return CMP(*a, *b); @@ -233,7 +233,7 @@ sd_int strverscmp_improved(const sd_char *a, const sd_char *b) { return r; } - /* The current segments are equivalent. Let's compare the next one. */ + /* The current segments are equivalent. Let's move to the next one. */ a = aa; b = bb; } From 53c26db4dac0f5b79ca2a57364ee7df78a14bbfd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Zbigniew=20J=C4=99drzejewski-Szmek?= Date: Wed, 25 May 2022 13:41:29 +0200 Subject: [PATCH 7/7] docs/BLS: clear up the confusion about what $BOOT means The text used was originally written for everything being on the ESP. It was later generalized for support XBOOTLDR, and "$BOOT" was introduced to mean something like "XBOOTLDR if present, the ESP otherwise", and most of the text was changed to talk about $BOOT. Sadly, this doesn't work, because the two partitions are not interchangeable. sd-boot loads entries from both partitions, and its configuration, random-seed, etc. only from the ESP. The terms are redefined: $BOOT now means either the ESP or the "boot partition" playing the same role on MBR systems, and $XBOOTLDR is XBOOTLDR. Like various previous commits, this makes the specification describe our current implementation. Also, the let's just accept the common practice of using /boot and /boot/efi. Since both partitions need to be read to gather configuration, it isn't a problem that one is mounted underneath the other one. I think having /boot and /efi is OK, but not better in any measureable way, so let's stop trying to push people towards this setup. A note that XBOOTLDR must be on the same disk as ESP is added. --- docs/BOOT_LOADER_SPECIFICATION.md | 161 +++++++++++++++--------------- 1 file changed, 82 insertions(+), 79 deletions(-) diff --git a/docs/BOOT_LOADER_SPECIFICATION.md b/docs/BOOT_LOADER_SPECIFICATION.md index d489208542c..e96aac69c48 100644 --- a/docs/BOOT_LOADER_SPECIFICATION.md +++ b/docs/BOOT_LOADER_SPECIFICATION.md @@ -35,52 +35,58 @@ The target audience for this specification is: ## The boot partition -Everything described below is located on a file system that will be called -`$BOOT`. The boot loader or user-space programs reading the boot loader -configuration should pick `$BOOT` according to the following rules: +Everything described below is located on one or two partitions. The boot loader +or user-space programs reading the boot loader configuration should locate them +in the following manner: * On disks with an MBR partition table: - * A partition with the type ID of 0xEA shall be used as `$BOOT`. + * The boot partition — partition with the type ID of 0xEA — shall be used + for boot loader configuration and entries. * On disks with GPT (GUID Partition Table) - * If an Extended Boot Loader Partition (XBOOTLDR partition for short) — a - partition with GPT type GUID of `bc13c2ff-59e6-4262-a352-b275fd6f7172` — - is found, it should be used as `$BOOT`. + * The EFI System Partition (ESP for short) — a partition with GPT type GUID + of `c12a7328-f81f-11d2-ba4b-00a0c93ec93b` — should be used for boot loader + configuration and boot entries. - * Otherwise, the EFI System Partition (ESP for short) — a partition with - GPT type GUID of `c12a7328-f81f-11d2-ba4b-00a0c93ec93b` — should - be used as `$BOOT`. + * Optionally, an Extended Boot Loader Partition (XBOOTLDR partition for + short) — a partition with GPT type GUID of + `bc13c2ff-59e6-4262-a352-b275fd6f7172` — may be used as an additional + location for boot loader entries. This partition must be located on the + same disk as the ESP. + +In the text below, `$BOOT` will be used to refer to (the root of) the first of +the two partitions (the boot partition on MBR disks and the ESP on GPT disks), +and `$XBOOTLDR` will be used to refer to (the root of) the optional second +partition. An installer for the operating system should use this logic when selecting or creating partitions: * If `$BOOT` is not found, a new suitably sized partition (let's say 500MB) - should be created and used as `$BOOT`, matching the characteristics - described above. On disks with GPT, just an ESP partition without a - XBOOTLDR partition should be created. + should be created, matching the characteristics described above. On disks + with GPT, only the ESP partition without the XBOOTLDR partition should be + created. * If the OS is installed on a disk with GPT and the ESP partition is found but is too small, a new suitably sized (let's say 500MB) XBOOTLDR partition - shall be created and used as `$BOOT`. + shall be created. -This placeholder file system shall be determined during _installation time_, -and an fstab entry may be created. It should be mounted to either `/boot/` or -`/efi/`. Additional locations like `/boot/efi/` (with `/boot/` being a separate -file system) might be supported by implementations. This is not recommended -because the mounting of `$BOOT` is then dependent on and requires the mounting -of the intermediate file system. +Those file systems shall be determined during _installation time_, and an fstab +entry may be created. If only one partition is used, it should be mounted on +`/boot/`. If both XBOOTLDR partition and the ESP are used, they should be +mounted on `/boot` and `/efi`, or on `/boot` and `/boot/efi`. -**Note:** _`$BOOT` is **shared** among all OS installations of a system. -Instead of maintaining one `$BOOT` per installed OS (as `/boot/` was -traditionally handled), all installed OSes use the same place for boot-time +**Note:** _Those file systems are **shared** among all OS installations on the +system. Instead of maintaining one boot partition per installed OS (as `/boot/` +was traditionally handled), all installed OSes use the same place for boot-time configuration._ -For systems where the firmware is able to read file systems directly, `$BOOT` -must be a file system readable by the firmware. For other systems and generic -installation and live media, `$BOOT` must be a VFAT (16 or 32) file system. -Applications accessing `$BOOT` should hence not assume that fancier file system +For systems where the firmware is able to read file systems directly, the ESP +must — and the XBOOTLDR partition should — be a file system readable by the +firmware. For most systems this means VFAT (16 or 32 bit). Applications +accessing both partitions should hence not assume that fancier file system features such as symlinks, hardlinks, access control or case sensitivity are supported. @@ -102,33 +108,32 @@ from the user. Only entries matching the feature set of boot loader and system shall be considered and displayed. This allows image builders to put together images that transparently support multiple different architectures. -Note that the `$BOOT` partition is not supposed to be exclusive territory of +Note that the boot partitions are not supposed to be the exclusive territory of this specification. This specification only defines semantics of the `/loader/` directory inside the file system (see below), but it doesn't intend to define -ownership of the whole file system exclusively. Boot loaders, firmware, and -other software implementing this specification may choose to place other files -and directories in the same file system. For example, boot loaders that -implement this specification might install their own boot code on the same -partition; on systems where `$BOOT` is the ESP this is a particularly common -setup. Implementations of this specification must be able to operate correctly -if files or directories other than `/loader/` are found in the top level -directory. Implementations that add their own files or directories to the file -systems should use well-named directories, to make name collisions between -multiple users of the file system unlikely. +ownership of the whole file system. Boot loaders, firmware, and other software +implementing this specification may choose to place other files and directories +in the same file system. For example, boot loaders that implement this +specification might install their own boot code on the same partition; this is +particularly common in the case of the ESP. Implementations of this specification +must be able to operate correctly if files or directories other than `/loader/` +are found in the top level directory. Implementations that add their own files +or directories to the file systems should use well-named directories, to make +name collisions between multiple users of the file system unlikely. ### Type #1 Boot Loader Specification Entries -`$BOOT/loader/` is the main directory containing the configuration for the boot +`$ESP/loader/` is the main directory containing the configuration for the boot loader. **Note:** _In all cases the `/loader/` directory should be located directly in -the root of the file system. Specifically, if `$BOOT` is the ESP, then -`/loader/` directory should be located directly in the root directory of the -ESP, and not in the `/EFI/` subdirectory._ +the root of the file system. Specifically, the `/loader/` directory should +**not** be located under the `/EFI/` subdirectory on the ESP._ -`$BOOT/loader/entries/` is the directory containing the drop-in snippets -defining boot entries, one `.conf` file for each boot menu item. Each OS may -provide one or more such entries. +`$BOOT/loader/entries/` and `$XBOOTLDR/loader/entries/` are the directories +containing the drop-in snippets defining boot entries, one `.conf` file for +each boot menu item. Each OS may provide one or more such entries. The boot +loader should enumerate both directories and provide a merged list. The file name is used for identification of the boot item but shall never be presented to the user in the UI. The file name may be chosen freely but should @@ -197,23 +202,21 @@ The following keys are recognized: Example: `sort-key fedora` -* `linux` is the Linux kernel to spawn and as a path relative to `$BOOT`. It - is recommended that every distribution creates a machine id and version - specific subdirectory below `$BOOT` and places its kernels and initial RAM - disk images there. +* `linux` is the Linux kernel to spawn and as a path relative to file system + root. It is recommended that every distribution creates a machine id and + version specific subdirectory and places its kernels and initial RAM disk + images there. Example: `linux /6a9857a393724b7a981ebb5b8495b9ea/3.8.0-2.fc19.x86_64/linux` -* `initrd` is the initrd to use when executing the kernel. This also - shall be a path relative to `$BOOT`. This key is optional. This key may - appear more than once in which case all specified images are used, in the - order they are listed. +* `initrd` is the initrd to use when executing the kernel. This key is + optional. This key may appear more than once in which case all specified + images are used, in the order they are listed. Example: `initrd 6a9857a393724b7a981ebb5b8495b9ea/3.8.0-2.fc19.x86_64/initrd` -* `efi` refers to an arbitrary EFI program. This also takes a path relative to - `$BOOT`. If this key is set, and the system is not an EFI system this entry - should be hidden. +* `efi` refers to an arbitrary EFI program. If this key is set, and the system + is not an EFI system, this entry should be hidden. * `options` shall contain kernel parameters to pass to the Linux kernel to spawn. This key is optional and may appear more than once in which case all @@ -222,8 +225,7 @@ The following keys are recognized: Example: `options root=UUID=6d3376e4-fc93-4509-95ec-a21d68011da2 quiet` * `devicetree` refers to the binary device tree to use when executing the - kernel. This also shall be a path relative to `$BOOT`. This key is - optional. + kernel. This key is optional. Example: `devicetree 6a9857a393724b7a981ebb5b8495b9ea/3.8.0-2.fc19.armv7hl/tegra20-paz00.dtb` @@ -268,8 +270,9 @@ i.e. don't hard depend on legacy BIOS calls or UEFI boot services. When Type #1 configuration snippets refer to other files (for `linux`, `initrd`, `efi`, `devicetree`, and `devicetree-overlay`), those files must be -located on the same partition. The naming of those files can be chosen by -the installer. A recommended scheme is described in the next section. +located on the same partition, and the paths must be absolute paths relative to +the root of that file system. The naming of those files can be chosen by the +installer. A recommended scheme is described in the next section. ### Recommended Directory Layout for Additional Files @@ -306,10 +309,10 @@ A unified kernel image is a single EFI PE executable combining an EFI stub loader, a kernel image, an initramfs image, and the kernel command line. See the description of the `--uefi` option in [dracut(8)](http://man7.org/linux/man-pages/man8/dracut.8.html). Such unified -images are installed in the`$BOOT/EFI/Linux/` directory and must have the -extension `.efi`. Support for images of this type is of course specific to -systems with EFI firmware. Ignore this section if you work on systems not -supporting EFI. +images are installed in the`$BOOT/EFI/Linux/` and `$XBOOTLDR/EFI/Linux/` +directories and must have the extension `.efi`. +Support for images of this type is of course specific to systems with EFI +firmware. Ignore this section if you work on systems not supporting EFI. Type #2 file names should be chosen from the same restricted character set as Type #1 described above (but with the file name suffix of `.efi` instead of @@ -355,23 +358,23 @@ EFI boot loaders. ## Locating boot entries -A _boot loader_ locates `$BOOT`, then simply reads all files -`$BOOT/loader/entries/*.conf`, and populates its boot menu with this. On EFI, -it then extends this with any unified kernel images found in -`$BOOT/EFI/Linux/*.efi`. It may also add additional entries, for example a -"Reboot into firmware" option. Optionally it may sort the menu based on the -`sort-key`, `machine-id` and `version` fields, and possibly others. It uses the -file name to identify specific items, for example in case it supports storing -away default entry information somewhere. A boot loader should generally not -modify these files. +A _boot loader_ locates `$BOOT` and `$XBOOTLDR`, then simply reads all the +files `$BOOT/loader/entries/*.conf` and `$XBOOTLDR/loader/entries/*.conf`, and +populates its boot menu. On EFI, it then extends this with any unified kernel +images found in `$BOOT/EFI/Linux/*.efi` and `$XBOOTLDR/EFI/Linux/*.efi`. It may +also add additional entries, for example a "Reboot into firmware" option. +Optionally it may sort the menu based on the `sort-key`, `machine-id` and +`version` fields, and possibly others. It uses the file name to identify +specific items, for example in case it supports storing away default entry +information somewhere. A boot loader should generally not modify these files. For "Boot Loader Specification Entries" (Type #1), the _kernel package -installer_ installs the kernel and initrd images to `$BOOT` (it is recommended -to place these files in a vendor and OS and installation specific directory) -and then generates a configuration snippet for it, placing this in -`$BOOT/loader/entries/xyz.conf`, with "xyz" as concatenation of machine id and -version information (see above). The files created by a kernel package are tied -to the kernel package and should be removed along with it. +installer_ installs the kernel and initrd images to `$XBOOTLDR` (if used) or +`$BOOT`. It is recommended to place these files in a vendor and OS and +installation specific directory. It then generates a configuration snippet, +placing it in `$BOOT/loader/entries/xyz.conf`, with "xyz" as concatenation of +machine id and version information (see above). The files created by a kernel +package are tied to the kernel package and should be removed along with it. For "EFI Unified Kernel Images" (Type #2), the vendor or kernel package installer should create the combined image and drop it into