110 Commits

Author SHA1 Message Date
Andrey Smirnov
6f7c3a8e5c
fix: build of talosctl on non-Linux arches
Move META constants out to machinery, and fix up imports. The internal
`pkg/meta` package shold not be consumed in public-facing commands.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-30 22:17:38 +04:00
Andrey Smirnov
b453385bd9
feat: support volume configuration, provisioning, etc
This implements the first round of changes, replacing the volume backend
with the new implementation, while keeping most of the external
interfaces intact.

See #8367

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-30 18:32:34 +04:00
Noel Georgi
19a44c2b0b
chore: drop console ttyS0 argument
Drop `console=ttyS0` argument for metal images/installer.

`console=ttyS0` causes lot of issues with bare metal hardware when
trying to use a physical serial port.

Ref:

* https://bugzilla.redhat.com/show_bug.cgi?id=1839923
* https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=763601;msg=17
* https://www.kernel.org/doc/html/latest/admin-guide/serial-console.html
* https://github.com/coreos/fedora-coreos-tracker/issues/567

Fixes: #8695
Fixes: #8657
Fixes: #8127

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-08-27 22:24:59 +05:30
Dmitriy Matrenichev
dad9c40c73
chore: simplify code
- replace `interface{}` with `any` using `gofmt -r 'interface{} -> any -w'`
- replace `a = []T{}` with `var a []T` where possible.
- replace `a = []T{}` with `a = make([]T, 0, len(b))` where possible.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-07-08 18:14:00 +03:00
Andrey Smirnov
be35f380cc
chore: update pkgs/tools/extras
This brings in Go 1.22.5 and new Flannel CNI plugin.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-03 20:38:55 +04:00
Andrey Smirnov
82d9cd3229
fix: add upgrade errata for arm64/zboot kernels
Fixes #8854

Talos 1.8.0 instroduces EFI ZBoot compression, and kexec from 1.7.0 to
compressed kernel doesn't work.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-06-03 20:10:24 +04:00
Andrey Smirnov
92a274e9a0
fix: workaround problems with udevd races
When `udevd` rescans block device partitions while Talos is doing
partitions, it might be that Talos can hit the following error
while trying to open/mount a partition:

```
no such device or address
```

Previous attempts to fix that were using `ENODEV`, while the proper code
is `ENXIO`.

Also take exclusive lock while working with user disks to prevent
concurrent udevd rescan.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-05-24 21:22:07 +04:00
Andrey Smirnov
4c0c626b78
feat: use zstd compression in place of xz
Initramfs and kernel are compressed with zstd.

Extensions are compressed with zstd for Talos 1.8+.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-29 18:09:12 +04:00
Andrey Smirnov
0a785802ea
fix: overlay installer operations
1. Use overlay installer to build the `cmdline` when running in
   install/upgrade mode.

2. Pull down the overlay installer with the arch specific to the
   installer being generated, vs. the arch of the `imager`.

3. Print a message when running an overlay installer.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-16 20:07:44 +04:00
Artem Chernyshev
3dd1f4e88c
chore: extract pkg/imager/quirks to pkg/machinery
To make it possible to use it without pulling the whole Talos.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2024-04-15 21:37:47 +03:00
Andrey Smirnov
090143b030
fix: allow platform cmdline args to be platform-specific
Fix Equnix Metal (where proper arm64 args are known) and metal platform
(using generic arm64 console arg).

Other platforms might need to be updated, but correct settings are not
known at the moment.

Fixes #8529

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-02 14:41:39 +04:00
Andrey Smirnov
7d43c9aa6b
chore: annotate installer errors
I want to catch a spurious error `ENODEV`, where exactly it comes from.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-21 16:58:34 +04:00
Dmitriy Matrenichev
19f15a840c
chore: bump golangci-lint to 1.57.0
Fix all discovered issues.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-03-21 01:06:53 +03:00
Noel Georgi
952801d8b2
fix: handle overlay partition options
Handling of Overlay PartitionOpts was missed in the previous code.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-03-14 15:39:59 +05:30
Noel Georgi
d118a852b9
feat: implement Install for imager overlays
Implement `Install` for imager overlays.
Also add support for generating installers.

Depends on: #8377

Fixes: #8350
Fixes: #8351
Fixes: #8350

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-03-12 22:46:29 +05:30
Artem Chernyshev
3c8f51d707
chore: move cli formatters and version modules to machinery
To be used in the `go-talos-support` module without importing the whole
Talos repo.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2024-03-07 16:29:15 +03:00
Andrey Smirnov
d81d490003
chore: update CoreDNS renovate source
As we're using a mirrored image from `registry.k8s.io`, use that as a
source instead of GitHub. Mirrored image appears with some delay after
an official CoreDNS release.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-27 17:12:25 +04:00
Andrey Smirnov
8872a7a210
fix: ignore 'no such device' in addition to 'no such file'
This errors pops up when `udevd` rescans the partition table with Talos
trying to mount a device concurrently.

This feels to be something new with Linux 6.6 probably.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-26 20:00:05 +04:00
Dmitriy Matrenichev
fa3b933705
chore: replace fmt.Errorf with errors.New where possible
This time use `eg` from `x/tools` repo tool to do this.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-14 17:39:30 +03:00
Dmitriy Matrenichev
5324d39167
chore: bump stuff
Also fix .golangci.yml file.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-09 19:19:25 +03:00
Andrey Smirnov
a5e13c696d
fix: retry blockdevice open in the installer
We had these retries in other places, but not here.

This seems to happen more frequently with Linux 6.6 update, the tl;dr is
same: `udevd` tries to rescan the partition table at the wrong moment,
preventing Talos installer to open the partition which was just created.

It's a race, so workaround it by retrying the call.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-01-31 22:17:20 +04:00
Andrey Smirnov
e0dfbb8fba
fix: allow META encoded values to be compressed
Fixes #8186

This is planned to be backported to Talos 1.6.3.

This allows to pass large META values (YAML for platform network
configuration) which might otherwise exceed the limit for kernel
command line params.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-01-23 17:24:18 +04:00
Andrey Smirnov
0a30ef7845
fix: imager should support different Talos versions
Add some quirks to make images generated with newer Talos compatible
with images generated by older Talos.

Specifically, reset options were adding in Talos 1.4, so we shouldn't
add them for older versions.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-22 16:13:34 +04:00
Andrey Smirnov
760f793d55
fix: use correct prefix when installing SBC files
When creating an image under non-default mount prefix, it should be
used explicitly when copying SBC files.

See https://github.com/siderolabs/image-factory/issues/65

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-15 19:46:10 +04:00
Andrey Smirnov
d8a435f0e4
fix: initialize boot assets with defaults early
The problem was that bootloaders were correctly picking up defaults for
`installer` mode (vs. `imager` mode), but DTB and other SBC stuff wasn't
properly initialized, so installing on SBC fails.

Now all options are properly initialized with defaults early in the
process.

Fixes #8009

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-01 17:47:05 +04:00
Noel Georgi
7a4a92854f
feat: support sanitized kernel args
Support dropping kernel args that start with `-`.

Fixes: #7613

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-11-28 16:23:05 +05:30
Andrey Smirnov
6dc776b8aa
fix: when writing to META in the installer/imager, use fixed name
Use fixed partition name instead of trying to auto-discover by label.

Auto-discovery by label might hit completely wrong blockdevice.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-01 20:34:41 +04:00
Andrey Smirnov
cbe6e7622d
fix: generate images for SBCs using imager
See https://github.com/siderolabs/image-factory/issues/43

Two fixes:

* pass path to the dtb, uboot and rpi-firmware explicitly
* include dtb, uboot and rpi-firmware into arm64 installer image when
  generated via imager (regular arm64 installer was fine)

(The generation of SBC images was not broken for Talos itself, but only
when used via Image Factory).

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-10-30 13:46:58 +04:00
Andrey Smirnov
b8fb55d5c2
fix: use a mount prefix when installing a bootloader
This is not a problem in general, but when running multiple image
generation procedures using the same mount point is a problem.

This is a no-op if `MountPrefix` is not set (when installing/upgrading
vs. creating an image).

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-08-31 22:21:41 +04:00
Andrey Smirnov
3c9f7a7de6
chore: re-enable nolintlint and typecheck linters
Drop startup/rand.go, as since Go 1.20 `rand.Seed` is done
automatically.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-08-25 01:05:41 +04:00
Andrey Smirnov
ccfa8de117
fix: automatically change rpi_4 board on upgrade
As `rpi_4` was removed in Talos 1.5, automatically migrate to
`rpi_generic`.

Fixes #7632

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-08-21 18:05:42 +04:00
Andrey Smirnov
e0f383598e
chore: clean up the output of the imager
Use `Progress`, and options to pass around the way messages are written.

Fixed some tiny issues in the code, but otherwise no functional changes.

To make colored output work with `docker run`, switched back image
generation to use volume mount for output (old mode is still
functioning, but it's not the default, and it works when docker is not
running on the same host).

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-08-07 16:00:14 +04:00
Andrey Smirnov
87fe8f1a2a
feat: implement image generation profiles
Support full configuration for image generation, including image
outputs, support most features (where applicable) for all image output
types, unify image generation process.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-08-02 19:13:44 +04:00
Artem Chernyshev
ce63abb219
feat: add KMS assisted encryption key handler
Talos now supports new type of encryption keys which rely on Sealing/Unsealing randomly generated bytes with a KMS server:

```
systemDiskEncryption:
  ephemeral:
    keys:
      - kms:
          endpoint: https://1.2.3.4:443
        slot: 0
```
gRPC API definitions and a simple reference implementation of the KMS server can be found in this
[repository](https://github.com/siderolabs/kms-client/blob/main/cmd/kms-server/main.go).

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2023-07-07 19:02:39 +03:00
Andrey Smirnov
35d6adcb9a
fix: provide stashed META values before installation
Previously, if META values were supplied to the Talos ISO via
environment variable, they will be written down and available after the
install. With this fix, values are also readable and available before
the installation runs (in maintenance mode).

Most of the PR is refactoring `meta.Value(s)` to be a shared library
which is used by the installer/imager and (now) Talos.

Also fixes an issue with not returning properly `NotExist` error when
META is not yet available as a partition on disk.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-06-27 20:57:43 +04:00
Andrey Smirnov
fe0f46980f
feat: implement secure boot from disk
This includes sd-boot handling, EFI variables, etc.

There are some TODOs which need to be addressed to make things smooth.

Install to disk, upgrades work.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-06-16 20:15:16 +05:30
Andrey Smirnov
19bc223de8
refactor: bootloader interface, labels
Move labels out of the bootloader interface, while moving copying assets
into the bootloader interface. GRUB is using one set of assets,
`sd-boot` will be using another one.

Fix the problem with `bootloader.Probe()` finding boot partition on the
host when it runs in a priv container, fixing issues with image creation
in the CI.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-06-14 17:33:11 +04:00
Noel Georgi
71a548d180
chore: generic boootloader implementation
This changes the bootloader code to be generic to support
multiple bootloader implementations.

Signed-off-by: Noel Georgi <git@frezbo.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-06-13 23:36:20 +04:00
Nanfei Chen
1865a0c296
chore: modify some usages that are not recommended
Struct MetaValue has methods on both value and pointer receivers. Such usage is not recommended by the Go Documentation. Modifies the receiver usage.

Variable config collides with imported package name. Renames the variable config.

Removes a redundant alias.

Empty slice declaration uses a literal. Replaces with nil slice declaration.

Signed-off-by: Nanfei Chen <chennanfei@yeah.net>
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-06-11 17:40:26 -04:00
Noel Georgi
1c0c7933df
chore: cleanup partition code
Cleanup partition code to be explicit about `Format` and `Partition`
options.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-06-08 00:35:09 +05:30
Noel Georgi
423a31ac9d
chore: deprectae bootloader installer option
Deprecate the `bootloader` installer option. This has not been used in a
long while.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-06-05 23:21:03 +05:30
Andrey Smirnov
bab484a405
feat: use stable network interface names
Use `udevd` rules to create stable interface names.

Link controllers should wait for `udevd` to settle down, otherwise link
rename will fail (interface should not be UP).

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-06-01 21:29:12 +04:00
Spencer Smith
c3fabb9829
chore: update default image sizes to 10GB for all "cloud" images
This PR adds a flag to imager that allows for tweaking the size of the created disk. Additionally, it sets the default value of that created disk to 10GB, as most images are cloud images that fail when uploaded b/c it only picks up a 1GB disk currently. Also adds some processing the makefile to make sure we set the default small value for metal images and SBCs.

Signed-off-by: Spencer Smith <spencer.smith@talos-systems.com>
2023-05-19 13:35:39 -04:00
Andrey Smirnov
eba8185642
release(v1.5.0-alpha.0): prepare release
This is the official v1.5.0-alpha.0 release.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-19 18:38:24 +04:00
Andrey Smirnov
02f0a4526d
feat: allow writing initial META values into the image
E.g. with the command:

```
make image-metal IMAGER_ARGS="--meta 0xc=abc --meta 0xd=abc"
```

This doesn't support ISO/PXE boot yet, it's going to come into the next
PR.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-28 21:27:00 +04:00
Noel Georgi
d1a61fd343
chore: bump golangci-lint
Bump golangci-lint and fixup new warnings. Ignore check that checks for
used function parameters, it's kind of noisy and makes it confusing to
read interface implementations.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-03-22 19:55:38 +05:30
Andrey Smirnov
1df841bb54
refactor: change the interface of META
Use a global instance, handle loading/saving META in global context.

Deprecate legacy syslinux ADV, provide an easier interface for
consumers.

Expose META as resources.

Fix the bootloader revert process (it was completely broken for quite a
while :sad:).

This is a first step which mostly does preparation work, real changes
will come in the next PRs:

* add APIs to write to META
* consume META keys for platform network config for `metal`
* custom key for URL `${code}`

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-15 15:43:16 +04:00
Dmitriy Matrenichev
22ef81c1e7
feat: add grub option to drop to maintenance mode
- [x] Support `talos.experimental.wipe=system:EPHEMERAL,STATE` boot kernel arg
- [x] GRUB option to wipe like above
- [x] update GRUB library to handle that

Closes #6842

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-03-07 12:37:59 +03:00
Noel Georgi
d4b8b35de7
feat: generate kernel module dependency tree
Run `depmod` during install/upgrades when extensions provide kernel
modules and `modules.dep` needs to be re-generated. This also allows
modules of same name from kernel to co-exist. Modules in `extras`
folder takes precedence over `in-built` ones.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-01-19 18:54:10 +05:30
Andrey Smirnov
1103c5ad24
feat: implement pre-flight checks in the installer
Host Talos mounts machined socket for API access into the installer
container (for upgrades).

Installer runs any check it might need to verify compatibility.

At the moment following checks are implemented:

* Talos version (whether upgrade from version X to Y is supported)
* Kubernetes version (whether Kubernetes version X is supported with
  Talos Y).

Fixes #6149

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-11-28 13:45:49 +04:00