671 Commits

Author SHA1 Message Date
Andrey Smirnov
3c9f7a7de6
chore: re-enable nolintlint and typecheck linters
Drop startup/rand.go, as since Go 1.20 `rand.Seed` is done
automatically.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-08-25 01:05:41 +04:00
Andrey Smirnov
8670450d28
release(v1.6.0-alpha.0): prepare release
This is the official v1.6.0-alpha.0 release.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-08-24 17:09:34 +04:00
Andrey Smirnov
ccfa8de117
fix: automatically change rpi_4 board on upgrade
As `rpi_4` was removed in Talos 1.5, automatically migrate to
`rpi_generic`.

Fixes #7632

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-08-21 18:05:42 +04:00
Dmitriy Matrenichev
c4a1ca8d61
chore: remove <-errCh where possible in grpc methods
Simplify code by passing error directly into the pipe closer.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-08-07 22:28:58 +03:00
Andrey Smirnov
e0f383598e
chore: clean up the output of the imager
Use `Progress`, and options to pass around the way messages are written.

Fixed some tiny issues in the code, but otherwise no functional changes.

To make colored output work with `docker run`, switched back image
generation to use volume mount for output (old mode is still
functioning, but it's not the default, and it works when docker is not
running on the same host).

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-08-07 16:00:14 +04:00
Dmitriy Matrenichev
80238a05a6
chore: unify semver under github.com/blang/semver/v4
Currently, we use `github.com/coreos/go-semver/semver` and `github.com/hashicorp/go-version`
for version parsing. As we use `github.com/blang/semver/v4` in our other projects, and it
has more features, it makes sense to use it across the projects. It also doesn't allocate
like crazy in `KubernetesVersion.SupportedWith`.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-08-04 00:29:52 +03:00
Andrey Smirnov
87fe8f1a2a
feat: implement image generation profiles
Support full configuration for image generation, including image
outputs, support most features (where applicable) for all image output
types, unify image generation process.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-08-02 19:13:44 +04:00
Noel Georgi
209c34801e
chore: drop with-secureboot talosctl flag
The code picks up firmware files in the order it's defined. The
secureboot QEMU firmware files are defined first, so this flag is a
no-op. This was leftover from when `ovmfctl` was used.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-07-31 17:33:12 +04:00
Andrey Smirnov
6d71bb8df2
refactor: replace google/gopacket with gopacket/gopacket
This new fork seems to be more active. The change itself doesn't fix any
memory allocation, but I submitted a PR for gopacket/gopacket:

https://github.com/gopacket/gopacket/pull/24

Also fix crazy alloc in `tui/components` (this is only relevant for
`talosctl`).

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-07-28 17:34:15 +04:00
Andrey Smirnov
5e13cafe5b
feat: enforce kernel lockdown for UKI
UKI is meant to be for UEFI Secure Boot, so it's expected to enforce
kernel lockdown. We might reconsider in the future to use a kernel patch
instead: b1a0314b08

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-07-22 13:52:46 +04:00
Dmitriy Matrenichev
130518de71
chore: change missing renames of KubePrism
For #7432

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-07-14 17:18:25 +03:00
Dmitriy Matrenichev
5f34f5b41f
chore: rename api load balancer to KubePrism
Closes #7432

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-07-14 15:23:53 +03:00
Andrey Smirnov
53873b8444
refactor: move ukify into Talos code
This is intemediate step to move parts of the `ukify` down to the main
Talos source tree, and call it from `talosctl` binary.

The next step will be to integrate it into the imager and move `.uki`
build out of the Dockerfile.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-07-13 19:14:32 +04:00
Noel Georgi
79365d9bac
feat: tpm2 based disk encryption
Support disk encryption using tpm2 and pre-calculated signed PCR values.

Fixes: #7266

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-07-12 20:41:28 +05:30
Andrey Smirnov
8017afb107
feat: implement CRI image management and pre-pull on K8s upgrade
Fixes #6391

Implement a set of APIs and commands to manage images in the CRI, and
pre-pull images on Kubernetes upgrades.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-07-11 19:25:10 +04:00
Noel Georgi
94e9891c1b
chore: bump sd-boot to v254-rc1
Bump sd-boot.
Fix parsing PE executable offsets.
Set the PE file alignment to be 512 bytes.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-07-11 15:52:57 +05:30
Artem Chernyshev
936111ce06
fix: properly set up tls for KMS endpoint
The condition was inverted 🤦

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2023-07-10 21:10:02 +03:00
Artem Chernyshev
ce63abb219
feat: add KMS assisted encryption key handler
Talos now supports new type of encryption keys which rely on Sealing/Unsealing randomly generated bytes with a KMS server:

```
systemDiskEncryption:
  ephemeral:
    keys:
      - kms:
          endpoint: https://1.2.3.4:443
        slot: 0
```
gRPC API definitions and a simple reference implementation of the KMS server can be found in this
[repository](https://github.com/siderolabs/kms-client/blob/main/cmd/kms-server/main.go).

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2023-07-07 19:02:39 +03:00
Andrey Smirnov
a4289e8703
chore: fix CLI docs generation stability
The problem first spotted by Artem, leads to spurious dirty checks.

The sort order was checking wrong (lowered) keys, so the order was
actually random.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-07-05 19:20:40 +04:00
Andrey Smirnov
c9a9f95611
refactor: extract secure boot certificate generation
Fixes #7412

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-07-03 16:55:02 +04:00
Andrey Smirnov
e241be85ba
fix: properly handle YAML comment stripping for multi-doc
Fixes #7425

The previously used method doesn't handle YAML multi-doc, incorrectly
stripping only the first document and throwing away everything else.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-06-30 15:59:06 +04:00
Andrey Smirnov
35d6adcb9a
fix: provide stashed META values before installation
Previously, if META values were supplied to the Talos ISO via
environment variable, they will be written down and available after the
install. With this fix, values are also readable and available before
the installation runs (in maintenance mode).

Most of the PR is refactoring `meta.Value(s)` to be a shared library
which is used by the installer/imager and (now) Talos.

Also fixes an issue with not returning properly `NotExist` error when
META is not yet available as a partition on disk.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-06-27 20:57:43 +04:00
Utku Ozdemir
0d313b9733
feat: add reboot-mode flag to talosctl upgrade
Allow specifying the reboot mode during upgrades by introducing `--reboot-mode` flag, similar to the `--mode` flag of the reboot command.

Closes siderolabs/talos#7302.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-06-26 17:37:19 +02:00
Tim Jones
53389b1e72
feat: auto-enroll secure boot keys
Uses the auto-enrollment feature of sd-boot to enroll required UEFI Secure
Boot keys.

Fixes: #7373

Signed-off-by: Tim Jones <tim.jones@siderolabs.com>
Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-06-24 00:44:56 +05:30
Noel Georgi
e3f3f5794d
feat: implement revert for sd-boot
Implement revert for sd-boot.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-06-22 20:20:31 +05:30
Andrey Smirnov
fe0f46980f
feat: implement secure boot from disk
This includes sd-boot handling, EFI variables, etc.

There are some TODOs which need to be addressed to make things smooth.

Install to disk, upgrades work.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-06-16 20:15:16 +05:30
Dmitriy Matrenichev
445f5ad542
feat: support API server load balancer
This commit adds support for API load balancer. Quick way to enable it is during cluster creation using new `api-server-balancer-port` flag (0 by default - disabled). When enabled all API request will be routed across
cluster control plane endpoints.

Closes #7191

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-06-16 10:09:20 -04:00
Andrey Smirnov
19bc223de8
refactor: bootloader interface, labels
Move labels out of the bootloader interface, while moving copying assets
into the bootloader interface. GRUB is using one set of assets,
`sd-boot` will be using another one.

Fix the problem with `bootloader.Probe()` finding boot partition on the
host when it runs in a priv container, fixing issues with image creation
in the CI.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-06-14 17:33:11 +04:00
Noel Georgi
71a548d180
chore: generic boootloader implementation
This changes the bootloader code to be generic to support
multiple bootloader implementations.

Signed-off-by: Noel Georgi <git@frezbo.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-06-13 23:36:20 +04:00
Nanfei Chen
1865a0c296
chore: modify some usages that are not recommended
Struct MetaValue has methods on both value and pointer receivers. Such usage is not recommended by the Go Documentation. Modifies the receiver usage.

Variable config collides with imported package name. Renames the variable config.

Removes a redundant alias.

Empty slice declaration uses a literal. Replaces with nil slice declaration.

Signed-off-by: Nanfei Chen <chennanfei@yeah.net>
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-06-11 17:40:26 -04:00
Nanfei Chen
d04cf19788
chore: clean up unnecessary self assignment
There is no need that the value of err is assigned to itself. So removes this self assignment.

Signed-off-by: Nanfei Chen <chennanfei@yeah.net>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-06-09 16:51:58 +04:00
Noel Georgi
1c0c7933df
chore: cleanup partition code
Cleanup partition code to be explicit about `Format` and `Partition`
options.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-06-08 00:35:09 +05:30
Christian Rolland
e6dde8ffc5
feat: add network chaos to qemu development environment
Add flags for configuring the qemu bridge interface with chaos options:
- network-chaos-enabled
- network-jitter
- network-latency
- network-packet-loss
- network-packet-reorder
- network-packet-corrupt
- network-bandwidth

These flags are used in /pkg/provision/providers/vm/network.go at the end of the CreateNetwork function to first see if the network-chaos-enabled flag is set, and then check if bandwidth is set.  This will allow developers to simulate clusters having a degraded WAN connection in the development environment and testing pipelines.

If bandwidth is not set, it will then enable the other options.
- Note that if bandwidth is set, the other options such as jitter, latency, packet loss, reordering and corruption will not be used.  This is for two reasons:
	- Restriction the bandwidth can often intoduce many of the other issues being set by the other options.
	- Setting the bandwidth uses a separate queuing discipline (Token Bucket Filter) from the other options (Network Emulator) and requires a much more complex configuration using a Heirarchial Token Bucket Filter which cannot be configured at a granular enough level using the vishvananda/netlink library.

Adding both queuing disciplines to the same interface may be an option to look into in the future, but would take more extensive testing and control over many more variables which I believe is out of the scope of this PR.  It is also possible to add custom profiles, but will also take more research to develop common scenarios which combine different options in a realistic manner.

Signed-off-by: Christian Rolland <christian.rolland@siderolabs.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-06-06 20:15:26 +04:00
Noel Georgi
3a865370f5
feat: qemu secureboot
Add qemu support for secureboot testing via `talosctl cluster create`.

Can be tested via:

```bash
sudo -E _out/talosctl-linux-amd64 cluster create --provisioner=qemu $REGISTRY_MIRROR_FLAGS --controlplanes=1 --workers=1 --iso-path=_out/talos-uki-amd64.iso --with-secureboot=true --with-tpm2=true --skip-injecting-config --with-apply-config
```

This currently only supports just booting Talos in SecureBoot mode.
Installation and Upgrade comes as extra PRs.

Fixes: #7324

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-06-06 19:20:07 +05:30
Noel Georgi
423a31ac9d
chore: deprectae bootloader installer option
Deprecate the `bootloader` installer option. This has not been used in a
long while.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-06-05 23:21:03 +05:30
Andrey Smirnov
cdfece7d64
chore: optimize image compression
Use `pigz` (parallel) instead of `gzip`.

Use `xz` compression `-0` instead of `-6`.

This has pros and cons:

* image size goes up (77M -> 79M) (+2.5%)
* image generation goes down 19s -> 10s (-50%).

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-06-05 18:55:56 +04:00
Noel Georgi
3f68485e44
feat: add uki iso generation
This adds code to generate a UKI ISO (UEFI only).

Fixes: #7261

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-06-02 22:44:27 +05:30
Andrey Smirnov
bab484a405
feat: use stable network interface names
Use `udevd` rules to create stable interface names.

Link controllers should wait for `udevd` to settle down, otherwise link
rename will fail (interface should not be UP).

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-06-01 21:29:12 +04:00
Andrey Smirnov
badbc51e63
refactor: rewrite code to include preliminary support for multi-doc
`config.Container` implements a multi-doc container which implements
both `Container` interface (encoding, validation, etc.), and `Conifg`
interface (accessing parts of the config).

Refactor `generate` and `bundle` packages to support multi-doc, and
provide backwards compatibility.

Implement a first (mostly example) machine config document for
SideroLink API URL.

Many places don't properly support multi-doc yet (e.g. config patches).

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-31 18:38:05 +04:00
Andrey Smirnov
3c64a5ffba
chore: optimize image generation time
Use `pigz` and `--sparse` to handle more efficiently compression of the
assets.

Also move tasks out of `setup-ci` step, as it runs always, including for
the promoted pipelines.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-31 17:47:23 +04:00
Andrey Smirnov
1fb29a56a8
fix: fail quickly if upgrade-k8s is used with multiple nodes
Fixes #7283

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-29 14:35:29 +04:00
Andrey Smirnov
dc6764871c
refactor: move around config interfaces, make RawV1Alpha1 typed
See #7230

Refactor more config interfaces, move config accessor interfaces
to different package to break the dependency loop.

Make `.RawV1Alpha1()` method typed to avoid type assertions everywhere.

No functional changes.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-23 22:08:58 +04:00
Andrey Smirnov
0bb7e8a5cf
refactor: split config.Provider into Config & Container
See #7230

This is a step towards preparing for multi-doc config.

Split the `config.Provider` interface into parts which have different
implementation:

* `config.Config` accesses the config itself, it might be implemented by
  `v1alpha1.Config` for example
* `config.Container` will be a set of config documents, which implement
  validation, encoding, etc.

`Version()` method dropped, as it makes little sense and it was almost
not used.

`Raw()` method renamed to `RawV1Alpha1()` to support legacy direct
access to `v1alpha1.Config`, next PR will refactor more to make it
return proper type.

There will be many more changes coming up.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-23 16:05:16 +04:00
Dmitriy Matrenichev
85d8a16194
chore: bump deps
Bump deps

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-05-22 16:02:15 -04:00
Spencer Smith
39b7a56f01
chore: use 8GiB instead of 10GiB for cloud images
This PR changes the default disk size for cloud images to be 8GiB
instead. This was prompted b/c the disk price in azure between tiers is
doubled and the cutoff for the tier is 8GiB.

Signed-off-by: Spencer Smith <spencer.smith@talos-systems.com>
2023-05-19 20:35:13 -04:00
Spencer Smith
c3fabb9829
chore: update default image sizes to 10GB for all "cloud" images
This PR adds a flag to imager that allows for tweaking the size of the created disk. Additionally, it sets the default value of that created disk to 10GB, as most images are cloud images that fail when uploaded b/c it only picks up a 1GB disk currently. Also adds some processing the makefile to make sure we set the default small value for metal images and SBCs.

Signed-off-by: Spencer Smith <spencer.smith@talos-systems.com>
2023-05-19 13:35:39 -04:00
Andrey Smirnov
eba8185642
release(v1.5.0-alpha.0): prepare release
This is the official v1.5.0-alpha.0 release.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-19 18:38:24 +04:00
Andrey Smirnov
bb02dd263c
chore: drop deprecated stuff for Talos 1.5
* drop old resources API, which was deprecated long time ago
* use bootstrapped event in `talosctl get --watch` to better align
  columns in the table output

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-18 19:46:37 +04:00
Dmitriy Matrenichev
45e6e27af7
chore: bump runtime
Use new functions and methods from runtime module.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-05-11 17:18:08 -04:00
Andrey Smirnov
7a004a6f7f
fix: parse errors correctly
Message metadata might be missing, the easiest usecase is contacting
worker directly using it both as an endpoint and a node.

Fixes #7108

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-04-19 17:08:36 +04:00