775 Commits

Author SHA1 Message Date
Andrey Smirnov
c8dedbe116
fix: filter out non-printable characters in process line
Otherwise the output might be distorted by characters like `\n`.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
(cherry picked from commit 8166a58b364f760212b2a610ce0d764b8b4c5c46)
2024-09-21 13:26:31 +04:00
Andrey Smirnov
bd91675121
test: add a test for inline machine config trusted roots
Run SideroLink API server via TLS with self-signed certificate, inject
that certificate into Talos via `talos.config.inline=`.

Fix a couple of place where our special TLS root CA provider supporting
reloading on the fly was not used.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
(cherry picked from commit 8d6884a8e28e1bfa29f9a479e0f7179819cf70cd)
2024-09-13 12:53:04 +04:00
Dmitriy Matrenichev
cd7c682662
chore: disallow duplicate documents on decoder level
Required for #9275

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-09-06 15:50:13 +03:00
Andrey Smirnov
3038ccfa88
feat: add configuration for EPHEMERAL volume
Fixes #9261

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-09-06 14:11:35 +04:00
Artem Chernyshev
faffa4c3f1
fix: never unarchive initramfs when loading boot assets in talosctl
The initramfs unarchive won't work as it's extension is `xz` while the
actual compression is `zst`.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2024-09-05 20:09:57 +03:00
Andrey Smirnov
6f7c3a8e5c
fix: build of talosctl on non-Linux arches
Move META constants out to machinery, and fix up imports. The internal
`pkg/meta` package shold not be consumed in public-facing commands.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-30 22:17:38 +04:00
Andrey Smirnov
b453385bd9
feat: support volume configuration, provisioning, etc
This implements the first round of changes, replacing the volume backend
with the new implementation, while keeping most of the external
interfaces intact.

See #8367

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-30 18:32:34 +04:00
Andrey Smirnov
be2ebf6b4d
chore: bump dependencies
Update tools, pkgs, extras, Go dependencies, Go tools, etc.

Linux 6.6.47 and containerd 2.0.0-rc.4.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-29 20:44:37 +04:00
Noel Georgi
19a44c2b0b
chore: drop console ttyS0 argument
Drop `console=ttyS0` argument for metal images/installer.

`console=ttyS0` causes lot of issues with bare metal hardware when
trying to use a physical serial port.

Ref:

* https://bugzilla.redhat.com/show_bug.cgi?id=1839923
* https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=763601;msg=17
* https://www.kernel.org/doc/html/latest/admin-guide/serial-console.html
* https://github.com/coreos/fedora-coreos-tracker/issues/567

Fixes: #8695
Fixes: #8657
Fixes: #8127

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-08-27 22:24:59 +05:30
Dmitry Sharshakov
4834a61a8e
feat: report SELinux labels
This will be useful for debugging SELinux implementation. Make API report other xattrs for further development like IMA/EVM

Signed-off-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
2024-08-26 16:19:38 +03:00
Noel Georgi
36f83eea9f
chore: make qemu check flag consistent with code
Restructure code as per changes from #9198.

This makes the flag name to be in sync with what it actually does.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-08-20 20:33:56 +05:30
Andrey Smirnov
61a1c946bf
feat: bundle (some) CNI plugins with Talos core
Fixes https://github.com/siderolabs/extensions/issues/448

Bundle some CNI standard plugins plus Flannel CNI plugin (as Flannel is
the default CNI in Talos) in the Talos `initramfs`.

With this change, no plugin install is required, so the `install-cni`
step is dropped from the Flannel default manifest.

The bundled plugins:

```
$ talosctl -n 172.20.0.2 ls -lH /opt/cni/bin/
NODE         MODE         UID   GID   SIZE(B)   LASTMOD       NAME
172.20.0.2   drwxr-xr-x   0     0     109 B     7 hours ago   .
172.20.0.2   -rwxr-xr-x   0     0     3.2 MB    7 hours ago   bridge
172.20.0.2   -rwxr-xr-x   0     0     3.3 MB    7 hours ago   firewall
172.20.0.2   -rwxr-xr-x   0     0     2.4 MB    7 hours ago   flannel
172.20.0.2   -rwxr-xr-x   0     0     2.4 MB    7 hours ago   host-local
172.20.0.2   -rwxr-xr-x   0     0     2.4 MB    7 hours ago   loopback
172.20.0.2   -rwxr-xr-x   0     0     2.8 MB    7 hours ago   portmap
```

The `initramfs` for amd64 grows 67 -> 73 MiB with this change.

The path `/opt/cni/bin` is still an overlay mount, so extra plugins can
be dropped to this directory (no change here).

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-14 14:33:18 +04:00
Jean-Francois Roy
fd54dc191d
feat(talosctl): append microsoft secure boot certs
This patch adds a flag to `secureboot.database.Generate` to append the
Microsoft UEFI secure boot DB and KEK certificates to the appropriate
ESLs, in addition to complimentary command line flags.

This patch also includes a copy of said Microsoft certificates. The
certificates are downloaded from an official Microsoft repo.

Signed-off-by: Jean-Francois Roy <jf@devklog.net>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-22 14:15:42 +04:00
Andrey Smirnov
1b8c9ccbb0
fix: enforce secureboot enroll option only for supported releases
Follow up for #9005

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-17 14:16:26 +04:00
Andrey Smirnov
b07338f547
feat: provide machine config document to update trusted CA roots
Fixes #8867

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-12 19:28:31 +04:00
Andrey Smirnov
cf5effabb2
feat: provide an option to enforce SecureBoot for TPM enrollment
Fixes #8995

There is no security impact, as the actual SecureBoot
state/configuration is measured into the PCR 7 and the disk encryption
key unsealing is tied to this value.

This is more to provide a way to avoid accidentally encrypting to the
TPM while SecureBoot is not enabled.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-11 22:21:47 +04:00
Noel Georgi
a727a1d97a
chore: make using action tracker easier
Refactor so that action tracker accepts an interface.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-07-11 21:51:29 +05:30
Dmitriy Matrenichev
dad9c40c73
chore: simplify code
- replace `interface{}` with `any` using `gofmt -r 'interface{} -> any -w'`
- replace `a = []T{}` with `var a []T` where possible.
- replace `a = []T{}` with `a = make([]T, 0, len(b))` where possible.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-07-08 18:14:00 +03:00
Dmitriy Matrenichev
d9db360ab4
fix: properly output multi-doc machine config in get mc
For `get mc -o json|yaml` we pretend that `spec` field is string and not an actual yaml map. That way you
can see the full spec in unformatted view using `talosctl -n <node> get mc -o yaml` or formatted using
`talosctl -n <node> get mc -o yaml | yq .spec`.

`edit mc` command is unaffected.

Fixes #8687

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-07-08 16:22:55 +03:00
Andrey Smirnov
be35f380cc
chore: update pkgs/tools/extras
This brings in Go 1.22.5 and new Flannel CNI plugin.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-03 20:38:55 +04:00
Dmitriy Matrenichev
2d054ad355
chore: handle documents diff in apply-config dry run
Before this PR diff generator only diffed the v1alpha1 config and nothing else. With this PR it also takes
separate docs into the account.

```shell
~ > <editor> controlplane.yaml
~ > talosctl -n talos-default-controlplane-1  apply-config --file controlplane.yaml --dry-run
Dry run summary:
Applied configuration without a reboot (skipped in dry-run).
Config diff:
No changes.
Documents diff:
[]config.Document{
+	&runtime.KmsgLogV1Alpha1{
+		Meta:       meta.Meta{MetaAPIVersion: "v1alpha1", MetaKind: "KmsgLogConfig"},
+		MetaName:   "omni-kmsg",
+		KmsgLogURL: s"tcp://[fdae:41e4:649b:9303::1]:8092",
+	},
}
~ > talosctl -n talos-default-controlplane-1  apply-config --file controlplane.yaml
Applied configuration without a reboot
~ >
~ >
~ >
~ > <editor> controlplane.yaml
~ > talosctl -n talos-default-controlplane-1  apply-config --file controlplane.yaml --dry-run
Dry run summary:
Applied configuration without a reboot (skipped in dry-run).
Config diff:
No changes.
Documents diff:
[]config.Document{
	&runtime.KmsgLogV1Alpha1{Meta: {MetaAPIVersion: "v1alpha1", MetaKind: "KmsgLogConfig"}, MetaName: "omni-kmsg", KmsgLogURL: {URL: &{Scheme: "tcp", Host: "[fdae:41e4:649b:9303::1]:8092"}}},
+	&network.DefaultActionConfigV1Alpha1{
+		Meta:    meta.Meta{MetaAPIVersion: "v1alpha1", MetaKind: "NetworkDefaultActionConfig"},
+		Ingress: s"block",
+	},
}
```

Closes #8885

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-06-27 21:15:36 +03:00
Andrey Smirnov
7fcb521a6a
feat: use hydrophone instead of sonobuoy
Fixes #8790

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-06-11 16:51:45 +04:00
Andrey Smirnov
f07b79f4a8
feat: provide disk detection based on new blockdevices
Uses go-siderolabs/go-blockdevice/v2 for all the hard parts,
provides new resource `Disk` which describes all disks in the system.

Additional resource `SystemDisk` always point to the system disk (based
on the location of `META` partition).

The `Disks` API (and `talosctl disks`) provides a view now into the
`talosctl get disks` to keep backwards compatibility.

QEMU provisioner can now create extra disks of various types: IDE, AHCI,
SCSI, NVME, this allows to test detection properly.

The new resource will be the foundation for volume provisioning (to pick
up the disk to provision the volume on).

Example:

```
talosctl -n 172.20.0.5 get disks
NODE         NAMESPACE   TYPE   ID        VERSION   SIZE          READ ONLY   TRANSPORT   ROTATIONAL   WWID                                                               MODEL            SERIAL
172.20.0.5   runtime     Disk   loop0     1         65568768      true
172.20.0.5   runtime     Disk   nvme0n1   1         10485760000   false       nvme                     nvme.1b36-6465616462656566-51454d55204e564d65204374726c-00000001   QEMU NVMe Ctrl   deadbeef
172.20.0.5   runtime     Disk   sda       1         10485760000   false       virtio      true                                                                            QEMU HARDDISK
172.20.0.5   runtime     Disk   sdb       1         10485760000   false       sata        true         t10.ATA     QEMU HARDDISK                           QM00013        QEMU HARDDISK
172.20.0.5   runtime     Disk   sdc       1         10485760000   false       sata        true         t10.ATA     QEMU HARDDISK                           QM00001        QEMU HARDDISK
172.20.0.5   runtime     Disk   vda       1         12884901888   false       virtio      true
```

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-06-07 20:18:32 +04:00
Andrey Smirnov
41f92e0ba4
chore: update Go to 1.22.4, other updates
Bump go modules, adjust the code.

New linter warnings.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-06-05 20:59:52 +04:00
Andrey Smirnov
82d9cd3229
fix: add upgrade errata for arm64/zboot kernels
Fixes #8854

Talos 1.8.0 instroduces EFI ZBoot compression, and kexec from 1.7.0 to
compressed kernel doesn't work.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-06-03 20:10:24 +04:00
Dmitriy Matrenichev
eb510d9fdf
chore: require enabled bootloader for docker provisioner
Otherwise, it doesn't make sense.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-05-24 23:15:04 +03:00
Andrey Smirnov
92a274e9a0
fix: workaround problems with udevd races
When `udevd` rescans block device partitions while Talos is doing
partitions, it might be that Talos can hit the following error
while trying to open/mount a partition:

```
no such device or address
```

Previous attempts to fix that were using `ENODEV`, while the proper code
is `ENXIO`.

Also take exclusive lock while working with user disks to prevent
concurrent udevd rescan.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-05-24 21:22:07 +04:00
Andrey Smirnov
e1711cd3c9
chore: stop using containerd package for cri namespace
In containerd 2.0 source tree, this constant is under `internal`, so we
can't import it directly.

So instead re-declare it as a Talos constant.

Doing this multi-staged, as `go-talos-support` is using it as well, and
to update it to stop importing old containerd library I need first to
declare the constant in Talos source tree.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-05-21 17:45:52 +04:00
Artem Chernyshev
42ac5cd0c2
fix: check for nil machine config during installation
Otherwise we get `nil reference` exception during maintenance mode
upgrade with partial machine configs.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2024-05-08 16:34:55 +03:00
Utku Ozdemir
478b862b4c
fix: do not fail cli action tracker when boot id cannot be read
If the `reboot/reset/shutdown/upgrade` action tracker cannot read the boot ID from the node under `/proc/sys/kernel/random/boot_id` due to insufficient permissions (e.g., when `talosctl reboot` is used over Omni), fall back to skipping boot ID check instead of hard-failing.

Closes siderolabs/talos#7197.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-05-07 13:51:28 +02:00
Andrey Smirnov
4c0c626b78
feat: use zstd compression in place of xz
Initramfs and kernel are compressed with zstd.

Extensions are compressed with zstd for Talos 1.8+.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-29 18:09:12 +04:00
Utku Ozdemir
dd1d279daa
fix: allow more flags in talosctl cluster create --input-dir
Some of the flags passed to `talosctl cluster create` were failing the input validation due to being incorrectly marked as mutually exclusive with the `--input-dir` flag.

Clean up the check to allow passing all flags along with the `--input-dir` flag if those flags impact the provisioning process in any way (i.e., not solely used in generating machine config).

Additionally, replace the mutual exclusion checks with Cobra's built-in function for that.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-04-24 10:49:24 +02:00
Utku Ozdemir
0821b9c50b
feat: add --non-masquerade-cidrs flag to talosctl cluster create
Allow skipping NAT for the given destinations from a cluster network. This option makes it possible to form an etcd cluster from clusters in different networks created by running `talosctl cluster create` command multiple times using different CIDRs: they simply should have the CIDR of the other clusters passed with `--non-masquerade-cidrs`.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-04-23 13:30:13 +02:00
Dmitry Sharshakov
6fbd1263cc
feat: report process MAC labels
This will be useful for debugging process access rights once we start implementing SELinux

Signed-off-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
2024-04-22 18:16:33 +03:00
Andrey Smirnov
0a785802ea
fix: overlay installer operations
1. Use overlay installer to build the `cmdline` when running in
   install/upgrade mode.

2. Pull down the overlay installer with the arch specific to the
   installer being generated, vs. the arch of the `imager`.

3. Print a message when running an overlay installer.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-16 20:07:44 +04:00
Artem Chernyshev
3dd1f4e88c
chore: extract pkg/imager/quirks to pkg/machinery
To make it possible to use it without pulling the whole Talos.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2024-04-15 21:37:47 +03:00
looklose
9550f5ff7a
docs: fix getAuthenticationMethod and completePathFromNode docs
Both of those contained incorrect comments.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-04-10 20:23:52 +03:00
Dmitry Sharshakov
653f838b09
feat: support multiple Docker cluster in talosctl cluster create
Dynamically map Kubernetes and Talos API ports to an available port on
the host, so every cluster gets its own unique set of parts.

As part of the changes, refactor the provision library and interfaces,
dropping old weird interfaces replacing with (hopefully) much more
descriprive names.

Signed-off-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-04 21:21:39 +04:00
Andrey Smirnov
951904554e
chore: bump dependencies (go 1.22.2)
Update Go to 1.22.2, update Go modules to resolve
[HTTP/2 issue](https://www.kb.cert.org/vuls/id/421644).

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-04 14:59:24 +04:00
Noel Georgi
f515741b52
chore: add equinix e2e-tests
Add equinix e2e-tests.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-04-02 17:16:59 +05:30
Andrey Smirnov
090143b030
fix: allow platform cmdline args to be platform-specific
Fix Equnix Metal (where proper arm64 args are known) and metal platform
(using generic arm64 console arg).

Other platforms might need to be updated, but correct settings are not
known at the moment.

Fixes #8529

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-02 14:41:39 +04:00
Andrey Smirnov
7a68504b6b
feat: support rotating Kubernetes CA
Fixes #8440

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-01 22:08:02 +04:00
Andrey Smirnov
fac3dd0430
fix: don't set default endpoints on gen config
Fixes #8500

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-01 21:21:58 +04:00
Dmitriy Matrenichev
8dc4910c48
chore: enable "WG over GRPC" testing in siderolink agent tests
Fixes https://github.com/siderolabs/talos/issues/8514
For https://github.com/siderolabs/talos/issues/8392

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-04-01 18:24:57 +03:00
Niklas Wik
0fc24eeb09
feat: provide insecure flag to imager
provides flag for imager to pull images insecurely from private registries

Signed-off-by: Niklas Wik <niklas.wik@nokia.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-01 15:53:56 +04:00
Dmitriy Matrenichev
949ad11a2d
chore: import siderolink as siderolink-launch subcommand
This PR ensures that we can test our siderolink communication using embedded siderolink-agent.
If `--with-siderolink` provided during `talos cluster create` talosctl will embed proper kernel string and setup `siderolink-agent` as a separate process. It should be used with combination of `--skip-injecting-config` and `--with-apply-config` (the latter will use newly generated IPv6 siderolink addresses which talosctl passes to the agent as a "pre-bind").

Fixes #8392

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-03-23 16:08:56 +03:00
Andrey Smirnov
8eacc4ba80
feat: support rotation of Talos API CA
This allows to roll all nodes to use a new CA, to refresh it, or e.g.
when the `talosconfig` was exposed accidentally.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-22 12:16:47 +04:00
Andrey Smirnov
7d43c9aa6b
chore: annotate installer errors
I want to catch a spurious error `ENODEV`, where exactly it comes from.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-21 16:58:34 +04:00
Dmitriy Matrenichev
19f15a840c
chore: bump golangci-lint to 1.57.0
Fix all discovered issues.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-03-21 01:06:53 +03:00
Artem Chernyshev
113fb646ec
chore: use go-talos-support library
The code for collecting Talos `support.zip` was extracted there.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2024-03-19 18:28:46 +03:00