622 Commits

Author SHA1 Message Date
Dmitriy Matrenichev
8dc4910c48
chore: enable "WG over GRPC" testing in siderolink agent tests
Fixes https://github.com/siderolabs/talos/issues/8514
For https://github.com/siderolabs/talos/issues/8392

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-04-01 18:24:57 +03:00
Andrey Smirnov
a6b2f54564
feat: update Kubernetes to 1.30.0-rc.0, etcd to 3.5.13
See:

* https://github.com/etcd-io/etcd/releases/tag/v3.5.13
* https://github.com/kubernetes/kubernetes/releases/tag/v1.30.0-rc.0

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-01 14:50:52 +04:00
Dmitriy Matrenichev
949ad11a2d
chore: import siderolink as siderolink-launch subcommand
This PR ensures that we can test our siderolink communication using embedded siderolink-agent.
If `--with-siderolink` provided during `talos cluster create` talosctl will embed proper kernel string and setup `siderolink-agent` as a separate process. It should be used with combination of `--skip-injecting-config` and `--with-apply-config` (the latter will use newly generated IPv6 siderolink addresses which talosctl passes to the agent as a "pre-bind").

Fixes #8392

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-03-23 16:08:56 +03:00
Dmitriy Matrenichev
55dd41c0df
chore: update coredns to v1.11.2 in required section
This change doesn't do anything actually since we already use replace with forked `github.com/siderolabs/coredns v1.11.52`.
But still this can silence some of our dep checkers.

Also update `github.com/siderolabs/coredns` to the latest upstream master (they made some minor changes).

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-03-22 11:59:29 +03:00
Andrey Smirnov
8eacc4ba80
feat: support rotation of Talos API CA
This allows to roll all nodes to use a new CA, to refresh it, or e.g.
when the `talosconfig` was exposed accidentally.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-22 12:16:47 +04:00
Jean-Tiare Le Bigot
92808e3bcf
feat: report Docker node resources in cluster show
`talosctl cluster show` currently reports '-' in the CPU and RAM
columns because these informations are not available directly from the
container list API call.

For consistency with `talosctl cluster create`, this commit fetches the
missing information through an inspect call for each node so that these
information are available from a `Reflect` initialization.

Signed-off-by: Jean-Tiare Le Bigot <jt@yadutaf.fr>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-21 23:43:04 +04:00
Artem Chernyshev
113fb646ec
chore: use go-talos-support library
The code for collecting Talos `support.zip` was extracted there.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2024-03-19 18:28:46 +03:00
Andrey Smirnov
15beb14780
feat: implement blockdevice watch controller
This controller combines kobject events, and scan of `/sys/block` to
build a consistent list of available block devices, updating resources
as the blockdevice changes.

Based on these resources the next step can run probe on the blockdevices
as they change to present a consistent view of filesystems/partitions.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-18 18:28:40 +04:00
Dmitriy Matrenichev
06e3bc0cbd
feat: implement Siderolink wireguard over GRPC
For #8064

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-03-18 15:38:13 +03:00
Andrey Smirnov
403ad93c35
feat: update dependencies
containerd 1.7.14
Linux 6.6.21

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-14 16:17:24 +04:00
Andrey Smirnov
465b9a4e6c
fix: update discovery client with the fix for keepalive interval
See https://github.com/siderolabs/discovery-client/pull/9

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-13 16:25:57 +04:00
Andrey Smirnov
1e9f866aca
feat: update Kubernetes to v1.30.0-beta.0
See https://github.com/kubernetes/kubernetes/releases/tag/v1.30.0-beta.0

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-13 15:35:44 +04:00
Andrey Smirnov
cd5a5a4474
chore: migrate to go-grpc-middleware/v2
See https://github.com/grpc-ecosystem/go-grpc-middleware

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-12 16:10:04 +04:00
Noel Georgi
1ec6683e0c
chore: use go-copy
Use go-copy and drop `pkg/copy`.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-03-07 19:51:28 +05:30
Andrey Smirnov
8152a6dd6b
feat: update Go to 1.22.1
Update Go and other dependencies as well.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-07 15:53:29 +04:00
Andrey Smirnov
0b9b4da12a
feat: update Kubernetes to 1.30.0-alpha.3
See https://github.com/kubernetes/kubernetes/releases/tag/v1.30.0-alpha.3

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-29 14:36:09 +04:00
Andrey Smirnov
b2ad5dc5f8
fix: workaround a race in CNI setup (talosctl cluster create)
When provisioning VMs, each launch process sets up CNI network, and from
time to time CNI setup fails with something like:

```
error provisioning CNI network: plugin type="firewall" failed (add): running [/sbin/iptables -t filter -N CNI-ADMIN --wait]: exit status 4: iptables v1.8.10 (nf_tables)
```

This a race condition in the CNI plugins, and it looks like there is no
fix for it (see e.g. https://github.com/hashicorp/nomad/issues/8838).

As a workaround, take a mutex around CNI operation to serialize them.
CNI setup happens in different processes, so use a file-based mutex.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-27 15:28:12 +04:00
Andrey Smirnov
6bf50fdc14
chore: disable x/net/trace in gRPC to enable dead code elimination
See https://github.com/grpc/grpc-go/pull/6954

Before:

```
-rw-r--r-- 1 smira smira 68241300 Feb 20 20:08 _out/initramfs-amd64.xz
```

After:

```
-rw-r--r-- 1 smira smira 66376648 Feb 20 20:08 _out/initramfs-amd64.xz
```

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-22 14:46:35 +04:00
Andrey Smirnov
66f3ffdd4a
fix: ensure that Talos runs in a pod (container)
Drop the Kubernetes manifests as static files clean up (this is only
needed for upgrades from 1.2.x).

Fix Talos handling of cgroup hierarchy: if started in container in a
non-root cgroup hiearachy, use that to handle proper cgroup paths.

Add a test for a simple TinK mode (Talos-in-Kubernetes).

Update the docs.

Fixes #8274

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-20 15:06:48 +04:00
Noel Georgi
9dbc33972a
feat: add basic syslog implementation
Add a basic syslog listening on `/dev/log`.

Fixes: #8087

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-02-20 15:02:06 +05:30
pardomue
53721883d5
feat: support AWS KMS for the SecureBoot signing
Fixes #8197

Signed-off-by: pardomue <edgar_ruben.pardo_munoz@roche.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-16 18:41:46 +04:00
Andrey Smirnov
7b87c7fe97
chore: bump Go dependencies
Update to address also some minor security alerts on the repo.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-16 14:53:54 +04:00
Andrey Smirnov
1366ce14a8
feat: update Kubernetes to v1.30.0-alpha.2
Talos Linux 1.7.0 will ship with Kubernetes v1.30.0.

Drop some compatibility for Kubernetes < 1.25, as 1.25 is the minimum
supported version now.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-15 21:56:56 +04:00
Dmitriy Matrenichev
83e0b0c19a
chore: adjust dns sockets settings
Enable some TCP optimization, set minimal TTL, set socket reuse.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-12 17:13:03 +03:00
Dmitriy Matrenichev
5324d39167
chore: bump stuff
Also fix .golangci.yml file.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-09 19:19:25 +03:00
Dmitriy Matrenichev
afa71d6b02
chore: use "handle-like" resource in DNSResolveCacheController
Rework (and simplify) `DNSResolveCacheController` to use `DNSUpstream` "handle-like" resources.

Depends on https://github.com/cosi-project/runtime/pull/400

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-08 21:40:57 +03:00
Andrey Smirnov
2ff81c06bc
feat: update runc 1.1.12, containerd 1.7.13
Also:

* Linux 6.6.14 + XDP enablement
* etcd 3.5.12

Various other bumps for the tools, utilities, and Go modules.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-01 17:01:04 +04:00
Dmitriy Matrenichev
9c14dea209
chore: bump coredns
Bump our CoreDNS fork.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-01-30 02:12:36 +03:00
Dmitriy Matrenichev
ebeef28525
feat: implement local caching dns server
This PR adds a new controller - `DNSServerController` that starts tcp and udp dns servers locally. Just like `EtcFileController` it monitors `ResolverStatusType` and updates the list of destinations from there.

Most of the caching logic is in our "lobotomized" "`CoreDNS` fork. We need this fork because default `CoreDNS` carries
full Caddy server and various other modules that we don't need in Talos. On our side we implement
random selection of the actual dns and request forwarding.

Closes #7693

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-01-29 20:26:38 +03:00
Andrey Smirnov
fb5ad05551
feat: update Kubernetes default to 1.29.1
See https://github.com/kubernetes/kubernetes/releases/v1.29.1

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-01-18 20:20:29 +04:00
Andrey Smirnov
1c2d10cccc
chore: bump dependencies
Go 1.21.6, update pkgs, tools, Go modules, etc.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-01-18 18:01:05 +04:00
Andrey Smirnov
1dbb4abf43
fix: update discovery service client to v0.1.6
This pulls in gRPC keepalive fix.

See https://github.com/siderolabs/discovery-client/pull/8

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-01-17 14:42:01 +04:00
Utku Ozdemir
6c5a0c2811
feat: generate a single JSON schema for multidoc config
Rework docgen to scan a whole directory for multidoc config types recursively and generate a single schema for all of them.

Annotate the files which need to be scanned by docgen while generating a schema by `//docgen:jsonschema`.

Move and rename the schema.

Bring back schema tests.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-01-16 12:25:15 +01:00
Andrey Smirnov
d5321e085e
fix: update kmsg with utf-8 fix
See: https://github.com/siderolabs/go-kmsg/pull/9

This fixes lots of `\xab` issues, specifically in:

* `talosctl dmesg` output
* `taloscl dashboard`
* embedded dashboard, including OAuth2 QR code display

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-01-12 18:33:43 +04:00
Andrey Smirnov
e6e422b92a
chore: bump dependencies
Go modules, tools, etc.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-21 19:01:16 +04:00
Andrey Smirnov
01f0cbe61c
feat: support iPXE direct booting in talosctl cluster create
This embeds a tiny TFTP server which serves UEFI iPXE which embeds a
script that chainloads a given iPXE script.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-19 17:56:08 +04:00
Andrey Smirnov
241bc9312e
fix: update the way secureboot signer fetches certificate (azure)
The previous code was a mistake, the public part of the certificate is
more easily available.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-18 17:54:51 +04:00
Andrey Smirnov
9a185a30f7
feat: update Kubernetes to v1.29.0
See https://github.com/kubernetes/kubernetes/releases/v1.29.0

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-13 22:59:17 +04:00
Artem Chernyshev
4547ad9afa
feat: send actor id to the SideroLink events sink
This might come handy to distinguish sequences, tasks initiated by a
particular API request.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2023-12-11 21:59:02 +03:00
Andrey Smirnov
4f9d3b975f
feat: update Kubernetes to v1.29.0-rc.2
See https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.29.md

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-08 19:41:28 +04:00
Andrey Smirnov
320064c5a8
feat: update Go 1.21.5, Linux 6.1.65, etcd 3.5.11
For main version, cut the release notes to start the 1.7 process.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-07 16:52:28 +04:00
Andrey Smirnov
db4e2539d4
feat: update Kubernetes 1.29.0-rc.1 and other bumps
Bump Go modules, final tools and semi-final pkgs.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-29 18:29:52 +04:00
Noel Georgi
7a4a92854f
feat: support sanitized kernel args
Support dropping kernel args that start with `-`.

Fixes: #7613

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-11-28 16:23:05 +05:30
Dmitriy Matrenichev
ba827bf8b8
chore: support getting multiple endpoints from the Provision rpc call
The code will rotate through the endpoints, until it reaches the end, and only then it will try to do the provisioning again.

Closes #7973

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-11-25 21:38:44 +03:00
Dmitriy Matrenichev
70d53ee13c
chore: deprecate .persist and .extensions
This commit deprecates those things:
- Removes the support of `.persist` flag. From now, it should always be enabled or not defined in the config.
- Removes the documentation for `.bootloader`. It never worked anyway.
- Adds a warning for `.machine.install.extensions`, suggests to use boot-assets.

Closes #7972
Closes #7507

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-11-22 20:35:38 +03:00
Andrey Smirnov
514e514ba6
feat: update Linux 6.1.63, containerd 1.7.9
Also various small bumps.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-21 18:01:36 +04:00
Andrey Smirnov
27d208c26b
feat: implement OAuth2 device flow for machine config
Fixes #7939

See documentation in the PR for the description of the feature.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-20 14:31:43 +04:00
Noel Georgi
0d3c3ed716
feat: support kube scheduler config
Support kube-scheduler config.

Fixes: #7905
Partially fixes: #7911

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-11-15 10:15:23 +05:30
Andrey Smirnov
f38eaaab87
feat: rework secureboot and PCR signing key
Support different providers, not only static file paths.

Drop `pcr-signing-key-public.pem` file, as we generate it on the fly
now.

See https://github.com/siderolabs/image-factory/issues/19

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-10 21:14:21 +04:00
Dmitriy Matrenichev
6eade3d5ef
chore: add ability to rewrite uuids and set unique tokens for Talos
This PR does those things:
- It allows API calls `MetaWrite` and `MetaRead` in maintenance mode.
- SystemInformation resource now waits for available META
- SystemInformation resource now overwrites UUID from META if there is an override
- META now supports "UUID override" and "unique token" keys
- ProvisionRequest now includes unique token and Talos version

For #7694

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-11-10 18:17:54 +03:00