605 Commits

Author SHA1 Message Date
Dmitriy Matrenichev
53f5489130
fix: increase host dns packet ttl for pods
This PR fixes incorrect packet TTL if `forwardKubeDNSToHost` is enabled.

Credits go to Julian Wiedmann.
Closes #8698.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-05-15 17:40:32 +03:00
Noel Georgi
1207054599
chore: handle I/O error for xfs_repair
Run `xfs_repair` on `unix.EIO` error.

```text
16T18:19:30.85674118Z]: XFS (sdb5): Mounting V5 Filesystem
109.200.197.196: kern:    info: [2024-04-16T18:19:30.92421418Z]: XFS (sdb5): Ending clean mount
109.200.197.196: kern:  notice: [2024-04-16T18:19:36.42651618Z]: XFS (sdb6): Mounting V5 Filesystem
109.200.197.196: kern:    info: [2024-04-16T18:19:36.49568918Z]: XFS (sdb6): Ending clean mount
109.200.197.196: kern:  notice: [2024-04-16T18:19:36.54484918Z]: XFS (sdb6): Quotacheck needed: Please wait.
109.200.197.196: kern:  notice: [2024-04-16T18:19:36.54586418Z]: XFS (sdb6): Quotacheck: Done.
109.200.197.196: kern:   alert: [2024-05-13T15:13:11.99476118Z]: XFS (sdb6): log I/O error -5
109.200.197.196: kern:   alert: [2024-05-13T15:13:11.99477118Z]: XFS (sdb6): Filesystem has been shut down due to log error (0x2).
109.200.197.196: kern:   alert: [2024-05-13T15:13:11.99477318Z]: XFS (sdb6): Please unmount the filesystem and rectify the problem(s).
```

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-05-13 21:19:50 +05:30
Andrey Smirnov
1d29111d43
chore: update Go to 1.22.3
Also bump dependencies.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-05-08 14:59:41 +04:00
Andrey Smirnov
07f78182c6
fix: use a fresh context for etcd unlock
By the time unlock is called, context might be already canceled.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-05-01 18:59:50 +04:00
Andrey Smirnov
4c0c626b78
feat: use zstd compression in place of xz
Initramfs and kernel are compressed with zstd.

Extensions are compressed with zstd for Talos 1.8+.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-29 18:09:12 +04:00
Dmitry Sharshakov
6fbd1263cc
feat: report process MAC labels
This will be useful for debugging process access rights once we start implementing SELinux

Signed-off-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
2024-04-22 18:16:33 +03:00
Dmitriy Matrenichev
908f67fa15
feat: add host dns support for resolving member addrs
Closes #8330

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-04-18 15:29:30 +03:00
Dmitriy Matrenichev
ec69d7a785
chore: replace math/rand with math/rand/v2
New package arrived in Go 1.22 which provides better rand primitives and functions.
Use it instead of the old one.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-04-18 13:20:59 +03:00
Dmitriy Matrenichev
ba7cdc8c8b
chore: optimize DNSResolveCacheController
Optimize `DNSResolveCacheController` type, including `dns.Server` optimization for easy start/stop. This PR ensures that we
delete server from runners on stop (even unexpected) and restart it properly. Also fixes incorrect assumption on unit-tests.

Fixes #8563

This PR also does those things:
- Removes `utils.Runner`
- Removes `ctxutil.MonitorFn`
- Removes `dns.Runner`
- Removes `network.dnsRunner`

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-04-10 17:24:19 +03:00
Utku Ozdemir
3735add87c
fix: reconnect to the logs stream in dashboard after reboot
The log stream displayed in the dashboard was stopping to work when a node was rebooted.
Rework the log data source to establish a per-node connection and use a retry loop to always reconnect until the dashboard is terminated.

Print the connection errors in the log stream in red color.

Closes siderolabs/talos#8388.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-04-10 10:43:45 +02:00
Dmitry Sharshakov
653f838b09
feat: support multiple Docker cluster in talosctl cluster create
Dynamically map Kubernetes and Talos API ports to an available port on
the host, so every cluster gets its own unique set of parts.

As part of the changes, refactor the provision library and interfaces,
dropping old weird interfaces replacing with (hopefully) much more
descriprive names.

Signed-off-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-04 21:21:39 +04:00
Andrey Smirnov
951904554e
chore: bump dependencies (go 1.22.2)
Update Go to 1.22.2, update Go modules to resolve
[HTTP/2 issue](https://www.kb.cert.org/vuls/id/421644).

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-04 14:59:24 +04:00
Andrey Smirnov
862c76001b
feat: add support for CoreDNS forwarding to host DNS
This PR adds the support for CoreDNS forwarding to host DNS. We try to bind on 9th address on the first element from
`serviceSubnets` and create a simple service so k8s will not attempt to rebind it.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
Co-authored-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-04-03 23:36:17 +03:00
Dmitry Sharshakov
84ec8c16f3
feat: support syncing to PTP clocks
Also abstract away from NTP types.

Signed-off-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-21 17:20:26 +04:00
Andrey Smirnov
7d43c9aa6b
chore: annotate installer errors
I want to catch a spurious error `ENODEV`, where exactly it comes from.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-21 16:58:34 +04:00
Dmitriy Matrenichev
19f15a840c
chore: bump golangci-lint to 1.57.0
Fix all discovered issues.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-03-21 01:06:53 +03:00
Noel Georgi
d118a852b9
feat: implement Install for imager overlays
Implement `Install` for imager overlays.
Also add support for generating installers.

Depends on: #8377

Fixes: #8350
Fixes: #8351
Fixes: #8350

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-03-12 22:46:29 +05:30
Utku Ozdemir
1bb6027ccd
fix: fix nil panic on maintenance upgrade with partial config
Fix the nil dereferences when a Talos node is attempted to be upgraded while in maintenance mode and having a partial machine config.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-03-08 12:52:21 +03:00
Artem Chernyshev
3c8f51d707
chore: move cli formatters and version modules to machinery
To be used in the `go-talos-support` module without importing the whole
Talos repo.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2024-03-07 16:29:15 +03:00
Dmitriy Matrenichev
f8c556a1ce
chore: listen for dns requests on 127.0.0.53
Turns out there is actually no black magic in systemd, they simply listen on 127.0.0.53 and forward dns requests there in resolv.conf.
Reason is the same as ours — to preserve compatibility with other applications. So we do the same in our code.

This PR also does two things:
- Adds `::1` into resolv.conf for IPv6 only resolvers.
- Drops `SO_REUSEPORT` from control options (it works without them).

Closes #8328

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-26 20:59:12 +03:00
Andrey Smirnov
8872a7a210
fix: ignore 'no such device' in addition to 'no such file'
This errors pops up when `udevd` rescans the partition table with Talos
trying to mount a device concurrently.

This feels to be something new with Linux 6.6 probably.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-26 20:00:05 +04:00
Andrey Smirnov
66f3ffdd4a
fix: ensure that Talos runs in a pod (container)
Drop the Kubernetes manifests as static files clean up (this is only
needed for upgrades from 1.2.x).

Fix Talos handling of cgroup hierarchy: if started in container in a
non-root cgroup hiearachy, use that to handle proper cgroup paths.

Add a test for a simple TinK mode (Talos-in-Kubernetes).

Update the docs.

Fixes #8274

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-20 15:06:48 +04:00
Dmitriy Matrenichev
fa3b933705
chore: replace fmt.Errorf with errors.New where possible
This time use `eg` from `x/tools` repo tool to do this.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-14 17:39:30 +03:00
Noel Georgi
2f0421b406
fix: run xfs_repair on invalid argument error
Run `xfs_repair` for invalid argument error.

Part of: #8292

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-02-13 23:01:33 +05:30
Dmitriy Matrenichev
fa2d34dd88
chore: enable v6 support on the same port
Replace `SO_REUSEPORT` with `SO_REUSEPORT`.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-13 01:02:27 +03:00
Dmitriy Matrenichev
83e0b0c19a
chore: adjust dns sockets settings
Enable some TCP optimization, set minimal TTL, set socket reuse.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-12 17:13:03 +03:00
Dmitriy Matrenichev
5324d39167
chore: bump stuff
Also fix .golangci.yml file.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-09 19:19:25 +03:00
Dmitriy Matrenichev
afa71d6b02
chore: use "handle-like" resource in DNSResolveCacheController
Rework (and simplify) `DNSResolveCacheController` to use `DNSUpstream` "handle-like" resources.

Depends on https://github.com/cosi-project/runtime/pull/400

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-08 21:40:57 +03:00
Andrey Smirnov
9d8cd4d058
chore: drop deprecated method EtcdRemoveMember
It was deprecated 16 months ago, time to cleanup.

(This is to prepare for the first v1.7 release)

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-01 15:54:29 +04:00
Andrey Smirnov
593afeea38
fix: run the interactive installer loop to report errors
In the previous implementation, even though `installer.err` was set, it
was never checked 🤦.

The run loop was stolen from the dashboard code.

Fixes #8205

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-01-31 19:20:46 +04:00
Dmitriy Matrenichev
ebeef28525
feat: implement local caching dns server
This PR adds a new controller - `DNSServerController` that starts tcp and udp dns servers locally. Just like `EtcFileController` it monitors `ResolverStatusType` and updates the list of destinations from there.

Most of the caching logic is in our "lobotomized" "`CoreDNS` fork. We need this fork because default `CoreDNS` carries
full Caddy server and various other modules that we don't need in Talos. On our side we implement
random selection of the actual dns and request forwarding.

Closes #7693

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-01-29 20:26:38 +03:00
Andrey Smirnov
9782319c31
fix: support KubePrism settings in Kubernetes Discovery
Fixes #8143

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-01-16 20:41:13 +04:00
Utku Ozdemir
7fa7362ddc
fix: fix nodes on dashboard footer when node names are used in --nodes
When the dashboard is used via the CLI through a proxy, e.g., through Omni, node names or IDs can be used in the `--nodes` flag instead of the IPs.

This caused rendering inconsistencies in the dashboard, as some parts of it used the IPs and some used the names passed in the context.

Fix this by collecting all node IPs on dashboard start, and map these IPs to the respective nodes passed as the `--nodes` flag.

On the dashboard footer, we always display the node names as they are passed in the `--nodes` flag.

As part of it, remove the node list change reactivity from the dashboard, so it will always take the passed nodes as the truth.

The IP to node mapping collection at dashboard startup also solves another issue where the first API call by the dashboard triggered the interactive API authentication (e.g., the OIDC flow). Previously, because the terminal was already switched to the raw mode, it was not possible to authenticate properly.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-01-12 12:00:08 +01:00
Andrey Smirnov
f38eaaab87
feat: rework secureboot and PCR signing key
Support different providers, not only static file paths.

Drop `pcr-signing-key-public.pem` file, as we generate it on the fly
now.

See https://github.com/siderolabs/image-factory/issues/19

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-10 21:14:21 +04:00
Dmitriy Matrenichev
6eade3d5ef
chore: add ability to rewrite uuids and set unique tokens for Talos
This PR does those things:
- It allows API calls `MetaWrite` and `MetaRead` in maintenance mode.
- SystemInformation resource now waits for available META
- SystemInformation resource now overwrites UUID from META if there is an override
- META now supports "UUID override" and "unique token" keys
- ProvisionRequest now includes unique token and Talos version

For #7694

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-11-10 18:17:54 +03:00
Andrey Smirnov
e22ab440d7
feat: update Linux 6.1.61, containerd 1.7.8, runc 1.1.10
Bump tools/pkgs/extras.

Update Go dependencies.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-09 20:17:28 +04:00
Noel Georgi
75d3987c05
chore: drop sha1 from genereated pcr json
Drop `sha1` algorithm from expected PCR json calculation.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-11-07 22:11:33 +05:30
Andrey Smirnov
6f3cd05935
refactor: update packet capture to use 'afpacket' interface
First of all, this interface is way more performant than `pcap`
interface. It is Linux-specific, but we don't care in Talos Linux :)

Second, this drop dependency of `machined` on `gopacket/layers` package,
which has huge issues with memory allocations and startup time.

This cuts around 20MiB of process RSS for all Talos processes.
(`talosctl` still requires this `gopacket/layers` library for decoding
packets).

Fixes #7880

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-07 15:52:04 +04:00
Andrey Smirnov
807a9950ac
fix: use custom Talos/kernel version when generating UKI
See https://github.com/siderolabs/image-factory/issues/44

Instead of using constants, use proper Talos version and kernel version
discovered from the image.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-03 11:14:02 +04:00
Andrey Smirnov
6dc776b8aa
fix: when writing to META in the installer/imager, use fixed name
Use fixed partition name instead of trying to auto-discover by label.

Auto-discovery by label might hit completely wrong blockdevice.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-01 20:34:41 +04:00
Artem Chernyshev
ffa5e05cb9
fix: make Talos work on Rockpi 4c boards again
Suppress `efivars` `ENODEV` errors: skip mount and proceed with boot
sequence.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2023-10-25 13:51:38 +03:00
Andrey Smirnov
9dfae8467d
chore: update dependencies
Containerd 1.7.7, Linux 6.1.58.

Fixes #7859

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-10-17 17:41:38 +04:00
Thomas Way
b87092ab69
fix: handle secure boot state policy pcr digest error
This does not fix the underlying digest mismatch issue, but does handle the error and should provide
further insight into issues (if present).

Refs: #7828

Signed-off-by: Thomas Way <thomas@6f.io>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-10-09 18:24:56 +04:00
Thomas Way
336aee0fdb
fix: use tpm2 hash algorithm constants and allow non-SHA-256 PCRs
The conversion from TPM 2 hash algorithm to Go crypto algorithm will fail for
uncommon algorithms like SM3256. This can be avoided by checking the constants
directly, rather than converting them. It should also be fine to allow some non
SHA-256 PCRs.

Fixes: #7810

Signed-off-by: Thomas Way <thomas@6f.io>
Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-10-04 01:02:20 +05:30
Andrey Smirnov
2b548ad0d9
feat: update containerd to 1.7.x
Also update Linux and other pkgs.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-09-28 16:33:57 +04:00
Andrey Smirnov
a52d3cda3b
chore: update gen and COSI runtime
No actual changes, adapting to use new APIs.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-09-22 12:13:13 +04:00
Andrey Smirnov
5ca4d58dc9
fix: generate of modules.dep when on the machine
When running on the machine, the extensionTreePath is not writeable, so
create and clean up a temporary directory to host `modules.dep`
extension.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-09-20 15:51:22 +04:00
Andrey Smirnov
e3b4940588
fix: build CPU ucode correctly for early loader
Closes #7729

This follows the steps described in
https://www.kernel.org/doc/html/v6.1/x86/microcode.html#early-load-microcode

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-09-18 14:03:41 +04:00
Andrey Smirnov
c5bd0ac5cf
refactor: reimplement the depmod extension rebuilder
Drop loop device/mounts completely, use userspace utilities to extract
and lay over module trees in the tmpfs.

Discover kernel version automatically instead of hardcoding it to be
current one (required for Image Service).

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-09-15 21:51:42 +04:00
Andrey Smirnov
a096f05a56
chore: update gRPC library and enable shared write buffers
Fixes #7576

See https://github.com/grpc/grpc-go/pull/6309

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-09-13 21:27:46 +04:00