222 Commits

Author SHA1 Message Date
Andrey Smirnov
dd4185b144
feat: add KubeSpan extra endpoint configuration
Fixes #9174

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-09-06 14:50:12 +04:00
Andrey Smirnov
3038ccfa88
feat: add configuration for EPHEMERAL volume
Fixes #9261

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-09-06 14:11:35 +04:00
Andrey Smirnov
b453385bd9
feat: support volume configuration, provisioning, etc
This implements the first round of changes, replacing the volume backend
with the new implementation, while keeping most of the external
interfaces intact.

See #8367

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-30 18:32:34 +04:00
Dmitry Sharshakov
4834a61a8e
feat: report SELinux labels
This will be useful for debugging SELinux implementation. Make API report other xattrs for further development like IMA/EVM

Signed-off-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
2024-08-26 16:19:38 +03:00
Eddie Zaneski
5ba1df4695
chore: add java package to protos
This option must be defined at the proto level in order to have an
import path that is reasonably usable

Signed-off-by: Eddie Zaneski <eddiezane@gmail.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-26 15:23:21 +04:00
Andrey Smirnov
ee4290f684
fix: bind HostDNS to 169.254.x link-local address
This is an attempt to fix many issues related with trying to use Service
IP for host DNS.

Fixes #9196

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-19 18:44:35 +04:00
Andrey Smirnov
61a1c946bf
feat: bundle (some) CNI plugins with Talos core
Fixes https://github.com/siderolabs/extensions/issues/448

Bundle some CNI standard plugins plus Flannel CNI plugin (as Flannel is
the default CNI in Talos) in the Talos `initramfs`.

With this change, no plugin install is required, so the `install-cni`
step is dropped from the Flannel default manifest.

The bundled plugins:

```
$ talosctl -n 172.20.0.2 ls -lH /opt/cni/bin/
NODE         MODE         UID   GID   SIZE(B)   LASTMOD       NAME
172.20.0.2   drwxr-xr-x   0     0     109 B     7 hours ago   .
172.20.0.2   -rwxr-xr-x   0     0     3.2 MB    7 hours ago   bridge
172.20.0.2   -rwxr-xr-x   0     0     3.3 MB    7 hours ago   firewall
172.20.0.2   -rwxr-xr-x   0     0     2.4 MB    7 hours ago   flannel
172.20.0.2   -rwxr-xr-x   0     0     2.4 MB    7 hours ago   host-local
172.20.0.2   -rwxr-xr-x   0     0     2.4 MB    7 hours ago   loopback
172.20.0.2   -rwxr-xr-x   0     0     2.8 MB    7 hours ago   portmap
```

The `initramfs` for amd64 grows 67 -> 73 MiB with this change.

The path `/opt/cni/bin` is still an overlay mount, so extra plugins can
be dropped to this directory (no change here).

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-14 14:33:18 +04:00
Andrey Smirnov
7a1c62b8bc
feat: publish installed extensions as node labels/annotations
Extensions are posted the following way:

`extensions.talos.dev/<name>=<version>`

The name should be valid as a label (annotation) key.

If the value is valid as a label value, use labels, otherwise use
annotations.

Also implements node annotations in the machine config as a side-effect.

Fixes #9089

Fixes #8971

See #9070

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-01 17:32:09 +04:00
Andrey Smirnov
be35f380cc
chore: update pkgs/tools/extras
This brings in Go 1.22.5 and new Flannel CNI plugin.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-03 20:38:55 +04:00
Andrey Smirnov
cc345c8c94
feat: add support for configuring vlan filtering on the bridge
Fixes #8941

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-01 20:20:28 +04:00
Utku Ozdemir
5ffc3f14bd
feat: show siderolink status on dashboard
Add a new resource, `SiderolinkStatus`, which combines the following info:
- The Siderolink API endpoint without the query parameters or fragments (potentially sensitive info due to the join token)
- The status of the Siderolink connection

This resource is not set as sensitive, so it can be retrieved by the users with `os:operator` role (e.g., using `talosctl dashboard` through Omni).

Make use of this resource in the dashboard to display the status of the Siderolink connection.

Additionally, rework the status columns in the dashboard to:
- Display a Linux terminal compatible "tick" or a "cross" prefix for statuses in addition to the red/green color coding.
- Move and combine some statuses to save rows and make them more even.

Closes siderolabs/talos#8643.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-06-18 12:31:54 +02:00
Andrey Smirnov
7cbdce73f7
fix: detect CD devices, fix user disks wipe test
Detect CD devices, and set size to 0 for CD without media.

In user disk wipe tests, skip device mapper devices and CD-ROM.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-06-10 18:00:06 +04:00
Andrey Smirnov
f07b79f4a8
feat: provide disk detection based on new blockdevices
Uses go-siderolabs/go-blockdevice/v2 for all the hard parts,
provides new resource `Disk` which describes all disks in the system.

Additional resource `SystemDisk` always point to the system disk (based
on the location of `META` partition).

The `Disks` API (and `talosctl disks`) provides a view now into the
`talosctl get disks` to keep backwards compatibility.

QEMU provisioner can now create extra disks of various types: IDE, AHCI,
SCSI, NVME, this allows to test detection properly.

The new resource will be the foundation for volume provisioning (to pick
up the disk to provision the volume on).

Example:

```
talosctl -n 172.20.0.5 get disks
NODE         NAMESPACE   TYPE   ID        VERSION   SIZE          READ ONLY   TRANSPORT   ROTATIONAL   WWID                                                               MODEL            SERIAL
172.20.0.5   runtime     Disk   loop0     1         65568768      true
172.20.0.5   runtime     Disk   nvme0n1   1         10485760000   false       nvme                     nvme.1b36-6465616462656566-51454d55204e564d65204374726c-00000001   QEMU NVMe Ctrl   deadbeef
172.20.0.5   runtime     Disk   sda       1         10485760000   false       virtio      true                                                                            QEMU HARDDISK
172.20.0.5   runtime     Disk   sdb       1         10485760000   false       sata        true         t10.ATA     QEMU HARDDISK                           QM00013        QEMU HARDDISK
172.20.0.5   runtime     Disk   sdc       1         10485760000   false       sata        true         t10.ATA     QEMU HARDDISK                           QM00001        QEMU HARDDISK
172.20.0.5   runtime     Disk   vda       1         12884901888   false       virtio      true
```

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-06-07 20:18:32 +04:00
Andrey Smirnov
8dbe2128a9
feat: implement Talos diagnostics
Talos diagnostics analyzes current system state and comes up with detailed
warnings on the system misconfiguration which might be tricky to figure
out other way.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-06-05 22:28:15 +04:00
Andrey Smirnov
4dd0aa7120
feat: implement PCI device bus enumeration
Fixes #8826

From the QEMU VM:

```shell
$ talosctl -n 172.20.0.5 get pcidevice
NODE         NAMESPACE   TYPE        ID             VERSION   CLASS                       SUBCLASS                    VENDOR              PRODUCT
172.20.0.5   hardware    PCIDevice   0000:00:00.0   1         Bridge                      Host bridge                 Intel Corporation   82G33/G31/P35/P31 Express DRAM Controller
172.20.0.5   hardware    PCIDevice   0000:00:01.0   1         Display controller          VGA compatible controller
172.20.0.5   hardware    PCIDevice   0000:00:02.0   1         Network controller          Ethernet controller         Red Hat, Inc.       Virtio network device
172.20.0.5   hardware    PCIDevice   0000:00:03.0   1         Unclassified device                                     Red Hat, Inc.       Virtio RNG
172.20.0.5   hardware    PCIDevice   0000:00:04.0   1         Unclassified device                                     Red Hat, Inc.       Virtio memory balloon
172.20.0.5   hardware    PCIDevice   0000:00:05.0   1         Communication controller    Communication controller    Red Hat, Inc.       Virtio console
172.20.0.5   hardware    PCIDevice   0000:00:06.0   1         Generic system peripheral   System peripheral           Intel Corporation   6300ESB Watchdog Timer
172.20.0.5   hardware    PCIDevice   0000:00:07.0   1         Mass storage controller     SCSI storage controller     Red Hat, Inc.       Virtio block device
172.20.0.5   hardware    PCIDevice   0000:00:1f.0   1         Bridge                      ISA bridge                  Intel Corporation   82801IB (ICH9) LPC Interface Controller
172.20.0.5   hardware    PCIDevice   0000:00:1f.2   1         Mass storage controller     SATA controller             Intel Corporation   82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode]
172.20.0.5   hardware    PCIDevice   0000:00:1f.3   1         Serial bus controller       SMBus                       Intel Corporation   82801I (ICH9 Family) SMBus Controller
```

```yaml
node: 172.20.0.5
metadata:
    namespace: hardware
    type: PCIDevices.hardware.talos.dev
    id: 0000:00:1f.3
    version: 1
    owner: hardware.PCIDevicesController
    phase: running
    created: 2024-05-30T12:09:05Z
    updated: 2024-05-30T12:09:05Z
spec:
    class: Serial bus controller
    subclass: SMBus
    vendor: Intel Corporation
    product: 82801I (ICH9 Family) SMBus Controller
    class_id: "0x0c"
    subclass_id: "0x05"
    vendor_id: "0x8086"
    product_id: "0x2930"
```

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-05-31 20:56:16 +04:00
Serge Logvinov
f4d7b9d9a9
feat: gather plaform dns names
Retrieve the DNS names of instances from the platform metadata.

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-05-08 00:11:24 +04:00
Dmitry Sharshakov
6fbd1263cc
feat: report process MAC labels
This will be useful for debugging process access rights once we start implementing SELinux

Signed-off-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
2024-04-22 18:16:33 +03:00
Dmitriy Matrenichev
908f67fa15
feat: add host dns support for resolving member addrs
Closes #8330

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-04-18 15:29:30 +03:00
Andrey Smirnov
862c76001b
feat: add support for CoreDNS forwarding to host DNS
This PR adds the support for CoreDNS forwarding to host DNS. We try to bind on 9th address on the first element from
`serviceSubnets` and create a simple service so k8s will not attempt to rebind it.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
Co-authored-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-04-03 23:36:17 +03:00
Andrey Smirnov
3195e5d15c
fix: force Flannel CNI to use KubePrism Kubernetes API endpoint
Fixes #8501

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-02 22:01:05 +04:00
Noel Georgi
917043fb55
chore: bump tools, pkgs and extra to stable
Bump tools, pkgs and extras to stable release.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-04-02 22:15:50 +05:30
Andrey Smirnov
7a68504b6b
feat: support rotating Kubernetes CA
Fixes #8440

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-01 22:08:02 +04:00
Noel Georgi
bac366e43e
chore: add ExtraInfo field for extensions
Add an extra field to extensions to store arbitrary info.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-04-01 19:30:29 +05:30
Dmitry Sharshakov
9456489147
feat: support hardware watchdog timers
Only enabled when activated by config, disabled on shutdown/reboot

Fixes #8284

Signed-off-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
Signed-off-by: Dmitry Sharshakov <d3dx12.xx@gmail.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-25 18:19:39 +03:00
Andrey Smirnov
8eacc4ba80
feat: support rotation of Talos API CA
This allows to roll all nodes to use a new CA, to refresh it, or e.g.
when the `talosconfig` was exposed accidentally.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-22 12:16:47 +04:00
Andrey Smirnov
89fc68b459
fix: service lifecycle issues
The core change is moving the context out of the `ServiceRunner` struct
to be a local variable, and using a channel to notify about shutdown
events.

Add more synchronization between Run and the moment service started to
avoid mis-identifying not running (yet) service as successfully finished.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
Co-authored-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-03-19 18:11:13 +04:00
Andrey Smirnov
15beb14780
feat: implement blockdevice watch controller
This controller combines kobject events, and scan of `/sys/block` to
build a consistent list of available block devices, updating resources
as the blockdevice changes.

Based on these resources the next step can run probe on the blockdevices
as they change to present a consistent view of filesystems/partitions.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-18 18:28:40 +04:00
Dmitriy Matrenichev
06e3bc0cbd
feat: implement Siderolink wireguard over GRPC
For #8064

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-03-18 15:38:13 +03:00
Dmitriy Matrenichev
32e0877607
chore: print all available logs containers in logs command completions
This is a small quality of life improvement that allows `logs` subcommand to suggest all available logs.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-03-11 17:48:01 +03:00
Noel Georgi
15e8bca2b2
feat: support environment in ExtensionServicesConfig
Support setting extension services environment variables in
`ExtensionServiceConfig` document.

Refactor `ExtensionServicesConfig` -> `ExtensionServiceConfig` and move extensions config under `runtime` pkg.

Fixes: #8271

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-02-15 20:16:29 +05:30
Dmitriy Matrenichev
5324d39167
chore: bump stuff
Also fix .golangci.yml file.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-09 19:19:25 +03:00
Dmitriy Matrenichev
afa71d6b02
chore: use "handle-like" resource in DNSResolveCacheController
Rework (and simplify) `DNSResolveCacheController` to use `DNSUpstream` "handle-like" resources.

Depends on https://github.com/cosi-project/runtime/pull/400

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-08 21:40:57 +03:00
Noel Georgi
1e6c8c4dec
feat: extensions services config
Support config files for extension services.

Fixes: #7791

Co-authored-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-02-06 17:12:01 +05:30
Andrey Smirnov
9d8cd4d058
chore: drop deprecated method EtcdRemoveMember
It was deprecated 16 months ago, time to cleanup.

(This is to prepare for the first v1.7 release)

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-01 15:54:29 +04:00
Dmitriy Matrenichev
ebeef28525
feat: implement local caching dns server
This PR adds a new controller - `DNSServerController` that starts tcp and udp dns servers locally. Just like `EtcFileController` it monitors `ResolverStatusType` and updates the list of destinations from there.

Most of the caching logic is in our "lobotomized" "`CoreDNS` fork. We need this fork because default `CoreDNS` carries
full Caddy server and various other modules that we don't need in Talos. On our side we implement
random selection of the actual dns and request forwarding.

Closes #7693

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-01-29 20:26:38 +03:00
Andrey Smirnov
131a1b1671
fix: add a KubeSpan option to disable extra endpoint harvesting
It works well for small clusters, but with bigger clusters it puts too
much load on the discovery service, as it has quadratic complexity in
number of endpoints discovered/reported from each member.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-12 14:07:31 +04:00
Dmitriy Matrenichev
6bb1e99aa3
chore: optimize pcap dump
Reimplement `gopacket.PacketSource.PacketsCtx` as `forEachPacket`.

- Use `ZeroCopyPacketDataSource` instead of `PacketDataSource`. I didn't find any specific reason why `PacketDataSource` exists at all, since `NewPacket` is doing copy inside if you don't explicitly tell it not to.
- Use `WillPool` to pool packet buffers. It doesn't fully remove allocations, but it's a safe start.
  Send packets back into the pool after we are done with them.
- Pass `Packet` directly to the closure instead of waiting for it on the channel. We don't store this packet anywhere so there is no reason to async this part.
- Drop `time.Sleep` code in `forEachPacket` body.
- Drop `SnapLen` support in client and server since it didn't work anyway (details in the PR).

Closes #7994

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-12-11 15:44:42 +03:00
Andrey Smirnov
e71e3e4161
feat: support extra arguments for flanneld
Fixes #7754

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-01 16:18:02 +04:00
Dmitriy Matrenichev
0b111ecb81
fix: support slices of enums and fix NfTablesConntrackStateMatch
We already have the code which supports custom enums, so let's extend it to support custom enums in slices and
fix the NfTablesConntrackStateMatch proto definition.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-11-30 00:23:16 +03:00
Andrey Smirnov
9a85217412
feat: improve nftables backend
Many changes to the nftables backend which will be used in the follow-up
PR with #4421.

1. Add support for chain policy: drop/accept.
2. Properly handle match on all IPs in the set (`0.0.0.0/0` like).
3. Implement conntrack state matching.
4. Implement multiple ifname matching in a single rule.
5. Implement anonymous counters.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-29 21:22:47 +04:00
Andrey Smirnov
db4e2539d4
feat: update Kubernetes 1.29.0-rc.1 and other bumps
Bump Go modules, final tools and semi-final pkgs.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-29 18:29:52 +04:00
Andrey Smirnov
e46e6a312f
feat: implement nftables backend
Implement initial set of backend controllers/resources to handle
nftables chains/rules etc.

Replace the KubeSpan nftables operations with controller-based.

See #4421

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-27 21:14:15 +04:00
Dmitriy Matrenichev
70d53ee13c
chore: deprecate .persist and .extensions
This commit deprecates those things:
- Removes the support of `.persist` flag. From now, it should always be enabled or not defined in the config.
- Removes the documentation for `.bootloader`. It never worked anyway.
- Adds a warning for `.machine.install.extensions`, suggests to use boot-assets.

Closes #7972
Closes #7507

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-11-22 20:35:38 +03:00
Oscar Utbult
020a0eb63e
docs: fix table formatting for bootstraprequest
Fixes formatting for https://www.talos.dev/v1.6/reference/api/#bootstraprequest

Signed-off-by: Oscar Utbult <oscar.utbult@gmail.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-20 20:11:43 +04:00
Oscar Utbult
de6caf5348
docs: fix table formatting for machineservice api
Fixes formatting for https://www.talos.dev/v1.6/reference/api/#machineservice

Signed-off-by: Oscar Utbult <oscar.utbult@gmail.com>
2023-11-20 17:13:10 +04:00
Noel Georgi
0d3c3ed716
feat: support kube scheduler config
Support kube-scheduler config.

Fixes: #7905
Partially fixes: #7911

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-11-15 10:15:23 +05:30
Noel Georgi
4f1ad16c76
feat: support kubelet credentialprovider config
Support configuring kubelet credential provider config.

Partially fixes: #7911

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-11-13 19:40:43 +05:30
Dmitriy Matrenichev
6eade3d5ef
chore: add ability to rewrite uuids and set unique tokens for Talos
This PR does those things:
- It allows API calls `MetaWrite` and `MetaRead` in maintenance mode.
- SystemInformation resource now waits for available META
- SystemInformation resource now overwrites UUID from META if there is an override
- META now supports "UUID override" and "unique token" keys
- ProvisionRequest now includes unique token and Talos version

For #7694

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-11-10 18:17:54 +03:00
Andrey Smirnov
2b548ad0d9
feat: update containerd to 1.7.x
Also update Linux and other pkgs.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-09-28 16:33:57 +04:00
Artem Chernyshev
2960f93baa
feat: add readonly information to the disks API response
Forward device readonly info from `go-blockdevice` library.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2023-09-12 18:09:59 +03:00