605 Commits

Author SHA1 Message Date
Utku Ozdemir
cbab12e3a1
refactor: rename outbound to connectivity on dashboard
Rename to be consistent between the `networkstatus` resource and the dashboard.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-03-31 15:31:35 +02:00
Utku Ozdemir
7967ccfc13
feat: add config code entry screen to dashboard
Implement a screen for entering/managing the config `${code}` variable.

Enable this screen only when the platform is `metal` and there is a `${code}` variable in the `talos.config` kernel cmdline URL query.

Additionally, remove the "Delete" button and its functionality from the network config screen to avoid users accidentally deleting PlatformNetworkConfig parts that are not managed by the dashboard.

Add some tests for the form data parsing on the network config screen.
Remove the unnecessary lock on the summary tab - all updates come from the same goroutine.

Closes siderolabs/talos#6993.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-03-31 10:33:28 +02:00
Nico Berlee
0af8fe2fb5
feat: netstat pod support
talosctl netstat -k show all host and non-hostnetwork pods sockets/connections.
talosctl netstat namespace/pod shows sockets/connections of a specific pod +
autocompletes in the shell.

Signed-off-by: Nico Berlee <nico.berlee@on2it.net>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-30 23:39:38 +04:00
Utku Ozdemir
aa662ff635
fix: apply small fixes on dashboard
* Clear the input form and switch to summary tab after the network config is saved.
* Use nodeaddress resource for detecting and displaying IPs. Improve the IP filtering logic.
* Fix the logic of gateway detection. Display all gateways instead of a single one.
* Use hostnamestatus resource to detect the hostname instead of an API call.
* Add hostname entry to the network info section on summary tab (as `HOST`).
* Enable `OUTBOUND` entry in network info section on summary tab.
* Display only the physical network interfaces in the interface dropdown on network config tab.
* Improve form input handling.
* Additional minor fixes & improvements.

Closes siderolabs/talos#6992.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-03-30 09:39:14 +02:00
Andrey Smirnov
ea0e9bdbe4
feat: environment variables via the kernel arguments
Unify getting environment variables, support passing environment
variables via kernel args.

Fixes #6984
See #6999

For META this will be used to pass environment variables to the
installer for ISO images (or PXE booting).

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-28 16:28:33 +04:00
Andrey Smirnov
9e8603f53b
feat: implement new download URL variable ${code}
New variable value is coming from `META`, and it might be set using the
interactive console (not implemented yet, but it will come soon).

I had to refactor the URL expansion implementation:

* simplify things where possible
* provide more unit-tests for smaller units
* handle expansion of all variables in parallel
* allow parallel expansion on multiple variables

Also I refactored download code to support proper passing of endpoint
function with context.

The end result:

* Talos will try to download config for 3 hours before rebooting
* Each attempt which includes URL expansion + download is limited to 3
  minutes

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-24 21:49:36 +04:00
Utku Ozdemir
a7b79ef1be
feat: add network config screen to dashboard
Implement the network config screen with input forms to configure the initial node networking by writing a config to the META partition.

Closes siderolabs/talos#6961.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-03-23 17:29:52 +04:00
Noel Georgi
d1a61fd343
chore: bump golangci-lint
Bump golangci-lint and fixup new warnings. Ignore check that checks for
used function parameters, it's kind of noisy and makes it confusing to
read interface implementations.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-03-22 19:55:38 +05:30
Noel Georgi
36a9a208ec
chore: bump deps
Bump deps

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-03-22 16:37:27 +05:30
Dmitriy Matrenichev
b246c90abd
fix: add uint32 to Magic1 and Magic2
Discovered in #6971. Go compiler cannot deduce proper type on 32bit architectures for those constants,
in `fmt.Print(f)` functions. Since we only compare them with uint32 variables, it makes sense to add proper
types to them.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-03-21 09:57:55 -03:00
Andrey Smirnov
bec89bf6e5
fix: use 'no block' etcd dial with multiple endpoints
The problem showed up on 'reset' of the Talos node which had multiple
endpoints for other control plane nodes, many of which weren't actually
available.

When 'grpc.WithBlock()' is used, etcd will try to dial the first
endpoint and return an error if the dial fails.

Use noblock mode by default with multiple endpoints, and blocking mode
with a single endpoint.

Pass the context to etcd to properly abort dial operations if the
context get canceled.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-21 15:35:31 +04:00
Utku Ozdemir
2dd0964c5f
refactor: use resource watches on dashboard
Instead of doing excessive get/list requests, do a watch per node in an infinite retry.

Additionally, refactor the dashboard code to make the various data listener namings more consistent and reorganize the packages.

Closes siderolabs/talos#6960.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-03-17 23:06:35 +01:00
Andrey Smirnov
64e3d24c6b
feat: provide platform network config for 'metal' in META
A special META key might contain optional platform network config for
the `METAL` platform.

It is completely optional, but if present, it works same way as in the
clouds: it is applied with low priority (can be overridden with machine
config), but provides some initial defaults for the machine.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-15 23:54:39 +04:00
Utku Ozdemir
9e07832db9
feat: implement summary dashboard
Implement the new summary dashboard with node info and logs.
Replace the previous metrics dashboard with the new dashboard which has multiple screens for node summary, metrics and editing network config.

Port the old metrics dashboard to the tview library and assign it to be a screen in the new dashboard, accessible by F2 key.

Add a new resource, infos.cluster.talos.dev which contains the cluster name and id of a node.

Disable the network config editor screen in the new dashboard until it is fully implemented with its backend.

Closes siderolabs/talos#4790.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-03-15 13:13:28 +01:00
Andrey Smirnov
1df841bb54
refactor: change the interface of META
Use a global instance, handle loading/saving META in global context.

Deprecate legacy syslinux ADV, provide an easier interface for
consumers.

Expose META as resources.

Fix the bootloader revert process (it was completely broken for quite a
while :sad:).

This is a first step which mostly does preparation work, real changes
will come in the next PRs:

* add APIs to write to META
* consume META keys for platform network config for `metal`
* custom key for URL `${code}`

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-15 15:43:16 +04:00
Dmitriy Matrenichev
ebc92f3c1d
chore: add container id to talosctl -k containers and talosctl -k logs
This PR adds first 12 symbols from container ID and adds them to `talosctl -k containers` each container output.
That way we can ensure that we get the logs from proper container even if there is a newer one.

Closes #6886

Co-authored-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-03-07 13:20:44 +03:00
Andrey Smirnov
40e69af224
fix: improve etcd leave on reset process
When removing a member from `etcd`, the server does a pre-check to make
sure the member is connected to a quorum of other members, and the
remove request might fail. Add a retry to wait for the etcd to be fully
connected before giving up, as some parts of the reset flow alrady ran.

Also fix an issue which appears in the integration test, when `reset` is
called early in the boot sequence when local etcd hasn't started fully yet.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-01 14:51:49 +04:00
Utku Ozdemir
f55f5df739
feat: move dashboard package & run it in tty2
Move dashboard package into a common location where both Talos and talosctl can use it.

Add support for overriding stdin, stdout, stderr and ctt in process runner.

Create a dashboard service which runs the dashboard on /dev/tty2.

Redirect kernel messages to tty1 and switch to tty2 after starting the dashboard on it.

Related to siderolabs/talos#6841, siderolabs/talos#4791.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-02-28 12:00:25 +01:00
Noel Georgi
426fe9687d
fix: extension base folder permission
The `modules.dep` kernel module dependency tree extension root path was
previously created with a permission of `0o700` which means the talos
root go a permission of `0o700` when the kernel module tree was re-built
when extensions providing kernel modules was enabled. This means that
any binaries lost the executable permission when ran as non-root
creating an `EACCES` error. Fix by making sure the temporary directory
created for building kernel modules tree has `0o755` permission
explicitly.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-02-27 19:49:06 +05:30
Andrey Smirnov
6e8f13529c
fix: add support for a fallback '*' mirror configuration
Talos always supported that, but CRI config lacked support for it.

Now with recent containerd the new `_default` host is used as a
fallback, so this re-enables the support and updates the docs.

See https://github.com/containerd/containerd/pull/8065

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-02-16 23:12:13 +04:00
Artem Chernyshev
dcd4eb1a93
fix: improve error message on single node upgrade
Fixes: https://github.com/siderolabs/talos/issues/6828

Propose a solution if the node upgrade fails.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2023-02-16 17:33:04 +03:00
Noel Georgi
7b75cd8b94
fix: kernel module dependency tree generation
This fixes the issue when the overlay mount target directory was used as
lowerdir for the mount, creating extra folders in the extension.

Fix the issue by adding support for normal overlay mounts to use a
source directory when specified.

Also fixes a small issue where messages was logged when error is nil.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-02-14 01:07:11 +05:30
Noel Georgi
cc6e37a47f
feat: use process wrapper for dropping capabilities
Use process wrapper introduced in #6814 to drop capabilities. This change
also means the capabilities are dropped per process level and not for
PID 1 (machined), which allows us to drop capabilities per process.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-02-07 00:49:56 +05:30
Noel Georgi
812a2877cd
chore: bump deps + renovate cleanup
Bump dependencies.
Disable renovate for PR's and skip un-needed update checks.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-01-24 00:42:58 +05:30
Andrey Smirnov
3e00571627
fix: unwrap gRPC errors on stop/remove pods check
As the client returns wrapped errors, unwrap them using our own method
which does `errors.As` instead of gRPC one which doesn't do unwrapping.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-01-23 14:29:04 +04:00
Noel Georgi
d4b8b35de7
feat: generate kernel module dependency tree
Run `depmod` during install/upgrades when extensions provide kernel
modules and `modules.dep` needs to be re-generated. This also allows
modules of same name from kernel to co-exist. Modules in `extras`
folder takes precedence over `in-built` ones.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-01-19 18:54:10 +05:30
Andrey Smirnov
a0c0352ddc
fix: send diagnostic output to stderr consistently
Fixes #6676

There was a mix of stdout/stderr, move more consistently to stderr.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-12-23 18:41:56 +04:00
Andrey Smirnov
5b2960efff
fix: introduce 'overridePath' setting and fix Talos resolver
There was inconsistency in the way `/v2` was appended to registry
endpoint path between containerd (CRI) and Talos:

* Talos only appended `/v2` to empty paths
* containerd appended `/v2` if it's not the suffix already

Fix Talos to act same as containerd, and introduce a setting
`overridePath` which stops both Talos and `containerd` from appending
`/v2` (should be required with e.g. Harbor registry mirror).

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-12-05 12:50:53 +04:00
Andrey Smirnov
1103c5ad24
feat: implement pre-flight checks in the installer
Host Talos mounts machined socket for API access into the installer
container (for upgrades).

Installer runs any check it might need to verify compatibility.

At the moment following checks are implemented:

* Talos version (whether upgrade from version X to Y is supported)
* Kubernetes version (whether Kubernetes version X is supported with
  Talos Y).

Fixes #6149

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-11-28 13:45:49 +04:00
Andrey Smirnov
ee7a4777af
chore: bump dependencies
Linux 5.15.79, containerd 1.6.10

Other changes come from:

* https://github.com/siderolabs/toolchain/pull/57
* https://github.com/siderolabs/tools/pull/244
* https://github.com/siderolabs/pkgs/pull/619
* https://github.com/siderolabs/extras/pull/67

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-11-22 23:47:05 +04:00
Andrey Smirnov
6ffc381c59
feat: implement CRI configuration customization
This is tricky, as containerd doesn't merge itself plugin configuration
across multiple files. TOML can't load configuration correctly from
concatenated files.

Fixes #6390

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-11-16 15:38:44 +04:00
Andrey Smirnov
1cfb6188bc
feat: implement support for cgroupsv1
Use boot kernel arg `talos.unified_cgroup_hierarchy=0` to force Talos to
use cgroups v1. Talos still defaults to cgroupsv2.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-11-11 15:49:25 +04:00
Andrey Smirnov
3866d0e334
feat: update Kubernetes to v1.26.0-beta.0
See https://github.com/kubernetes/kubernetes/releases/tag/v1.26.0-beta.0

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-11-11 15:06:34 +04:00
Philipp Sauter
4e114ca120
feat: use the etcd member id for etcd operations instead of hostname
We add a controller that provides the etcd member id as a resource
and change the etcd related commands to support member ids next to
hostnames.

Fixes: #6223

Signed-off-by: Philipp Sauter <philipp.sauter@siderolabs.com>
2022-11-10 19:17:56 +04:00
Andrey Smirnov
96aa9638f7
chore: rename talos-systems/talos to siderolabs/talos
There's a cyclic dependency on siderolink library which imports talos
machinery back. We will fix that after we get talos pushed under a new
name.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-11-03 16:50:32 +04:00
Andrey Smirnov
30bbf6463a
refactor: use siderolabs/net version with netip.Addr
Replace most of `net.IP` usage in Talos with `netip.Addr`, refactor code
accordingly.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-11-02 14:21:03 +04:00
Andrey Smirnov
343c55762e
chore: replace talos-systems Go modules with siderolabs
This the first step towards replacing all import paths to be based on
`siderolabs/` instead of `talos-systems/`.

All updates contain no functional changes, just refactorings to adapt to
the new path structure.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-11-01 12:55:40 +04:00
Andrey Smirnov
d7edd0e2e6
refactor: use go-circular, go-kubeconfig, and go-tail
Remove Talos versions, use new extracted Go modules.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-10-25 20:20:44 +04:00
Dmitriy Matrenichev
93e55b85f2
chore: bump golangci-lint to v1.50.0
I had to do several things:
- contextcheck now supports Go 1.18 generics, but I had to disable it because of this https://github.com/kkHAIKE/contextcheck/issues/9
- dupword produces to many false positives, so it's also disabled
- revive found all packages which didn't have a documentation comment before. And tehre is A LOT of them. I updated some of them, but gave up at some point and just added them to exclude rules for now.
- change lint-vulncheck to use `base` stage as base

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2022-10-20 18:33:19 +03:00
Andrew Rynhard
b7b1d4fd6a
feat: use readonly containers
Containers created with `talosctl cluster create` are ran with readonly
filesystem. This more accurately mimics standard Talos.

Signed-off-by: Andrew Rynhard <andrew@rynhard.io>
2022-10-11 15:24:38 +00:00
Andrey Smirnov
06f76bfebb
chore: bump dependencies
Update to some dependencies moved to siderolabs/ path.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-10-04 14:47:27 +04:00
Andrey Smirnov
139c62d762
feat: allow upgrades in maintenance mode (only over SideroLink)
This implements a simple way to upgrade Talos node running in
maintenance mode (only if Talos is installed, i.e. if `STATE` and
`EPHEMERAL` partitions are wiped).

Upgrade is only available over SideroLink for security reasons.

Upgrade in maintenance mode doesn't support any options, and it works
without machine configuration, so proxy environment variables are not
available, registry mirrors can't be used, and extensions are not
installed.

Fixes #6224

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-09-30 21:16:15 +04:00
Dmitriy Matrenichev
fc48849d00
chore: move maps/slices/ordered to gen module
Use github.com/siderolabs/gen

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2022-09-21 20:22:43 +03:00
Andrey Smirnov
8b09bd4b04
feat: update Kubernetes to v1.26.0-alpha.1
Talos 1.3.0 will ship with Kubernetes 1.26.0.

See https://github.com/kubernetes/kubernetes/releases/tag/v1.26.0-alpha.1

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-09-21 18:42:31 +04:00
Noel Georgi
357b770cb5
fix: cryptsetup delete slot
Fix cryptsetup delete slot.

Fixes: #6298

Signed-off-by: Noel Georgi <git@frezbo.dev>
2022-09-21 16:37:54 +05:30
Andrey Smirnov
f424e53404
fix: stop containers more thoroughly
Don't skip pods which are not ready, try still to stop containers inside
not ready pod sandboxes.

Re-enable the test with Canal CNI (upstream Calico got fixed).

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-09-12 20:16:40 +04:00
Andrey Smirnov
f62d17125b
chore: update crypto to use new import path siderolabs/crypto
No functional changes in this PR, just updating import paths.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-09-07 23:02:50 +04:00
Andrey Smirnov
4e9c322564
fix: correctly render hosts.toml with multiple endpoints
We should preserve the order of keys in generated `hosts.toml`, but
go-toml library has no real way to do that on marshaling, so fix the
previous workaround, as it was generating invalid TOML.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-08-29 15:34:42 +04:00
Dmitriy Matrenichev
b59ca5810e
chore: move from inet.af/netaddr to net/netip and go4.org/netipx
Closes #6007

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2022-08-25 17:51:32 +03:00
Andrey Smirnov
053af1d59e
fix: update etcd certificates when node addresses changes
Fixes #6110

I somehow missed the fact that etcd certs were not made fully reactive
to node address changes (I wrongly assume it was already the fact).

This PR refactors etcd certificate generation process to be
resource-based and introduces unit-tests for the controller.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-08-25 00:27:52 +04:00