1859 Commits

Author SHA1 Message Date
Noel Georgi
36a9a208ec
chore: bump deps
Bump deps

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-03-22 16:37:27 +05:30
Noel Georgi
c63cf90e32
feat: update k8s to v1.27.0-beta.0
Update k8s to v1.27.0-beta.0

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-03-21 23:59:17 +05:30
Dmitriy Matrenichev
b246c90abd
fix: add uint32 to Magic1 and Magic2
Discovered in #6971. Go compiler cannot deduce proper type on 32bit architectures for those constants,
in `fmt.Print(f)` functions. Since we only compare them with uint32 variables, it makes sense to add proper
types to them.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-03-21 09:57:55 -03:00
Andrey Smirnov
bec89bf6e5
fix: use 'no block' etcd dial with multiple endpoints
The problem showed up on 'reset' of the Talos node which had multiple
endpoints for other control plane nodes, many of which weren't actually
available.

When 'grpc.WithBlock()' is used, etcd will try to dial the first
endpoint and return an error if the dial fails.

Use noblock mode by default with multiple endpoints, and blocking mode
with a single endpoint.

Pass the context to etcd to properly abort dial operations if the
context get canceled.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-21 15:35:31 +04:00
Utku Ozdemir
2dd0964c5f
refactor: use resource watches on dashboard
Instead of doing excessive get/list requests, do a watch per node in an infinite retry.

Additionally, refactor the dashboard code to make the various data listener namings more consistent and reorganize the packages.

Closes siderolabs/talos#6960.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-03-17 23:06:35 +01:00
Dmitriy Matrenichev
a14a0aba04
fix: nil pointer exception in syncLink
If link has no `Info` field we can't do anything meaningful, so we'll just log and skip.
Also fix race in test.

For #6956

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-03-17 15:33:20 +04:00
Noel Georgi
cf101e56fb
fix: add --force flag for talosctl gen
Error out if file(s) already exists and warn user to use
`--force` to overwrite.

Fixes: #6963

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-03-17 15:07:12 +05:30
Utku Ozdemir
ea2aa06116
fix: fix data race on network config read
Fix a data race caused by the metadata field of PlatformNetworkConfig being edited after it was sent to the channel. It caused test failures.

Fix it by setting a copy of the metadata instead.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-03-17 00:24:22 +01:00
Andrey Smirnov
64e3d24c6b
feat: provide platform network config for 'metal' in META
A special META key might contain optional platform network config for
the `METAL` platform.

It is completely optional, but if present, it works same way as in the
clouds: it is applied with low priority (can be overridden with machine
config), but provides some initial defaults for the machine.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-15 23:54:39 +04:00
Andrey Smirnov
442cb9c1b0
feat: implement APIs to write to META
This allows to put keys to META partition.

META contents can be viewed with `talosctl get metakeys`.

There is not real usecase for it yet, but the next PRs will introduce
two special keys which can be written:

* platform network config for `metal`
* `${code}` variable

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-15 22:17:52 +04:00
Utku Ozdemir
9e07832db9
feat: implement summary dashboard
Implement the new summary dashboard with node info and logs.
Replace the previous metrics dashboard with the new dashboard which has multiple screens for node summary, metrics and editing network config.

Port the old metrics dashboard to the tview library and assign it to be a screen in the new dashboard, accessible by F2 key.

Add a new resource, infos.cluster.talos.dev which contains the cluster name and id of a node.

Disable the network config editor screen in the new dashboard until it is fully implemented with its backend.

Closes siderolabs/talos#4790.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-03-15 13:13:28 +01:00
Andrey Smirnov
1df841bb54
refactor: change the interface of META
Use a global instance, handle loading/saving META in global context.

Deprecate legacy syslinux ADV, provide an easier interface for
consumers.

Expose META as resources.

Fix the bootloader revert process (it was completely broken for quite a
while :sad:).

This is a first step which mostly does preparation work, real changes
will come in the next PRs:

* add APIs to write to META
* consume META keys for platform network config for `metal`
* custom key for URL `${code}`

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-15 15:43:16 +04:00
Andrey Smirnov
02b0ff35ee
feat: generate Flannel CNI manifest from upstream
Fixes #6730

`go generate`-based step downloads the upstream manifest, transforms it
to match our requirements, and it is compiled in as the Flannel
manifest.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-13 20:00:35 +04:00
Serge Logvinov
9948a646d2
feat: coredns node uninitialized toleration
Launch CoreDNS even if the node is not initialized.
Network is ready already, but CCM didn't finish their job.

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-13 14:29:14 +04:00
Erik Lund
230cfaf803
feat: use network information from guestinfo.metadata
Add VMware GuestInfo metadata to network configuration.

Fixes #6708

Signed-off-by: Erik Lund Jensen <info@erikjensen.it>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-09 16:51:08 +04:00
Nico Berlee
97048f7c37
feat: netstat in API and client
Implements netstat in Talos API and client (talosctl).

Signed-off-by: Nico Berlee <nico.berlee@on2it.net>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-09 15:48:30 +04:00
Andrey Smirnov
fda6da6929
fix: successful ACPI shutdown in maintenance mode
Fixes #6817

The original problem wasn't reproducible with `main`, but there was a
set of bugs in the shutdown sequence which was preventing it from
completing successfully, as in the maintenance mode nothing is running
and initialized yet.

Most of the bugs were `nil` pointer dereferences.

Fixed a small issue with final 'RebootError' printed as a failure in the
ACPI shutdown path.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-07 23:52:02 +04:00
Dmitriy Matrenichev
ebc92f3c1d
chore: add container id to talosctl -k containers and talosctl -k logs
This PR adds first 12 symbols from container ID and adds them to `talosctl -k containers` each container output.
That way we can ensure that we get the logs from proper container even if there is a newer one.

Closes #6886

Co-authored-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-03-07 13:20:44 +03:00
Dmitriy Matrenichev
22ef81c1e7
feat: add grub option to drop to maintenance mode
- [x] Support `talos.experimental.wipe=system:EPHEMERAL,STATE` boot kernel arg
- [x] GRUB option to wipe like above
- [x] update GRUB library to handle that

Closes #6842

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-03-07 12:37:59 +03:00
Dmitriy Matrenichev
e71cc6619b
fix: redo assertHostnames in HostnameMergeSuite.TestMerge
Use `rtestutils.AssertResources` for hostnames test.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-03-06 15:09:50 +03:00
Andrey Smirnov
8ea4bfad8f
refactor: improve the kubernetes upgrade flow
Use new version of go-kubernetes, and move the `kube-proxy` DaemonSet
update to follow common logic of bootstrap manifests update.

This fixes a confusing behavior when after `k8s-upgrade` the version of
`kube-proxy` is not updated in the machine config.

See https://github.com/siderolabs/go-kubernetes/pull/3

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-06 15:01:29 +04:00
Tim Jones
061640cccf
feat: add pod ip to kube-proxy spec
Exposes the pod IP as the `POD_IP` environment variable via the downward
API in the kube-proxy pod for use in e.g. metrics-bind-addr.

Signed-off-by: Tim Jones <tim.jones@siderolabs.com>
2023-03-03 12:52:30 +01:00
Andrey Smirnov
337aaba7a7
feat: add 'os:operator' role
This introduces a new role for Talos API which fills the gap between
`os:reader` and `os:admin` roles.

Fixes #6898

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-01 16:12:25 +04:00
Andrey Smirnov
40e69af224
fix: improve etcd leave on reset process
When removing a member from `etcd`, the server does a pre-check to make
sure the member is connected to a quorum of other members, and the
remove request might fail. Add a retry to wait for the etcd to be fully
connected before giving up, as some parts of the reset flow alrady ran.

Also fix an issue which appears in the integration test, when `reset` is
called early in the boot sequence when local etcd hasn't started fully yet.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-01 14:51:49 +04:00
Dmitriy Matrenichev
638dc9128f
fix: fix "defer" leak in ResetUserDisks
Also, print error if we failed to close the device.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-02-28 21:51:37 +03:00
Dmitriy Matrenichev
bfba3677b0
chore: handle grub option - "wipe"
This PR ensures that we can handle third grub option - "wipe". We will use it in 1.4.

For #6842

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-02-28 21:21:28 +03:00
Artem Chernyshev
b520710810
feat: introduce new flag in reset API that makes Talos reset user disks
Fixes: https://github.com/siderolabs/talos/issues/6815

Additionally, make it possible to run reset in maintenance mode: to
enable a way for resetting system disk and remove all traces of Talos
from it.

The new reset flow works in a separate sequence, changed disk probe
lookup to check the boot partition instead of the ephemeral one.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2023-02-28 15:10:41 +03:00
Utku Ozdemir
f55f5df739
feat: move dashboard package & run it in tty2
Move dashboard package into a common location where both Talos and talosctl can use it.

Add support for overriding stdin, stdout, stderr and ctt in process runner.

Create a dashboard service which runs the dashboard on /dev/tty2.

Redirect kernel messages to tty1 and switch to tty2 after starting the dashboard on it.

Related to siderolabs/talos#6841, siderolabs/talos#4791.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-02-28 12:00:25 +01:00
Dmitriy Matrenichev
36e077ead4
chore: bump deps
- github.com/aws/aws-sdk-go to v1.44.209
- github.com/stretchr/testify to v1.8.2
- github.com/jsimonetti/rtnetlink to v1.3.1
- google.golang.org/genproto to v0.0.0-20230223222841-637eb2293923
- github.com/emicklei/dot to v1.3.1
- github.com/gdamore/tcell/v2 to v2.6.0
- github.com/insomniacslk/dhcp to v0.0.0-20230220063916-5369909a5de7
- github.com/jsimonetti/rtnetlink to v1.3.1
- github.com/opencontainers/runtime-spec to v1.1.0-rc.1.0.20230215090456-58ec43f9fc39
- github.com/rivo/tview to v0.0.0-20230226195229-47e7db7885b4

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-02-28 00:14:59 +03:00
Noel Georgi
426fe9687d
fix: extension base folder permission
The `modules.dep` kernel module dependency tree extension root path was
previously created with a permission of `0o700` which means the talos
root go a permission of `0o700` when the kernel module tree was re-built
when extensions providing kernel modules was enabled. This means that
any binaries lost the executable permission when ran as non-root
creating an `EACCES` error. Fix by making sure the temporary directory
created for building kernel modules tree has `0o755` permission
explicitly.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-02-27 19:49:06 +05:30
Andrey Smirnov
230e46e567
refactor: extract parts of kubernetes libraries
The shared code is going out to the
github.com/siderolabs/go-kubernetes library.

The code will be used in Talos and other projects using same features.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-02-22 14:56:49 +04:00
Utku Ozdemir
5ac9f43e45
feat: start machined earlier & in maintenance mode
Load & start machined earlier and in initialize sequence, so that it is possible to use its API over its unix socket in maintenance mode.

Additionally, do not return features from Version API  if a config is not yet available.

Related to siderolabs/talos#4791.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-02-21 12:21:36 +01:00
Dmitriy Matrenichev
3d55bd80f4
fix: add --force flag to talosctl gen config
Only overwrite existing files if explicitly demanded.

Closes #6847

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-02-20 23:44:00 +03:00
Serge Logvinov
660b8874da
feat: cmdline integer netmask
Can set netmask as number.

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-02-20 20:55:56 +04:00
Andrey Smirnov
6e8f13529c
fix: add support for a fallback '*' mirror configuration
Talos always supported that, but CRI config lacked support for it.

Now with recent containerd the new `_default` host is used as a
fallback, so this re-enables the support and updates the docs.

See https://github.com/containerd/containerd/pull/8065

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-02-16 23:12:13 +04:00
Artem Chernyshev
dcd4eb1a93
fix: improve error message on single node upgrade
Fixes: https://github.com/siderolabs/talos/issues/6828

Propose a solution if the node upgrade fails.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2023-02-16 17:33:04 +03:00
Noel Georgi
2d01480180
feat: automatically load modules based on hw info
Fixes: #6802

Automatically load kernel modules based on hardware info and modules
alias info. udevd would automatically load modules based on HW
information present.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-02-14 19:57:13 +05:30
Noel Georgi
7b75cd8b94
fix: kernel module dependency tree generation
This fixes the issue when the overlay mount target directory was used as
lowerdir for the mount, creating extra folders in the extension.

Fix the issue by adding support for normal overlay mounts to use a
source directory when specified.

Also fixes a small issue where messages was logged when error is nil.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-02-14 01:07:11 +05:30
Noel Georgi
65d02e5ade
fix: dbus shutdown when it's not initialized
If dbus is not started and a shutdown was called talos panics, fix by
checking if the mock is nil.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-02-13 21:12:54 +05:30
Andrey Smirnov
a7079ce85c
fix: quote the ampersand character in GRUB config
Not sure how I missed it in the first PR, but that's the only character
which was not quoted properly.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-02-13 18:58:34 +04:00
Andrey Smirnov
dcbcf5a93c
fix: wait for network and retry in platform get config funcs
Wait for the network before trying to access the metadata service.

Retry the calls when appropriate (most platforms use `download.Download`
function which does proper retries).

Co-authored-by: Noel Georgi <git@frezbo.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-02-09 21:04:43 +04:00
Andrey Smirnov
e09e106665
fix: default dns domain to 'cluster.local' in local case
One case was missing: when network section is present, but value is
omitted.

Fixes #6825

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-02-08 14:35:28 +04:00
Noel Georgi
cc6e37a47f
feat: use process wrapper for dropping capabilities
Use process wrapper introduced in #6814 to drop capabilities. This change
also means the capabilities are dropped per process level and not for
PID 1 (machined), which allows us to drop capabilities per process.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-02-07 00:49:56 +05:30
Noel Georgi
5cb2915d8e
feat: use wrapper for starting processes
Use a wrapper for starting processes which can setup proper cgroups,
OOMscore, and also drop capabilities for the process, then it calls
`execve`.

The containerd tests is also fixed to support cgroups when
running tests in buildkit. It used to pass previously as we did not
error if cgroup setup failed.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-02-03 18:32:09 +05:30
Andrey Smirnov
38a51191e4
fix: correctly expand parameters in the URL
This fixes multiple issues:

* `log.Fatalf` in the machined code leads to kernel panic
* return URL if some expansion fails
* correctly handle destroyed event (wait for the next one)

Fixes #6807

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-02-02 18:42:45 +04:00
Andrey Smirnov
54f7d4c923
fix: correctly quote and unquote strings in GRUB config
One of the fields in the GRUB config - boot arguments - contains
user-controlled input. Talos supports variable expansion in
`talos.config` parameter, and uses `${var}` syntax.

In GRUB config, `}` is a special character, and introduction of `}`
breaks config parsing both for GRUB and Talos.

Correctly escape and unescape special characters.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-02-02 17:11:22 +04:00
Noel Georgi
590a393de9
fix: udevd healthcheck
The previous `udevd` healthcheck was incomplete and if `udevd` took more
time to startup the initial `udevadm trigger` would have silently failed
failing to setup proper devices. `udevadm trigger` returns an exit code
of zero even if `udevd` is not running. This PR fixes by first checking
if the `udevd` control socket exists, which is a faster check, then
making sure `udevd` is up by running `udevadm control` command. This
ensures that `udevd` is properly initialized before running any `udevadm
trigger` commands even if `udevd` is restarted/killed.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-01-30 16:12:41 +05:30
Noel Georgi
812a2877cd
chore: bump deps + renovate cleanup
Bump dependencies.
Disable renovate for PR's and skip un-needed update checks.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-01-24 00:42:58 +05:30
Andrey Smirnov
aa9f66c1c8
fix: mark DigitalOcean anchor IP as scope link
This excludes it out of the `NodeAddress`.

Needs extra testing to confirm that it actually still works as anchor
IP.

Fixes #6760

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-01-23 20:35:52 +04:00
Andrey Smirnov
3e00571627
fix: unwrap gRPC errors on stop/remove pods check
As the client returns wrapped errors, unwrap them using our own method
which does `errors.As` instead of gRPC one which doesn't do unwrapping.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-01-23 14:29:04 +04:00