633 Commits

Author SHA1 Message Date
Andrey Smirnov
badbc51e63
refactor: rewrite code to include preliminary support for multi-doc
`config.Container` implements a multi-doc container which implements
both `Container` interface (encoding, validation, etc.), and `Conifg`
interface (accessing parts of the config).

Refactor `generate` and `bundle` packages to support multi-doc, and
provide backwards compatibility.

Implement a first (mostly example) machine config document for
SideroLink API URL.

Many places don't properly support multi-doc yet (e.g. config patches).

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-31 18:38:05 +04:00
Andrey Smirnov
3c64a5ffba
chore: optimize image generation time
Use `pigz` and `--sparse` to handle more efficiently compression of the
assets.

Also move tasks out of `setup-ci` step, as it runs always, including for
the promoted pipelines.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-31 17:47:23 +04:00
Andrey Smirnov
1fb29a56a8
fix: fail quickly if upgrade-k8s is used with multiple nodes
Fixes #7283

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-29 14:35:29 +04:00
Andrey Smirnov
dc6764871c
refactor: move around config interfaces, make RawV1Alpha1 typed
See #7230

Refactor more config interfaces, move config accessor interfaces
to different package to break the dependency loop.

Make `.RawV1Alpha1()` method typed to avoid type assertions everywhere.

No functional changes.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-23 22:08:58 +04:00
Andrey Smirnov
0bb7e8a5cf
refactor: split config.Provider into Config & Container
See #7230

This is a step towards preparing for multi-doc config.

Split the `config.Provider` interface into parts which have different
implementation:

* `config.Config` accesses the config itself, it might be implemented by
  `v1alpha1.Config` for example
* `config.Container` will be a set of config documents, which implement
  validation, encoding, etc.

`Version()` method dropped, as it makes little sense and it was almost
not used.

`Raw()` method renamed to `RawV1Alpha1()` to support legacy direct
access to `v1alpha1.Config`, next PR will refactor more to make it
return proper type.

There will be many more changes coming up.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-23 16:05:16 +04:00
Dmitriy Matrenichev
85d8a16194
chore: bump deps
Bump deps

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-05-22 16:02:15 -04:00
Spencer Smith
39b7a56f01
chore: use 8GiB instead of 10GiB for cloud images
This PR changes the default disk size for cloud images to be 8GiB
instead. This was prompted b/c the disk price in azure between tiers is
doubled and the cutoff for the tier is 8GiB.

Signed-off-by: Spencer Smith <spencer.smith@talos-systems.com>
2023-05-19 20:35:13 -04:00
Spencer Smith
c3fabb9829
chore: update default image sizes to 10GB for all "cloud" images
This PR adds a flag to imager that allows for tweaking the size of the created disk. Additionally, it sets the default value of that created disk to 10GB, as most images are cloud images that fail when uploaded b/c it only picks up a 1GB disk currently. Also adds some processing the makefile to make sure we set the default small value for metal images and SBCs.

Signed-off-by: Spencer Smith <spencer.smith@talos-systems.com>
2023-05-19 13:35:39 -04:00
Andrey Smirnov
eba8185642
release(v1.5.0-alpha.0): prepare release
This is the official v1.5.0-alpha.0 release.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-19 18:38:24 +04:00
Andrey Smirnov
bb02dd263c
chore: drop deprecated stuff for Talos 1.5
* drop old resources API, which was deprecated long time ago
* use bootstrapped event in `talosctl get --watch` to better align
  columns in the table output

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-05-18 19:46:37 +04:00
Dmitriy Matrenichev
45e6e27af7
chore: bump runtime
Use new functions and methods from runtime module.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-05-11 17:18:08 -04:00
Andrey Smirnov
7a004a6f7f
fix: parse errors correctly
Message metadata might be missing, the easiest usecase is contacting
worker directly using it both as an endpoint and a node.

Fixes #7108

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-04-19 17:08:36 +04:00
Andrey Smirnov
2c55550a66
fix: quote ISO kernel args for GRUB
Use GRUB quoting function to the kernel args passed to Talos.

This fixes passing `${variable}` to `talos.config=` kernel argument.

Also fix a problem with `ONBUILD` being exected for `imager` image.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-04-07 12:29:49 +04:00
Utku Ozdemir
c3a595d5b7
fix: improve action tracking post checks
In the tracking of the `reset --reboot`, `reboot` and `upgrade` lifecycle commands, verify that the node was actually rebooted in the post check by comparing the pre- and post-check boot IDs.

In the `reset --reboot` post-check, try both maintenance and normal mode, since the reset might be issued to only remove `EPHEMERAL` partition, which will not put the node into the maintenance mode.

Fixes siderolabs/talos#7009.

Additionally, if an action tracking fails, return the error instead of swallowing it. This way the command erminates with a non-zero exit code. Suppress the re-printing this error after the command was run.

Fixes siderolabs/talos#6966.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-04-04 15:28:33 +02:00
Artem Chernyshev
07c3c5d59e
feat: return disk subsystem in the Disks API
Fixes: https://github.com/siderolabs/talos/issues/7017

Should allow external services to detect which user block devices might
need to be wiped during reset.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2023-03-31 16:10:59 +03:00
Nico Berlee
0af8fe2fb5
feat: netstat pod support
talosctl netstat -k show all host and non-hostnetwork pods sockets/connections.
talosctl netstat namespace/pod shows sockets/connections of a specific pod +
autocompletes in the shell.

Signed-off-by: Nico Berlee <nico.berlee@on2it.net>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-30 23:39:38 +04:00
Andrey Smirnov
289b41fe4b
fix: output of talosctl logs might be corruped
The line was not copied properly, so sometimes `talosctl logs` output
was garbled as the buffer was re-used.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-29 15:52:42 +04:00
Andrey Smirnov
02f0a4526d
feat: allow writing initial META values into the image
E.g. with the command:

```
make image-metal IMAGER_ARGS="--meta 0xc=abc --meta 0xd=abc"
```

This doesn't support ISO/PXE boot yet, it's going to come into the next
PR.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-28 21:27:00 +04:00
Utku Ozdemir
a7b79ef1be
feat: add network config screen to dashboard
Implement the network config screen with input forms to configure the initial node networking by writing a config to the META partition.

Closes siderolabs/talos#6961.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-03-23 17:29:52 +04:00
Noel Georgi
d1a61fd343
chore: bump golangci-lint
Bump golangci-lint and fixup new warnings. Ignore check that checks for
used function parameters, it's kind of noisy and makes it confusing to
read interface implementations.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-03-22 19:55:38 +05:30
Noel Georgi
cf101e56fb
fix: add --force flag for talosctl gen
Error out if file(s) already exists and warn user to use
`--force` to overwrite.

Fixes: #6963

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-03-17 15:07:12 +05:30
Andrey Smirnov
442cb9c1b0
feat: implement APIs to write to META
This allows to put keys to META partition.

META contents can be viewed with `talosctl get metakeys`.

There is not real usecase for it yet, but the next PRs will introduce
two special keys which can be written:

* platform network config for `metal`
* `${code}` variable

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-15 22:17:52 +04:00
Utku Ozdemir
9e07832db9
feat: implement summary dashboard
Implement the new summary dashboard with node info and logs.
Replace the previous metrics dashboard with the new dashboard which has multiple screens for node summary, metrics and editing network config.

Port the old metrics dashboard to the tview library and assign it to be a screen in the new dashboard, accessible by F2 key.

Add a new resource, infos.cluster.talos.dev which contains the cluster name and id of a node.

Disable the network config editor screen in the new dashboard until it is fully implemented with its backend.

Closes siderolabs/talos#4790.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-03-15 13:13:28 +01:00
Andrey Smirnov
1df841bb54
refactor: change the interface of META
Use a global instance, handle loading/saving META in global context.

Deprecate legacy syslinux ADV, provide an easier interface for
consumers.

Expose META as resources.

Fix the bootloader revert process (it was completely broken for quite a
while :sad:).

This is a first step which mostly does preparation work, real changes
will come in the next PRs:

* add APIs to write to META
* consume META keys for platform network config for `metal`
* custom key for URL `${code}`

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-15 15:43:16 +04:00
Steffen Windoffer
c8f8579f2d
fix: upgrade-k8s to flag should not be required since there is a default
Having a default and still requiring it confuses the user.

Signed-off-by: Steffen Windoffer <steffen@wind0r.de>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-09 17:33:21 +04:00
Nico Berlee
97048f7c37
feat: netstat in API and client
Implements netstat in Talos API and client (talosctl).

Signed-off-by: Nico Berlee <nico.berlee@on2it.net>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-09 15:48:30 +04:00
Seán C McCord
b97e1abaa6
feat: set default image, validate empty image
Adds a default image URL and ensures that an empty image URL is not
sent when calling `talosctl upgrade`.

Fixes #6912

Signed-off-by: Seán C McCord <ulexus@gmail.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-07 18:21:54 +04:00
Dmitriy Matrenichev
22ef81c1e7
feat: add grub option to drop to maintenance mode
- [x] Support `talos.experimental.wipe=system:EPHEMERAL,STATE` boot kernel arg
- [x] GRUB option to wipe like above
- [x] update GRUB library to handle that

Closes #6842

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-03-07 12:37:59 +03:00
Andrey Smirnov
8ea4bfad8f
refactor: improve the kubernetes upgrade flow
Use new version of go-kubernetes, and move the `kube-proxy` DaemonSet
update to follow common logic of bootstrap manifests update.

This fixes a confusing behavior when after `k8s-upgrade` the version of
`kube-proxy` is not updated in the machine config.

See https://github.com/siderolabs/go-kubernetes/pull/3

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-06 15:01:29 +04:00
Noel Georgi
a78281214d
feat: add cilium e2e tests
Add cilium e2e tests. The existing cilium check was very old, update to
latest cilium version and also add a test for KPR strict mode.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-03-03 20:03:25 +05:30
Andrey Smirnov
337aaba7a7
feat: add 'os:operator' role
This introduces a new role for Talos API which fills the gap between
`os:reader` and `os:admin` roles.

Fixes #6898

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-03-01 16:12:25 +04:00
Artem Chernyshev
b520710810
feat: introduce new flag in reset API that makes Talos reset user disks
Fixes: https://github.com/siderolabs/talos/issues/6815

Additionally, make it possible to run reset in maintenance mode: to
enable a way for resetting system disk and remove all traces of Talos
from it.

The new reset flow works in a separate sequence, changed disk probe
lookup to check the boot partition instead of the ephemeral one.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2023-02-28 15:10:41 +03:00
Utku Ozdemir
f55f5df739
feat: move dashboard package & run it in tty2
Move dashboard package into a common location where both Talos and talosctl can use it.

Add support for overriding stdin, stdout, stderr and ctt in process runner.

Create a dashboard service which runs the dashboard on /dev/tty2.

Redirect kernel messages to tty1 and switch to tty2 after starting the dashboard on it.

Related to siderolabs/talos#6841, siderolabs/talos#4791.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-02-28 12:00:25 +01:00
Andrey Smirnov
230e46e567
refactor: extract parts of kubernetes libraries
The shared code is going out to the
github.com/siderolabs/go-kubernetes library.

The code will be used in Talos and other projects using same features.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-02-22 14:56:49 +04:00
Dmitriy Matrenichev
8711eea962
fix: use passed --context in talosctl config cmd
Use context from command line flags. Also some minor fixes.

Closes #6846

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-02-21 15:00:04 +03:00
Dmitriy Matrenichev
3d55bd80f4
fix: add --force flag to talosctl gen config
Only overwrite existing files if explicitly demanded.

Closes #6847

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-02-20 23:44:00 +03:00
Andrey Smirnov
c449cb736b
fix: talosctl reboot command passing mode in wait mode
The reboot mode was not passed correctly in wait mode.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-02-13 16:20:07 +04:00
Steffen Windoffer
0c6c888745
fix: trackable action flag usage text. --no-wait does not exist
--wait gets set to true

Signed-off-by: Steffen Windoffer <steffen@wind0r.de>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-02-06 15:26:38 +04:00
Noel Georgi
d4b8b35de7
feat: generate kernel module dependency tree
Run `depmod` during install/upgrades when extensions provide kernel
modules and `modules.dep` needs to be re-generated. This also allows
modules of same name from kernel to co-exist. Modules in `extras`
folder takes precedence over `in-built` ones.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-01-19 18:54:10 +05:30
Andrey Smirnov
b0775ebf2c
feat: add ISO wipe GRUB boot option
This simply uses existing `talos.experimental.wipe` kernel flag to wipe
the currently installed Talos.

This allows to use Talos ISO to reset the system disk and revert back to
maintenance mode.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-01-10 16:41:45 +04:00
Andrey Smirnov
96629d5ba6
feat: implement etcd maintenance commands
This allows to safely recover out of space quota issues, and perform
degragmentation as needed.

`talosctl etcd status` command provides lots of information about the
cluster health.

See docs for more details.

Fixes #4889

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-01-03 23:25:28 +04:00
Murtaza Udaipurwala
ba8265bc5c
feat: new talosctl config remove to remove context
Adds a new sub-command to talosctl config. It takes in the context to be
deleted as argument and supports glob matching.

A local flag --noconfirm|-y can be passed to bypass the confirmation
prompt.

It also supports dry run by passing the --dry-run flag similar to
apply-config and edit commands.

Example:

    $ talosctl config remove 'ctx-*'
    Remove context ctx-a? (y/N): y
    Remove context ctx-b? (y/N): y

Signed-off-by: Murtaza Udaipurwala <murtaza@murtazau.xyz>
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2022-12-29 13:39:12 +03:00
Andrey Smirnov
89dbb0ecf0
release(v1.4.0-alpha.0): prepare release
This is the official v1.4.0-alpha.0 release.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-12-23 22:32:09 +04:00
Andrey Smirnov
a0c0352ddc
fix: send diagnostic output to stderr consistently
Fixes #6676

There was a mix of stdout/stderr, move more consistently to stderr.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-12-23 18:41:56 +04:00
Andrey Smirnov
ff83d9fd7b
fix: improve talosctl completion
This got broken with a sequence of changes related to not unmarshalling
resources when using `get`.

Rewrite the completion to be more straightforward and use easier
methods.

Make shell completion work even when `nodes` are not set.

Fixes #6646

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-12-19 18:55:24 +04:00
Andrey Smirnov
a9643b4774
fix: use proper key usage for apid client certificate
This command is referenced from PKI recovery documentation.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-12-19 14:33:42 +04:00
Andrey Smirnov
f0dddca2a3
docs: expand help for 'talosctl get'
Make it more obvious how to get list of all resources.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-12-05 17:42:28 +04:00
Utku Ozdemir
7ab140a94a
feat: add talosctl machineconfig patch command
Add talosctl machineconfig patch command which accepts a machine config as input and a list of patches, applying the patches and writing the result to a file or to stdout.

Link `talosctl machineconfig gen` to `talosctl gen config`, so they work the same way.

Closes siderolabs/talos#6562.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2022-12-02 15:42:48 +01:00
Utku Ozdemir
44e2799b8c
feat: add stdout and single config type support to talosctl gen config
Add support to specify the types of outputs to be generated by talosctl gen config.

Add support for writing a single type of output to stdout instead of a file.

Related to siderolabs/talos#6562.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2022-12-01 16:55:22 +01:00
Andrey Smirnov
2ebe410e93
feat: update COSI to v0.2.0
This brings many fixes, including a new Watch with support for
Bootstapped and Errored event types.

`talosctl` from before this change is still compatible, as there's gRPC
API level backwards compatibility versioning.

New client doesn't yet depend on new event types, so it will work
against Talos 1.2.x.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-11-29 21:21:59 +04:00