775 Commits

Author SHA1 Message Date
Andrey Smirnov
30bbf6463a
refactor: use siderolabs/net version with netip.Addr
Replace most of `net.IP` usage in Talos with `netip.Addr`, refactor code
accordingly.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-11-02 14:21:03 +04:00
Andrey Smirnov
343c55762e
chore: replace talos-systems Go modules with siderolabs
This the first step towards replacing all import paths to be based on
`siderolabs/` instead of `talos-systems/`.

All updates contain no functional changes, just refactorings to adapt to
the new path structure.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-11-01 12:55:40 +04:00
ankitm123
0301bbe936
fix: check if processes is nil to avoid panic
- A check for nodeData.Processes to check if there are nil before accessing nodeData.Processes.Processes
- Unit tests to catch the panic
- make fmt to format proto files

Signed-off-by: ankitm123 <ankitmohapatra123@gmail.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-11-01 12:07:41 +04:00
Serge Logvinov
29f2195e13
feat: support exoscale cloud
Add Exoscale cloud-init support.

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-10-28 17:52:55 +04:00
Jon Stelly
8b4ae08d1c
fix: etcd snapshot command on Windows
Close file before renaming

Signed-off-by: Jon Stelly <967068+jonstelly@users.noreply.github.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-10-28 15:17:05 +04:00
Andrey Smirnov
d7edd0e2e6
refactor: use go-circular, go-kubeconfig, and go-tail
Remove Talos versions, use new extracted Go modules.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-10-25 20:20:44 +04:00
Tim Jones
e6fba7d3bc
chore: update dependencies
Updates:
* pkgs v1.3.0-alpha.0-33-g8fe5cbc
* tools v1.3.0-alpha.0-20-g3b5f89a
* aws-sdk-go v1.44.120
* docker v20.10.20+incompatible
* fsnotify v1.6.0
* nftables v0.0.0-20221015190445-4f5cd5826fbd
* gen v0.4.0
* grpc-proxy v0.4.0
* spf13/cobra v1.6.0
* u-root v0.10.0
* x/net v0.1.0
* x/sync v0.1.0
* x/sys v0.1.0
* x/term v0.1.0
* x/time v0.1.0
* grpc v1.50.1
* genproto v0.0.0-20221018160656-63c7b68cfc55
* Linux kernel 5.15.74

Signed-off-by: Tim Jones <tim.jones@siderolabs.com>
2022-10-21 15:20:01 +04:00
Dmitriy Matrenichev
93e55b85f2
chore: bump golangci-lint to v1.50.0
I had to do several things:
- contextcheck now supports Go 1.18 generics, but I had to disable it because of this https://github.com/kkHAIKE/contextcheck/issues/9
- dupword produces to many false positives, so it's also disabled
- revive found all packages which didn't have a documentation comment before. And tehre is A LOT of them. I updated some of them, but gave up at some point and just added them to exclude rules for now.
- change lint-vulncheck to use `base` stage as base

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2022-10-20 18:33:19 +03:00
Andrey Smirnov
d210338e33
fix: skip protobuf full unmarshaling for some talosctl commands
The following commands:

* `talosctl get`
* `talosctl edit mc`, `talosctl patch mc`

Now these commands don't fully unmarshal resources from protobuf
representation, but rather use YAML representation.

This allows `talosctl` version to be out of sync with Talos version.

Still other commands do full unmarshaling (e.g. `talosctl upgrade-k8s`),
so `talosctl` should match Talos to avoid issues.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-10-11 13:32:25 +04:00
Andrey Smirnov
06f76bfebb
chore: bump dependencies
Update to some dependencies moved to siderolabs/ path.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-10-04 14:47:27 +04:00
Andrey Smirnov
139c62d762
feat: allow upgrades in maintenance mode (only over SideroLink)
This implements a simple way to upgrade Talos node running in
maintenance mode (only if Talos is installed, i.e. if `STATE` and
`EPHEMERAL` partitions are wiped).

Upgrade is only available over SideroLink for security reasons.

Upgrade in maintenance mode doesn't support any options, and it works
without machine configuration, so proxy environment variables are not
available, registry mirrors can't be used, and extensions are not
installed.

Fixes #6224

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-09-30 21:16:15 +04:00
Philipp Sauter
f17cdee167
feat: jsonpath filter for talosctl get outputs
We add a filter to the `talosctl get` command that allows users to
specify a jsonpath filter. Now they can reduce the information that is
printed to only the parts they are interested in.

Fixes #6109

Signed-off-by: Philipp Sauter <philipp.sauter@siderolabs.com>
2022-09-27 20:47:11 +02:00
Serge Logvinov
c90e20251d
fix: kubeconfig permission
Set kubeconfig permission to `-rw-------`

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-09-23 15:00:43 +04:00
Dmitriy Matrenichev
fc48849d00
chore: move maps/slices/ordered to gen module
Use github.com/siderolabs/gen

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2022-09-21 20:22:43 +03:00
Noel Georgi
357b770cb5
fix: cryptsetup delete slot
Fix cryptsetup delete slot.

Fixes: #6298

Signed-off-by: Noel Georgi <git@frezbo.dev>
2022-09-21 16:37:54 +05:30
Andrey Smirnov
9df8f1ff1a
fix: list COSI APIs for the apid authenticator
As APIs were not listed explicitly, access with `os:reader` was denied
by default, while it should have been checked down in the access filter.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-09-08 21:05:36 +04:00
Andrey Smirnov
f62d17125b
chore: update crypto to use new import path siderolabs/crypto
No functional changes in this PR, just updating import paths.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-09-07 23:02:50 +04:00
Andrey Smirnov
36c1f1d6e6
fix: flip the client-server version check
It should have been the opposite: it's a problem if the server version
is _older_ than the client verion.

E.g. using talosctl 1.2.0 against Talos 1.1.2 is a problem, not vice
versa.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-09-02 19:14:44 +04:00
Utku Ozdemir
0b339a9dc5
feat: track progress of action API calls
Track the progress of the long-running actions `reboot`, `reset`, `upgrade` and `shutdown` on the client side by default, unless `--no-wait=true` is specified.

Use the events API to follow the events using the actor ID of the action and display it using an stderr reporter with a spinner.

Closes siderolabs/talos#5499.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2022-08-29 22:54:40 +02:00
Andrey Smirnov
d7ef346db8
fix: get command in the case 'nodes' are not set in the context
For maintenance mode (`talosctl get --insecure`), there's no 'nodes'
set, so we run the loop for a single "current" node client is connected
to.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-08-29 18:48:59 +04:00
Andrey Smirnov
cdd0f08bc5
feat: check client <> server version in some Talos commands
Talos commands which are sensitive to resource API changes:

* `get`
* `edit`, `patch`
* `upgrade-k8s`

Commands with upcoming changes for actorID:

* `reboot`
* `reset`
* `shutdown`
* `upgrade`

Fixes #6101

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-08-26 18:37:51 +04:00
Dmitriy Matrenichev
29bd632401
chore: remove old build tags syntax
This commit removes lines contains old build tag syntax.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2022-08-24 17:27:01 +03:00
Andrey Smirnov
2f2d97b6b5
fix: don't wait for the hostname in maintenance mode
Fixes #6119

With new stable default hostname feature, any default hostname is
disabled until the machine config is available.

Talos enters maintenance mode when the default config source is empty,
so it doesn't have any machine config available at the moment
maintenance service is started.

Hostname might be set via different sources, e.g. kernel args or via
DHCP before the machine config is available, but if all these sources
are not available, hostname won't be set at all.

This stops waiting for the hostname, and skips setting any DNS names in
the maintenance mode certificate SANs if the hostname is not available.

Also adds a regression test via new `--disable-dhcp-hostname` flag to
`talosctl cluster create`.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-08-23 17:52:20 +04:00
Philipp Sauter
7d43fc79b1
fix: make 'ca', 'crt' and 'key' flags optional for 'talosctl config add'
As the 'ca', 'crt' and 'key' parameters are now optional for the talos
client, requiring them for the 'talosctl config add' command no longer
makes sense.

Signed-off-by: Philipp Sauter <philipp.sauter@siderolabs.com>
2022-08-17 16:51:10 +02:00
Noel Georgi
61e3eb2eaa
fix: talosctl edit mc loop
Fixes re-opening editor forever when using `talosctl edit mc`.
Also fixes the temp dir getting filled up with temporary files created
for editing machine config.

Fixes: #6098

Signed-off-by: Noel Georgi <git@frezbo.dev>
2022-08-16 05:37:19 +05:30
Philipp Sauter
f37da96ef3
feat: enable talos client to connect to Talos through an auth proxy
Talos client can connect to Talos API via a proxy with basic auth.
Additionally it is now optional to specify a TLS CA,key or crt. Optionally
Developers can build talosctl with WITH_DEBUG=1 to allow insecure
connections when http:// endpoints are specified.

Fixes #5980

Signed-off-by: Philipp Sauter <philipp.sauter@siderolabs.com>
2022-08-15 18:05:26 +02:00
Utku Ozdemir
586e29dfca
feat: add event actor id to client api and events cmd
Add the missing actor id on the event and a way to filter by it to the events cli command.

Related to siderolabs/talos#5499.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2022-08-12 22:01:34 +02:00
Andrey Smirnov
9baca49662
refactor: implement COSI resource API for Talos
Overview: deprecate existing Talos resource API, and introduce new COSI
API.

Consequences:

* COSI API can only go via one-2-one proxy (`client.WithNode`)
* client-side API access is way easier with `state.State` wrappers
* lots of small changes on the client side to use new APIs

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-08-12 22:31:54 +04:00
Artem Chernyshev
5c6648e3d2
fix: make talosctl command return nonzero error codes if it had errors
Multinode requests were printing out the errors for each node to stderr,
but they didn't set the global error.

Refactor the code a bit to use a single function for handling that logic
to avoid rewriting it in many other places.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2022-08-12 14:19:45 +03:00
Artem Chernyshev
13499fc302
feat: support patching the machine config in the apply-config cmd
Fixes: https://github.com/siderolabs/talos/issues/6045

`talosctl apply-config` now supports `--config-patch` flag that takes
machine config patches as the input.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2022-08-11 13:56:23 +03:00
Noel Georgi
b62b18a972
feat: bump k8s to v1.25.0-beta.0
Bump k8s to v1.25.0-beta.0

Update most kubernetes `master` references to `controlplane`

Signed-off-by: Noel Georgi <git@frezbo.dev>
2022-08-10 22:17:53 +05:30
Utku Ozdemir
84e712a9f1
feat: introduce Talos API access from Kubernetes
We add a new CRD, `serviceaccounts.talos.dev` (with `tsa` as short name), and its controller which allows users to get a `Secret` containing a short-lived Talosconfig in their namespaces with the roles they need. Additionally, we introduce the `talosctl inject serviceaccount` command to accept a YAML file with Kubernetes manifests and inject them with Talos service accounts so that they can be directly applied to Kubernetes afterwards. If Talos API access feature is enabled on Talos side, the injected workloads will be able to talk to Talos API.

Closes siderolabs/talos#4422.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2022-08-08 18:27:26 +02:00
Andrey Smirnov
f9b664c947
fix: reload trusted CA list when client is recreated
Fixes #5652

This reworks and unifies HTTP client/transport management in Talos:

* cleanhttp is used everywhere consistently
* DefaultClient is using pooled client, other clients use regular
  transport
* like before, Proxy vars are inspected on each request (but now
  consistently)
* manifest download functions now recreate the client on each run to
  pick up latest changes
* system CA list is picked up from a fixed locations, and supports
  reloading on changes

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-08-04 20:01:35 +04:00
Andrey Smirnov
a6b010a8b4
chore: update Go to 1.19, Linux to 5.15.58
See https://go.dev/doc/go1.19

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-08-03 17:03:58 +04:00
Eng Zer Jun
fb058a7c92
test: use T.TempDir to create temporary test directory
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-08-03 16:31:55 +04:00
Andrey Smirnov
fe2ee3b100
feat: implement MachineStatus resource
Fixes #5789

Example:

```yaml
spec:
    stage: running
    status:
        ready: false
        unmetConditions:
            - name: staticPods
              reason: kube-system/kube-controller-manager-talos-default-master-1 not ready, kube-system/kube-scheduler-talos-default-master-1 not ready
```

As events (CLI doesn't show full contents):

```
172.20.0.2   cbhf2l6f9lrs738hehfg   talos/runtime/machine.MachineStatusEvent   BOOTING   ready: false, unmet conditions: [time network services]
```

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-08-01 18:36:10 +04:00
Dmitriy Matrenichev
30f7851d2a
chore: bump golangci-lint from 1.45.2 to 1.47.2
Minor linter upgrade.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2022-07-22 17:49:44 +03:00
Noel Georgi
d650afb6cd
chore: fix typo in powercycle
Fix typo in `powercycle`

Signed-off-by: Noel Georgi <git@frezbo.dev>
2022-07-20 21:33:17 +05:30
Andrey Smirnov
80444a43d9
fix: remove data race in pcap capture
Capture handle should be closed in the same goroutine with packet
reading.

Fix a spurious error which might appear in `talosctl pcap`.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-07-20 01:10:59 +04:00
Andrey Smirnov
065b59276c
feat: implement packet capture API
This uses the `go-packet` library with native bindings for the packet
capture (without `libpcap`). This is not the most performant way, but it
allows us to avoid CGo.

There is a problem with converting network filter expressions (like
`tcp port 3222`) into BPF instructions, it's only available in C
libraries, but there's a workaround with `tcpdump`.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-07-19 01:23:09 +04:00
Utku Ozdemir
a75fe7600d
feat: gen secrets from kubernetes pki dir
This PR allows the ability to generate `secrets.yaml` (`talosctl gen secrets`) using a Kubernetes PKI directory path (e.g. `/etc/kubernetes/pki`) as input. Also introduces the flag `--kubernetes-bootstrap-token` to be able to set a static Kubernetes bootstrap token to the generated `secrets.yaml` file instead of a randomly-generated one. Closes siderolabs/talos#5894.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2022-07-16 13:06:32 +02:00
Andrey Smirnov
641f6a1e4e
feat: expose strategic merge config patches
The end result is that every Talos CLI accepts both JSON and strategic
patches to patch machine configuration.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-07-12 15:38:01 +04:00
Utku Ozdemir
d924901b79
feat: add cli subcommand to generate secrets
Adds a new command `talosctl gen secrets` to generate a `secrets.yaml` file with Talos and Kubenetes secrets. This file can later be used like `talosctl gen config ... --with-secrets secrets` to generate a config with these pre-generated secrets. Closes siderolabs/talos#5861.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2022-07-06 20:00:35 +02:00
Dennis Marttinen
c2a5126088
fix: avoid double append of talos.platform kernel argument
The example configuration generated by talosctl contains

```yaml
extraKernelArgs:
  - talos.platform=metal
```

in the install section, which, if uncommented, causes the installer to append the
`talos.platform` option twice. Thus, if the platform is set/changed here, it will
not be respected. This change allows the existing value to be overridden.

Signed-off-by: Dennis Marttinen <twelho@welho.tech>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-07-05 16:18:38 +04:00
Utku Ozdemir
ae3840dbc3
refactor: move kubeconfig package under public api
Move the kubeconfig package under pkg/ so that other projects can reuse parts of it.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2022-07-01 19:22:16 +02:00
RyanSquared
63caa281ae
fix: create native image format for DigitalOcean
This change replaces the gzipped tar with a gzipped image, which
DigitalOcean natively supports uploading, and means that it can be
linked directly to the image when using, for example, Terraform.

Signed-off-by: Ryan Heywood <me@ryansquared.pub>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-06-23 16:24:00 +04:00
Utku Ozdemir
6759fcd4ae
feat: use discovery service on cluster health checks
Query the discovery service to fetch the node list and use the results in health checks. Closes siderolabs#5554.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2022-06-15 16:01:38 +02:00
Utku Ozdemir
8d2be5e315
feat: extend node definition used in health checks
Introduce `cluster.NodeInfo` to represent the basic info about a node which can be used in the health checks. This information, where possible, will be populated by the discovery service in following PRs. Part of siderolabs#5554.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2022-06-13 14:13:42 +02:00
Philipp Sauter
7a11b4def7
fix: make talosctl bootstrap accept only single node
Previously talosctl would accept multiple nodes for the bootstrap
command which is a strictly single-node operation. Talosctl will abort
the bootstrap command if more than one node is specified either as a
command-line flag or in talosconfig.

Fixes #5636

Signed-off-by: Philipp Sauter <philipp.sauter@siderolabs.com>
2022-06-10 22:27:15 +02:00
Andrey Smirnov
da2985fe1b
fix: respect local API server port
It wasn't used when building an endpoint to the local API server, so
Talos couldn't talk to the local API server when port was changed from
the default one.

Fixes #5706

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-06-09 00:33:49 +04:00