2451 Commits

Author SHA1 Message Date
Artem Chernyshev
14e696d068 feat: update COSI runtime and add support for tail in the Talos gRPC
Updated protobufs to expose tail length option.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-06-03 11:46:39 -07:00
Andrew Rynhard
a71053fcd8 feat: default to bootstrap workflow
Changes `gen config` to output `controlplane` and `join` machine config
types only. Users can manually set the `type` to `init` if they need to.

Signed-off-by: Andrew Rynhard <andrew@rynhard.io>
2021-06-03 11:29:56 -07:00
Artem Chernyshev
76aac4bb25 feat: implement CPU and Memory stats controller
Now the latest value for CPU and Memory is also represented as COSI
resources.

Was going back and forth in the implementation but in the end decided to
use dedicated yaml structures for both CPU and Memory stats because:

- JSON tags are ignored by `go-yaml`, so the output is not really great.
- protobuf Talos definition contains fields which we don't really need
in the YAML output of `talosctl get`.
- current state of Talos resource service does not support protobuf
encoding for resources.

So the plan for Theila is to just use the structure as a dynamic object
without relying on protobufs. At least for now.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-06-03 10:19:27 -07:00
Andrey Smirnov
8f90c6a8e1 feat: parse Talos-specific cmdline params
This parses Talos cmdline args in addition to standard `ip=` cmdline
params.

GC'ed unused constant.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-06-02 14:04:12 -07:00
Andrey Smirnov
ed10e139c1 feat: implement NodeAddress controller
This controller provides three important aggregated resources to be
consumed by different interested parties:

* "default" node IP
* "current" addresses (node can be reached on these at the moment)
* "accumulative" addresses (for certSANs)

Example:

```
$ talosctl get nodeaddresses -n 172.20.0.2
NODE         NAMESPACE   TYPE          ID             VERSION   ADDRESSES
172.20.0.2   network     NodeAddress   accumulative   4         ["10.244.0.0","10.244.0.1","172.20.0.2"]
172.20.0.2   network     NodeAddress   current        6         ["10.244.0.0","10.244.0.1","172.20.0.2"]
172.20.0.2   network     NodeAddress   default        1         ["172.20.0.2"]
```

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-06-02 14:03:59 -07:00
Andrey Smirnov
33db8857aa fix: use COSI runtime DestroyReady input type
See https://github.com/cosi-project/runtime/pull/35

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-06-01 12:30:52 -07:00
Andrey Smirnov
6e77536392 refactor: rename *.Status() to *.TypedSpec() in the resources
It was a bad choice to name the method which returns resource spec
typecasted to the proper Go type as `Status()`, as certainly `Status`
has a different meaning for resources. So rename the method accordingly
to be `TypedSpec()`.

No functional or user-visbile changes, just renames.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-06-01 12:30:13 -07:00
Serge Logvinov
97627061d7 docs: set static IP on ISO install mode
Set static ip when the network dhcp server are not exist

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
2021-06-01 10:11:43 -07:00
Andrey Smirnov
5811f4dda1 feat: implement link (interface) controllers
The structure of the controllers is really similar to addresses and
routes:

* `LinkSpec` resource describes desired link state
* `LinkConfig` controller generates `LinkSpecs` based on machine
configuration and kernel cmdline
* `LinkMerge` controller merges multiple configuration sources into a
single `LinkSpec` paying attention to the config layer priority
* `LinkSpec` controller applies the specs to the kernel state

Controller `LinkStatus` (which was implemented before) watches the
kernel state and publishes current link status.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-06-01 09:36:25 -07:00
Andrey Smirnov
046b229b13 chore: skip building multi-arch installer for race-enabled build
Go cross-compilation is not quite compatible with Go race detector, as C
toolchain is not cross-compilation ready.

Workaround is really simple: for race-enabled builds, don't build
multi-arch installer image (that is installer image which contains both
amd64 and arm64 Talos artifacts), but build installer artifacts only for
the target arch (skipping cross-compilation).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-31 11:04:55 -07:00
Kevin Hellemun
73fbb4b523 fix: only fetch machine uuid if it's not set
During boot sequence, if `talos.config`'s url has the uuid parameter, the uuid
value is retrieved via SMBIOS. However, at this part of the code it can happen
that the uuid is already set and valid. If this is the case, instead of
re-fetching the uuid, the one that is already set can be used.

closes #3676

Signed-off-by: Kevin Hellemun <17928966+OGKevin@users.noreply.github.com>
2021-05-31 10:19:07 -07:00
Andrey Smirnov
f112a540b0 fix: clean up stale snapshots on container start
I wasn't able to reproduce this case, but looks like containerd can get
into a state when the snapshot was left behind, but container record is
missing. This prevents the container from being started with the error:

```
failed to create container kubelet: snapshot kubelet: already exists
```

This PR should help to fix this issue by trying to remove the snapshot
by name even if the container record is not found.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-31 08:46:38 -07:00
Alexey Palazhchenko
c036b94948 chore: bump dependencies
Closes #3699, #3668, #3698, #3697, #3696, #3695, #3694, #3693, #3692.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-05-31 06:12:06 -07:00
Artem Chernyshev
a4d67a0182 feat: add the ability to disable CoreDNS
Fixes: https://github.com/talos-systems/talos/issues/3556

Introduce `coredns.disabled` flag to the machine config that allows
turning off coredns deployment during cluster bootstrap.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-05-28 10:33:14 -07:00
Artem Chernyshev
76dbfb3699 feat: add ability to mark MBR partition bootable
Fixes: https://github.com/talos-systems/talos/issues/3532

Machine install section now has `markMBRBootable` option.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-05-27 12:44:50 -07:00
Alexey Palazhchenko
e0f5b1e20a chore: split mgmt/gen.go into several files
No functional changes in this PR, to make future PRs easier.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-05-26 12:54:48 -07:00
Andrey Smirnov
fad1b4f1fd chore: fix go generate for the machinery
The problem is that `pkg/machinery` is a submodule, so regular go walk
over the pattern `./...` doesn't include submodules, so we have to call
explicitly for the submodule.

This also enables go generate for all `./pkg`.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-26 10:19:28 -07:00
Andrey Smirnov
1117294ad2 release(v0.11.0-alpha.0): prepare release
This is the official v0.11.0-alpha.0 release.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-26 10:19:00 -07:00
Andrey Smirnov
c096294663 chore: prepare for 0.11 release series
Remove `parsebool.sh` script as it's causing issues when the
release-tool is running `Makefile` commands to extract variable values.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-26 08:49:22 -07:00
Artem Chernyshev
723597657a feat: enable GORACE=halt_on_panic=1 in machined binary
Fixes: https://github.com/talos-systems/talos/issues/3533

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-05-25 11:24:07 -07:00
Andrey Smirnov
0acb04ad7a feat: implement route network controllers
Route handling is very similar to addresses:

* `RouteStatus` describes kernel routing table state,
`RouteStatusController` reflects kernel state into resources
* `RouteSpec` defines routes to be configured
* `RouteConfigController` creates `RouteSpec`s based on cmdline and
machine configuration
* `RouteMergeController` merges different configuration layers into the
final representation
* `RouteSpecController` applies the specs to the kernel routing table

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-25 11:09:21 -07:00
Alexey Palazhchenko
f5bf88a4c2 feat: create certificates with os:admin role
Minimal change for backporting into 0.10.

Refs #3421.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-05-25 09:38:23 -07:00
Artem Chernyshev
1db301edf6 feat: switch controller-runtime to zap.Logger
Enable logging using default development config with some fine tuning.
Additionally, now `info` and below logs go to kmsg.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-05-25 02:15:31 -07:00
Joost Coelingh
f7cf64d42e fix: add talos.config to the vApp Properties in VMware OVA
VMware vSphere doens't allow talos.config to be set when deploying the OVA
due to missing vApp properties. Added talos.config to the ovf template to include
talos.config to fix this.

Fixes talos-systems#3669

Signed-off-by: Joost Coelingh <joost.coelingh@eu.equinix.com>
2021-05-24 14:31:53 -07:00
Andrey Smirnov
209527eccc docs: add AMIs for Talos 0.10.3
Late, but still good to add.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-24 12:23:34 -07:00
Andrey Smirnov
59cfd312c1 chore: bump dependencies via dependabot
There were some upstream code changes in etcd, some code got moved
around.

PRs #3651 #3652 #3653 #3654 #3655 #3655 #3656 #3657 #3658
    #3659 #3660 #3661 #3662 #3663

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-24 12:15:15 -07:00
Boran Car
1edb20cf98 feat: extract config generation
Extract config generation into a separate function. This reduces
cyclomatic complexity with the added benefit of being able to use
generation in another tool (e.g. Terraform provider to generate config).

Signed-off-by: Boran Car <boran.car@gmail.com>
2021-05-24 09:41:36 -07:00
Kevin Hellemun
af77c29565 docs: update wirguard guide
The wireguard guide has a small typo in the structure example of the config.

Signed-off-by: Kevin Hellemun <17928966+OGKevin@users.noreply.github.com>
2021-05-21 05:08:26 -07:00
Alexey Palazhchenko
4fe6912143 test: better talosctl ls tests
Refs #3018.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-05-20 03:29:21 -07:00
Andrey Smirnov
04ddda962f feat: update containerd to 1.5.2, runc to 1.0.0-rc95
This also updates libseccomp and add support for `netxen` networkd card.

This addresses[CVE-2021-30465](https://github.com/opencontainers/runc/security/advisories/GHSA-c3xm-pvg7-gh7r).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-19 15:24:28 -07:00
Alexey Palazhchenko
49c7276b16 chore: fix markdown linting
* without `-print0` / `-0`, paths with spaces were skipped sometimes;
* `--stdin-filename` flag incorrectly consumed the first file.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-05-19 06:08:14 -07:00
Spencer Smith
7270495ace docs: add mayastor quickstart
This PR adds some documentation on how to use mayastor with talos and
links the video showing it in action.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2021-05-18 15:15:22 -07:00
Andrey Smirnov
d3d9112f28 docs: fix spelling/grammar in What's New for Talos 0.9
From PR #3269

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-18 13:38:41 -07:00
Andrey Smirnov
82804414fc test: provide a way to force different boot order in provision library
There's no change to the default behavior. This change is going to be
used with Sidero/Sfyra.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-18 13:38:22 -07:00
Andrew LeCody
a1c0e99a17 docs: add guide for deploying metrics-server
Resolves talos-systems#3166

Signed-off-by: Andrew LeCody <andrew@beyondweb.net>
2021-05-18 09:05:51 -07:00
Andrey Smirnov
6bc6658b51 feat: update containerd to 1.5.1
See https://github.com/containerd/containerd/releases/tag/v1.5.1

Also brings Talos kernel with Geneve encapsulation for Openvswitch (see
talos-systems/pkgs#278).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-17 10:33:49 -07:00
Andrey Smirnov
c6567fae9c chore: dependabot updates
PRs #3622 #3623 #3624 #3625 #3627 #3628

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-17 07:46:24 -07:00
Alexey Palazhchenko
61ccbb3f5a chore: keep debug symbols in debug builds
Makes debugging easier.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-05-17 04:58:46 -07:00
Serge Logvinov
1ce362e05e docs: update customizing kernel build steps
How to customizing the Talos kernel documentation update.

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
2021-05-14 14:05:56 -07:00
Artem Chernyshev
a26174b548 fix: properly compose pattern and header in etcd members output
Header was printed more than once.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-05-14 13:17:46 -07:00
Andrey Smirnov
0825cf11f4 fix: stop networkd and pods before leaving etcd on upgrade
The change is essentially same as #3590, but applied to the upgrade path
which is very similar to the reset path.

We have to stop networkd (and remove the VIP/lease on the VIP) before we
leave and stop etcd. Plus we stop the kube-apiserver before the etcd is
stopped, so that we don't have unhealthy kube-apiserver.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-14 12:03:29 -07:00
Artem Chernyshev
bed6b15d6f fix: properly populate AllowSchedulingOnMasters option in gen config RPC
Previosly it was set only if NetworkConfig is not nil.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-05-14 07:04:49 -07:00
Andrey Smirnov
071f044562 feat: implement AddressSpec handling
This includes multiple controllers responsible for different stages of
`AddressSpec` conversion:

* `AddressConfigController` produces initial unmerged configuration from
multiple sources (more sources coming later, e.g. DHCP)
* `AddressMergeController` merges address configuration into final
representation
* `AddressSpecController` syncs resources with kernel state

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-14 02:27:12 -07:00
Andrey Smirnov
76e38b7b82 feat: update Kubernetes to 1.21.1
See https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.21.md

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-13 08:05:08 -07:00
Alexey Palazhchenko
9b1338d989 chore: parse "boolean" variables
Makes `make WITH_DEBUG=1` work.

Refs #3534.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-05-13 03:54:33 -07:00
Alexey Palazhchenko
c81cfb2167 chore: allow building with debug handlers
Refs #3534.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-05-13 02:20:15 -07:00
Spencer Smith
c9651673b9 feat: update go-smbios library
This pulls in a newer version of smbios so that we can detect lower
smbios version and handle endianness if necessary.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2021-05-12 10:49:59 -07:00
Andrey Smirnov
95c656fb72 feat: update containerd to 1.5.0, runc to 1.0.0-rc94
Fixes #3538

See also talos-systems/pkgs#276

As new containerd is now Go module-based, it pulls many more
dependencies if simply imported in `go.mod`, so I had to replace the
reference to the constant in `pkg/machinery/` to `containerd` volume
with simple value to avoid pulling Kubernetes dependencies into
`pkg/machinery`.

Also updates the kernel to include PR talos-systems/pkgs#275 for AES-NI
support.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-11 14:43:27 -07:00
Andrey Smirnov
db9c35b570 feat: implement AddressStatusController
This controller queries addresses of all the interfaces in the system
and presents them as resources. The idea is that can be a source for
many decisions - e.g. whether network is ready (physical interface has
scope global address assigned).

This is also good for debugging purposes.

Examples:

```
$ talosctl -n 172.20.0.2 get addresses
NODE         NAMESPACE   TYPE            ID                                          VERSION
172.20.0.2   network     AddressStatus   cni0/10.244.0.1/24                          1
172.20.0.2   network     AddressStatus   cni0/fe80::9c87:cdff:fe8e:5fdc/64           2
172.20.0.2   network     AddressStatus   eth0/172.20.0.2/24                          1
172.20.0.2   network     AddressStatus   eth0/fe80::ac1b:9cff:fe19:6b47/64           2
172.20.0.2   network     AddressStatus   flannel.1/10.244.0.0/32                     1
172.20.0.2   network     AddressStatus   flannel.1/fe80::440b:67ff:fe99:c18f/64      2
172.20.0.2   network     AddressStatus   lo/127.0.0.1/8                              1
172.20.0.2   network     AddressStatus   lo/::1/128                                  1
172.20.0.2   network     AddressStatus   veth178e9b31/fe80::6040:1dff:fe5b:ae1a/64   2
172.20.0.2   network     AddressStatus   vethb0b96a94/fe80::2473:86ff:fece:1954/64   2
```

```
$ talosctl -n 172.20.0.2 get addresses -o yaml eth0/172.20.0.2/24
node: 172.20.0.2
metadata:
    namespace: network
    type: AddressStatuses.net.talos.dev
    id: eth0/172.20.0.2/24
    version: 1
    owner: network.AddressStatusController
    phase: running
spec:
    address: 172.20.0.2/24
    local: 172.20.0.2
    broadcast: 172.20.0.255
    linkIndex: 4
    linkName: eth0
    family: inet4
    scope: global
    flags: permanent
```

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-11 13:32:17 -07:00
Andrey Smirnov
1cf011a809 chore: bump dependencies via dependabot
See PRs #3596 #3593 #3592

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-11 11:20:23 -07:00