501 Commits

Author SHA1 Message Date
Philipp Sauter
9ee470f955
feat: set /etc/localtime to UTC
Adds a timezone file for UTC to the rootfs and create a symlink for it
at /etc/localtime

Fixes #4840

Signed-off-by: Philipp Sauter <sauterp@protonmail.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-02-07 15:44:19 +03:00
Andrey Smirnov
abfb258128
feat: share /lib/firmware across initramfs and rootfs
See #4816

Depending on the hardware and firmware type, firmware might be either
needed during initial boot (`initramfs`) or in the Talos running phase
(`rootfs`). As we don't want to have two copies of same firmware, share
the firmware by bind-mounting it from the `initramfs` down to `rootfs`
on switchroot.

This also cleans up `Dockerfile` to keep firmware only in `initramfs`.

Eventually we might get rid of some of the firmware and move it to the
system extensions.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-01-27 15:41:46 +03:00
Andrey Smirnov
c5e5922e53
chore: bump dependencies
dependabot + go-mod-outdated

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-01-26 17:07:11 +03:00
Andrey Smirnov
b3c3ef29bd
feat: install system extensions
Fixes #4815

This implements the following steps:

* machine configuration updates
* pulling and unpacking system extension images
* validating, listing system extensions
* re-packing system extensions
* preserving installed extensions in `/etc/extensions.yaml`

Once extension is enabled, raw information can be queried with:

```
$ talosctl -n 172.20.0.2 cat /etc/extensions.yaml
layers:
    - image: 000.ghcr.io-smira-gvisor-c927b54-dirty.sqsh
      metadata:
        name: gvisor
        version: 20220117.0-v1.0.0
        author: Andrew Rynhard
        description: |
            This system extension provides gVisor using containerd's runtime handler.
        compatibility:
            talos:
                version: '> v0.15.0-alpha.1'
```

This was tested with the `gvisor` system extension.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-01-26 16:24:28 +03:00
Andrey Smirnov
a0889600fb
chore: fix golangci-lint install
Version was never passed properly, plus the goreleaser is deprecated
now, switch to golangci-lint equivalent script.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-01-26 16:02:31 +03:00
Andrey Smirnov
d4b8445935
feat: support CRI configuration merging and reimplement registry config
Containerd doesn't support merging plugin configuration from multiple
sources, and Talos has several pieces which configure CRI plugin:
(see https://github.com/containerd/containerd/issues/5837)

* base config
* registry mirror config
* system extensions
* ...

So we implement our own simple way of merging config parts (by simply
concatenating text files) to build a final `cri.toml`.

At the same time containerd migrated to a new format to specify registry
mirror configuration, while old way (via CRI config) is going to be
removed in 1.7.0. New way also allows to apply most of registry
configuration (except for auth) on the fly.

Also, containerd was updated to 1.6.0-rc.0 and runc to 1.1.0.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-01-20 23:05:20 +03:00
Noel Georgi
6d8bea5d55
feat: jetson nano SoC
Support Jetson Nano SoC

Depends on https://github.com/talos-systems/pkgs/pull/380

Signed-off-by: Noel Georgi <git@frezbo.dev>
2022-01-18 20:12:54 +05:30
Andrey Smirnov
43b2d81371
chore: bump dependencies
Dependabot + go-mod-outdated.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-01-17 16:25:18 +03:00
Andrey Smirnov
d02d944ec7
chore: provide umarshal from YAML methods for network resource specs
This will be required to preserve platform network configuration in a
cache across boots. Network configuration will be marshaled to YAML and
unmarshaled back.

Changes are pretty simple and most of the code is auto-generated:
replace `stringer` codegen with `enumer` which produces also methods
which convert from string back to enum values.

Added tests to verify YAML marshaling/unmarshaling.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-01-10 17:00:53 +03:00
Andrey Smirnov
2e735714d9
fix: derive machine-id from node identity
Fixes #4759

This uses existing features: Talos always generates 32 bytes random node
identity, we use first 16 bytes of that to generate `machine-id` in
compliant format and mount that into the `kubelet` container.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-01-10 15:59:07 +03:00
Andrey Smirnov
f4516c7d84
chore: bump dependencies
Some via dependabot, some via go-mod-outdated.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-12-30 19:38:03 +03:00
Andrey Smirnov
942c8074fd
docs: fork docs for 0.15
Also updated upgrade guide for 0.14 as I noticed it had some TBD.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-12-22 00:06:34 +03:00
Andrey Smirnov
36c9a65ac0
feat: update deps and Kubernetes to 1.23.0-rc.1
Update Go modules to the latest versions including talos-systems/
released modules.

In preparation for 0.14-beta release.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-12-06 18:43:26 +03:00
Andrey Smirnov
2a0da06247
feat: split installer and imager images
The `installer` image is used for initial install and upgrades, so it
contains only Talos artifacts for the same arch as the image

The `imager` image is used to produce Talos images, so it contains artifacts
for all arhitectures, so that it's possible e.g. to build Raspberry PI
Talos image on amd64 machine

Unpacked size comparison for amd64 (before this change, size of
`installer` was same as `imager` size):

```
REPOSITORY                               TAG                                      IMAGE ID       CREATED         SIZE
ghcr.io/smira/installer                  v0.14.0-alpha.2-8-g73293bc2-dirty        264995cf56df   4 minutes ago   176MB
ghcr.io/smira/imager                     v0.14.0-alpha.2-8-g73293bc2-dirty        3ba8e0176565   6 minutes ago   616MB
```

As the `installer` image is pulled into `tmpfs` on upgrade, it's important
to have it as small as possible to avoid putting too much pressure on
node memory.

Both images are generated anyways for both `amd64` and `arm64` target
architectures.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-12-03 14:37:45 +03:00
Alexey Palazhchenko
8d1cbeef9f
chore: add API breaking changes detector
Closes #4576.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-11-30 15:06:05 +00:00
Andrey Smirnov
f3149780e6
feat: update Kubernetes to 1.23.0-rc.0
Still no proper tags for Kubernetes modules (known issue upstream).

Also bump dependencies.

Closes #4605 closes #4606

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-11-29 16:22:46 +03:00
Alexey Palazhchenko
5ac64b2d97
chore: set version in unit-tests
Refs #4576.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-11-23 18:07:08 +00:00
Alexey Palazhchenko
20d39c0b48
chore: format .proto files
Refs #2722.

Co-authored-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-11-23 15:05:25 +00:00
Serge Logvinov
caa76be2c9
fix: containerd failed to load plugin
Solve warning message: failed to load plugin io.containerd.internal.v1.opt

Co-authored-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-11-19 17:55:05 +03:00
Andrey Smirnov
753a82188f
refactor: move pkg/resources to machinery
Fixes #4420

No functional changes, just moving packages around.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-11-15 19:50:35 +03:00
Alexey Palazhchenko
7462733bcb
chore: update golangci-lint
Fix context propagation.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-11-15 14:55:25 +00:00
Alexey Palazhchenko
189221d589
chore: update dependencies
Closes #4538.
Closes #4539.
Closes #4540.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-11-15 12:10:03 +00:00
Alexey Palazhchenko
3e7d4df990
chore: bump dependencies
Closes #4473.
Closes #4474.
Closes #4475.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-11-01 13:20:09 +00:00
Andrey Smirnov
c97becdd95
chore: remove interfaces and routes APIs
Fixes #4279

These APIs were deprecated in 0.13, now it's time to drop them for 0.14.

They were not used anywhere in Talos, so no changes on Talos side.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-10-27 15:34:17 +03:00
Alexey Palazhchenko
72f62ac27b
chore: bump Go and Docker dependencies
`go mod tidy -compat=1.17`.

Closes #4426.
Closes #4427.
Closes #4428.
Closes #4429.
Closes #4430.
Closes #4431.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-10-25 08:55:57 +00:00
Alexey Palazhchenko
431e4fb4b6
chore: bump Go and Docker dependencies
Closes #4387.
Closes #4388.
Closes #4389.
Closes #4390.
Closes #4391.
Closes #4392.
Closes #4393.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-10-18 13:43:36 +00:00
Andrey Smirnov
d6309eed66
docs: create docs for Talos 0.14
Docs generation switched to v0.14 folder.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-10-14 15:42:24 +03:00
Andrey Smirnov
ce65ca4e4a
chore: build using only amd64 builders
Our CI amd64 builders are 3-4 times faster to build Talos than our arm64
builders when building Talos.

Our Dockerfile was restructured a while ago to support cross-compilation
on all platforms but CI was still using amd64/arm64 workers, so arm64
part was done on arm64 builders.

As our CI runs on Talos, `binfmt_misc` is not enabled in the kernel, but
buildkit has built-in QEMU emulation layer which works just fine for
those small pieces which actually need to run arm64 binaries on amd64
(mostly `apk add` in the installer container). Interesting enough,
buildkit QEMU support fails for `ca-certificates` script which runs
after install. At the same time I believe we don't need
`ca-certificates` in the installer, as installer doesn't download
anything from the network, and `ca-certificates` were added a while ago
when installer was actually downloading configuration on its own.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-10-05 23:02:37 +03:00
Andrey Smirnov
0e18e2800f
chore: bump dependencies
Some via dependabot, some via go-mod-outdated.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-09-27 16:35:50 +03:00
Andrey Smirnov
9d803d75bf
chore: bump dependencies and drop firecracker support
Note: Talos can be still run under `Firecracker`, support for
Firecracker was only removed for `talosctl cluster create`.

Reason:

* code is untested/unmaintained, and probably doesn't work correctly
* firecracker Go SDK pulls lots of dependencies and it blocks CNI Go
module update

Bonus: `talosctl-linux-amd64` shrinks by 2 MiB.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-09-20 17:13:34 +03:00
Andrey Smirnov
1c05089bb2
feat: implement KubeSpan manager for Wireguard peer state
KubeSpan manager uses list of KubeSpan peers prepared from the discovery
and local KubeSpan identity to set up and update configuration of the
Wireguard interface.

As new peers are getting added or deleted, manager takes care of
updating the Wireguard config.

Manager also keeps track of all peers and their state coming from the
Wireguard link status: whether the connection is up or not, some stats,
last actually used endpoint, etc.

Manager cycles through the available peer endpoints until it finds the
one which works.

Manager exposes peer status as `PeerStatus` resources.

Example:

```
$ talosctl -n 172.20.0.2 get kubespanpeerstatuses
NODE         NAMESPACE   TYPE                 ID                                             VERSION   LABEL                    ENDPOINT           STATE   RX    TX
172.20.0.2   kubespan    KubeSpanPeerStatus   GpO3gs5n09WpoiVANbzRL5nwrkRi+9Q19qoeC8RTkQ4=   30        talos-default-worker-2   172.20.0.6:51820   up      640   1920
172.20.0.2   kubespan    KubeSpanPeerStatus   j4CRlKByMcTWOBS2ifZcPzcUr3lXdBOc/I4AxGmhXxI=   30        talos-default-worker-1   172.20.0.5:51820   up      672   1888
172.20.0.2   kubespan    KubeSpanPeerStatus   o5EPScFrD895A5EpVyKU8hFR+vi25D0CJMYsoaXN3Qk=   28        talos-default-master-3   172.20.0.4:51820   up      640   1920
172.20.0.2   kubespan    KubeSpanPeerStatus   rBp5wyHdxqZkq5CWher2DcPcGgwHrFOwB6fP/ReFRlE=   16        talos-default-master-2   172.20.0.3:51820   up      432   2088
```

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
Signed-off-by: Seán C McCord <ulexus@gmail.com>
Co-authored-by: Seán C McCord <ulexus@gmail.com>
2021-09-15 16:09:38 +03:00
Andrey Smirnov
a059454045
chore: build using Go 1.17
`initramfs` size for amd64 shrinks by 1.3 MiB.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-09-13 22:33:47 +03:00
Andrey Smirnov
ee2dce6c1a
chore: bump dependencies
PRs:

* #4215
* #4216
* #4217
* #4218
* #4219
* #4220
* #4221

+ go-mod-outdated

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-09-13 16:39:17 +03:00
Andrey Smirnov
3c3c281bff
chore: bump dependencies via dependabot
PRs:

* #4170
* #4171
* #4172
* #4173

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-30 21:41:46 +03:00
Olli Janatuinen
ba169c6f91
feat: provide talosctl.exe for Windows
amd64 version only

Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2021-08-26 17:06:50 +03:00
Andrey Smirnov
c9af8f7ff1
docs: fork docs for 0.13
s/0.12/0.13/ mostly

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-25 21:43:05 +03:00
Andrey Smirnov
83fdb7721f
feat: provide first NIC hardware addr as a resource
This will be used to derive e.g. KubeSpan address.

Fixes #4132

Example:

```
$ talosctl -n 172.20.0.2 get hardwareaddresses -o yaml
node: 172.20.0.2
metadata:
    namespace: network
    type: HardwareAddresses.net.talos.dev
    id: first
    version: 1
    owner: network.HardwareAddrController
    phase: running
    created: 2021-08-24T20:30:43Z
    updated: 2021-08-24T20:30:43Z
spec:
    name: eth0
    hardwareAddr: 6a:2b:bd:b2:fc:e0
```

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-25 17:11:24 +03:00
Alexey Palazhchenko
244b08cc19
chore: bump dependencies
Also downgrade some indirect dependencies that don't have a good reason to be updated.

Closes #4111.
Closes #4112.
Closes #4116.
Closes #4117.
Closes #4121.
Closes #4122.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-08-23 08:49:38 +00:00
Andrey Smirnov
8bef41e4ba
fix: make sure file mode is same (reproducibility issue)
This fixes the diff between local and CI builds, apparently different
`umask` for the file checked out on the host:

```
--- initramfs-amd64.xz
+++ initramfs-amd64.ci.xz
│   --- initramfs-amd64
├── +++ initramfs-amd64.ci
│ ├── rootfs.sqsh
│ │ ├── unsquashfs -d  -lls {}
│ │ │ @@ -9,15 +9,15 @@
│ │ │  -rwxr-xr-x root/root          11441016 2021-08-17 16:57 /bin/runc
│ │ │  drwxr-xr-x root/root                 3 2021-08-17 16:57 /boot
│ │ │  drwxr-xr-x root/root                 3 2021-08-17 16:57 /dev
│ │ │  drwxr-xr-x root/root               186 2021-08-17 16:57 /etc
│ │ │  lrwxrwxrwx root/root                 8 2021-08-17 16:57 /etc/ca-certificates -> /etc/ssl
│ │ │  drwxr-xr-x root/root                 3 2021-08-17 16:57 /etc/cni
│ │ │  drwxr-xr-x root/root                38 2021-08-17 16:57 /etc/cri
│ │ │ --rw-rw-r-- root/root               385 2021-08-17 16:57 /etc/cri/containerd.toml
│ │ │ +-rw-r--r-- root/root               385 2021-08-17 16:57 /etc/cri/containerd.toml
│ │ │  -rw-r--r-- root/root                 0 2021-08-17 16:57 /etc/hosts
│ │ │  drwxr-xr-x root/root                75 2021-08-17 16:57 /etc/iscsi
│ │ │  drwxr-xr-x root/root                36 2021-08-17 16:57 /etc/iscsi/ifaces
│ │ │  -rw-r--r-- root/root              5984 2021-08-17 16:57 /etc/iscsi/ifaces/iface.example
│ │ │  -rw-r--r-- root/root                54 2021-08-17 16:57 /etc/iscsi/initiatorname.iscsi
│ │ │  -rw-r--r-- root/root             13338 2021-08-17 16:57 /etc/iscsi/iscsid.conf
│ │ │  drwxr-xr-x root/root                72 2021-08-17 16:57 /etc/isns
```

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-18 22:27:24 +03:00
Artem Chernyshev
fcfca55a05
chore: do not check that go mod tidy gives empty output
It's flaky and we have another dirty check, so this one can be safely removed.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2021-08-18 20:59:48 +03:00
Andrey Smirnov
0ef8f83acf
chore: bump dependencies via dependabot
PRs #4079 #4080 #4081 #4082 #4083 #4084 #4085

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-16 16:33:52 +03:00
Andrey Smirnov
faecae44fd feat: make ISO builds reproducible
This relies on changes in GRUB and other utilities to respect
`SOURCE_DATE_EPOCH`.

Variable `SOURCE_DATE_EPOCH` is set to the timestamp of the last git
commit which makes it deterministic, but still changes for each
release/commit.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-11 09:20:07 -07:00
Alexey Palazhchenko
fdf6b2433c chore: revert "improve artifacts generation reproducibility"
GCP does not consider generated .tar file to be valid.

This reverts commit b2507b41d250b989b9c13ad23e16202cd53a18d2.
Refs #4023.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-08-10 11:50:42 -07:00
Alexey Palazhchenko
4dae9ea55c chore: use vtprotobuf compiled marshaling in Talos API
Use `vtprobuf` optimized Marshal/Unmarshal methods which do not depend
on reflection to reduce memory and CPU usage while using Talos API.

See https://github.com/planetscale/vtprotobuf and
https://vitess.io/blog/2021-06-03-a-new-protobuf-generator-for-go/

Co-authored-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-08-09 08:42:13 -07:00
Andrey Smirnov
7ca5749ad4 chore: bump dependencies via dependabot
PRs #4033 #4034 #4035 #4036 #4037 #4038 #4039 #4040

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-09 08:32:02 -07:00
Andrey Smirnov
b2507b41d2 chore: improve artifacts generation reproducibility
Sparse file generation replaced with Go native calls.

Final artifact `.tar` reproducible with new tar flags and using GNU tar
instead of busybox one, but as the image itself is not reproducible,
this only helps a bit.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-08-09 06:54:47 -07:00
Andrey Smirnov
539f42090e chore: bump dependencies via dependabot
Fixes #3993

Fixes #3994

Fixes #3995

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-08-03 10:25:17 -07:00
Artem Chernyshev
55e17ccdd1 chore: bump dependencies
Fixes: https://github.com/talos-systems/talos/pull/3954 https://github.com/talos-systems/talos/pull/3955 https://github.com/talos-systems/talos/pull/3956 https://github.com/talos-systems/talos/pull/3957 https://github.com/talos-systems/talos/pull/3958 https://github.com/talos-systems/talos/pull/3959 https://github.com/talos-systems/talos/pull/3960 https://github.com/talos-systems/talos/pull/3961 https://github.com/talos-systems/talos/pull/3962 https://github.com/talos-systems/talos/pull/3963 https://github.com/talos-systems/talos/pull/3964

And update kubelet to 1.21.3.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-07-19 06:06:01 -07:00
dependabot[bot]
2ea28f62d8 chore: bump node from 16.3.0-alpine to 16.4.2-alpine
Bumps node from 16.3.0-alpine to 16.4.2-alpine.

---
updated-dependencies:
- dependency-name: node
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-07-12 03:20:49 -07:00
Andrey Smirnov
d930a26502 chore: implement DeepCopy for machine configuration
Resources code extensively uses DeepCopy to prevent in-memory copy of
the resource to be mutated outside of the resource model.

Previous implementation relied on YAML serialization to copy the
machine configuration which was slow, potentially might lead to panics
and it generates pressure on garbage collection.

This implementation uses k8s code generator to generate DeepCopy methods
with some manual helpers when code generator can't handle it.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-07-08 07:21:24 -07:00