775 Commits

Author SHA1 Message Date
Noel Georgi
952801d8b2
fix: handle overlay partition options
Handling of Overlay PartitionOpts was missed in the previous code.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-03-14 15:39:59 +05:30
Noel Georgi
d118a852b9
feat: implement Install for imager overlays
Implement `Install` for imager overlays.
Also add support for generating installers.

Depends on: #8377

Fixes: #8350
Fixes: #8351
Fixes: #8350

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-03-12 22:46:29 +05:30
Dmitriy Matrenichev
32e0877607
chore: print all available logs containers in logs command completions
This is a small quality of life improvement that allows `logs` subcommand to suggest all available logs.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-03-11 17:48:01 +03:00
Utku Ozdemir
f02aeec922
fix: do not fail cluster create when input dir does not contain talosconfig
As `--input-dir` flag now supports partial configs, it should not fail when there is no talosconfig in the directory.

This was the missing part in siderolabs/talos#8333.

Additionally, allow the `--cidr` flag when `--input-dir` is used - it is used even when the input configs are provided.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-03-07 23:13:10 +03:00
Artem Chernyshev
3c8f51d707
chore: move cli formatters and version modules to machinery
To be used in the `go-talos-support` module without importing the whole
Talos repo.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2024-03-07 16:29:15 +03:00
Noel Georgi
8125e754b8
feat: imager overlay
Support overlays for imager.
The `Install` interface is not wired yet, it will be done as a different
PR.

This should be a no-op for existing imager.

Part of: #8350

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-02-29 20:44:31 +05:30
Andrey Smirnov
d81d490003
chore: update CoreDNS renovate source
As we're using a mirrored image from `registry.k8s.io`, use that as a
source instead of GitHub. Mirrored image appears with some delay after
an official CoreDNS release.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-27 17:12:25 +04:00
Andrey Smirnov
8872a7a210
fix: ignore 'no such device' in addition to 'no such file'
This errors pops up when `udevd` rescans the partition table with Talos
trying to mount a device concurrently.

This feels to be something new with Linux 6.6 probably.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-26 20:00:05 +04:00
Dmitry Sharshakov
4575dd8e74
chore: allow not preallocated disks for QEMU cluster
Preallocation still done by default for correct max usage estimates, but
in development environment it could be beneficial not to use up that
space, so I added a flag to disable preallocation

Signed-off-by: Dmitry Sharshakov <d3dx12.xx@gmail.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-23 16:45:44 +04:00
Utku Ozdemir
815a8e9cc5
feat: add partial config support to talosctl cluster create
Allow passing a partial config to the machines using the `--input-dir` flag of `talosctl cluster create`.
With this change, it is not required to have talosconfig inside the input directory, neither for the configuration documents to contain a `v1alpha1` config document. They can solely contain maintenance config, e.g., `KmsgLogConfig`.

This is useful for testing scenarios such as partial machine configs being provided via cloud user data.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-02-20 17:08:58 +01:00
Noel Georgi
15e8bca2b2
feat: support environment in ExtensionServicesConfig
Support setting extension services environment variables in
`ExtensionServiceConfig` document.

Refactor `ExtensionServicesConfig` -> `ExtensionServiceConfig` and move extensions config under `runtime` pkg.

Fixes: #8271

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-02-15 20:16:29 +05:30
Matthieu S
3fe82ec461
feat: custom image settings for k8s upgrade
Allows to use custom registry/images.

Fixes: #8275

Co-authored-by:  @g3offrey
Signed-off-by: Matthieu STROHL <mstrohl@dive-in-it.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-15 17:54:01 +04:00
Dmitriy Matrenichev
fa3b933705
chore: replace fmt.Errorf with errors.New where possible
This time use `eg` from `x/tools` repo tool to do this.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-14 17:39:30 +03:00
Dmitriy Matrenichev
5324d39167
chore: bump stuff
Also fix .golangci.yml file.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-09 19:19:25 +03:00
Andrey Smirnov
383e528df8
chore: allow uuid-based hostnames in talosctl cluster create
This is useful when the VMs are booted without machine config,
so default hostnames based on controlplanes/workers no longer make
sense.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-07 16:22:53 +04:00
Andrey Smirnov
9d8cd4d058
chore: drop deprecated method EtcdRemoveMember
It was deprecated 16 months ago, time to cleanup.

(This is to prepare for the first v1.7 release)

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-01 15:54:29 +04:00
Andrey Smirnov
a5e13c696d
fix: retry blockdevice open in the installer
We had these retries in other places, but not here.

This seems to happen more frequently with Linux 6.6 update, the tl;dr is
same: `udevd` tries to rescan the partition table at the wrong moment,
preventing Talos installer to open the partition which was just created.

It's a race, so workaround it by retrying the call.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-01-31 22:17:20 +04:00
Andrey Smirnov
87be76b878
fix: be more tolerant to error handling in Mounts API
Fixes #8202

If some mountpoint can't be queried successfully for 'diskfree'
information, don't treat that as an error, and report zero values for
disk usage/size instead.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-01-31 18:24:38 +04:00
Dmitriy Matrenichev
9c14dea209
chore: bump coredns
Bump our CoreDNS fork.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-01-30 02:12:36 +03:00
Andrey Smirnov
e0dfbb8fba
fix: allow META encoded values to be compressed
Fixes #8186

This is planned to be backported to Talos 1.6.3.

This allows to pass large META values (YAML for platform network
configuration) which might otherwise exceed the limit for kernel
command line params.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-01-23 17:24:18 +04:00
Andrey Smirnov
4e9b688d3f
fix: use correct TTL for talosconfig in talosctl config new
See #8152

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-01-22 15:39:41 +04:00
Utku Ozdemir
cc06b5d7a6
fix: fix .der output in talosctl gen secureboot
PEM was converted to DER incorrectly when the output was a X509 certificate and not a public key.

Skip unnecessary parsing of it to an RSA public key before writing it in DER format as output.

Simplify the code as we do not generate `*-signing-public-key.pem` anymore.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-01-17 14:02:03 +01:00
Andrey Smirnov
e8758dcbad
chore: support http downloads for assets in talosctl cluster create
This allows to pass direct URLs to Image Factory assets for disk
image/ISO/vmlinuz/initramfs, so that we can test Image Factory with
Talos.

Also add an integration test for Image Factory.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-25 18:58:25 +04:00
Andrey Smirnov
0a30ef7845
fix: imager should support different Talos versions
Add some quirks to make images generated with newer Talos compatible
with images generated by older Talos.

Specifically, reset options were adding in Talos 1.4, so we shouldn't
add them for older versions.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-22 16:13:34 +04:00
Andrey Smirnov
01f0cbe61c
feat: support iPXE direct booting in talosctl cluster create
This embeds a tiny TFTP server which serves UEFI iPXE which embeds a
script that chainloads a given iPXE script.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-19 17:56:08 +04:00
Andrey Smirnov
760f793d55
fix: use correct prefix when installing SBC files
When creating an image under non-default mount prefix, it should be
used explicitly when copying SBC files.

See https://github.com/siderolabs/image-factory/issues/65

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-15 19:46:10 +04:00
Dmitriy Matrenichev
6bb1e99aa3
chore: optimize pcap dump
Reimplement `gopacket.PacketSource.PacketsCtx` as `forEachPacket`.

- Use `ZeroCopyPacketDataSource` instead of `PacketDataSource`. I didn't find any specific reason why `PacketDataSource` exists at all, since `NewPacket` is doing copy inside if you don't explicitly tell it not to.
- Use `WillPool` to pool packet buffers. It doesn't fully remove allocations, but it's a safe start.
  Send packets back into the pool after we are done with them.
- Pass `Packet` directly to the closure instead of waiting for it on the channel. We don't store this packet anywhere so there is no reason to async this part.
- Drop `time.Sleep` code in `forEachPacket` body.
- Drop `SnapLen` support in client and server since it didn't work anyway (details in the PR).

Closes #7994

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-12-11 15:44:42 +03:00
Andrey Smirnov
4f9d3b975f
feat: update Kubernetes to v1.29.0-rc.2
See https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.29.md

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-08 19:41:28 +04:00
Andrey Smirnov
46121c9fec
docs: rework machine config documentation generation
Generate a structured table of contents following the structure of the
config.

Make high-level examples follow the full structure of the config.

Document new multi-doc machine config.

Fixes #8023

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-08 14:16:40 +04:00
Andrey Smirnov
e128d3c827
fix: talosctl cluster create not to enforce kubeprism always
The command should be able to deploy old versions of Talos as well,
even before KubePrism.

The version contract correctly enables/disables KubePrism by default, so
take default flag value as "don't change defaults".

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-07 18:15:54 +04:00
Andrey Smirnov
d8a435f0e4
fix: initialize boot assets with defaults early
The problem was that bootloaders were correctly picking up defaults for
`installer` mode (vs. `imager` mode), but DTB and other SBC stuff wasn't
properly initialized, so installing on SBC fails.

Now all options are properly initialized with defaults early in the
process.

Fixes #8009

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-01 17:47:05 +04:00
Andrey Smirnov
36c8ddb5e1
feat: implement ingress firewall rules
Fixes #4421

See documentation for details on how to use the feature.

With `talosctl cluster create`, firewall can be easily test with
`--with-firewall=accept|block` (default mode).

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-30 22:58:16 +04:00
Noel Georgi
7a4a92854f
feat: support sanitized kernel args
Support dropping kernel args that start with `-`.

Fixes: #7613

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-11-28 16:23:05 +05:30
Oscar Utbult
8e23074665
docs: fix talosctl pcap argument
Fixes a formatting issue on the website

Signed-off-by: Oscar Utbult <oscar.utbult@gmail.com>
Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-11-23 21:56:02 +05:30
Oscar Utbult
e4a050cb1d
docs: fix talosctl inspect dependencies example indentation
Adds missing whitespace to ensure correct rendering on web

Signed-off-by: Oscar Utbult <oscar.utbult@gmail.com>
Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-11-23 19:16:18 +05:30
Oscar Utbult
fbcf4264ff
docs: fix talosctl dashboard cli docs
Fixes the shortcuts for the dashboard command by encoding < and >

Signed-off-by: Oscar Utbult <oscar.utbult@gmail.com>
Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-11-23 15:18:59 +05:30
Oscar Utbult
0eb245e043
docs: fix talosctl pcap example indentation
Fixes whitespaces in examples.

Signed-off-by: Oscar Utbult <oscar.utbult@gmail.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-20 18:45:27 +04:00
Andrey Smirnov
f38eaaab87
feat: rework secureboot and PCR signing key
Support different providers, not only static file paths.

Drop `pcr-signing-key-public.pem` file, as we generate it on the fly
now.

See https://github.com/siderolabs/image-factory/issues/19

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-10 21:14:21 +04:00
Dmitriy Matrenichev
6eade3d5ef
chore: add ability to rewrite uuids and set unique tokens for Talos
This PR does those things:
- It allows API calls `MetaWrite` and `MetaRead` in maintenance mode.
- SystemInformation resource now waits for available META
- SystemInformation resource now overwrites UUID from META if there is an override
- META now supports "UUID override" and "unique token" keys
- ProvisionRequest now includes unique token and Talos version

For #7694

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2023-11-10 18:17:54 +03:00
Andrey Smirnov
e22ab440d7
feat: update Linux 6.1.61, containerd 1.7.8, runc 1.1.10
Bump tools/pkgs/extras.

Update Go dependencies.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-09 20:17:28 +04:00
Utku Ozdemir
6f32d2990f
feat: add .der output talosctl gen secureboot pcr
Output the PCR public key in `.der` format in addition to the `.pem` format.

Closes siderolabs/talos#7742.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-11-07 16:25:54 +01:00
Andrey Smirnov
6dc776b8aa
fix: when writing to META in the installer/imager, use fixed name
Use fixed partition name instead of trying to auto-discover by label.

Auto-discovery by label might hit completely wrong blockdevice.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-01 20:34:41 +04:00
Andrey Smirnov
cbe6e7622d
fix: generate images for SBCs using imager
See https://github.com/siderolabs/image-factory/issues/43

Two fixes:

* pass path to the dtb, uboot and rpi-firmware explicitly
* include dtb, uboot and rpi-firmware into arm64 installer image when
  generated via imager (regular arm64 installer was fine)

(The generation of SBC images was not broken for Talos itself, but only
when used via Image Factory).

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-10-30 13:46:58 +04:00
Utku Ozdemir
5dff164f1c
fix: fix error output of cli action tracker
Before we started a reboot/shutdown/reset/upgrade action with the action tracker (`--wait`), we were setting a flag to prevent cobra from printing the returned error from the command.

This was to prevent the error from being printed twice, as the reporter of the action tracker already prints any errors occurred during the action execution.

But if the error happens too early - i.e. before we even started the status printer goroutine, then that error wouldn't be printed at all, as we have suppressed the errors.

This PR moves the suppression flag to be set after the status printer is started - so we still do not double-print the errors, but neither do we suppress any early-stage error from being printed.

Closes siderolabs/talos#7900.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2023-10-27 21:16:54 +02:00
Andrey Smirnov
8eba4c5999
feat: generate secrets bundle from the machine config
This allows to "recover" secrets if the machine config was generated
first without explicitly saving secrets bundle.

Fixes #7895

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-10-25 13:44:14 +04:00
Serge Logvinov
401e894112
feat: customize image size
Can possible to change boot image size.
Change the default image size for some cloud platform.

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-10-16 15:51:13 +04:00
Andrey Smirnov
390137447f
feat: enable KubePrism by default
Fixes #7787

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-09-25 23:12:33 +04:00
Andrey Smirnov
a52d3cda3b
chore: update gen and COSI runtime
No actual changes, adapting to use new APIs.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-09-22 12:13:13 +04:00
Artem Chernyshev
2960f93baa
feat: add readonly information to the disks API response
Forward device readonly info from `go-blockdevice` library.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2023-09-12 18:09:59 +03:00
Serge Logvinov
3f52320752
feat: upgrade-k8s without comments
This feature allows us to remove any comments from the machineconfig after
upgrading Kubernetes.

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-09-12 14:50:56 +04:00