Commit Graph

98 Commits

Author SHA1 Message Date
Andrey Smirnov
883d401f9f
chore: rename github organization to siderolabs
Go module import paths still use talos-systems, packages use new
siderolabs name.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-03-23 21:07:46 +03:00
Tim Jones
d1294d014f
chore: add day-two tests for e2e-qemu
Uses Sidero Labs d2ctl to install a small sample of
common cluster services to ensure the Talos cluster
is capable.

Signed-off-by: Tim Jones <tim.jones@siderolabs.com>
2022-03-22 20:53:07 +01:00
Andrey Smirnov
18a21b5f24
chore: add dependency images-essential -> images
This is needed to make sure that those steps don't run in parallel on
release builds, as they write same output files.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-03-14 18:32:03 +03:00
Andrey Smirnov
09efa62f68
chore: re-enable kexec and default to UEFI booting in tests
Fixes #4947

It turns out there's something related to boot process in BIOS mode
which leads to initramfs corruption on later `kexec`.

Booting via GRUB is always successful.

Problem with kexec was confirmed with:

* direct boot via QEMU
* QEMU boot via iPXE (bundled with QEMU)

The root cause is not known, but the only visible difference is the
placement of RAMDISK with UEFI and BIOS boots:

```
[    0.005508] RAMDISK: [mem 0x312dd000-0x34965fff]
```

or:

```
[    0.003821] RAMDISK: [mem 0x711aa000-0x747a7fff]
```

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-03-02 21:52:18 +03:00
Andrey Smirnov
b2bf3117ff
feat: implement extension services
Fixes #4694

User services run alongside with Talos system services.
Every user service container root filesystem should be already present
in the Talos root filesystem.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-02-22 23:11:20 +03:00
Andrey Smirnov
063a9e1657
test: pre-fetch CLI tools
As `_out` is a shared volume across Drone steps, we should prefetch CLI
dependencies early, as fetching them concurrently from multiple steps
might lead to surpising results.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-02-22 22:17:11 +03:00
Andrey Smirnov
df0e388a4f
feat: extract firmware part of system extensions into initramfs
Fixes #4816

This changes the way system extensions are packaged into the squashfs
images: `/lib/firmware` is now moved out of the future squashfs images
and becomes part of `initramfs` to make firmware available in the early
boot.

Talos will bind-mount `/lib/firmware` into rootfs as well, so it will be
available in the rootfs as well.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-02-09 22:58:45 +03:00
Noel Georgi
151c9df091
chore: add CSI tests for e2e-qemu
Add tests for using rook as CSI for e2e-qemu
Allow specifying cpu/memory for workers

Signed-off-by: Noel Georgi <git@frezbo.dev>
2022-01-27 20:06:10 +05:30
Andrey Smirnov
0bf161dffb
test: add integration test for system extensions
This verifies system extensions via the gVisor system extension.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-01-26 23:29:15 +03:00
Noel Georgi
6d8bea5d55
feat: jetson nano SoC
Support Jetson Nano SoC

Depends on https://github.com/talos-systems/pkgs/pull/380

Signed-off-by: Noel Georgi <git@frezbo.dev>
2022-01-18 20:12:54 +05:30
Serge Logvinov
2869b5eeac
feat: add oraclecloud.com platform support
* cloud-init for oraclecloud (IMDSv2)
* amd64/arm64 arch
* set DHCPv6 on if IPv6 subnet allocated

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2022-01-14 16:56:37 +03:00
Andrey Smirnov
944f13221d
chore: fix release pipeline
Looks like I missed adding new steps.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-12-30 17:46:00 +03:00
Andrey Smirnov
b3fbb2f312
test: don't build all images in the default CI pipeline
Instead, build all images for a release and in the integration pipeline
which is triggered `nightly` and can be promoted to with
`/promote integration-images`.

This saves time and resources on regular builds.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-12-24 22:56:34 +03:00
Andrey Smirnov
a5a6c720e9
chore: remove boot-{arch}.tar.gz artifact
Fixes #3311

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-12-07 17:21:00 +03:00
Andrey Smirnov
2a0da06247
feat: split installer and imager images
The `installer` image is used for initial install and upgrades, so it
contains only Talos artifacts for the same arch as the image

The `imager` image is used to produce Talos images, so it contains artifacts
for all arhitectures, so that it's possible e.g. to build Raspberry PI
Talos image on amd64 machine

Unpacked size comparison for amd64 (before this change, size of
`installer` was same as `imager` size):

```
REPOSITORY                               TAG                                      IMAGE ID       CREATED         SIZE
ghcr.io/smira/installer                  v0.14.0-alpha.2-8-g73293bc2-dirty        264995cf56df   4 minutes ago   176MB
ghcr.io/smira/imager                     v0.14.0-alpha.2-8-g73293bc2-dirty        3ba8e0176565   6 minutes ago   616MB
```

As the `installer` image is pulled into `tmpfs` on upgrade, it's important
to have it as small as possible to avoid putting too much pressure on
node memory.

Both images are generated anyways for both `amd64` and `arm64` target
architectures.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-12-03 14:37:45 +03:00
Andrey Smirnov
6dcce20e6f
test: set proper pod CIDR for Cilium tests
This fixes the issue with kubelet picking up wrong IP on restart, as
Talos doesn't know pod IPs (Cilium is using its own pod CIDR, it doesn't
look up Kubernetes settings).

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-11-15 23:50:00 +03:00
Andrey Smirnov
e4e8e8737f
test: disable e2e-misc test with Canal CNI
Canal latest is broken right now:
https://github.com/projectcalico/cni-plugin/issues/1214

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-11-12 13:25:58 +03:00
Andrey Smirnov
68c420e3c9
feat: enable cluster discovery by default
This enables cluster discovery by default for Talos 0.14. KubeSpan is
not enabled by default.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-10-15 14:46:32 +03:00
Andrey Smirnov
9bd4838ac1
chore: stop using sonobuoy CLI
We already have (better) way to run short conformance tests via
`talosctl`.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-10-15 12:34:53 +03:00
Artem Chernyshev
877a2b6fc0
test: bump CAPI components to v1alpha4
Alpha versions of talos providers for now.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2021-10-12 12:48:15 +03:00
Andrey Smirnov
8fcd421967
chore: fix integration-qemu-race
We don't need to build for arm64, as the test runs on amd64.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-10-08 21:56:28 +03:00
Andrey Smirnov
ce65ca4e4a
chore: build using only amd64 builders
Our CI amd64 builders are 3-4 times faster to build Talos than our arm64
builders when building Talos.

Our Dockerfile was restructured a while ago to support cross-compilation
on all platforms but CI was still using amd64/arm64 workers, so arm64
part was done on arm64 builders.

As our CI runs on Talos, `binfmt_misc` is not enabled in the kernel, but
buildkit has built-in QEMU emulation layer which works just fine for
those small pieces which actually need to run arm64 binaries on amd64
(mostly `apk add` in the installer container). Interesting enough,
buildkit QEMU support fails for `ca-certificates` script which runs
after install. At the same time I believe we don't need
`ca-certificates` in the installer, as installer doesn't download
anything from the network, and `ca-certificates` were added a while ago
when installer was actually downloading configuration on its own.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-10-05 23:02:37 +03:00
Andrey Smirnov
e9b0f010d2
chore: update docker image in the pipeline
We use hacked version with a workaround for capability issues with
`--privileged` in Docker.

See moby/moby#42906

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-10-05 21:29:55 +03:00
Serge Logvinov
353d632ae5
feat: add nocloud platform support
* fetch cdrom/net nocloud config
* apply simple network configuration

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-09-28 16:32:12 +03:00
Serge Logvinov
19a8ae97c6
feat: add vultr.com cloud support
* cloud-init for vultr.com
* ipv4/v6 support
* set static IPs for private interface

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-09-14 22:58:30 +03:00
Serge Logvinov
3b5f4038de
feat: add scaleway.com cloud support
* cloud-init for scaleway
* set ipv6 to the interface

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-09-09 23:01:50 +03:00
Serge Logvinov
f156ab1847
feat: add upcloud.com cloud support
* cloud-init for upcloud.com
* ipv4/v6 support

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
2021-09-09 17:00:05 +03:00
Serge Logvinov
812d59c700
feat: add hetzner.com cloud support
* cloud-init for hcloud
* set ipv6 to the interface

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-09-07 21:33:15 +03:00
Olli Janatuinen
ba169c6f91
feat: provide talosctl.exe for Windows
amd64 version only

Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2021-08-26 17:06:50 +03:00
Andrey Smirnov
82731124b2
chore: run e2e-qemu test against Talos with race-detector enabled
Race-detector enabled image is pushed with a tag next to normal image,
regular integration test is ran against the image.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-16 20:13:46 +03:00
Alexey Palazhchenko
e030b2e8bb chore: use k8s 1.21.3 in CAPI tests for now
Refs #4046.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-08-10 13:28:37 -07:00
Andrey Smirnov
571f7db1bb chore: workaround GitHub new release notes limit
GitHub recently introduced limit of 25000 byte for the release notes,
which breaks our release process, as our notes are over the limit.

PR https://github.com/talos-systems/talos/pull/4021 introduced support
for GFM release notes, which are at least 50% smaller.

This PR enables GFM, adds a check for release notes size and pushes
release-notes step to every build so that we known in advance if release
notes are over the limit (vs. the moment we push the release).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-08-06 11:56:26 -07:00
Andrey Smirnov
77fabaceca chore: ignore future pkg/machinery/vX.Y.Z tags
Drone shouldn't build releases for `pkg/machinery/vX.Y.Z` tags.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-07-07 10:33:10 -07:00
Spencer Smith
0731be908b feat: add cloud images to releases
This PR updates our CI so that when we release talos, a json file
containing our cloud images for AWS will be published as a release
asset.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2021-06-23 16:40:54 -04:00
Artem Chernyshev
2dc27d9964 fix: do not format state partition in the initialize sequence
Initialize state should be only reading the config.
So now if it detects that the partition is not even formatted it will
skip it and will consider the state to be empty.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-06-17 07:17:42 -07:00
Andrey Smirnov
807497ec20 chore: make conformance pipeline depend on cron-default
This fixes the dependency in the nightly run to make sure artifacts are
rebuilt fresh before triggering conformance run (every other step did
that, but not this one).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-07 12:14:05 -07:00
Jorik Jonker
5de8dbc06c fix: repair pine64 support
Apparently, I submitted non working board support earlier. This commit
fixes that and also adds the boards to both `sbcs` target and drone CI.

Signed-off-by: Jorik Jonker <jorik@kippendief.biz>
2021-05-05 11:11:53 -07:00
Andrey Smirnov
5fb38d3e5f chore: refactor Dockerfile for cross-compilation
This has two big visible changes:

* `installer` image now contains assets for both `amd64` and `arm64`, so
it can be used to generate any Talos image (including RPi on amd64 host)
* Talos is using cross-compilation instead of emulation to build
non-native architectures: on amd64, Go amd64 compiler produces binaries
for both arm64 and amd64
(before this change: Go arm64 compiler via QEMU produces arm64 binaries on amd64)

CI implications: we no longer require arm64 nodes.

Changes walkthrough:

* `installer` container now keeps assets under `/usr/install/<arch>`
* Dockerfile build starts forcing toolchain/base image to use the build
host native architecture, not target architecture
* lots of duplication for amd64/arm64 as we want to combine assets for
both arches in a single image (e.g. we have multi-arch amd64/arm64
installer image, each arch has native installer binary, but both arches
contain full set of amd64/arm64 assets)
* fixed a small bug preventing arm64 on amd64 talosctl cluster create

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-04-20 13:11:38 -07:00
Andrey Smirnov
a8f1e526bf chore: build talosctl for Darwin / Apple Silicon
This adds a new target `darwin/arm64` which is supported since Go 1.16.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-04-20 08:05:40 -07:00
Alexey Palazhchenko
da92049c0b chore: use codecov from the build container
That's the best we can do now with https://about.codecov.io/security-update/

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-04-19 10:56:05 -07:00
Andrey Smirnov
e7a9164b1e test: implement talosctl conformance command to run e2e tests
Command implements two modes:

* `fast`: conformance suite is run at maximum speed
* `certified`: conformance suite is run in serial mode, results
  are capture to produce artifacts ready for CNCF submission process

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-04-16 09:17:51 -07:00
Artem Chernyshev
e16d6d3468 fix: publish rockpi4 image to release artifacts
Attempt #2. Forgot to add it to .drone.jsonnet also 🤦

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-04-03 18:20:54 -07:00
Andrey Smirnov
f3a32fff99 chore: expire objects in CI S3 bucket
Otherwise we can quickly overflow our storage backend.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-03-04 06:18:38 -08:00
Andrey Smirnov
d7cdc8cc15 feat: implement simple layer 2 shared IP for CP
This adds a VIP (virtual IP) option to the network configuration of an
interface, which will allow a set of nodes to share a floating IP
address among them.  For now, this is restricted to control plane use
and only a single shared IP is supported.

Fixes #3111

Signed-off-by: Seán C McCord <ulexus@gmail.com>
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-02-26 14:14:34 -08:00
Andrey Smirnov
c29cfaa09b chore: build both Darwin and Linux versions of talosctl
This showed up as missing Darwin talosctl in the release.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-02-25 10:47:57 -08:00
Andrey Smirnov
4e19b597ab test: add integration test with Canal CNI and reset API
Canal CNI is known to be trying to reach out to k8s control plane on pod
teardown.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-02-24 11:34:02 -08:00
Artem Chernyshev
7108bb3f5b test: upgrade master to master tests
Verify upgrade flow using the same version of the installer.
Run that with disk encryption enabled.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-02-24 07:56:44 -08:00
Andrey Smirnov
84ad6cbb1a chore: switch CI to stop embedding local registry into the builds
This adds new `IMAGE_REGISTRY` variable (similar to `IMAGE_TAG`) which
affects only the registry image gets pushed to, but it's not built into
the binaries and images as a default registry.

This fixes a problem when release builds reference our CI local
registry.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-02-24 18:05:37 +03:00
Andrey Smirnov
c2d109637b chore: add default cron pipeline to the list of pipelines
Follow-up fix for #3177

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-02-18 10:20:13 -08:00
Andrey Smirnov
ce6bfbdbb7 chore: run default pipeline as part of the cron pipeline
This makes sure artifacts will allways be built as part of
the scheduled run, even if latest merge to master happened just before
the scheduled build.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-02-18 09:00:35 -08:00