IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
This was preferred way to push images before multi-arch was introduced.
This way doesn't work with multi-platform images, so we had two paths.
Changes:
* `PUSH=true` works now always for `make installer`, `make talos`
* for `image-aws` and friends make sure to push the installer image
before doing that (either local registry or different username)
* `talosctl cluster create` for docker - make sure image is pushed and
`talosctl` knows about the image reference.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
The variable $(TAG) was used both during linking phase to set the
version of Talos built into binaries and as an image reference for
pushing. This leads to to `:latest` image being built with version
`latest`.
Separate image push reference and Talos version by introducing
`$(IMAGE_TAG)` which defaults to `$(TAG)`, but can be overridden.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This introduces the notion of a "board" in Talos. A board is an interface that is capable
of modifying the installation in specific ways for a given SBC. This also adds support for the
libretech_all_h3_cc_h5.
Signed-off-by: Andrew Rynhard <andrew@rynhard.io>
This PR adds the ability for us to deploy Talos in openstack. Tested in
local devstack with a supplied userdata file. It also adds support to
the Makefile for building the openstack image so it'll be published with
next release.
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
Talos 0.8 is going to ship with K8s 1.20.x.
Changes to support new `control-plane` label,
upgrade-k8s supports automated fixups for 1.20.
See also: https://github.com/talos-systems/bootkube-plugin/pull/22
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Bump to 0.7.0 as we have a new release.
Clean up the tests we do: 0.6.3 is a previous release, 0.7.0 is a stable
release, current version (0.8.x) is the "next" release.
We test the following:
* 0.6.3 -> 0.7.0
* 0.7.0 -> 0.8-current
* 0.7.0 -> 0.8-current (single node)
This tests upgrades always between two releases.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This changes installer image/iso output to be tar via stdout
(optionally), so that we can copy back artifacts back from remote docker
daemon.
Fixes#2776
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This builds a bundle with CNI plugins for talosctl which is
automatically downloaded by `talosctl` if CNI plugins are missing.
CNI directories are moved by default to the `~/.talos/cni` path.
Also add a bunch of pre-flight checks to the QEMU provisioner to make it
easier to bootstrap the Talos QEMU cluster.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
For 0.6 -> 0.7 upgrade, in any case config.yaml is preserved and moved
from `/boot` to `/system/state`.
For single node upgrade, `EPHEMERAL` partition is not touched and other
partitions are re-created as needed.
Bump provision tests to 0.6/0.7 upgrades as we get closer to the new
release.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Comments encoding works, defaults encoding works.
Docgen was revamped: now it generates go files.
While markdown files are all handled by
`pkg/machinery/config/encoder/markdown.go`.
Changed scheme for docs. Now it no longer relies on a single `doc.go` in
the root of a package. Instead it can generate separate `*_doc.go` files
for each file in the package. `docgen` now expects to get 3 params
instead of 2. 3rd parameter is used to define a unique method name for
getting the list of structs in the file.
Backward compatibility is supported if we define package name as the 3rd
parameter.
1st parameter no longer scans whole current directory, instead it points
to the particular file that should be processed by docgen.
`talosctl docs` command now supports two flags: `--config` and `--cli`.
They allow generating only docs for v1alpha1 configs or for talosctl. If
no flags are defined, all docs are generated.
Additionally made field types clickable in the output markdown file.
Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
This introduces newly built multi-arch images from
`talos-systems/extras`.
Also fixed default installer image if config is missing.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This uses `talosctl images`. This way it's easy to find installer image
or talos image for any release.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
By default, build outside of Drone works the same and builds only amd64
version, loads images back into dockerd, etc.
If multiple platforms are used, multi-arch images are built which can't
be exported to docker or to `.tar` image, they're always pushed to the
registry (even for PR builds to our internal CI registry).
Artifacts as files (initramfs, kernel) now have `-arch` suffix:
`vmlinuz-amd64`, `initramfs-amd64.xz`. "Magic" script normalizes output
paths depending on whether single platform or multiple platforms were
given.
VM provisioners accept magic `${ARCH}` in initramfs/kernel paths which
gets replaced by cluster architecture.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Add sonobuoy runner code with log fetching on failure. Use hand-picked
set of e2e tests to run: verify basic pod functionality, verify service
connectivity.
Add option `--run-e2e` to the `talosctl health` to run quick e2e test to
verify cluster health.
Add option to run provision tests with custom CNI, run one track of
provision tests with Cilium.
Bump Cilium to 1.8.2.
Talos 0.6 won't uncordon node automatically after upgrade from 0.5, as
0.5 doesn't put annotation. Workaround that in upgrade tests.
Bump upgrade test version to 0.6.0 release.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
There's no way to use variable substitution in `COPY --from=`, but we
can use that in `FROM`.
Also bump TOOLS version to finish bump from #2497.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This moves to using grub instead of syslinux.
BREAKING CHANGE: Single node upgrades will fail in this change. This
will also break the A/B fallback setup since this version introduces
an entirely new partition scheme, that any fallback will not know about.
We plan on addressing these issues in a follow up change.
Signed-off-by: Andrew Rynhard <andrew@rynhard.io>
`vmlinux` (uncompressed kernel) shouldn't be much useful to the people
as it can't be booted directly with most of the boot tools (it's too
huge). We had it enabled for firecracker, but now firecracker has
support for decompressing `vmlinuz` on the fly.
So, to reduce confusion, make assets smaller, remove `vmlinux`.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This moves `pkg/config`, `pkg/client` and `pkg/constants`
under `pkg/machinery` umbrella.
And `pkg/machinery` is published as Go module inside Talos repository.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This integrates [importvet](https://github.com/talos-systems/importvet)
into `lint` target.
First rule file was added for public packages `pkg/` which shouldn't
depend on other parts of Talos tree (except for the API definitions).
Only one change: `internal/cis` was moved under single user -
`pkg/config/internal/cis` to satisfy the rules.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Fixes#2363#2364#2370#2371
Several changes packed together:
* use compressed `vmlinuz` everywhere, firecracker provisioner
uncompresses it before first use, drop `vmlinux`
* handle reboots in qemu launcher to support reset API case, update
empty disk check to handle reset behavior (erasing partition table)
* make bootloader support default in provisioners, and flag to disable
that
* early support for target architecture for qemu provisioner
This should allow us to use `qemu` in CI/CD (not included into this PR):
integration test passes with qemu.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This bumps next version to the latest 0.6 alpha and latest 0.5.
This also enables single node preserve test.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
We're not using load balancer for `apid` (always using client-side load
balancing), so we can remove this safely.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Fixes#2272
`gofumpt` is now included into `golangci-lint`, but not the
`gofumports`, so we keep it using it as separate binary, but we keep
versions in sync with `golangci-lint`.
This contains fixes from:
* `gofumpt` (automated, mostly around octal constants)
* `exhaustive` in `switch` statements
* `noctx` (adding context with default timeout to http requests)
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
go1.14.5 (released 2020/07/14) includes security fixes to the
crypto/x509 and net/http packages.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This PR brings in the latest version of clusterctl that has built-in
support for the talos repos. I'll be chasing this with a move to using
the control-plane provider as well!
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
This adds new flug to `cluster create` to launch cluster with custom
CNI, `integration` pipeline gets a new step to run short test with
Cilium 1.8.0 CNI.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This moves full integratation test and provision tests to
the `integration` pipeline.
Docker test wasn't affected much, as anyways docker can't run long
integration tests, so it mostly affects firecracker and provision tests.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This moves our test scripts to using the bootstrap API. Some
automation around invoking the bootstrap API was also added
to give the same ease of use when creating clusters with the
CLI.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
This PR brings in all changes necessary to deploy kubernetes 1.19.x.
It relies on an update to our bootkube-plugin project, as well as
implementation of some Image() functions for our various control plane
components, since they are all distinct images and not just hyperkube.
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
This PR will update k8s to the latest 1.18 release and bump sonobuoy to
help resolve some e2e flakes. Also adds some retry logic around the
sonobuoy run.
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
With Go 1.14.3 we can run race-enabled code on muslc, so this opens path
to run unit-tests-race under Talos environment with rootfs, enabling all
the tests to run under race detector.
Also fixed the tests run by specifying platform in the test environment.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This PR will update our testing ocde to make use of the clusterctl tool,
as well as use the newer versions of various providers and updated
manifests.
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
This is a rewrite of machined. It addresses some of the limitations and
complexity in the implementation. This introduces the idea of a
controller. A controller is responsible for managing the runtime, the
sequencer, and a new state type introduced in this PR.
A few highlights are:
- no more event bus
- functional approach to tasks (no more types defined for each task)
- the task function definition now offers a lot more context, like
access to raw API requests, the current sequence, a logger, the new
state interface, and the runtime interface.
- no more panics to handle reboots
- additional initialize and reboot sequences
- graceful gRPC server shutdown on critical errors
- config is now stored at install time to avoid having to download it at
install time and at boot time
- upgrades now use the local config instead of downloading it
- the upgrade API's preserve option takes precedence over the config's
install force option
Additionally, this pulls various packes in under machined to make the
code easier to navigate.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
This PR does the following:
- updates the conform config
- cleans up conform scopes
- moves slash commands to the talos-bot
- adds a check list to the pull request template
- disables codecov comments
- uses `BOT_TOKEN` so all actions are performed as the talos-bot user
- adds a `make conformance` target to make it easy for contributors to
check their commit before creating a PR
- bumps golangci-lint to v1.24.0
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
`make docs` removes and then regenerates contents of some docs, so it
might cause random `-dirty` issue when running concurrently with build
steps.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This PR will bring in the latest sonobuoy that is designed for the 1.18
branch on kubernetes.
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
This PR will pull in the latest release of k8s 1.18 so we can start
validating it through our test suite.
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
This updates upgrade tests to run two flows with 3+1 clusters:
1. 0.3 -> current (testing upgrade with partition wiping)
2. 0.4-alpha.7 -> current (testing upgrade without partition wiping,
boot-a/boot-b)
And small upgrade with preserve enabled for single-node cluster.
Provision tests are now split into two parallel tracks in Drone.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This is a rename of the osctl binary. We decided that talosctl is a
better name for the Talos CLI. This does not break any APIs, but does
make older documentation only accurate for previous versions of Talos.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
This class of tests is included/excluded by build tags, but as it is
pretty different from other integration tests, we build it as separate
executable. Provision tests provision cluster for the test run, perform
some actions and verify results (could be upgrade, reset, scale up/down,
etc.)
There's now framework to implement upgrade tests, first of the tests
tests upgrade from latest 0.3 (0.3.2 at the moment) to current version
of Talos (being built in CI). Tests starts by booting with 0.3
kernel/initramfs, runs 0.3 installer to install 0.3.2 cluster, wait for
bootstrap, followed by upgrade to 0.4 in rolling fashion. As Firecracker
supports bootloader, this boots 0.4 system from boot disk (as installed
by installer).
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This enables a way to run the matching installer image in firecracker
tests. New image is used in firecracker tests and bootloader support to
use installed kernel/initramfs, which opens path for upgrade tests.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Container images for `apid`, `networkd`, etc. are now built inside the
buildkit using the `img` tool. This means that all the dependencies are
now controlled in `buildkit` and many more stages can run in parallel
without problems (overwriting content in `_out/images`).
This also simplifies Drone configuration, as we can let buildkit handle
the dependencies. I also enabled more stages to run in parallel.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This is initial PR to push the initial code, it has several known
problems which are going to be addressed in follow-up PRs:
1. there's no "cluster destroy", so the only way to stop the VMs is to
`pkill firecracker`
2. provisioner creates state in `/tmp` and never deletes it, that is
required to keep cluster running when `osctl cluster create` finishes
3. doesn't run any controller process around firecracker to support
reboots/CNI cleanup (vethxyz interfaces are lingering on the host as
they're never cleaned up)
The plan is to create some structure in `~/.talos` to manage cluster
state, e.g. `~/.talos/clusters/<name>` which will contain all the
required files (disk images, file sockets, VM logs, etc.). This
directory structure will also work as a way to detect running clusters
and clean them up.
For point number 3, `osctl cluster create` is going to exec lightweight
process to control the firecracker VM process and to simulate VM reboots
if firecracker finishes cleanly (when VM reboots).
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
There are few workarounds for Drone way of running integration test:
DinD runs as a separate pod, and we can only access its exposed on the
"host" ports, while from Talos cluster this endpoint is not reachable.
So internally Talos nodes still use addresses like "10.5.0.2", while
test is using "docker" to access it (that's name of the `docker` service
in the pipeline).
When running locally, 127.0.0.1 is used as endpoint, which should work
fine both on OS X and Linux.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This adds a help menu to the Makefile. It documents all build
dependencies, and how to get started.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
Primarily doc/constant changes.
Added additionnal bits to `docs` target in makefile to generate osctl
docs as well as config files. Explicitly define a HOME variable so we
get consistent home directories for talosconfig variables in our docs.
Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
This PR will be the start of several. It rewrites the basic integration
in go. We'll do these one at a time.
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
This adds a step to the conformance pipeline that pushes all containers
with the tag "edge." This Will allow us to start using and edge
"channel" for upgrades.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
This adds a small program to parse our config structs and generate
markdown from them. This will allow us to enforce a standard and require
documentation for fields as they get added.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
This PR adds in the necessary manifests and fixes to deploy aws clusters
as part of e2e testing.
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
Also refactored `integration-test` build as a generic step to be shared
by basic-integration and e2e-integration steps.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This PR will re-enable e2e testing by using the new cluster api
bootstrap provider and various infra providers.
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
This is just first steps and core foundation.
It can be used like:
```
make integration.test
osctl cluster create
build/integration.test -test.v
```
This should run the test against the Docker instance.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This PR introduces APId. This service replaces the frontend functionality
previously provided by OSD. The main driver for this is two fold:
1. Create a single purpose application to expose the talos api
2. Make use of code generation to DRY api changes
Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
This removes the github.com/pkg/errors package in favor of the official
error wrapping in go 1.13.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
We have decided that proxyd is not the best architectue for HA
Kubernetes. Our recommendation to users will be to create a load
balancer instead.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
The gofumports does everything that gofumpt does with the addition of
formatting imports. This change proposes the use of the `-local` flag so
that we can have imports separated in the following order:
- standard library
- third party
- Talos specific
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
This provides a target that can be useful for developers. It will format
code according to our standards.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
This PR will make sure that each platform gets the console settings it
needs by setting them as extra flags in the makefile. This should ensure
that we have console logs flowing properly for each cloud.
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
This moves to making AWS releases align with Azure, and GCP. We no
longer need packer since we will now release an artifact that users can
import.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
This cache was more important back when builds of Talos took upwards of
40 minutes. Since this is no longer the case, and I have seen
performance issues by mounting a host path into the container, I think
we should drop this.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
This is a major rewrite of our network subsystem.
- This changes networkd to run as a standalone app versus internal goroutine
- This changes out the netlink package with the more idiomatic netlink/rtnetlink
packages
- This changes the initial network bootstrap/discovery from using a single
interface to attempting to bring up all interfaces
- This moves us back on to the upstream dhcp library
Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
In order to DRY up all installation methods and mount methods, this PR
introduces a few more runtime modes. The modes are then used to
determine the strategy for creating and or mounting the paritions.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
Should prevent flakes when we merge fixes on release branches where they unintentionally
get tagged as `latest`.
Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
This should fix a race conditions where two independent image creation steps
run `losetup -f` and discover the same 'next available' loopback device and
attempt to use it.
Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
This is not ideal, but it works. We essentially need to start using
replace statements in order to pull in the modules we need.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
In order to run more than one instance of E2E testing at a time, we need
to ensure that all resources are unique to the run.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
Instead of building platform specific images in the default pipeline, we
should build just one image as part of our basic testing to make sure
installations work as expected.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
This change improves the drone jsonnet file by making it more DRY and
structuring it in a way that makes it much easier to follow.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
The machined dependency is not needed in the rootfs target. The
dependency is handled by buildkit.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
This PR will allow us to run an azure e2e test in parallel with our
current GCE implementation.
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
This PR will break cluster create apart from the other steps in
integration tests. It will allow us to run the cluster create, then use
it for parallel e2e builds in different cloud environments.
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
