733 Commits

Author SHA1 Message Date
Andrey Smirnov
ff2267eb99 test: update versions used for upgrade tests
We should stick to the latest version in each release series.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-04-07 15:51:56 -07:00
Andrey Smirnov
2d5c6f4c10 test: serialize docs step execution
`make docs` removes and then regenerates contents of some docs, so it
might cause random `-dirty` issue when running concurrently with build
steps.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-04-07 23:46:16 +03:00
Spencer Smith
502611f28e chore: update sonobuoy to v0.18.0
This PR will bring in the latest sonobuoy that is designed for the 1.18
branch on kubernetes.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-04-06 15:11:40 -04:00
Spencer Smith
3a4eaeeef0 feat: upgrade kubernetes to 1.18
This PR will pull in the latest release of k8s 1.18 so we can start
validating it through our test suite.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-03-26 14:59:43 -04:00
Andrey Smirnov
e38cde9b48 chore: update upgrade tests for new version, split into two tracks
This updates upgrade tests to run two flows with 3+1 clusters:

1. 0.3 -> current (testing upgrade with partition wiping)
2. 0.4-alpha.7 -> current (testing upgrade without partition wiping,
boot-a/boot-b)

And small upgrade with preserve enabled for single-node cluster.

Provision tests are now split into two parallel tracks in Drone.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-03-24 15:30:00 -07:00
Spencer Smith
3485ea9f09 fix: update k8s to 1.17.3
This PR will update k8s to v1.17.3 to address CVEs mentioned in https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!topic/kubernetes-security-announce/2UOlsba2g0s

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-03-23 17:08:52 -07:00
Andrew Rynhard
5dbc26c7a3 feat: rename osctl to talosctl
This is a rename of the osctl binary. We decided that talosctl is a
better name for the Talos CLI. This does not break any APIs, but does
make older documentation only accurate for previous versions of Talos.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-03-20 19:07:39 -07:00
Andrey Smirnov
a1350aa819 feat: upgrade Go to version 1.14.1
Fixes #1934

See talos-systems/pkgs#106, talos-systems/tools#90

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-03-20 21:42:47 +03:00
Andrey Smirnov
e6dc87dfa4 chore: update pkgs & tools for Go 1.14
See also:

* https://github.com/talos-systems/tools/pull/89
* https://github.com/talos-systems/pkgs/pull/103

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-02-27 01:15:46 +03:00
Andrey Smirnov
923ef4537b test: implement new class of tests: provision tests (upgrades)
This class of tests is included/excluded by build tags, but as it is
pretty different from other integration tests, we build it as separate
executable. Provision tests provision cluster for the test run, perform
some actions and verify results (could be upgrade, reset, scale up/down,
etc.)

There's now framework to implement upgrade tests, first of the tests
tests upgrade from latest 0.3 (0.3.2 at the moment) to current version
of Talos (being built in CI). Tests starts by booting with 0.3
kernel/initramfs, runs 0.3 installer to install 0.3.2 cluster, wait for
bootstrap, followed by upgrade to 0.4 in rolling fashion. As Firecracker
supports bootloader, this boots 0.4 system from boot disk (as installed
by installer).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-02-21 07:04:03 -08:00
Niklas Wik
08b1a782cd feat: support proxy in docker buildx
This allows building when http(s) proxy is enforced to download content on the build machine

Signed-off-by: Niklas Wik <niklas.wik@nokia.com>
2020-02-20 05:35:17 -08:00
Andrey Smirnov
5f330f1f64 chore: push installer & talos images to the CI registry on every build
This enables a way to run the matching installer image in firecracker
tests. New image is used in firecracker tests and bootloader support to
use installed kernel/initramfs, which opens path for upgrade tests.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-02-18 07:32:45 -08:00
Andrey Smirnov
f51e9a14fe chore: build app container images skipping export to host
Container images for `apid`, `networkd`, etc. are now built inside the
buildkit using the `img` tool. This means that all the dependencies are
now controlled in `buildkit` and many more stages can run in parallel
without problems (overwriting content in `_out/images`).

This also simplifies Drone configuration, as we can let buildkit handle
the dependencies. I also enabled more stages to run in parallel.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-02-14 13:17:25 -08:00
Andrew Rynhard
88667641df chore: refactor E2E scripts
This PR aims to simplify our E2E scripts.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-26 20:47:25 -08:00
Andrew Rynhard
f87c6d74d3 chore: use firecracker in basic-integration
This adds a basic integration step that uses firecracker.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-23 05:52:22 -08:00
Andrew Rynhard
a0d8656ca0 chore: use v0.1.0 tools and pkgs
This brings in the official v0.1.0 releases of tools and pkgs.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-20 07:53:08 -08:00
Andrey Smirnov
2bf8540855 test: provision Talos clusters via Firecracker VMs
This is initial PR to push the initial code, it has several known
problems which are going to be addressed in follow-up PRs:

1. there's no "cluster destroy", so the only way to stop the VMs is to
`pkill firecracker`

2. provisioner creates state in `/tmp` and never deletes it, that is
required to keep cluster running when `osctl cluster create` finishes

3. doesn't run any controller process around firecracker to support
reboots/CNI cleanup (vethxyz interfaces are lingering on the host as
they're never cleaned up)

The plan is to create some structure in `~/.talos` to manage cluster
state, e.g. `~/.talos/clusters/<name>` which will contain all the
required files (disk images, file sockets, VM logs, etc.). This
directory structure will also work as a way to detect running clusters
and clean them up.

For point number 3, `osctl cluster create` is going to exec lightweight
process to control the firecracker VM process and to simulate VM reboots
if firecracker finishes cleanly (when VM reboots).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-01-16 00:27:08 +03:00
Andrey Smirnov
810e9b418b chore: bump tools/pkgs for Go 1.13.6
Ref: https://github.com/talos-systems/tools/pull/85,
https://github.com/talos-systems/pkgs/pull/87

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-01-13 20:55:17 +03:00
Andrew Rynhard
d824d0bfdb chore: publish boot.tar.gz
This adds a convenience tarball that includes vmlinuz, and initramfs.xz
in a single tarball.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-09 12:38:21 -08:00
Andrew Rynhard
794d9e6066 chore: update all target in Makefile
We should build the most common things by default.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-06 11:08:27 -08:00
Andrew Rynhard
288d4d0b51 chore: push latest tag on tag events
This ensures that the latest tag is updated on git tag events.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-01 11:41:49 -08:00
Andrey Smirnov
ebd40bd0eb chore: use osctl cluster --wait in basic-integration
There are few workarounds for Drone way of running integration test:
DinD runs as a separate pod, and we can only access its exposed on the
"host" ports, while from Talos cluster this endpoint is not reachable.

So internally Talos nodes still use addresses like "10.5.0.2", while
test is using "docker" to access it (that's name of the `docker` service
in the pipeline).

When running locally, 127.0.0.1 is used as endpoint, which should work
fine both on OS X and Linux.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-30 15:15:42 -08:00
Andrew Rynhard
5a7eb631b2 feat: add installer command to installer container
This replaces the entrypoint.sh shell script with a go binary.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-26 06:41:25 -08:00
Andrew Rynhard
e4a1bc3cf9 chore: add help menu to the Makefile
This adds a help  menu to the Makefile. It documents all build
dependencies, and how to get started.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-25 11:11:41 -08:00
Andrew Rynhard
831f5524a1 chore: refactor Makefile to be more DRY
This PR aims to make the Makefile more DRY.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-24 10:48:32 -08:00
Andrew Rynhard
6602a85976 chore: use docker buildx
This replaces buildkit and buildctl with the docker buildx plugin.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-24 08:30:39 -08:00
Brad Beam
9584b47cd7 feat: Upgrade kubernetes to 1.17.0
Primarily doc/constant changes.

Added additionnal bits to `docs` target in makefile to generate osctl
docs as well as config files. Explicitly define a HOME variable so we
get consistent home directories for talosconfig variables in our docs.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-12-10 16:03:35 -08:00
Spencer Smith
264c5440ef chore: rewrite basic integration in go instead of bash
This PR will be the start of several. It rewrites the basic integration
in go. We'll do these one at a time.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-12-05 15:55:19 -05:00
Andrew Rynhard
70b9186be0 chore: push edge tag on succesful conformance
This adds a step to the conformance pipeline that pushes all containers
with the tag "edge." This Will allow us to start using and edge
"channel" for upgrades.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-27 08:10:25 -08:00
Andrew Rynhard
4680f66bc5 docs: add autogenerated config reference
This adds a small program to parse our config structs and generate
markdown from them. This will allow us to enforce a standard and require
documentation for fields as they get added.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-11 08:38:39 -08:00
Spencer Smith
6d5bbaf7c8 chore: re-enable e2e for aws clusters
This PR adds in the necessary manifests and fixes to deploy aws clusters
as part of e2e testing.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-11-07 15:32:14 -05:00
Andrey Smirnov
8fdf71789e test: add 'integration-test' to e2e runs
Also refactored `integration-test` build as a generic step to be shared
by basic-integration and e2e-integration steps.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-11-07 06:30:34 -08:00
Spencer Smith
ce7a0e36cc chore: re-enable e2e testing
This PR will re-enable e2e testing by using the new cluster api
bootstrap provider and various infra providers.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-11-05 16:53:38 -05:00
Andrey Smirnov
b0aef2cf22 test: add integration test framework
This is just first steps and core foundation.

It can be used like:

```
make integration.test
osctl cluster create
build/integration.test -test.v
```

This should run the test against the Docker instance.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-11-05 17:21:38 +03:00
Andrew Rynhard
3887b1e5b6 chore: force overwrite of output file
This adds the force option to gzip.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-03 10:31:17 -08:00
Andrew Rynhard
7cd9ba588c chore: remove RAW disk
We need to remove this so that it is not published in a release.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-31 15:35:06 -07:00
Andrew Rynhard
2cad745292 docs: add customization guide
This adds a section on customizing Talos.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-31 14:47:17 -07:00
Brad Beam
ca76ccd4af feat: Add support for creating VMware images
This PR adds support for generating VMware compatible images as an ova.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-10-31 13:39:54 -05:00
Andrew Rynhard
3c6d0135d0 feat: upgrade Kubernetes to 1.16.2
This brings in 1.16.2 modules and bumps the default hyperkube image.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-30 06:35:12 -07:00
Andrew Rynhard
0d1c5ac305 feat: add support for Digital Ocean
This adds a Digital Ocean platform implementation.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-28 10:46:15 -07:00
Andrey Smirnov
c3e353aa45 chore: bump tools/pkgs for toolchain refactor
This also pulls in Go 1.13.3

See  talos-systems/toolchain#8, talos-systems/tools#82,
https://github.com/talos-systems/pkgs/pull/69

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-10-25 21:59:41 +03:00
Brad Beam
573cce8d18 feat: Add APId
This PR introduces APId. This service replaces the frontend functionality
previously provided by OSD. The main driver for this is two fold:

1. Create a single purpose application to expose the talos api

2. Make use of code generation to DRY api changes

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-10-25 13:02:33 -05:00
Andrey Smirnov
0d09e6f530 chore: bump tools & pkgs for Go 1.13.2
Ref:

* https://github.com/talos-systems/pkgs/pull/68
* https://github.com/talos-systems/tools/pull/80

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-10-18 00:20:34 +03:00
Brad Beam
c3d43a8171 chore: Update gitmeta to latest release
Fix issue inconsistency between build-container and talos

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-10-16 14:31:37 -07:00
Andrew Rynhard
d430a37e46 refactor: use go 1.13 error wrapping
This removes the github.com/pkg/errors package in favor of the official
error wrapping in go 1.13.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-15 22:20:50 -07:00
Andrew Rynhard
80e3876df5 feat: remove proxyd
We have decided that proxyd is not the best architectue for HA
Kubernetes. Our recommendation to users will be to create a load
balancer instead.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-14 08:11:00 -07:00
Spencer Smith
0d694f88c3 chore: upgrade tools for go v1.13.1
This PR will upgrade the tools container used for building talos.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-10-01 17:15:53 -04:00
Andrew Rynhard
82c706a0fb feat: upgrade Kubernetes to v1.16.0
Brings in Kubernetes v1.16.0.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-19 20:19:29 -07:00
Andrew Rynhard
ab4e058489 feat: upgrade Kubernetes to v1.16.0-rc.2
This brings in the release candidate for Kubernetes v1.16.0.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-16 14:56:55 -07:00
Andrew Rynhard
75746266ce feat: upgrade Kubernetes to v1.16.0-rc.1
This brings in the latest RC of 1.16.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-12 20:20:48 -07:00