Commit Graph

153 Commits

Author SHA1 Message Date
Spencer Smith
6d5bbaf7c8 chore: re-enable e2e for aws clusters
This PR adds in the necessary manifests and fixes to deploy aws clusters
as part of e2e testing.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-11-07 15:32:14 -05:00
Andrey Smirnov
8fdf71789e test: add 'integration-test' to e2e runs
Also refactored `integration-test` build as a generic step to be shared
by basic-integration and e2e-integration steps.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-11-07 06:30:34 -08:00
Spencer Smith
ce7a0e36cc chore: re-enable e2e testing
This PR will re-enable e2e testing by using the new cluster api
bootstrap provider and various infra providers.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-11-05 16:53:38 -05:00
Andrey Smirnov
b0aef2cf22 test: add integration test framework
This is just first steps and core foundation.

It can be used like:

```
make integration.test
osctl cluster create
build/integration.test -test.v
```

This should run the test against the Docker instance.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-11-05 17:21:38 +03:00
Andrew Rynhard
3887b1e5b6 chore: force overwrite of output file
This adds the force option to gzip.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-03 10:31:17 -08:00
Andrew Rynhard
7cd9ba588c chore: remove RAW disk
We need to remove this so that it is not published in a release.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-31 15:35:06 -07:00
Andrew Rynhard
2cad745292 docs: add customization guide
This adds a section on customizing Talos.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-31 14:47:17 -07:00
Brad Beam
ca76ccd4af feat: Add support for creating VMware images
This PR adds support for generating VMware compatible images as an ova.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-10-31 13:39:54 -05:00
Andrew Rynhard
3c6d0135d0 feat: upgrade Kubernetes to 1.16.2
This brings in 1.16.2 modules and bumps the default hyperkube image.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-30 06:35:12 -07:00
Andrew Rynhard
0d1c5ac305 feat: add support for Digital Ocean
This adds a Digital Ocean platform implementation.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-28 10:46:15 -07:00
Andrey Smirnov
c3e353aa45 chore: bump tools/pkgs for toolchain refactor
This also pulls in Go 1.13.3

See  talos-systems/toolchain#8, talos-systems/tools#82,
https://github.com/talos-systems/pkgs/pull/69

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-10-25 21:59:41 +03:00
Brad Beam
573cce8d18 feat: Add APId
This PR introduces APId. This service replaces the frontend functionality
previously provided by OSD. The main driver for this is two fold:

1. Create a single purpose application to expose the talos api

2. Make use of code generation to DRY api changes

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-10-25 13:02:33 -05:00
Andrey Smirnov
0d09e6f530 chore: bump tools & pkgs for Go 1.13.2
Ref:

* https://github.com/talos-systems/pkgs/pull/68
* https://github.com/talos-systems/tools/pull/80

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-10-18 00:20:34 +03:00
Brad Beam
c3d43a8171 chore: Update gitmeta to latest release
Fix issue inconsistency between build-container and talos

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-10-16 14:31:37 -07:00
Andrew Rynhard
d430a37e46 refactor: use go 1.13 error wrapping
This removes the github.com/pkg/errors package in favor of the official
error wrapping in go 1.13.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-15 22:20:50 -07:00
Andrew Rynhard
80e3876df5 feat: remove proxyd
We have decided that proxyd is not the best architectue for HA
Kubernetes. Our recommendation to users will be to create a load
balancer instead.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-14 08:11:00 -07:00
Spencer Smith
0d694f88c3 chore: upgrade tools for go v1.13.1
This PR will upgrade the tools container used for building talos.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-10-01 17:15:53 -04:00
Andrew Rynhard
82c706a0fb feat: upgrade Kubernetes to v1.16.0
Brings in Kubernetes v1.16.0.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-19 20:19:29 -07:00
Andrew Rynhard
ab4e058489 feat: upgrade Kubernetes to v1.16.0-rc.2
This brings in the release candidate for Kubernetes v1.16.0.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-16 14:56:55 -07:00
Andrew Rynhard
75746266ce feat: upgrade Kubernetes to v1.16.0-rc.1
This brings in the latest RC of 1.16.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-12 20:20:48 -07:00
Andrew Rynhard
5ee554128e chore: move from gofumpt to gofumports
The gofumports does everything that gofumpt does with the addition of
formatting imports. This change proposes the use of the `-local` flag so
that we can have imports separated in the following order:

- standard library
- third party
- Talos specific

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-12 07:49:12 -07:00
Andrew Rynhard
2e8b570302 chore: add fmt target
This provides a target that can be useful for developers. It will format
code according to our standards.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-11 15:19:53 -07:00
Spencer Smith
473df84cf6 fix: move to per-platform console setup
This PR will make sure that each platform gets the console settings it
needs by setting them as extra flags in the makefile. This should ensure
that we have console logs flowing properly for each cloud.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-09-10 07:50:34 -07:00
Andrew Rynhard
44dd2fc7c9 chore: remove packer from installer
This moves to making AWS releases align with Azure, and GCP. We no
longer need packer since we will now release an artifact that users can
import.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-09 18:54:37 -07:00
Brad Beam
f21d1244bd test(ci): Add aws for e2e and conformance targets
Add additional scripts and steps to enable doing tests against aws.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-09-09 13:56:19 -05:00
Andrew Rynhard
fd3521649f chore: remove buildkit cache directory
This cache was more important back when builds of Talos took upwards of
40 minutes. Since this is no longer the case, and I have seen
performance issues by mounting a host path into the container, I think
we should drop this.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-04 10:43:41 -07:00
Andrew Rynhard
295cbf9dc6 chore: remove generated raw disk
This was mistakenly removed.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-27 19:08:51 -07:00
Andrew Rynhard
bf8fc1dcbd chore: lint protobuf definitions
This adds linting to our protobuf definitions via prototool.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-27 18:12:36 -07:00
Andrew Rynhard
15cfd42168 chore: upgrade tools
This brings in Go v1.12.9 to address CVEs and bugs.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-26 15:57:30 -07:00
Brad Beam
313c118ad0 refactor(networkd): Replace networkd with a standalone app
This is a major rewrite of our network subsystem.

- This changes networkd to run as a standalone app versus internal goroutine
- This changes out the netlink package with the more idiomatic netlink/rtnetlink
  packages
- This changes the initial network bootstrap/discovery from using a single
  interface to attempting to bring up all interfaces
- This moves us back on to the upstream dhcp library

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-08-21 13:24:51 -05:00
Andrew Rynhard
0af1eba159 refactor: add more runtime modes
In order to DRY up all installation methods and mount methods, this PR
introduces a few more runtime modes. The modes are then used to
determine the strategy for creating and or mounting the paritions.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-19 20:23:45 -07:00
Brad Beam
03228c7401 chore(ci): Only push latest tags if branch is master.
Should prevent flakes when we merge fixes on release branches where they unintentionally
get tagged as `latest`.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-08-17 16:03:01 -05:00
Brad Beam
af47edf1ad chore: Make losetup atomic during installation
This should fix a race conditions where two independent image creation steps
run `losetup -f` and discover the same 'next available' loopback device and
attempt to use it.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-08-17 15:23:42 -05:00
Andrew Rynhard
7970f977b7 chore: add markdownlint
This will give us a standard tool for linting Markdown files.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-17 03:53:52 -07:00
Andrew Rynhard
15408963ac chore: update tools image
This image upgrades golang to v1.12.8 to address security
vulnerabilities.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-14 19:07:52 -07:00
Andrew Rynhard
09693a26c9 chore: update go modules to use Kubernetes v1.16.0-alpha.3
This is not ideal, but it works. We essentially need to start using
replace statements in order to pull in the modules we need.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-14 15:34:09 -07:00
Andrew Rynhard
a420b85b07 chore: run unique E2E tests
In order to run more than one instance of E2E testing at a time, we need
to ensure that all resources are unique to the run.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-12 14:14:08 -07:00
Andrew Rynhard
2ee769d19e chore: add image test step
Instead of building platform specific images in the default pipeline, we
should build just one image as part of our basic testing to make sure
installations work as expected.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-11 10:51:33 -07:00
Andrew Rynhard
817380bad6 chore: refactor the Jsonnet file
This change improves the drone jsonnet file by making it more DRY and
structuring it in a way that makes it much easier to follow.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-11 09:23:30 -07:00
Andrew Rynhard
8786916fd0 chore: build drone YAML via jsonnet
This PR aims to DRY the drone config file by using Jsonnet to generate
it.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-09 22:30:37 -07:00
Andrey Smirnov
2227d1f6b6 chore: add race-enabled test run
As Go race detector doesn't work under libmuscl, use stock glibc-based
golang container.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-08-09 17:12:34 +03:00
Andrew Rynhard
3383e72d37 chore: remove machined from rootfs target
The machined dependency is not needed in the rootfs target. The
dependency is handled by buildkit.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-08 12:26:30 -07:00
Brad Beam
bfc1646cd9 chore(ci): Add e2e promotion pipeline
Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-08-08 11:27:57 -05:00
Spencer Smith
9e02c77c0a chore: add azure e2e testing
This PR will allow us to run an azure e2e test in parallel with our
current GCE implementation.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-08-07 12:16:32 -04:00
Spencer Smith
38dfddbab3 feat: break up osctl cluster create and basic/e2e tests
This PR will break cluster create apart from the other steps in
integration tests. It will allow us to run the cluster create, then use
it for parallel e2e builds in different cloud environments.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-08-01 10:55:24 -04:00
Andrey Smirnov
3024c26a55 chore: update dockerfile/buildkit versions
New buildkit release: https://github.com/moby/buildkit/releases/tag/v0.6.0

New release was published for buildkit's dockerfile:
https://github.com/moby/buildkit/releases/tag/dockerfile%2F1.1.2-experimental,
so we can stick to release version now.

These releases include fixes/implementation for `RUN --security=insecure`.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-08-01 01:05:42 +03:00
Andrew Rynhard
12486ef0e2 chore: remove rootfs output param
This removes the `--output` flag from the rootfs target. With the output
specified it was outputting the file directory structure to the build
directory.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-07-29 20:26:54 -07:00
Andrew Rynhard
e63c882b89 refactor: split machined into phases
This change aims to standardize the boot process. It introduces the
concept of a phase, which is comprised of tasks. Phases are ran in serial and
the tasks that make up a phase are ran concurrently.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-07-29 12:40:03 -07:00
Andrew Rynhard
0ec17e4169 feat: run rootfs from squashfs
This change moves the rootfs to a squashfs image.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-07-25 08:38:31 -07:00
Andrey Smirnov
8c59adb9dc chore: allow to run tests only for specified packages
This allows to do `make test TESTPKGS=./internal/app/machined`.

Also update Dockerfile slug as
https://github.com/moby/buildkit/pull/1081 was merged into master.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-07-23 22:17:22 +03:00