131 Commits

Author SHA1 Message Date
Andrew Rynhard
9ea041c7d9 feat: upgrade packages
This brings in fix patches for:

- ca-certificates
- eudev
- libressl
- libseccomp

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-27 13:24:25 -08:00
Andrew Rynhard
3f49a15c06 feat: enable IMA measurement and appraisal
This updates the kernel to make use of a version that has IMA
measurement and appraisal enabled. It is not yet enforced. Additionally,
this adds the securityfs mount at /sys/kernel/security.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-26 03:15:58 -08:00
Andrew Rynhard
e78e1655f1 feat: upgrade packages
This brings in the following changes:

- Linux 5.3.13
- Containerd 1.3.1

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-25 10:41:47 -08:00
Andrew Rynhard
b2ec56ca92 refactor: rename version label
Since we are standardizing on talos.dev everywhere else, we should align
version label with this.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-23 06:54:51 -08:00
Andrew Rynhard
2dd90a226d refactor: remove CNI bundle
The common pattern is for CNIs to install everything required. I don't
think we need to do this beforehand anymore. If we end up finding that
we do we can always add it back.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-23 06:54:19 -08:00
Andrew Rynhard
4680f66bc5 docs: add autogenerated config reference
This adds a small program to parse our config structs and generate
markdown from them. This will allow us to enforce a standard and require
documentation for fields as they get added.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-11 08:38:39 -08:00
Andrey Smirnov
6231b7db3c chore: run gofumports after protoc-gen
This fixes import order and guarantess clean diff after `make generate`.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-11-09 00:20:45 +03:00
Brad Beam
7897374ff1 feat: Add support for streaming apis in apid
This brings in the recent updates to protoc-gen-proxy to allow support
for proxying streaming api requests. We artificially limit it to only the first
target specified in the list while we work through what multi target stream
support looks like.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-11-08 14:22:30 -06:00
Andrey Smirnov
cdda81df66 test: add k8s integration tests
Once again, mostly groundwork and one simple test for node versions.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-11-06 17:08:44 -08:00
Andrey Smirnov
551fa45d33 test: add CLI integration test
This starts with a very simple test for `osctl version` using regexps as
output of the command depends a lot on current version.

We might use more of 'gold' matches for other commands potentially.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-11-05 17:59:23 -08:00
Andrey Smirnov
b0aef2cf22 test: add integration test framework
This is just first steps and core foundation.

It can be used like:

```
make integration.test
osctl cluster create
build/integration.test -test.v
```

This should run the test against the Docker instance.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-11-05 17:21:38 +03:00
Andrew Rynhard
f43e42d845 chore: install customization requirements with ONBUILD
There is no need for these packages to be in the base image. This moves
to installing them using ONBUILD.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-03 22:51:05 -08:00
Andrew Rynhard
2cad745292 docs: add customization guide
This adds a section on customizing Talos.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-31 14:47:17 -07:00
Brad Beam
ca76ccd4af feat: Add support for creating VMware images
This PR adds support for generating VMware compatible images as an ova.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-10-31 13:39:54 -05:00
Andrew Rynhard
f26a4ce040 chore: update pkgs SHA
This brings in pkgs that have been built using a prefix of /usr.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-29 07:10:25 -07:00
Andrey Smirnov
c3e353aa45 chore: bump tools/pkgs for toolchain refactor
This also pulls in Go 1.13.3

See  talos-systems/toolchain#8, talos-systems/tools#82,
https://github.com/talos-systems/pkgs/pull/69

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-10-25 21:59:41 +03:00
Brad Beam
573cce8d18 feat: Add APId
This PR introduces APId. This service replaces the frontend functionality
previously provided by OSD. The main driver for this is two fold:

1. Create a single purpose application to expose the talos api

2. Make use of code generation to DRY api changes

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-10-25 13:02:33 -05:00
Brad Beam
251ab16e07 feat: Add node metadata wrapper to machine api
- Added common.proto to host NodeMetadata
- go_package names were fixed up so imports are generated with the proper
  package names
- fixed up build work (dockerfile) to prevent copying the previously
  generated go proto files. This fixes a bug where we could incorrectly
  copy the previously generated protobuf instead of a new one generated
  at an incorrect location/name/etc.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-10-22 14:42:34 -05:00
Brad Beam
5b58d8b7df chore: Rename proto files into more appropriate names
This is partially driven by the upcoming api changes, but when we tell protoc to look for api.proto,
itll find the first match in the includes(`-I`) directive.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-10-21 14:08:41 -05:00
Andrey Smirnov
0d09e6f530 chore: bump tools & pkgs for Go 1.13.2
Ref:

* https://github.com/talos-systems/pkgs/pull/68
* https://github.com/talos-systems/tools/pull/80

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-10-18 00:20:34 +03:00
Brad Beam
e6bf92ce31 feat(osd): Enable hitting multiple OSD endpoints
This enables the ability to specify additional <talos> endpoints to connect to
to pull back data.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-10-16 15:30:25 -05:00
Andrey Smirnov
e6f980463f chore: bump golangci-lint to 1.21
This release comes with support for caching lint results. It should
allow for speed ups when some packages don't change between the runs, so
cached information about packages might be used instead of running
linter collect phase on the package.

In my non-scientific test:

1. Fresh: 40s
2. One file change: 15s

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-10-15 14:01:58 -07:00
Andrew Rynhard
10b6202c4f refactor: improve metal platform
This brings in a few minor improvements to the metal platform. The first
is to use talos.config=metal-iso to indicate that the machine's config
can be found in an ISO image. The second is a fix to ensure that /mnt
exists.

This adds support for creating more than one node using the qemu-boot.sh
script.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-14 22:05:56 -07:00
Andrew Rynhard
80e3876df5 feat: remove proxyd
We have decided that proxyd is not the best architectue for HA
Kubernetes. Our recommendation to users will be to create a load
balancer instead.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-14 08:11:00 -07:00
Andrew Rynhard
fef151748b feat: use the unified pkgs repo artifacts
This moves to using a single revision of pkgs. It includes a few
changes:

- kernel with KVM host support
- containerd v1.3.0

This change brings in a kernel with host KVM support. This will allow us
to use VMs within Talos for things like integrations tests. This also
allows users to do things with KVM as they see fit.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-14 07:18:17 -07:00
Andrey Smirnov
bb5f5cc754 chore: bump golangci-lint to 1.20
Memory usage reduced around 8-10x: now it stays stable at 1GB.

I disabled some of the new linters, and one rule which is violated a
lot.

I might make sense to go back and enable `wsl` fixing all the issues
(leaving that for another PR).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-10-09 22:21:08 +03:00
Andrew Rynhard
b29391f0be feat: use bootkube for cluster creation
This replaces kubeadm with bootkube.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-07 17:17:57 -07:00
Spencer Smith
dfe429a8b3 chore: bump kernel to 5.2.18
This PR pulls in the latest kernel container, containing v5.2.18.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-10-01 17:00:52 -04:00
Andrew Rynhard
8f10647d3f fix: set extra kernel args for all platforms
This change ensures that the installer has access to the machine config
so that it can set the extra kernel arguments when installing.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-23 11:50:13 -07:00
Andrew Rynhard
79672fb251 chore: add version label to installer image
This adds a label to the installer image that indicates the version. We
can build automation around this in a number of different ways, but one
of the use cases we have immediately is to use this label to determine
which version of Talos is at a given channel. For example, if we were to
implement an "edge" channel, we could periodically check for an image
with the tag "edge" and use the version label to determine if a node is
running the current version of edge. Even if we don't use the labels for our
channel implementation, its' still useful information to have.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-23 02:30:22 -07:00
Andrew Rynhard
d4260f6918 fix: add kerenel config required by Cilium
This adds CONFIG_NETFILTER_XT_TARGET_CT=y to the kernel.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-20 09:58:48 -07:00
Andrew Rynhard
82c706a0fb feat: upgrade Kubernetes to v1.16.0
Brings in Kubernetes v1.16.0.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-19 20:19:29 -07:00
Andrew Rynhard
6efd6fbe08 chore: move gRPC API to public
In order for other projects to make use of our APIs, they must not
reside underneath the internal directory. This moves the protobuf
definitions to a top-level "api" directory and scopes them according to
their domain. This change also removes generated code from the gitignore
file so that users don't have to generate the code themseleves.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-19 08:55:13 -07:00
Andrew Rynhard
ab4e058489 feat: upgrade Kubernetes to v1.16.0-rc.2
This brings in the release candidate for Kubernetes v1.16.0.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-16 14:56:55 -07:00
Andrew Rynhard
75746266ce feat: upgrade Kubernetes to v1.16.0-rc.1
This brings in the latest RC of 1.16.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-12 20:20:48 -07:00
Andrew Rynhard
5ee554128e chore: move from gofumpt to gofumports
The gofumports does everything that gofumpt does with the addition of
formatting imports. This change proposes the use of the `-local` flag so
that we can have imports separated in the following order:

- standard library
- third party
- Talos specific

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-12 07:49:12 -07:00
Andrey Smirnov
980829708e chore: upgrade golancgi-lint to 1.18.0
New linter 'funlen' was disabled as too many functions break the default
limit, but might be considered for the future.

To limit peak memory usage, `GOGC=50` was added to the golangci-lint run
to make Go's garbage collector more aggressive. With this setting peak
seems to be around 8Gb.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-09-11 15:18:57 -07:00
Andrew Rynhard
2955428850 chore: format code with gofumpt
The gofumpt linter is a stricter drop-in replacement for gofmt. The
rules are ones that I strongly agree with and I think it would be better
if we added this linter instead of nit picking every PR.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-11 11:03:29 -07:00
Andrew Rynhard
38690d72df chore: remove unneeded packages
This removes packages we don't need anymore.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-10 08:12:07 -07:00
Andrew Rynhard
44dd2fc7c9 chore: remove packer from installer
This moves to making AWS releases align with Azure, and GCP. We no
longer need packer since we will now release an artifact that users can
import.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-09 18:54:37 -07:00
Brad Beam
1373806165 fix(init): Enable containerd subreaper
Should take care of our issue with Zombies

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-08-30 14:32:13 -07:00
Andrey Smirnov
029374f07d chore: disable go test result cache
Go by default caches unit-tests results via build cache, so if source
code doesn't have any changes, test results are cached on package level.
As our unit-tests are not that pure and depend on the environment, it
would be more helpful to make sure all the unit-tests during each build.

Setting number of test runs to one disable test result cache (but build
cache is still being used).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-08-30 22:03:00 +03:00
Andrey Smirnov
de49903a5f chore: fix location of Go build cache mount for unit-tests-race
This step is based on `golang` image, so `GOCACHE` is set in a bit of a
different way.

No big deal, but should speed up subsequent runs a bit.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-08-29 16:35:14 -07:00
Andrew Rynhard
bf8fc1dcbd chore: lint protobuf definitions
This adds linting to our protobuf definitions via prototool.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-27 18:12:36 -07:00
Spencer Smith
739e232896 feat: upgrade kubernetes to v1.16.0-beta.1
This PR will upgrade to the latest beta of v1.16 in order to get us
closer to catching the v1.16.0 release as soon as it drops.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-08-27 13:25:33 -04:00
Brad Beam
692571bdec feat(networkd): Add grpc endpoint
Allows us to list routes and interface details

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-08-25 19:48:08 -07:00
Brad Beam
313c118ad0 refactor(networkd): Replace networkd with a standalone app
This is a major rewrite of our network subsystem.

- This changes networkd to run as a standalone app versus internal goroutine
- This changes out the netlink package with the more idiomatic netlink/rtnetlink
  packages
- This changes the initial network bootstrap/discovery from using a single
  interface to attempting to bring up all interfaces
- This moves us back on to the upstream dhcp library

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-08-21 13:24:51 -05:00
Andrew Rynhard
8c73c38b8a chore: enforce one sentence per line in Markdown files
This is widely considered best practice, we should enforce it.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-17 10:15:27 -07:00
Andrew Rynhard
7970f977b7 chore: add markdownlint
This will give us a standard tool for linting Markdown files.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-17 03:53:52 -07:00
Brad Beam
76a9c15044 feat: Add gRPC server for ntp
Part of the API refactor; this introduces a gRPC server for ntp.
This allows the ability to query node time and check time against
specific ntp servers.

This refactor also moves the ntp functionality into a sub package for
better project organization.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-08-16 09:46:43 -07:00