226 Commits

Author SHA1 Message Date
Andrew Rynhard
7cf28dc805 refactor: rename ntpd to timed
This renames the ntpd application to timed.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-04-13 15:02:26 -07:00
Andrew Rynhard
a10acd592a chore: address random CI nits
This PR does the following:

- updates the conform config
- cleans up conform scopes
- moves slash commands to the talos-bot
- adds a check list to the pull request template
- disables codecov comments
- uses `BOT_TOKEN` so all actions are performed as the talos-bot user
- adds a `make conformance` target to make it easy for contributors to
check their commit before creating a PR
- bumps golangci-lint to v1.24.0

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-04-13 13:01:14 -07:00
Andrey Smirnov
7fd19fd3b6 feat: upgrade Go to 1.14.2
https://github.com/talos-systems/tools/pull/91

https://github.com/talos-systems/pkgs/pull/114

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-04-09 10:15:58 -07:00
Spencer Smith
8bf05d38ec fix: add hpsa drivers
This PR will add in the drivers for HP Smart Array RAID controllers so
that Talos users can use raid setups.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-04-09 10:59:12 -04:00
Andrew Rynhard
681b1a8cb2 feat: upgrade Linux to v5.5.15
This brings in the latest 5.5 version of Linux.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-04-07 09:06:18 -07:00
Andrew Rynhard
3a89d79f84 fix: add bnx2 and bnx2x firmware
This adds the firmware for the bnx2 and bnx2x drivers.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-04-07 08:09:43 -07:00
Andrew Rynhard
675a0eea0e feat: add BNX drivers
This brings in a kernel with BNX drivers enabled.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-04-03 09:53:56 -07:00
Andrey Smirnov
2294a65972 feat: move bootkube out as full service
This moves `bootkube start` into container which runs pretty much like
any other Talos service.

This resolves issues with bootkube log and provides better isolation.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-03-27 23:07:11 +03:00
Spencer Smith
69e2ebbb15 feat: add usb storage support
This PR will pull in a newer kernel config that enables the
`CONFIG_USB_STORAGE` options for folks wanting to interact with USB
storage devices from talos.

This PR also pulls in a small change to our kernel decompression logic
to try 7z decompression first.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-03-24 15:16:54 -04:00
Andrew Rynhard
c6581fabac feat: build talosctl for ARM v7
This adds an ARM v7 build of `talosctl`.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-03-21 18:35:00 -07:00
Andrew Rynhard
43662e4a24 feat: build talosctl for ARM64
This adds an ARM64 build of `talosctl`.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-03-21 16:40:52 -07:00
Andrew Rynhard
5dbc26c7a3 feat: rename osctl to talosctl
This is a rename of the osctl binary. We decided that talosctl is a
better name for the Talos CLI. This does not break any APIs, but does
make older documentation only accurate for previous versions of Talos.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-03-20 19:07:39 -07:00
Andrey Smirnov
0babc39653 feat: split osctl commands into Talos API and cluster management
This keeps backwards compatibility with `osctl` CLI binary with the
exception of `osctl config generate` which was renamed to `osctl
gen config` to avoid confusion with other `osctl config`
commands which operate on client config, not Talos server config.

Command implementation and helpers were split into subpackages for
cleaner code and more visible boundaries. The resulting binary still
combines commands from both sections into a single binary.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-03-20 22:45:04 +03:00
Andrey Smirnov
a1350aa819 feat: upgrade Go to version 1.14.1
Fixes #1934

See talos-systems/pkgs#106, talos-systems/tools#90

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-03-20 21:42:47 +03:00
Spencer Smith
fa82454be4 chore: fix formatting of imports
This PR cleans up the formatting for various package imports as they
were causing the linter to throw errors.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-03-19 15:06:05 -04:00
Spencer Smith
1cbbf9cd5a feat: update talos base packages
This PR will update the base packages to the latest versions. Updated
packages are:

- ca-certificates
- cni
- iptables
- kernel
- kmod
- libseccomp
- musl
- runc
- socat
- util-linux
- xfsprogs

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-03-17 19:08:13 -04:00
Andrey Smirnov
a068acfbe4 feat: split routerd from apid
New service `routerd` performs exactly single task: based on incoming
API call service name, it routes the requests to the appropriate Talos
service (`networkd`, `osd`, etc.) Service `routerd` listens of file
socket and routes requests to file sockets.

Service `apid` now does single task as well:

* it either fans out request to other `apid` services running on other
nodes and aggregates responses
* or it forwards requests to local `routerd` (when request destination
is local node)

Cons:

* one more proxying layer on request path

Pros:

* more clear service roles
* `routerd` is part of core Talos, services should register with it to
expose their API; no auth in the service (not exposed to the world)
* `apid` might be replaced with other implementation, it depends on TLS infra,
auth, etc.
* `apid` is better segregated from other Talos services (can only access
`routerd`, can't talk to other Talos services directly, so less exposure
in case of a bug)

This change is no-op to the end users.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-03-05 22:05:56 +03:00
Andrey Smirnov
e6dc87dfa4 chore: update pkgs & tools for Go 1.14
See also:

* https://github.com/talos-systems/tools/pull/89
* https://github.com/talos-systems/pkgs/pull/103

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-02-27 01:15:46 +03:00
Andrey Smirnov
923ef4537b test: implement new class of tests: provision tests (upgrades)
This class of tests is included/excluded by build tags, but as it is
pretty different from other integration tests, we build it as separate
executable. Provision tests provision cluster for the test run, perform
some actions and verify results (could be upgrade, reset, scale up/down,
etc.)

There's now framework to implement upgrade tests, first of the tests
tests upgrade from latest 0.3 (0.3.2 at the moment) to current version
of Talos (being built in CI). Tests starts by booting with 0.3
kernel/initramfs, runs 0.3 installer to install 0.3.2 cluster, wait for
bootstrap, followed by upgrade to 0.4 in rolling fashion. As Firecracker
supports bootloader, this boots 0.4 system from boot disk (as installed
by installer).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-02-21 07:04:03 -08:00
Andrew Rynhard
c9a8605f87 chore: move golangci-lint.yaml to .golangci.yml
This allows local runs of golangci-lint to use the default config path.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-02-18 07:10:21 -08:00
Andrey Smirnov
f51e9a14fe chore: build app container images skipping export to host
Container images for `apid`, `networkd`, etc. are now built inside the
buildkit using the `img` tool. This means that all the dependencies are
now controlled in `buildkit` and many more stages can run in parallel
without problems (overwriting content in `_out/images`).

This also simplifies Drone configuration, as we can let buildkit handle
the dependencies. I also enabled more stages to run in parallel.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-02-14 13:17:25 -08:00
Andrew Rynhard
d57598ebe1 chore: update pkgs
This brings in a number of kernel improvements.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-02-14 12:10:38 -08:00
Andrey Smirnov
01d696ed10 chore: update golangci-lint-1.23.3
`gomnd` disabled, as it complains about every number used in the code,
and `wsl` became much more thorough.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-02-04 08:56:39 -08:00
Brad Beam
88df1b50b8 feat(networkd): Add health api
This introduces a health/ready api for networkd. This
will allow us to better determine the state of networkd
and allow for some level of monitoring.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2020-01-29 09:09:27 -06:00
Brad Beam
4e22295324 chore: Clean up generated path for protoc
Came across an issue that mentioned `paths=source_relative` to address
the full go import path generation.

ref: https://github.com/golang/protobuf/issues/515#issuecomment-487988591
Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2020-01-23 13:57:17 -08:00
Andrew Rynhard
4f39907b6e feat: update kernel
This brings in a kernel with VLAN filtering enabled.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-23 07:06:58 -08:00
Andrew Rynhard
a0d8656ca0 chore: use v0.1.0 tools and pkgs
This brings in the official v0.1.0 releases of tools and pkgs.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-20 07:53:08 -08:00
Andrey Smirnov
2bf8540855 test: provision Talos clusters via Firecracker VMs
This is initial PR to push the initial code, it has several known
problems which are going to be addressed in follow-up PRs:

1. there's no "cluster destroy", so the only way to stop the VMs is to
`pkill firecracker`

2. provisioner creates state in `/tmp` and never deletes it, that is
required to keep cluster running when `osctl cluster create` finishes

3. doesn't run any controller process around firecracker to support
reboots/CNI cleanup (vethxyz interfaces are lingering on the host as
they're never cleaned up)

The plan is to create some structure in `~/.talos` to manage cluster
state, e.g. `~/.talos/clusters/<name>` which will contain all the
required files (disk images, file sockets, VM logs, etc.). This
directory structure will also work as a way to detect running clusters
and clean them up.

For point number 3, `osctl cluster create` is going to exec lightweight
process to control the firecracker VM process and to simulate VM reboots
if firecracker finishes cleanly (when VM reboots).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-01-16 00:27:08 +03:00
Andrey Smirnov
810e9b418b chore: bump tools/pkgs for Go 1.13.6
Ref: https://github.com/talos-systems/tools/pull/85,
https://github.com/talos-systems/pkgs/pull/87

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-01-13 20:55:17 +03:00
Andrew Rynhard
e66ac62877 feat: upgrade Linux to v5.4.11
This brings in the latest stable linux.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-12 09:09:57 -08:00
Andrew Rynhard
7edd96947a feat: upgrade Linux to v5.4.10
This brings in the latest stable Linux.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-10 20:51:07 -08:00
Andrew Rynhard
4242acd085 feat: upgrade linux to v5.4.8
This brings in the latest 5.4 kernel.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-08 11:59:05 -06:00
Andrew Rynhard
794d9e6066 chore: update all target in Makefile
We should build the most common things by default.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-06 11:08:27 -08:00
Andrew Rynhard
93ba252e42 fix: use the correct TLD for the container version label
This fixes the container label to use the correct top level domain. It
was mistakenly reverted in a previous commit.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-29 20:15:33 -08:00
Andrew Rynhard
5a7eb631b2 feat: add installer command to installer container
This replaces the entrypoint.sh shell script with a go binary.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-26 06:41:25 -08:00
Andrew Rynhard
e4a1bc3cf9 chore: add help menu to the Makefile
This adds a help  menu to the Makefile. It documents all build
dependencies, and how to get started.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-25 11:11:41 -08:00
Andrew Rynhard
831f5524a1 chore: refactor Makefile to be more DRY
This PR aims to make the Makefile more DRY.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-24 10:48:32 -08:00
Andrew Rynhard
907f87d8e0 feat: upgrade Linux to v5.4.5
This brings in the latest stable version of Linux.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-19 17:43:34 -08:00
Andrew Rynhard
0347286511 feat: upgrade Linux to v5.3.15
This brings in the latest 5.3 version of Linux.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-04 20:15:42 -08:00
Andrew Rynhard
d4c202438c refactor: set CRI config to /etc/cri/containerd.toml
This changes the CRI specific containerd instance's config to a
different path.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-04 19:32:00 -08:00
Andrew Rynhard
1f4c17269d feat: add universal TUN/TAP device driver support
This is required when doing anything with KVM.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-04 15:25:39 -08:00
Andrew Rynhard
1d3cc0038b feat: use containerd-shim-runc-v2
This configures the CRI containerd to use containerd-shim-runc-v2.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-04 14:36:18 -08:00
Andrew Rynhard
43e6703b8b feat: upgrade containerd to v1.3.2
This brings in the latest version of Containerd.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-04 10:19:51 -08:00
Andrew Rynhard
9c0b0a1c46 chore: upgrade packages
This brings in the latest set of packages with the following changes:

- Linux v5.3.14
- Pinned ca-certificates (2019-11-27)

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-01 21:10:19 -08:00
Andrey Smirnov
5b7bea2471 feat: use grpc-proxy in apid
This replaces codegen version of apid proxying with
talos-systems/grpc-proxy based version. Proxying is transparent, it
doesn't require exact information about methods and response types. It
requires some common layout response to enhance it properly with node
metadata or errors.

There should be no signifcant changes to the API with the previous
version, but it's worth mentioning a few changes:

1. grpc.ClientConn is established just once per upstream (either local
service or remote apid instance).

2. When called without `-t` (`targets`), apid proxies immediately down
to local service skipping proxying to itself (as before), which results
in empty node metadata in response (before it had local node IP). Might
revert this later to proxy to itself (?).

3. Streaming APIs are now fully supported with multiple targets, but
message definition doesn't contain `ResponseMetadata`, so streaming APIs
are broken now with targets (needs a fix).

4. Errors are now returned as responses with `Error` field set in
`ResponseMetadata`, this requires client library update and `osctl` to
handle it properly.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-11-29 22:57:25 +03:00
Andrew Rynhard
9ea041c7d9 feat: upgrade packages
This brings in fix patches for:

- ca-certificates
- eudev
- libressl
- libseccomp

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-27 13:24:25 -08:00
Andrew Rynhard
3f49a15c06 feat: enable IMA measurement and appraisal
This updates the kernel to make use of a version that has IMA
measurement and appraisal enabled. It is not yet enforced. Additionally,
this adds the securityfs mount at /sys/kernel/security.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-26 03:15:58 -08:00
Andrew Rynhard
e78e1655f1 feat: upgrade packages
This brings in the following changes:

- Linux 5.3.13
- Containerd 1.3.1

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-25 10:41:47 -08:00
Andrew Rynhard
b2ec56ca92 refactor: rename version label
Since we are standardizing on talos.dev everywhere else, we should align
version label with this.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-23 06:54:51 -08:00
Andrew Rynhard
2dd90a226d refactor: remove CNI bundle
The common pattern is for CNIs to install everything required. I don't
think we need to do this beforehand anymore. If we end up finding that
we do we can always add it back.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-23 06:54:19 -08:00