440 Commits

Author SHA1 Message Date
Spencer Smith
718fd28264 chore: update kernel to support CONFIG_CRYPTO_USER_API_HASH
This PR pulls in a newer kernel with CONFIG_CRYPTO_USER_API_HASH
support. This is needed for cilium 1.7.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-06-17 13:47:34 -07:00
Andrew Rynhard
65d02f9ddb feat: add open-iscsi
A number of storage providers depend on having open-iscsi installed
on the host. This adds open-iscsi along with any dependencies.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-06-14 14:16:27 -07:00
Andrew Rynhard
2ab2ec296e fix: update LVM2
This brings in an updated build of LVM2 that addresses a
segfault error. We were also missing libaio.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-06-12 13:18:03 -07:00
Spencer Smith
0adfd81bd4 feat: update linux kernel (with 32 bit support) and talos pkgs for v0.6
This PR bundles three commits from the pkgs repo:

- c09825e05a
- 393291ddf2
- 908b75b81f

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-06-11 15:43:56 -07:00
Andrew Rynhard
336f983c21 docs: add v0.6 docs
This adds the documentation for v0.6 and removes v0.3 since
it is no longer supported.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-06-10 10:39:38 -07:00
Andrew Rynhard
90efb0a74e feat: add LVM2
This adds LVM2 to the rootfs. This is required by rook.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-05-26 06:07:16 -07:00
Andrey Smirnov
2fb00344ab chore: upgrade Go to 1.14.3 and use toolchain for race detector
With Go 1.14.3 we can run race-enabled code on muslc, so this opens path
to run unit-tests-race under Talos environment with rootfs, enabling all
the tests to run under race detector.

Also fixed the tests run by specifying platform in the test environment.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-05-25 08:35:11 -07:00
Andrew Rynhard
00b7176a8a feat: upgrade Linux to v5.6.13
This brings in the latest version of Linux.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-05-18 14:41:59 -07:00
Andrew Rynhard
cc7ee56852 feat: enable rpfilter
This brings in an updated kernel that enables rpfilter.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-05-11 08:42:49 -07:00
Andrew Rynhard
fa5299b924 fix: update kernel package
This updates packages to include a kernel built with
CONFIG_NETFILTER_XT_MATCH_SOCKET=y since it is recommended
by cilium.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-05-08 06:04:39 -07:00
Andrew Rynhard
1c20a32ab4 chore: pin markdown linting libraries
A new version of markdownlint-cli was released that breaks the
lint-markdown target. This pins all the libraries we use for
linting markdown files to specific versions that are known to
work.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-05-07 10:19:01 -04:00
Andrew Rynhard
0204feeb0b chore: fix prototool lint
We were never copying the protobuf files in, so prototool was never
really linting anything. This ensures that the `api` directory is copied
in.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-04-29 08:14:39 -07:00
Andrew Rynhard
37a7906f09 chore: fix markdown linting issues
This fixes random markdown linting issues. The previous `sentences-per-line`
library seems to be broken now, and unmaintained. This moves to using
`textlint` instead.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-04-26 20:38:03 -07:00
Andrew Rynhard
5e822d02a6 chore: update pkgs tag to v0.2.0
This change updates the pkgs tag to v0.2.0. There is no difference in
the content of the packages.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-04-14 08:58:31 -07:00
Andrew Rynhard
7cf28dc805 refactor: rename ntpd to timed
This renames the ntpd application to timed.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-04-13 15:02:26 -07:00
Andrew Rynhard
a10acd592a chore: address random CI nits
This PR does the following:

- updates the conform config
- cleans up conform scopes
- moves slash commands to the talos-bot
- adds a check list to the pull request template
- disables codecov comments
- uses `BOT_TOKEN` so all actions are performed as the talos-bot user
- adds a `make conformance` target to make it easy for contributors to
check their commit before creating a PR
- bumps golangci-lint to v1.24.0

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-04-13 13:01:14 -07:00
Andrey Smirnov
7fd19fd3b6 feat: upgrade Go to 1.14.2
https://github.com/talos-systems/tools/pull/91

https://github.com/talos-systems/pkgs/pull/114

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-04-09 10:15:58 -07:00
Spencer Smith
8bf05d38ec fix: add hpsa drivers
This PR will add in the drivers for HP Smart Array RAID controllers so
that Talos users can use raid setups.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-04-09 10:59:12 -04:00
Andrew Rynhard
681b1a8cb2 feat: upgrade Linux to v5.5.15
This brings in the latest 5.5 version of Linux.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-04-07 09:06:18 -07:00
Andrew Rynhard
3a89d79f84 fix: add bnx2 and bnx2x firmware
This adds the firmware for the bnx2 and bnx2x drivers.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-04-07 08:09:43 -07:00
Andrew Rynhard
675a0eea0e feat: add BNX drivers
This brings in a kernel with BNX drivers enabled.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-04-03 09:53:56 -07:00
Andrey Smirnov
2294a65972 feat: move bootkube out as full service
This moves `bootkube start` into container which runs pretty much like
any other Talos service.

This resolves issues with bootkube log and provides better isolation.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-03-27 23:07:11 +03:00
Spencer Smith
69e2ebbb15 feat: add usb storage support
This PR will pull in a newer kernel config that enables the
`CONFIG_USB_STORAGE` options for folks wanting to interact with USB
storage devices from talos.

This PR also pulls in a small change to our kernel decompression logic
to try 7z decompression first.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-03-24 15:16:54 -04:00
Andrew Rynhard
c6581fabac feat: build talosctl for ARM v7
This adds an ARM v7 build of `talosctl`.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-03-21 18:35:00 -07:00
Andrew Rynhard
43662e4a24 feat: build talosctl for ARM64
This adds an ARM64 build of `talosctl`.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-03-21 16:40:52 -07:00
Andrew Rynhard
5dbc26c7a3 feat: rename osctl to talosctl
This is a rename of the osctl binary. We decided that talosctl is a
better name for the Talos CLI. This does not break any APIs, but does
make older documentation only accurate for previous versions of Talos.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-03-20 19:07:39 -07:00
Andrey Smirnov
0babc39653 feat: split osctl commands into Talos API and cluster management
This keeps backwards compatibility with `osctl` CLI binary with the
exception of `osctl config generate` which was renamed to `osctl
gen config` to avoid confusion with other `osctl config`
commands which operate on client config, not Talos server config.

Command implementation and helpers were split into subpackages for
cleaner code and more visible boundaries. The resulting binary still
combines commands from both sections into a single binary.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-03-20 22:45:04 +03:00
Andrey Smirnov
a1350aa819 feat: upgrade Go to version 1.14.1
Fixes #1934

See talos-systems/pkgs#106, talos-systems/tools#90

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-03-20 21:42:47 +03:00
Spencer Smith
fa82454be4 chore: fix formatting of imports
This PR cleans up the formatting for various package imports as they
were causing the linter to throw errors.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-03-19 15:06:05 -04:00
Spencer Smith
1cbbf9cd5a feat: update talos base packages
This PR will update the base packages to the latest versions. Updated
packages are:

- ca-certificates
- cni
- iptables
- kernel
- kmod
- libseccomp
- musl
- runc
- socat
- util-linux
- xfsprogs

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-03-17 19:08:13 -04:00
Andrey Smirnov
a068acfbe4 feat: split routerd from apid
New service `routerd` performs exactly single task: based on incoming
API call service name, it routes the requests to the appropriate Talos
service (`networkd`, `osd`, etc.) Service `routerd` listens of file
socket and routes requests to file sockets.

Service `apid` now does single task as well:

* it either fans out request to other `apid` services running on other
nodes and aggregates responses
* or it forwards requests to local `routerd` (when request destination
is local node)

Cons:

* one more proxying layer on request path

Pros:

* more clear service roles
* `routerd` is part of core Talos, services should register with it to
expose their API; no auth in the service (not exposed to the world)
* `apid` might be replaced with other implementation, it depends on TLS infra,
auth, etc.
* `apid` is better segregated from other Talos services (can only access
`routerd`, can't talk to other Talos services directly, so less exposure
in case of a bug)

This change is no-op to the end users.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-03-05 22:05:56 +03:00
Andrey Smirnov
e6dc87dfa4 chore: update pkgs & tools for Go 1.14
See also:

* https://github.com/talos-systems/tools/pull/89
* https://github.com/talos-systems/pkgs/pull/103

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-02-27 01:15:46 +03:00
Andrey Smirnov
923ef4537b test: implement new class of tests: provision tests (upgrades)
This class of tests is included/excluded by build tags, but as it is
pretty different from other integration tests, we build it as separate
executable. Provision tests provision cluster for the test run, perform
some actions and verify results (could be upgrade, reset, scale up/down,
etc.)

There's now framework to implement upgrade tests, first of the tests
tests upgrade from latest 0.3 (0.3.2 at the moment) to current version
of Talos (being built in CI). Tests starts by booting with 0.3
kernel/initramfs, runs 0.3 installer to install 0.3.2 cluster, wait for
bootstrap, followed by upgrade to 0.4 in rolling fashion. As Firecracker
supports bootloader, this boots 0.4 system from boot disk (as installed
by installer).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-02-21 07:04:03 -08:00
Andrew Rynhard
c9a8605f87 chore: move golangci-lint.yaml to .golangci.yml
This allows local runs of golangci-lint to use the default config path.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-02-18 07:10:21 -08:00
Andrey Smirnov
f51e9a14fe chore: build app container images skipping export to host
Container images for `apid`, `networkd`, etc. are now built inside the
buildkit using the `img` tool. This means that all the dependencies are
now controlled in `buildkit` and many more stages can run in parallel
without problems (overwriting content in `_out/images`).

This also simplifies Drone configuration, as we can let buildkit handle
the dependencies. I also enabled more stages to run in parallel.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-02-14 13:17:25 -08:00
Andrew Rynhard
d57598ebe1 chore: update pkgs
This brings in a number of kernel improvements.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-02-14 12:10:38 -08:00
Andrey Smirnov
01d696ed10 chore: update golangci-lint-1.23.3
`gomnd` disabled, as it complains about every number used in the code,
and `wsl` became much more thorough.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-02-04 08:56:39 -08:00
Brad Beam
88df1b50b8 feat(networkd): Add health api
This introduces a health/ready api for networkd. This
will allow us to better determine the state of networkd
and allow for some level of monitoring.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2020-01-29 09:09:27 -06:00
Brad Beam
4e22295324 chore: Clean up generated path for protoc
Came across an issue that mentioned `paths=source_relative` to address
the full go import path generation.

ref: https://github.com/golang/protobuf/issues/515#issuecomment-487988591
Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2020-01-23 13:57:17 -08:00
Andrew Rynhard
4f39907b6e feat: update kernel
This brings in a kernel with VLAN filtering enabled.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-23 07:06:58 -08:00
Andrew Rynhard
a0d8656ca0 chore: use v0.1.0 tools and pkgs
This brings in the official v0.1.0 releases of tools and pkgs.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-20 07:53:08 -08:00
Andrey Smirnov
2bf8540855 test: provision Talos clusters via Firecracker VMs
This is initial PR to push the initial code, it has several known
problems which are going to be addressed in follow-up PRs:

1. there's no "cluster destroy", so the only way to stop the VMs is to
`pkill firecracker`

2. provisioner creates state in `/tmp` and never deletes it, that is
required to keep cluster running when `osctl cluster create` finishes

3. doesn't run any controller process around firecracker to support
reboots/CNI cleanup (vethxyz interfaces are lingering on the host as
they're never cleaned up)

The plan is to create some structure in `~/.talos` to manage cluster
state, e.g. `~/.talos/clusters/<name>` which will contain all the
required files (disk images, file sockets, VM logs, etc.). This
directory structure will also work as a way to detect running clusters
and clean them up.

For point number 3, `osctl cluster create` is going to exec lightweight
process to control the firecracker VM process and to simulate VM reboots
if firecracker finishes cleanly (when VM reboots).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-01-16 00:27:08 +03:00
Andrey Smirnov
810e9b418b chore: bump tools/pkgs for Go 1.13.6
Ref: https://github.com/talos-systems/tools/pull/85,
https://github.com/talos-systems/pkgs/pull/87

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-01-13 20:55:17 +03:00
Andrew Rynhard
e66ac62877 feat: upgrade Linux to v5.4.11
This brings in the latest stable linux.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-12 09:09:57 -08:00
Andrew Rynhard
7edd96947a feat: upgrade Linux to v5.4.10
This brings in the latest stable Linux.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-10 20:51:07 -08:00
Andrew Rynhard
4242acd085 feat: upgrade linux to v5.4.8
This brings in the latest 5.4 kernel.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-08 11:59:05 -06:00
Andrew Rynhard
794d9e6066 chore: update all target in Makefile
We should build the most common things by default.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-06 11:08:27 -08:00
Andrew Rynhard
93ba252e42 fix: use the correct TLD for the container version label
This fixes the container label to use the correct top level domain. It
was mistakenly reverted in a previous commit.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-29 20:15:33 -08:00
Andrew Rynhard
5a7eb631b2 feat: add installer command to installer container
This replaces the entrypoint.sh shell script with a go binary.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-26 06:41:25 -08:00
Andrew Rynhard
e4a1bc3cf9 chore: add help menu to the Makefile
This adds a help  menu to the Makefile. It documents all build
dependencies, and how to get started.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-25 11:11:41 -08:00