slipenkomk/valkey
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Delete TLS.md and update README.md about tests (#960)

Browse Source 
Most of the content of TLS.md has already been copied to README.md in
#927.

The description of how to run tests with TLS is moved to
tests/README.md.

Descriptions of the additional scripts runtest-cluster, runtest-sentinel
and runtest-module are added in tests/README.md.

Links to tests/README.md and src/unit/README.md are added in the
top-level README.md along with a brief overview of the `make test-*`
commands.

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
This commit is contained in:
Viktor Söderqvist 2024-08-28 21:17:04 +02:00 committed by Madelyn Olson
parent 314ee9b30f
commit be7f15c7b7
3 changed files with 31 additions and 113 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
README.md
View File

@ -60,12 +60,16 @@ After building Valkey, it is a good idea to test it using:
% make test
If TLS is built, running the tests with TLS enabled (you will need `tcl-tls`
installed):
The above runs the main integration tests. Additional tests are started using:
% ./utils/gen-test-certs.sh
% ./runtest --tls
% make test-unit # Unit tests
% make test-modules # Tests of the module API
% make test-sentinel # Valkey Sentinel integration tests
% make test-cluster # Valkey Cluster integration tests
More about running the integration tests can be found in
[tests/README.md](tests/README.md) and for unit tests, see
[src/unit/README.md](src/unit/README.md).
Fixing build problems with dependencies or cached build options
---------

106
TLS.md
View File

@ -1,106 +0,0 @@
TLS Support
===========
Getting Started
---------------
### Building
To build with TLS support you'll need OpenSSL development libraries (e.g.
libssl-dev on Debian/Ubuntu).
To build TLS support as Valkey built-in:
Run `make BUILD_TLS=yes`.
Or to build TLS as Valkey module:
Run `make BUILD_TLS=module`.
Note that sentinel mode does not support TLS module.
### Tests
To run Valkey test suite with TLS, you'll need TLS support for TCL (i.e.
`tcl-tls` package on Debian/Ubuntu).
1. Run `./utils/gen-test-certs.sh` to generate a root CA and a server
certificate.
2. Run `./runtest --tls` or `./runtest-cluster --tls` to run Valkey and Valkey
Cluster tests in TLS mode.
3. Run `./runtest --tls-module` or `./runtest-cluster --tls-module` to
run Valkey and Valkey cluster tests in TLS mode with Valkey module.
### Running manually
To manually run a Valkey server with TLS mode (assuming `gen-test-certs.sh` was
invoked so sample certificates/keys are available):
For TLS built-in mode:
./src/valkey-server --tls-port 6379 --port 0 \
--tls-cert-file ./tests/tls/valkey.crt \
--tls-key-file ./tests/tls/valkey.key \
--tls-ca-cert-file ./tests/tls/ca.crt
For TLS module mode:
./src/valkey-server --tls-port 6379 --port 0 \
--tls-cert-file ./tests/tls/valkey.crt \
--tls-key-file ./tests/tls/valkey.key \
--tls-ca-cert-file ./tests/tls/ca.crt \
--loadmodule src/valkey-tls.so
To connect to this Valkey server with `valkey-cli`:
./src/valkey-cli --tls \
--cert ./tests/tls/valkey.crt \
--key ./tests/tls/valkey.key \
--cacert ./tests/tls/ca.crt
Specifying `port 0` will disable TCP. It's also possible to have
both TCP and TLS available, but you'll need to assign different ports.
To make a Replica connect to the master using TLS, use `--tls-replication yes`,
and to make Valkey Cluster use TLS across nodes use `--tls-cluster yes`.
Connections
-----------
All socket operations now go through a connection abstraction layer that hides
I/O and read/write event handling from the caller.
**Multi-threading I/O is not currently supported for TLS**, as a TLS connection
needs to do its own manipulation of AE events which is not thread safe. The
solution is probably to manage independent AE loops for I/O threads and longer
term association of connections with threads. This may potentially improve
overall performance as well.
Sync IO for TLS is currently implemented in a hackish way, i.e. making the
socket blocking and configuring socket-level timeout. This means the timeout
value may not be so accurate, and there would be a lot of syscall overhead.
However I believe that getting rid of syncio completely in favor of pure async
work is probably a better move than trying to fix that. For replication it would
probably not be so hard. For cluster keys migration it might be more difficult,
but there are probably other good reasons to improve that part anyway.
To-Do List
----------
- [ ] valkey-benchmark support. The current implementation is a mix of using
hiredis for parsing and basic networking (establishing connections), but
directly manipulating sockets for most actions. This will need to be cleaned
up for proper TLS support. The best approach is probably to migrate to hiredis
async mode.
- [ ] valkey-cli `--slave` and `--rdb` support.
Multi-port
----------
Consider the implications of allowing TLS to be configured on a separate port,
making Valkey listening on multiple ports:
1. Startup banner port notification
2. Proctitle
3. How slaves announce themselves
4. Cluster bus port calculation

26
tests/README.md
View File

@ -21,7 +21,8 @@ enabled using the `--host` and `--port` parameters. When executing against an
external server, tests tagged `external:skip` are skipped.
There are additional runtime options that can further adjust the test suite to
match different external server configurations:
match different external server configurations. All options are listed by
`./runtest --help`. The following table is just a subset of the options:
| Option | Impact |
| -------------------- | -------------------------------------------------------- |
@ -29,7 +30,26 @@ match different external server configurations:
| `--ignore-encoding` | Skip all checks for specific encoding. |
| `--ignore-digest` | Skip key value digest validations. |
| `--cluster-mode` | Run in strict Valkey Cluster compatibility mode. |
| `--large-memory` | Enables tests that consume more than 100mb |
| `--large-memory` | Enables tests that consume more than 100MB |
| `--tls` | Run tests with TLS. See below. |
| `--tls-module` | Run tests with TLS, when TLS support is built as a module. |
| `--help` | Displays the full set of options. |
Running with TLS requires the following preparations:
* Build Valkey is TLS support, e.g. using `make BUILD_TLS=yes`, or `make BUILD_TLS=module`.
* Run `./utils/gen-test-certs.sh` to generate a root CA and a server certificate.
* Install TLS support for TCL, e.g. the `tcl-tls` package on Debian/Ubuntu.
Additional tests
----------------
Not all tests are included in the `./runtest` scripts. Some additional entry points are provided by the following scripts, which support a subset of the options listed above:
* `./runtest-cluster` runs more extensive tests for Valkey Cluster.
Some cluster tests are included in `./runtest`, but not all.
* `./runtest-sentinel` runs tests of Valkey Sentinel.
* `./runtests-module` runs tests of the module API.
Debugging
---------
@ -69,7 +89,7 @@ The following compatibility and capability tags are currently used:
| --------------------- | --------- |
| `external:skip` | Not compatible with external servers. |
| `cluster:skip` | Not compatible with `--cluster-mode`. |
| `large-memory` | Test that requires more than 100mb |
| `large-memory` | Test that requires more than 100MB |
| `tls:skip` | Not compatible with `--tls`. |
| `needs:repl` | Uses replication and needs to be able to `SYNC` from server. |
| `needs:debug` | Uses the `DEBUG` command or other debugging focused commands (like `OBJECT REFCOUNT`). |