pve-docs/certificate-management.adoc

[[sysadmin_certificate_management]]
Certificate Management
----------------------
ifdef::wiki[]
:pve-toplevel:
endif::wiki[]


Certificates for communication within the cluster
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Each {PVE} cluster creates its own (self-signed) Certificate Authority (CA) and
generates a certificate for each node which gets signed by the aforementioned
CA. These certificates are used for encrypted communication with the cluster's
`pveproxy` service and the Shell/Console feature if SPICE is used.

The CA certificate and key are stored in the xref:chapter_pmxcfs[Proxmox Cluster File System (pmxcfs)].


Certificates for API and web GUI
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The REST API and web GUI are provided by the `pveproxy` service, which runs on
each node.

You have the following options for the certificate used by `pveproxy`:

1. By default the node-specific certificate in
`/etc/pve/nodes/NODENAME/pve-ssl.pem` is used. This certificate is signed by
the cluster CA and therefore not automatically trusted by browsers and
operating systems.
2. use an externally provided certificate (e.g. signed by a commercial CA).
3. use ACME (Let's Encrypt) to get a trusted certificate with automatic
renewal, this is also integrated in the {pve} API and Webinterface.

For options 2 and 3 the file `/etc/pve/local/pveproxy-ssl.pem` (and
`/etc/pve/local/pveproxy-ssl.key`, which needs to be without password) is used.

NOTE: Keep in mind that `/etc/pve/local` is a node specific symlink to
`/etc/pve/nodes/NODENAME`.

Certificates are managed with the {PVE} Node management command
(see the `pvenode(1)` manpage).

WARNING: Do not replace or manually modify the automatically generated node
certificate files in `/etc/pve/local/pve-ssl.pem` and
`/etc/pve/local/pve-ssl.key` or the cluster CA files in
`/etc/pve/pve-root-ca.pem` and `/etc/pve/priv/pve-root-ca.key`.


Getting trusted certificates via ACME
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

{PVE} includes an implementation of the **A**utomatic **C**ertificate
**M**anagement **E**nvironment **ACME** protocol, allowing {pve} admins to
interface with Let's Encrypt for easy setup of trusted TLS certificates which
are accepted out of the box on most modern operating systems and browsers.

Currently the two ACME endpoints implemented are Let's Encrypt (LE) and its
staging environment (see https://letsencrypt.org). Our ACME client supports
validation of `http-01` challenges using a built-in webserver and validation of
`dns-01` challenges using a DNS plugin.

Because of https://letsencrypt.org/docs/rate-limits/[rate-limits] you should use
LE `staging` for experiments.

ACME Plugin configuration is stored in `/etc/pve/priv/acme/plugins.cfg`. There
is always an implicitly configured `standalone` plugin for validating `http-01`
challenges via the built-in webserver spawned on port 80.

There are a few prerequisites to use Let's Encrypt:

* You have to accept the ToS of Let's Encrypt to register an account.

For `http-01` challenges:

* **Port 80** of the node needs to be reachable from the internet.
* There **must** be no other listener on port 80.
* The requested (sub)domain needs to resolve to a public IP of the Node.

For `dns-01` challenges:

* DNS needs to be handled by a server that allows automatic provisioning of
TXT records

At the moment the GUI uses only the default ACME account.

.Example: Sample `pvenode` invocation for using Let's Encrypt certificates

----
root@proxmox:~# pvenode acme account register default mail@example.invalid
Directory endpoints:
0) Let's Encrypt V2 (https://acme-v02.api.letsencrypt.org/directory)
1) Let's Encrypt V2 Staging (https://acme-staging-v02.api.letsencrypt.org/directory)
2) Custom
Enter selection:
1

Attempting to fetch Terms of Service from 'https://acme-staging-v02.api.letsencrypt.org/directory'..
Terms of Service: https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf
Do you agree to the above terms? [y|N]y

Attempting to register account with 'https://acme-staging-v02.api.letsencrypt.org/directory'..
Generating ACME account key..
Registering ACME account..
Registration successful, account URL: 'https://acme-staging-v02.api.letsencrypt.org/acme/acct/xxxxxxx'
Task OK
root@proxmox:~# pvenode acme account list
default
root@proxmox:~# pvenode config set --acme domains=example.invalid
root@proxmox:~# pvenode acme cert order
Loading ACME account details
Placing ACME order
Order URL: https://acme-staging-v02.api.letsencrypt.org/acme/order/xxxxxxxxxxxxxx

Getting authorization details from
'https://acme-staging-v02.api.letsencrypt.org/acme/authz/xxxxxxxxxxxxxxxxxxxxx-xxxxxxxxxxxxx-xxxxxxx'
... pending!
Setting up webserver
Triggering validation
Sleeping for 5 seconds
Status is 'valid'!

All domains validated!

Creating CSR
Finalizing order
Checking order status
valid!

Downloading certificate
Setting pveproxy certificate and key
Restarting pveproxy
Task OK
----

Switching from the `staging` to the regular ACME directory
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Changing the ACME directory for an account is unsupported. If you want to switch
an account from the `staging` ACME directory to the regular, trusted, one you
need to deactivate it and recreate it.

This procedure is also needed to change the default ACME account used in the GUI.

.Example: Changing the `default` ACME account from the `staging` to the regular directory

----
root@proxmox:~# pvenode acme account info default
Directory URL: https://acme-staging-v02.api.letsencrypt.org/directory
Account URL: https://acme-staging-v02.api.letsencrypt.org/acme/acct/6332194
Terms Of Service: https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf

Account information:
ID: xxxxxxx
Contact:
        - mailto:example@proxmox.com
Creation date: 2018-07-31T08:41:44.54196435Z
Initial IP: 192.0.2.1
Status: valid

root@proxmox:~# pvenode acme account deactivate default
Renaming account file from '/etc/pve/priv/acme/default' to '/etc/pve/priv/acme/_deactivated_default_4'
Task OK

root@proxmox:~# pvenode acme account register default example@proxmox.com
Directory endpoints:
0) Let's Encrypt V2 (https://acme-v02.api.letsencrypt.org/directory)
1) Let's Encrypt V2 Staging (https://acme-staging-v02.api.letsencrypt.org/directory)
2) Custom
Enter selection:
0

Attempting to fetch Terms of Service from 'https://acme-v02.api.letsencrypt.org/directory'..
Terms of Service: https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf
Do you agree to the above terms? [y|N]y

Attempting to register account with 'https://acme-v02.api.letsencrypt.org/directory'..
Generating ACME account key..
Registering ACME account..
Registration successful, account URL: 'https://acme-v02.api.letsencrypt.org/acme/acct/39335247'
Task OK
----

Automatic renewal of ACME certificates
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If a node has been successfully configured with an ACME-provided certificate
(either via pvenode or via the GUI), the certificate will be automatically
renewed by the pve-daily-update.service. Currently, renewal will be attempted
if the certificate has expired already, or will expire in the next 30 days.

Configuring ACME DNS APIs for validation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

On systems where external access for validation via the `http-01` method is
not possible or desired, it is possible to use the `dns-01` validation method.
This validation method requires a DNS server that allows provisioning of `TXT`
records via an API.

{PVE} re-uses the DNS plugins developed for the `acme.sh`
footnote:[acme.sh https://github.com/acmesh-official/acme.sh]
project, please refer to its documentation for details on configuration of
specific APIs.

Combining `http-01` and `dns-01` validation is possible in case your node is
reachable via multiple domains with different requirements / DNS provisioning
capabilities. Mixing DNS APIs from multiple providers or instances is also
possible by specifying different plugin instances per domain.

A special `alias` mode can be used to handle the validation on a different
domain/DNS server, in case your primary/real DNS does not support provisioning
via an API. Manually set up a permanent `CNAME` record for
`_acme-challenge.domain1.example` pointing to `_acme-challenge.domain2.example`
and set the `alias` property in the {PVE} node configuration file to
`domain2.example` to allow the DNS server of `domain2.example` to validate all
challenges for `domain1.example`.

.Example: Setting up the OVH API for validating a domain

NOTE: the account registration steps are the same no matter which plugins are
used, and are not repeated here.

NOTE: `OVH_AK` and `OVH_AS` need to be obtained from OVH according to the OVH
API documentation


First you need to get all information so you and {pve} can access the API.

----
root@proxmox:~# cat /path/to/api-token
OVH_AK=XXXXXXXXXXXXXXXX
OVH_AS=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
root@proxmox:~# source /path/to/api-token
root@proxmox:~# curl -XPOST -H"X-Ovh-Application: $OVH_AK" -H "Content-type: application/json" \
https://eu.api.ovh.com/1.0/auth/credential  -d '{
  "accessRules": [
    {"method": "GET","path": "/auth/time"},
    {"method": "GET","path": "/domain"},
    {"method": "GET","path": "/domain/zone/*"},
    {"method": "GET","path": "/domain/zone/*/record"},
    {"method": "POST","path": "/domain/zone/*/record"},
    {"method": "POST","path": "/domain/zone/*/refresh"},
    {"method": "PUT","path": "/domain/zone/*/record/"},
    {"method": "DELETE","path": "/domain/zone/*/record/*"}
]
}'
{"consumerKey":"ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ","state":"pendingValidation","validationUrl":"https://eu.api.ovh.com/auth/?credentialToken=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"}

(open validation URL and follow instructions to link Application Key with account/Consumer Key)

root@proxmox:~# echo "OVH_CK=ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ" >> /path/to/api-token
----

Now you can setup the the ACME plugin:

----
root@proxmox:~# pvenode acme plugin add dns example_plugin --api ovh --data /path/to/api_token
root@proxmox:~# pvenode acme plugin config example_plugin
┌────────┬──────────────────────────────────────────┐
│ key    │ value                                    │
╞════════╪══════════════════════════════════════════╡
│ api    │ ovh                                      │
├────────┼──────────────────────────────────────────┤
│ data   │ OVH_AK=XXXXXXXXXXXXXXXX                  │
│        │ OVH_AS=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY  │
│        │ OVH_CK=ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ  │
├────────┼──────────────────────────────────────────┤
│ digest │ 867fcf556363ca1bea866863093fcab83edf47a1 │
├────────┼──────────────────────────────────────────┤
│ plugin │ example_plugin                           │
├────────┼──────────────────────────────────────────┤
│ type   │ dns                                      │
└────────┴──────────────────────────────────────────┘
----

At last you can configure the domain you want to get certitficates for and
place the certificate order for it:

----
root@proxmox:~# pvenode config set -acmedomain0 example.proxmox.com,plugin=example_plugin
root@proxmox:~# pvenode acme cert order
Loading ACME account details
Placing ACME order
Order URL: https://acme-staging-v02.api.letsencrypt.org/acme/order/11111111/22222222

Getting authorization details from 'https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/33333333'
The validation for example.proxmox.com is pending!
[Wed Apr 22 09:25:30 CEST 2020] Using OVH endpoint: ovh-eu
[Wed Apr 22 09:25:30 CEST 2020] Checking authentication
[Wed Apr 22 09:25:30 CEST 2020] Consumer key is ok.
[Wed Apr 22 09:25:31 CEST 2020] Adding record
[Wed Apr 22 09:25:32 CEST 2020] Added, sleep 10 seconds.
Add TXT record: _acme-challenge.example.proxmox.com
Triggering validation
Sleeping for 5 seconds
Status is 'valid'!
[Wed Apr 22 09:25:48 CEST 2020] Using OVH endpoint: ovh-eu
[Wed Apr 22 09:25:48 CEST 2020] Checking authentication
[Wed Apr 22 09:25:48 CEST 2020] Consumer key is ok.
Remove TXT record: _acme-challenge.example.proxmox.com

All domains validated!

Creating CSR
Checking order status
Order is ready, finalizing order
valid!

Downloading certificate
Setting pveproxy certificate and key
Restarting pveproxy
Task OK
----