3998 Commits

Author SHA1 Message Date
Dominik Csapak
dd13eab34b ceph: move service_cmd and MDS related code to Services.pm
Also adapts the calls to the relevant subs.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-12-20 09:44:01 +01:00
Dominik Csapak
3cdc2ff0af ceph: move CephTools into Ceph/Tools.pm
It makes more sense to have it there, especially since we want to
split out the service parts into a seperate file.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-12-20 09:44:01 +01:00
Dominik Csapak
7b89bbfdd9 ceph: use cfs_read/write_file for ceph.conf
The parser is now registered, and ceph.conf is a tracked file in pmxcfs.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-12-20 09:44:01 +01:00
Thomas Lamprecht
37b3c8dfc3 bump version to 5.3-6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-12-14 12:25:43 +01:00
Thomas Lamprecht
2555aa1c4e fixup: whitespace cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-12-13 10:15:46 +01:00
Tim Marx
e36137eb66 ceph api: added check for /etc/pve/ceph.conf to remaining/new endpoints
Signed-off-by: Tim Marx <t.marx@proxmox.com>
2018-12-13 10:14:39 +01:00
Dominik Csapak
0c4db2fb47 fix #2017: add help button to storage input panels
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-12-07 14:09:14 +01:00
Thomas Lamprecht
69aa353e34 ui: storage/cephfs: fix maxfiles default enable
Most other storage types do not have backup in their default content
selection when creating a new storage, so there it make sense to
initially have maxfiles disabled. But with CephFS we have default
_only_ backup as content type, so let it enabled here - as else one
must change something in the content selector to enable the field.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-12-07 14:03:06 +01:00
Alwin Antreich
fa415a2438 Fix #2019: CephFS storage misses Max Backups
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-12-07 14:03:06 +01:00
Thomas Lamprecht
18115e9323 ui: fixup jslint warnings
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-12-07 14:03:06 +01:00
Wolfgang Bumiller
b1a7fa8a10 ui: lxc/features: add fuse
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
2018-12-06 15:42:28 +01:00
Wolfgang Bumiller
9fb7b347e0 ui: lxc/features: disable nfs and cifs for unprivileged
While we can allow them via the apparmor profile, they still
won't be usable as the kernel doesn't have FS_USERNS_MOUNT
set on those.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
2018-12-06 15:42:13 +01:00
Dominik Csapak
e606d3227e do not send 'undefined' as romfile
when adding a pcidevice, we added 'romfile=undefined' by accident

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-12-05 14:25:24 +01:00
Thomas Lamprecht
97ae681d6b bump version to 5.3-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-12-03 12:14:06 +01:00
Thomas Lamprecht
253c3e4f12 ui: CephFS: fix onlineHelp buttons
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-30 18:43:00 +01:00
Thomas Lamprecht
fbb1554ef8 bump version to 5.3-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-29 13:02:33 +01:00
Thomas Lamprecht
a3e7ff34b6 ceph: update all pg_num defaults to 128
As the last patch missed a few places. see pg calc or also:
http://docs.ceph.com/docs/luminous/rados/operations/placement-groups/

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-29 12:36:25 +01:00
Alwin Antreich
51a3046bdf Raise the default pg_num to 128
PGCalc and the ceph recommendation say 128 PGs for up to 5 OSDs

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-11-29 12:19:22 +01:00
Thomas Lamprecht
0f706ed86c ui: cluster: require ring0_addr if joinee's ring and node addr differ
Pass the ring addresses from the node we use to join a cluster with
the encoded cluster info. Use it to determine if the nodes public
address (i.e., the onei t's node name resolves to) differs from it'S
ring0 address, if that's the case and the cluster was created with
PVE tooling then we know that it's highly probable that the ring0 is
on another network than the public net, so require the ring0_addrss
field in this case. If we have a false positive, it's not to bad
here.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-28 12:39:40 +01:00
Thomas Lamprecht
1f409296fb ui: ceph pools: we now add only one storage, so signular
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-28 12:06:54 +01:00
Dominik Csapak
2a67983029 add pciutils as dependency
the package is not big, installed by default in the installer,
makes the pci scan much prettier, and we call lspci in the pvereport

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-11-28 11:41:50 +01:00
Thomas Lamprecht
161ea42ce4 fixup: values is not needed for iterating a list
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-28 10:31:32 +01:00
Thomas Lamprecht
8354741294 fixup: s/wipe_disk/wipe_disks/ as we can to multiple at once
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-28 10:30:58 +01:00
Alwin Antreich
b4bf4a376f Factoring the wipe_disk out into CephTools
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-11-28 10:25:16 +01:00
Thomas Lamprecht
cccaf366a2 ceph: fixup s/add_storage/add-storage/
it's just the nicer interface and we want to go away from underscore
usage in new calls

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-28 10:07:08 +01:00
Thomas Lamprecht
8181673e71 fix #1430: ceph init: allow to specify separate cluster network
Allow to specify a separate cluster network when initializing ceph.
Ceph docs[0] imply a possibility for performance increase and
enhanced security in environments where the public network serves not
fully trusted peers, which could else provoke a DOS to the cluster
traffic[0].

Make this optional, but if passed `network` is required too.

[0]: http://docs.ceph.com/docs/luminous/rados/configuration/network-config-ref/

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-28 09:29:22 +01:00
Thomas Lamprecht
b565ae16da ui: cephfs: only allow CephFS creation if MDS is configured
Introduce the mdsCount again, I know remember again why I had it in
v3 of my CephFS series.. Use this to disable the CephFS create button
if we have no MDS configured, as this is a requirement.
Further change the gettext for 'No XY configure' to a format string
so that it can be reused.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
2018-11-27 15:24:13 +01:00
Thomas Lamprecht
5d7747290c api: cephfs: reuse rados connection when polling for active MDS
no point in recreating one, we have an active one from earlier

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-27 15:24:08 +01:00
Thomas Lamprecht
20783672bd pveceph: ensure ceph-fuse gets uptdated when installing ceph server
as we depend on ceph-fuse elsewhere (pve-storage) this gets installed
from Debians repositories with the Ceph 10 version.
So ensure that an up to date version, from our current supported Ceph
release, gets installed when doing `pveceph install` else you may
fall into certain issues which would have been already resolved with
a newer version.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-27 15:24:08 +01:00
Thomas Lamprecht
83a7c6e5a6 update shipped aplinfo index
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-26 18:45:55 +01:00
Thomas Lamprecht
614a0084c1 pveceph: mds destroy: fixup parameter s/id/name/
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-26 17:11:37 +01:00
Thomas Lamprecht
b51bfddaca is_any_mds_active: check if state is correct
a MDS gets transferred to the active MDS info once it got selected as
active, not once it really _is_ active, it can be there in the
'up:creating' state. So ensure that a MDS with 'u:active' could be
found

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-26 16:55:37 +01:00
Dominik Csapak
123273bb92 gui: refactor render_full_name
this also fixes a missing  htmlEncode in UserSelector

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-11-26 12:30:30 +01:00
Thomas Lamprecht
c460f95dc1 bump version to 5.1-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-23 19:16:40 +01:00
Thomas Lamprecht
b9031ded3f api: cephfs: wait for MDS to become active
A MDS gets only active once a FS is there, and we need an MDS active
to be able to add a storage, as the CephFS plugin does an immediate
mount check. As an MDS needs some time to get active we had a
problematic time window where this mounting could fail.

Wait for a MDS to get in active state.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-23 18:49:48 +01:00
Thomas Lamprecht
2ed99b32ff api: cephfs: check if SID is free when add_storage is set
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-23 18:35:16 +01:00
Thomas Lamprecht
a75068b184 update bump to 5.3-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-23 16:17:46 +01:00
Thomas Lamprecht
174daedcb8 api/ceph: import file_set_contents
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-23 16:17:13 +01:00
Thomas Lamprecht
57d471964d bump version to 5.3-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-23 15:13:45 +01:00
Thomas Lamprecht
1563353fd6 pveceph: also install ceph-mds for hyper converged install
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-23 15:11:03 +01:00
Thomas Lamprecht
3311f92cca bump version to 5.3-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-23 14:11:27 +01:00
Thomas Lamprecht
845956ccaf ui: add CephFS integration
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-23 13:59:49 +01:00
Thomas Lamprecht
db5acea2c3 pveceph: move to sub commands
add aliases for the existing ones, ignore the ones for MDS and
CephFS, they did never hit any repo.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-23 13:33:12 +01:00
Thomas Lamprecht
d7d317a562 ceph: add CephFS create and list API
Allow to create a new CephFS instance and allow to list them.

As deletion requires coordination between the active MDS and all
standby MDS next in line this needs a bit more work. One could mark
the MDS cluster down and stop the active, that should work but as
destroying is quite a sensible operation, in production not often
needed I deemed it better to document this only, and leaving API
endpoints for this to the future.

For index/list I slightly transform the result of an RADOS `fs ls`
monitor command, this would allow relative easy display of a CephFS
and it's backing metadata and data pools in a GUI.

While for now it's not enabled by default and marked as experimental,
this API is designed to host multiple CephFS instances - we may not
need this at all, but I did not want to limit us early. And anybody
liking to experiment can use it after the respective ceph.conf
settings.

When encountering errors try to rollback. As we verified at the
beginning that we did not reused pools, destroy the ones which we
created.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Co-authored-by: Alwin Antreich <a.antreich@proxmox.com>
2018-11-23 13:33:12 +01:00
Thomas Lamprecht
9700a50f53 ceph: add MDS create/delete/list API
Allow to create, list and destroy and Ceph Metadata Server (MDS) over
the API and the CLI `pveceph` tool.

Besides setting up the local systemd service template and the MDS
data directory we also add a reference to the MDS in the ceph.conf
We note the backing host (node) from the respective MDS and set up a
'mds standby for name' = 'pve' so that the PVE created ones are a
single group. If we decide to add integration for rank/path specific
MDS (possible useful for CephFS with quite a bit of load) then this
may help as a starting point.

On create, check early if a reference already exists in ceph.conf and
abort in that case. If we only see existing data directories later
on we abort but do not remove them, they could well be from an older
manual create - where it's possible dangerous to just remove it. Let
the user handle it themself in that case.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Co-authored-by: Alwin Antreich <a.antreich@proxmox.com>
2018-11-23 13:33:12 +01:00
Thomas Lamprecht
5870ae1b8b ceph: move create/destroy pool to CephTools
We will reuse this in the future, e.g., when creating a data and
metadata pool for CephFS.

Allow to pass a $rados object (to reuse it, as initializing is not
that cheap) but also create it if it's undefined, fro convenience.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-23 13:33:12 +01:00
Thomas Lamprecht
b24080c0ee api: document default for ceph service start/stop/restart
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-11-23 12:40:46 +01:00
Dominik Csapak
6140c3c815 gui: add ceph monitor restart button
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-11-23 12:23:42 +01:00
Dominik Csapak
81b29681d8 gui: add osd restart button
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-11-23 12:23:42 +01:00
Dominik Csapak
b0317ede2c ceph: add service restart api call
like start/stop but for restart, this makes maintenance a bit easier

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-11-23 12:23:42 +01:00