1
0
mirror of git://sourceware.org/git/lvm2.git synced 2024-12-21 13:34:40 +03:00
Commit Graph

18106 Commits

Author SHA1 Message Date
Peter Rajnoha
6cba28a335 config: recognize 'json_std' for report/output_format config setting 2022-08-11 11:10:11 +02:00
Peter Rajnoha
73ec3c954b libdm: report: use proper JSON array for string list output in JSON_STD format
In JSON format, we print string list this way:

  "key" = "item1,item2,...,itemN"

while in JSON_STD format, we print string list this way:

  "key" = ["item1","item2",...,"itemN"]
2022-08-11 11:10:11 +02:00
Peter Rajnoha
fbee18f6e5 libdm: report: separate basic and JSON+JSON_STD format in _output_field
Use separate functions to handle basic and JSON+JSON_STD format.
It's clearer this way than interleaving both in the same function.
2022-08-11 11:10:11 +02:00
Peter Rajnoha
31cd8346ae libdm: report: enhance the way string list is stored internally
Before, we stored only the report string itself for a string list
in field->report_string. The field->report_string has either
sorted items or not, depending on what we need for a field -
some report fields have sorted output, some don't...

The field->sort_value.value then contains pointer to the exact
field->report_string. The field->sort_value.items ALWAYS keeps
sorted array of individual items, represented as '[position,length]'
pairs pointing to the field->sort_value.value string.

This approach was fine as far as we didn't need to apply further
formatting to field->report_string. However, if we need to apply
further formatting to field->report_string content, taking into
account individual items, we also need to know where each item
starts and what is its length. Before, we only knew this when
items in report string were sorted, but not in the unsorted version.

We can't rely on the delimiter (default ",") only to separate items
back out of report string, because that delimiter can be contained
in the item value itself.

So this patch enhances the field->report_string for a string list so
it also contains '[position,length]' pairs for each individual item
inside field->report_string. We store this array right beyond the
string itself and we encode it in the same manner we already did for
field->sort_value.items before.

If field->report_string has sorted items, the field->sort_value.items
just points to the array of items we store beyond the report string.
If field->report_string has unsorted items, we store separate array
of items for both field->report_string and field->sort_value.

This patch also cleans up the _report_field_string_list function a bit
so it's easier and more straightforward to follow than the original
version.

Example. If we have "abc", "xy", "defgh" as list on input with ","
as delimiter, then:

  - field->report_string will have:

    - if we need field->report_string unsorted:

        abc,xy,defgh\0{[3,12],[0,3],[4,2],[7,5]}
        |____________||________________________|
           string      array of [pos,len] pairs
                       |____||________________|
                       #items      items

    - if we need field->report_string sorted:

                 repstr_extra
                      |
                      V
        abc,defgh,xy\0{[3,12],[0,3],[4,5],[10,2]}
        |____________||________________________|
           string      array of [pos,len] pairs
                       |____||________________|
                       #items      items

  - field->sort_value will have:

    - if field->report_string is unsorted:

        field->sort_value.value = field->report_string
        field->sort_value.items = {[0,3],[0,3],[7,5],[4,2]}
                                    (that is 'abc,defgh,xy')

    - if field->report_string is sorted already:

        field->sort_value.value = field->report_string
        field->sort_value.items = repstr_extra
                                  (that is also 'abc,defgh,xy')
2022-08-11 11:10:11 +02:00
Peter Rajnoha
1e31621ceb libdm: report: use 'null' for undefined numeric values in JSON_STD output
For JSON_STD format, use 'null' if a field has no value at all.

In JSON format, we print undefined numeric values this way:

  "key" = ""

while in JSON_STD format, we print undefined numeric values this way:

  "key" = null

(Keep in mind that 'null' is different from 0 (zero value) which is
a defined value.)
2022-08-11 11:10:11 +02:00
Peter Rajnoha
99299eb728 libdm: report: remove double quotes around numeric values in JSON_STD output
In JSON format, we print numeric values this way:

  "key" = "N"

while in JSON_STD format, we print numeric value this way:

  "key" = N

(Where N is a numeric value.)
2022-08-11 11:10:11 +02:00
Peter Rajnoha
02f015990b libdm: report: add DM_REPORT_GROUP_JSON_STD group
The original JSON formatting will be still available using the original
DM_REPORT_GROUP_JSON identifier. Subsequent patches will add enhancements
to JSON formatting code so that it adheres more to JSON standard - this
will be identified by new DM_REPORT_GROUP_JSON_STD identifier.
2022-08-11 11:10:11 +02:00
Marian Csontos
2fc52b6c41 config: add correct unconfigured value for use_devicesfile 2022-08-09 16:17:45 +02:00
Marian Csontos
4e6ed2c458 spec: use --with-default-use-devices-file=1 for rhel9+ 2022-08-09 16:17:43 +02:00
Marian Csontos
31f9c4bfab configure: fix typo 2022-08-09 16:17:40 +02:00
Marian Csontos
9cdd258646 spec: Use libedit for newer distributions 2022-08-09 16:17:36 +02:00
Peter Rajnoha
2fa9916493 shell: also output error message about max number of args hit with JSON format
If using JSON format for lvm shell's output, the error message about
exceeding the maximum number of arguments was not reported on output if
this condition was ever hit.

This is because the JSON format (as well as any other future format)
requires extra formatting compared to "basic" format and so it also
requires extra calls when it comes to reporting. The report needs to
be added to a report group and then popped and put on output with
specialized "dm_report_group_output_and_pop_all".

This "output and pop" is normally executed after we execute the command
in the lvm shell. When we didn't get to the command exection at all because
some precondition was not met (like hitting the limit for the number of
arguments for the command here), we skipped this important call and
so there was no log report output.

Right now, it's only this exact error message for which we need to call
"output and pop" directly, all the other error messages are about
initializing and setting the log report itself which we can't report
obviously.

Before this patch:

  lvm> pvs 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
  lvm>

With this patch applied:

  lvm> pvs 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
  {
      "log": [
          {"log_seq_num":"1", "log_type":"error", "log_context":"shell", "log_object_type":"cmd", "log_object_name":"", "log_object_id":"", "log_object_group":"", "log_object_group_id":"", "log_message":"Too many arguments, sorry.", "log_errno":"-1", "log_ret_code":"0"}
      ]
  }

If there's any other error message in the future before we execute the
command itself, we also need to call the "output and pop" directly.
2022-08-08 15:46:52 +02:00
David Teigland
99ce09ae77 apply multipath_component_detection=0 to duplicate PV handling
multipath_component_detection=0 has always applied to the filter-based
component detection.  Also apply this setting to the duplicate-PV
handling which also eliminates multipath components (based on duplicate
PVs having the same wwid.)
2022-07-25 13:50:43 -05:00
Zdenek Kabelac
c0f8e6675c make: generate 2022-07-11 01:18:24 +02:00
Zdenek Kabelac
4d2f9a4ff3 cov: restore disable_dm_devs also for error path
Keep the structure correct for failing error path,
alhtough likely this particual var will not be used.
2022-07-11 01:18:24 +02:00
Zdenek Kabelac
5c463584f6 cov: remove unused headers 2022-07-11 01:18:24 +02:00
Zdenek Kabelac
493acb9195 vdo: suffle code for better error path handling
For failing dm_ no need to report 2nd. error,
but we missed to report error with 'updated==NULL'.
2022-07-11 01:18:24 +02:00
Zdenek Kabelac
e2e31d9acf vdo: enhance lvcreate validation
When creating VDO pool based of  % values, lvm2 is now more clever
and avoids to create 'unsupportable' sizes of physical backend
volumes as 16TiB is maximum size supported by VDO target
(and also limited by maximum supportable slabs (8192) based on slab
size.

If the requested virtual size is approaching max supported size 4PiB,
switch header size to 0.
2022-07-11 01:18:24 +02:00
Zdenek Kabelac
1c18ed3b4a vdo: support v4 kernel target line
Check and use new available table line v4, if kernel supports it.
2022-07-11 01:18:24 +02:00
Zdenek Kabelac
a477490e81 vdo: add reformating to extent size aligned virtual size
Newer VDO kernel target require to have matching virtual size - this
however cause incompatiblity when lvcreate is let to format VDO data
device and read the usable size from vdoformat.
Altough this is a kernel regression and will likely get fixed,
lvm2 can actually reformat VDO device to use properly aligned VDO LV
size to make this problem disappear.
2022-07-11 01:18:24 +02:00
Zdenek Kabelac
ebad057579 vdo: check vdo memory constrains
Add function to check for avaialble memory for particular VDO
configuration - to avoid unnecessary machine swapping for configs
that will not fit into memory (possibly in locked section).

Formula tries to estimate RAM size machine can use also with
swapping for kernel target - but still leaving some amount of
usable RAM.

Estimation is based on documented RAM usage of VDO target.

If the /proc/meminfo would be theoretically unavailable, try to use
'sysinfo()' function, however this is giving only free RAM without
the knowledge about how much RAM could be eventually swapped.

TODO: move _get_memory_info() into generic lvm2 API function used
by other targets with non-trivial memory requirements.
2022-07-11 01:18:24 +02:00
Zdenek Kabelac
9f3eff002c vdo: report supported range in error path 2022-07-11 01:18:24 +02:00
Zdenek Kabelac
b5c8e591ed vdo: use defines also for configuration defines
Keep single source for most of values printed in lvm.conf
(still needs some conversion)

Correct max for logical threads to 60
(we may refuse some older configuration which might eventually
user higher numbers - but so far let's assume no user have ever set this
as it's been non-trivial and if would complicate code unnecessarily.)

Accept maximum of 4PiB for virtual size of VDO LV
(lvm2 will drop 'header borders to 0 for this case').
2022-07-11 01:18:24 +02:00
Zdenek Kabelac
f445624c33 vdo: update info about memory
Add more info about kernel target memory allocation associated with
VDO pool usage.
2022-07-11 01:18:24 +02:00
Zdenek Kabelac
8ca2b1bc21 vdo: use single validator
Add era lenght validation into dm_vdo_validate_target_params()
and reuse this validator also for _check_lv_segment().
2022-07-11 01:18:24 +02:00
Zdenek Kabelac
fe6fb1ec52 man: space after size
Put space between size and SI unit.
Automatically make this 'space' as fixed size by Makefile sed script.
2022-07-11 01:18:24 +02:00
Zdenek Kabelac
d2667bc25b vdo: fix conversion of vdo_slab_size_mb 2nd
Patch 1b070f366b should have
been already fixing this issue but since it the incorrect
patch rebasing the change to vdo_slabSize got lost.
So again now with explicit one-line patch.
2022-07-11 01:18:24 +02:00
David Teigland
92b4fcf57f exit with error when --devicesfile name doesn't exist 2022-07-06 10:10:58 -05:00
Tony Asleson
d0f94e763d lvmdbustest: Add test for property "Get"
We typically use "GetAll", so add test for "Get" and check values.
2022-06-30 10:55:16 -05:00
Tony Asleson
01ef2f2525 lvmdbusd: Remove try/except for mkfifo
We should never run into this error condition when using tempfile.mkdtemp.
2022-06-30 10:55:16 -05:00
Tony Asleson
0d957dcacc lvmdusd: Remove non lvm JSON output support 2022-06-30 10:55:16 -05:00
Tony Asleson
73121e3f07 lvmdbustest: Increase number of LVs
As storage is getting faster, we need to create more LVs to pass this test.
2022-06-30 10:55:16 -05:00
Tony Asleson
8fa8dfdb8c lvmdbustest: Correct comment spelling/grammar 2022-06-30 10:55:16 -05:00
Tony Asleson
55059e002a lvmdbustest: Test job remove path when job not complete 2022-06-30 10:55:16 -05:00
Tony Asleson
d393436727 lvmdbusd: Correct grammar in lvm shell proxy comments 2022-06-30 10:55:16 -05:00
Tony Asleson
6914942685 lvmdbusd: Don't require "lvm> " prompt for shell
Depending on how lvm is compiled, it may not present the "lvm> " prompt
when using the lvm shell.  Don't require it to be present.

Addresses: https://bugzilla.redhat.com/show_bug.cgi?id=2090391
2022-06-30 10:55:16 -05:00
Tony Asleson
eee89a941e lvmdbusd: Job prop. Get/GetAll exec. immediately
This allows API user the ability to check on the status of a long running
job without blocking in the request queue.
2022-06-30 10:55:16 -05:00
Tony Asleson
7a2090655d lvmdbusd: Remove the use of sub shell for lvm shell
This reduces the number of processes and improves security.
2022-06-30 10:55:16 -05:00
Tony Asleson
b3d7aff6a3 lvmdbusd: Fix env variable LVM_DBUSD_TEST_MODE
Make it more logical.
2022-06-30 10:55:16 -05:00
Tony Asleson
47c61907b4 lvmdbusd: Change unit test vdo minimum size 2022-06-30 10:55:16 -05:00
Tony Asleson
51d9b686c0 lvmdbusd: Add debug output for which lvm binary is used 2022-06-30 10:55:16 -05:00
Tony Asleson
b3407b16c1 lvmdbusd: re-work lvm shell main
Add an optional single argument "bisect" to use with git bisect for
automation.  Normal case is no arguments when running stand-alone.
2022-06-30 10:55:16 -05:00
Tony Asleson
58c6c9e9aa lvmdbusd: Simplify child process env
We don't need to duplicate the entire env from the parent, supply only what
is needed.
2022-06-30 10:55:16 -05:00
Tony Asleson
37733cd4eb lvmdbusd: Correct conditional for lvm child process running
Poll returns None when process is running, else exit value.  If poll returns
0 we will fail to exit the select loop.
2022-06-30 10:55:16 -05:00
David Teigland
db5277c971 pvdisplay: restore --reportformat option
Fixes commit b8f4ec846 "display: ignore --reportformat"
by restoring the --reportformat option to pvdisplay.
Adding -C to pvdisplay turns the command into a reporting
command (like pvs, vgs, lvs) in which --reportformat can
be useful.
2022-06-24 10:40:54 -05:00
David Teigland
3b0f9cec7e filter-mpath: get wwids from sysfs vpd_pg83
to compare with wwids in /etc/multipath/wwids when
excluding multipath components.  The wwid printed
from the sysfs wwid file may not be the wwid used
in multipath wwids.  Save the wwids found for each
device on dev->wwids to avoid repeating reading
and parsing the sysfs files.
2022-06-08 15:06:01 -05:00
Zdenek Kabelac
2bea95764e tests: skip running tests for non root user
Testing needs 'root' privileges.
Only 'make run-unit-test' can work without them.
2022-06-07 17:14:09 +02:00
Zdenek Kabelac
4a49851207 tests: update for wrapper
Update calling vdo manager since our vdo wrapper has a simple shell
arg parser so it needs args without '='

Also correct using  DM_DEV_DIR for 'pvcreate'
2022-06-07 17:14:09 +02:00
Zdenek Kabelac
2ecfd503ed tests: add lvm_vdo_wrapper
Introduce a replacement vdo manager wrapper for testing.
When using test suite on a system without vdo manager (which has got
deprecated) - we still need its functionality to prepare 'vdo volume'
for testing lvm_import_vdo.

Wrapper currently need 2 binaries from older 'vdo 6.2' package -
to be named:
oldvdoformat - format VDO metadata with older format
oldvdoprepareforlvm - shift vdo metadata by 1MiB
2022-06-07 17:14:08 +02:00
Zdenek Kabelac
1b070f366b vdo: fix conversion of vdo_slab_size_mb
When converting VDO volume, the parameter vdo_slabSize was
incorrectly copied as vdo_blockMapCacheSize, however this parameter
is then no longer used for any table line creation so the wrong
value was only stored in metadata.

Also use just single get_kb_size_with_unit_ and remove it's duplicate
functionality with get_mb_size_with_unit_.

Use $VERB for vdo remove call.
2022-06-07 17:14:08 +02:00