From 2a75dc746541ec929430deff3adfa6e88d9d7412 Mon Sep 17 00:00:00 2001 From: David Teigland Date: Mon, 6 May 2024 12:01:52 -0500 Subject: [PATCH] man: lvmthin update major refresh of text and organization --- man/lvmthin.7_main | 1776 +++++++++++++++++++------------------------- 1 file changed, 774 insertions(+), 1002 deletions(-) diff --git a/man/lvmthin.7_main b/man/lvmthin.7_main index 415efa720..dcd304edc 100644 --- a/man/lvmthin.7_main +++ b/man/lvmthin.7_main @@ -8,20 +8,21 @@ lvmthin \(em LVM thin provisioning . Blocks in a standard \fBlvm\fP(8) Logical Volume (LV) are allocated when the LV is created, but blocks in a thin provisioned LV are allocated as -they are written. Because of this, a thin provisioned LV is given a -virtual size, and can then be much larger than physically available -storage. The amount of physical storage provided for thin provisioned LVs -can be increased later as the need arises. +they are written. Because of this, a thin provisioned LV has a virtual +size that can be much larger than the available physical storage. The +amount of physical storage provided for thin provisioned LVs can be +increased later as the need arises. .P Blocks in a standard LV are allocated (during creation) from the Volume Group (VG), but blocks in a thin LV are allocated (during use) from a -special "thin pool LV". The thin pool LV contains blocks of physical -storage, and blocks in thin LVs just reference blocks in the thin pool LV. +"thin pool". The thin pool contains blocks of physical storage, and +thin LV blocks reference blocks in the thin pool. .P -A thin pool LV must be created before thin LVs can be created within it. -A thin pool LV is created by combining two standard LVs: a large data LV -that will hold blocks for thin LVs, and a metadata LV that will hold -metadata. The metadata tracks which data blocks belong to each thin LV. +A special "thin pool LV" must be created before thin LVs can be created +within it. A thin pool LV is created by combining two standard LVs: a +data LV that will hold blocks for thin LVs, and a metadata LV that will +hold metadata. Thin pool metadata is created and used by the dm-thin +kernel module to track the data blocks used by thin LVs. .P Snapshots of thin LVs are efficient because the data blocks common to a thin LV and any of its snapshots are shared. Snapshots may be taken of @@ -30,407 +31,632 @@ are also shared in the thin pool. There is no limit to or degradation from sequences of snapshots. .P As thin LVs or snapshot LVs are written to, they consume data blocks in -the thin pool. As free data blocks in the pool decrease, more free blocks -may need to be supplied. This is done by extending the thin pool data LV -with additional physical space from the VG. Removing thin LVs or -snapshots from the thin pool can also free blocks in the thin pool. -However, removing LVs is not always an effective way of freeing space in a -thin pool because the amount is limited to the number of blocks not shared -with other LVs in the pool. +the thin pool. As free data blocks in the pool decrease, more physical +space may need to be added to the pool. This is done by extending the +thin pool with additional physical space from the VG. Removing thin LVs +or snapshots from the thin pool can also make more space available. +However, removing thin LVs is not always an effective way of freeing space +in a thin pool because blocks may be shared by snapshots, and free blocks +may be too fragmented to make available. .P -Incremental block allocation from thin pools can cause thin LVs to become -fragmented. Standard LVs generally avoid this problem by allocating all -the blocks at once during creation. -. -.SH THIN TERMS -. -.TP -ThinDataLV -.br -thin data LV -.br -large LV created in a VG -.br -used by thin pool to store ThinLV blocks -. -.TP -ThinMetaLV -.br -thin metadata LV -.br -small LV created in a VG -.br -used by thin pool to track data block usage -. -.TP -ThinPoolLV -.br -thin pool LV -.br -combination of ThinDataLV and ThinMetaLV -.br -contains ThinLVs and SnapLVs -. -.TP -ThinLV -.br -thin LV -.br -created from ThinPoolLV -.br -appears blank after creation -. -.TP -SnapLV -.br -snapshot LV -.br -created from ThinPoolLV -.br -appears as a snapshot of another LV after creation -.TP -ExternalOriginLV -.br -external origin LV -.br -read-only LV used for unprovisioned blocks of a thin LV -.br -. -.SH THIN USAGE -. -The primary method for using lvm thin provisioning: -.nr step 1 1 -. -.SS \n[step]. Create ThinDataLV -. -Create an LV that will hold thin pool data. +On-demand block allocation can cause thin LV blocks to be fragmented in +the thin pool, which can cause reduced performance compared to standard +fully provisioned LV. .P -.B lvcreate -n ThinDataLV -L LargeSize VG +.SH DEFINITIONS .P -.I Example +Thin LV .br -# lvcreate -n pool0 -L 10G vg -. -.SS \n+[step]. Create ThinMetaLV -. -Create an LV that will hold thin pool metadata. +A thin LV is an LVM logical volume for which storage is allocated on +demand. As a thin LV is written, blocks are allocated from a thin pool to +hold the data. A thin LV has a virtual size that can be larger than the +physical space in the thin pool. .P -.B lvcreate -n ThinMetaLV -L SmallSize VG -.P -.I Example +Thin Pool .br -# lvcreate -n pool0meta -L 1G vg +A thin pool is a special LV containing physical extents from which thin +LVs are allocated. The thin pool LV is not used as a block device, but +the thin pool name is referenced when creating thin LVs. The thin pool LV +must be extended with additional physical extents before it runs out of +space. A thin pool has two hidden component LVs: one for holding thin +data and another for holding thin metadata. .P -# lvs - LV VG Attr LSize - pool0 vg -wi-a----- 10.00g - pool0meta vg -wi-a----- 1.00g +Thin Pool Data LV +.br +A component of a thin pool that holds thin LV data. The data LV is a +hidden LV with a _tdata suffix, and is not used directly. The physical +size of the data LV is displayed as the thin pool size. +.P +Thin Pool Metadata LV +.br +A component of a thin pool that holds metadata for the dm-thin kernel +module. dm-thin generates and uses this metadata to track data blocks +used by thin LVs. The metadata LV is a hidden LV with a _tmeta suffix, +and is not used directly. +.P +Thin Snapshot +.br +A thin snapshot is a thin LV that is created in reference to an existing +thin LV or other thin snapshot. The thin snapshot initially refers to the +same blocks as the existing thin LV. It acts as a point in time copy of +the thin LV it referenced. +.P +External Origin +.br +A read-only LV that is used as a snapshot origin for thin LVs. +Unwritten portions of the thin LVs are read from the external origin. +.P +.SH USAGE . -.SS \n+[step]. Create ThinPoolLV +.SS Thin Pool Creation . +A thin pool can be created with the lvcreate command. The data and +metadata component LVs are each allocated from the VG, and combined into a +thin pool. The lvcreate -L|--size will be the size of the thin pool data +LV, and the size of the metadata LV will be calculated automatically (or, +can be optionally specified with --poolmetadatasize.) + .nf -Combine the data and metadata LVs into a thin pool LV. -ThinDataLV is renamed to hidden ThinPoolLV_tdata. -ThinMetaLV is renamed to hidden ThinPoolLV_tmeta. -The new ThinPoolLV takes the previous name of ThinDataLV. +$ lvcreate --type thin-pool -n ThinPool -L Size VG +.fi +. +.SS Thin Pool Conversion +. +For a customized thin pool layout, data and metadata LVs can be created +separately, and then combined into a thin pool with lvconvert. This +allows specific LV types, or specific devices, to be used for +data/metadata LVs. Combining the data and metadata LVs into a thin pool +erases the content of both LVs. The resulting thin pool takes the name +and size of the data LV. (If a metadata LV is not specified, lvconvert +will automatically create one to use in the thin pool.) + +.nf +$ lvcreate -n DataLV -L Size VG DataDevices +$ lvcreate -n MetadataLV -L MetadataSize VG MetadataDevices +$ lvconvert --type thin-pool --poolmetadata MetadataLV VG/DataLV +.fi + +(DataLV would now be referred to as ThinPool, and can be used for creating +thin LVs.) +. +.SS Thin LV Creation +. +Thin LVs are created in a thin pool, and are created with a virtual size +using the option -V|--virtualsize. The virtual size may be larger than +the physical space available in the thin pool. + +.nf +$ lvcreate --type thin -n ThinLV -V VirtualSize --thinpool ThinPool VG +.fi +. +.SS Thin Snapshot Creation +. +Snapshots of thin LVs are thin LVs themselves, but the snapshot LV +initially refers to the same blocks as the origin thin LV. The origin +thin LV and its snapshot thin LVs will diverge as either are written. The +origin thin LV can be removed without affecting snapshots that reference +it. Snapshots can be taken of thin LVs that were themselves created as +snapshots. (A size option must not be used when creating a thin snapshot, +otherwise a COW snapshot will be created.) +.P +.nf +$ lvcreate --snapshot -n SnapLV VG/ThinLV +.fi +. +.SS Thin Pool Data Percent and Metadata Percent +. +For active thin pool LVs, the 'lvs' command displays "Data%" (-o +data_percent) and "Meta%" (-o metadata_percent). Data percent is the +percent of space in the data LV that is currently used by thin LVs. +Metadata percent is the percent of space in the metadata LV that is +currently used by the dm-thin module. The thin pool should be extended +before either of these values reach 100%. +.P +.nf +$ lvs -o data_percent VG/ThinPool +$ lvs -o metadata_percent VG/ThinPool +.fi +. +.SS Thin Pool Extension +. +When lvextend is run on a thin pool, it will extend the internal data LV +by the specified amount, and the internal metadata LV will also be +extended, if needed, relative to the new data size. +.P +.nf +$ lvextend --size Size VG/ThinPool .fi .P -.B lvconvert --type thin-pool --poolmetadata VG/ThinMetaLV VG/ThinDataLV +A specific new metadata size can also requested when extending the thin +pool data. .P -.I Example -.br -# lvconvert --type thin-pool --poolmetadata vg/pool0meta vg/pool0 -.P -# lvs vg/pool0 - LV VG Attr LSize Pool Origin Data% Meta% - pool0 vg twi-a-tz-- 10.00g 0.00 0.00 -.P -# lvs -a - LV VG Attr LSize - pool0 vg twi-a-tz-- 10.00g - [pool0_tdata] vg Twi-ao---- 10.00g - [pool0_tmeta] vg ewi-ao---- 1.00g -. -.SS \n+[step]. Create ThinLV -. .nf -Create a new thin LV from the thin pool LV. -The thin LV is created with a virtual size. -Multiple new thin LVs may be created in the thin pool. -Thin LV names must be unique in the VG. -The '--type thin' option is inferred from the virtual size option. -The --thinpool argument specifies which thin pool will -contain the ThinLV. +$ lvextend --size Size --poolmetadatasize MetadataSize VG/ThinPool .fi .P -.B lvcreate -n ThinLV -V VirtualSize --thinpool ThinPoolLV VG +The metadata LV can also be extended individually without extending the +data LV. .P -.I Example -.br -Create a thin LV in a thin pool: -.br -# lvcreate -n thin1 -V 1T --thinpool pool0 vg -.P -Create another thin LV in the same thin pool: -.br -# lvcreate -n thin2 -V 1T --thinpool pool0 vg -.P -# lvs vg/thin1 vg/thin2 - LV VG Attr LSize Pool Origin Data% - thin1 vg Vwi-a-tz-- 1.00t pool0 0.00 - thin2 vg Vwi-a-tz-- 1.00t pool0 0.00 +.nf +$ lvextend --poolmetadatasize MetadataSize VG/ThinPool +.fi . -.SS \n+[step]. Create SnapLV +.SS Thin Pool Automatic Extension . -Create snapshots of an existing ThinLV or SnapLV. -.br -Do not specify -.BR -L ", " --size -when creating a thin snapshot. -.br -A size argument will cause an old COW snapshot to be created. +It is important to extend a thin pool before it runs out of space, +otherwise it may be damaged, and difficult or impossible to repair. LVM +can be configured so that dmeventd automatically extends thin pools when +they run low on space. Free extents must be available in the VG to use +for extending the thin pools. .P -.B lvcreate -n SnapLV --snapshot VG/ThinLV -.br -.B lvcreate -n SnapLV --snapshot VG/PrevSnapLV +dmeventd is usually started by the lvm2-monitor service. dmeventd +receives notifications from the kernel indicating when thin pool data or +metadata are becoming full. In response, dmeventd runs the command +"lvextend --use-policies VG/ThinPool", which compares the current usage of +data and metadata with the autoextend threshold. The data LV and/or +metadata LV may be extended in response. System messages will show when +these extensions have happened. .P -.I Example -.br -Create first snapshot of an existing ThinLV: -.br -# lvcreate -n thin1s1 -s vg/thin1 +To enable thin pool automatic extension, set lvm.conf: .P -Create second snapshot of the same ThinLV: +.IP \[bu] 2 +.BR thin_pool_autoextend_threshold .br -# lvcreate -n thin1s2 -s vg/thin1 +Extend the thin pool when the current usage reaches this percentage. The +chosen value should depend on the rate at which new data may be written. +If space is consumed more quickly, then a lower threshold will provide +dmeventd and lvextend more time to react and extend the pool. The minimum +is 50. Setting to 100 disables autoextend. .P -Create a snapshot of the first snapshot: +.IP \[bu] 2 +.BR thin_pool_autoextend_percent .br -# lvcreate -n thin1s1s1 -s vg/thin1s1 +A thin pool is extended by this percent of its current size. .P -# lvs vg/thin1s1 vg/thin1s2 vg/thin1s1s1 - LV VG Attr LSize Pool Origin - thin1s1 vg Vwi---tz-k 1.00t pool0 thin1 - thin1s2 vg Vwi---tz-k 1.00t pool0 thin1 - thin1s1s1 vg Vwi---tz-k 1.00t pool0 thin1s1 +The thin pool itself must be monitored by dmeventd to be automatically +extended. When activating a thin pool, lvm normally requests monitoring +by dmeventd. To verify this, run: +.P +.nf +$ lvs -o+seg_monitor VG/ThinPool +.fi +.P +To begin monitoring a thin pool in dmeventd: +.P +.nf +$ lvchange --monitor y VG/ThinPool +.fi . -.SS \n+[step]. Create ThinLV with ThinPoolLV +.SS Thin LV Activation . -Create a new thin LV together with thin pool LV and let lvm2 -to allocate data and metadata volume. +A thin LV that is created as a snapshot is given the "skip activation" +property. It is reported with lvs -o skip_activation, or 'k' in the tenth +lv_attr. This property causes vgchange -ay and lvchange -ay commands to +skip activating the thin LV unless the -K|--ignoreactivationskip option is +also set. .P -.B lvcreate -T -n ThinLV -V VirtualSize -L PoolSize VG/ThinPoolLV +.nf +$ lvchange -ay -K VG/SnapLV +.fi .P -.I Example +The skip activation property on a thin LV can be cleared, so that -K is +not required to activate it (or enabled so -K is required.) +.P +.nf +$ lvchange --setactivationskip y|n VG/SnapLV +.fi +.P +To configure the "skip activation" setting that lvcreate applies to new +snapshots, set lvm.conf: .br -Create a 50MiB thin LV and 10MiB thin pool: -.br -# lvcreate -T -n thin -V 52M -L 12M vg/pool +.B auto_set_activation_skip +. +.SS Thick LV to Thin LV Conversion +. +A thick LV (e.g. linear, striped) can be converted to a thin LV in a new +thin pool. The new thin pool is created using the existing thick LV as +thin pool data. New thin pool metadata is generated and written to a new +metadata LV. The new thin LV references the original thick data now +located in the thin pool data LV. Note: This conversion cannot be +reversed; the thin volume cannot be reverted back to the thick LV. .P +.nf +$ lvconvert --type thin VG/ThickLV +.fi +.P +(ThickLV would now be referred to as ThinLV, and a new thin pool will +exist named ThinLV_tpool0.) +.P +After the conversion, the resulting thin LV and thin pool will look +somewhat different from ordinary thin LVs/pools: the new thin LV will be +fully provisioned in the thin pool, and the thin pool data usage will be +100%. The thin pool will require extension before new thin LVs or +snapshots are used. +. +.SS Thin Pool on LVM RAID +. +Thin pool data or metadata component LVs can use LVM RAID by first +creating RAID LVs for data and/or metadata component LVs, and then +converting these RAID LVs into a thin pool. +.P +.nf +$ lvcreate --type raidN -n DataLV -L Size VG DataDevices +$ lvcreate --type raidN -n MetadataLV -L MetadataSize VG MetadataDevices +$ lvconvert --type thin-pool --poolmetadata MetadataLV VG/DataLV +.fi +.P +(DataLV would now be referred to as ThinPool, and can be used for creating +thin LVs.) +.P +To use MD RAID instead of LVM RAID, create linear data/metadata LVs on MD +devices, and refer to the MD devices for DataDevices/MetadataDevices. +. +.SS Thin Pool on LVM VDO +. +Thin pool data can be compressed and deduplicated using VDO. Data for all +thin LVs in the thin pool will be compressed and deduplicated using the +dm-vdo module. +.P +.nf +$ lvcreate --type thin-pool -n ThinPool -L Size --pooldatavdo y VG +.fi +.P +Or, convert an existing LV (e.g. linear, striped) into a thin-pool that +uses VDO compression/deduplication for thin data. Existing content on the +LV will be erased. +.P +.nf +$ lvconvert --type thin-pool --pooldatavdo y VG/LV +.fi +.P +(LV would now be referred to as ThinPool, and can be used for creating +thin LVs.) +. +.SS Thin Pool and Thin LV Combined Creation +. +One command can be used to create a new thin pool with a new thin LV. +.P +.nf +$ lvcreate --type thin -n ThinLV -V VirtualSize \\ + --thinpool ThinPool -L ThinPoolSize VG +.fi +.P +First, a new thin pool is created: +.br +Thin Pool name is from --thinpool ThinPool +.br +Thin Pool size is from -L|--size ThinPoolSize +.P +Second, a new thin LV is created: +.br +Thin LV name is from -n|--name ThinLV +.br +Thin LV size is from -V|--virtualsize VirtualSize +.P +Other thin LVs can then be created in the thin pool using standard +lvcreate commands for thin LVs. +. +.SS Thin Snapshot Creation of an External Origin +. +Thin snapshots are typically taken of other thin LVs within the same thin +pool. But, it is also possible to create a thin snapshot of an external +LV (e.g. linear, striped, thin LV in another thin pool.) The external +LV must be read-only (lvchange --permission r) and inactive to be used as +a thin external origin. Writes to the thin snapshot LV are stored in its +thin pool, and unwritten parts are read from the external origin. One +external origin LV can be used for multiple thin snapshots. +.P +.nf +$ lvcreate --snapshot -n SnapLV --thinpool ThinPool VG/ExternalOrigin +.fi +. +.SS Thin Snapshot and External Origin Conversion +. +In this case, an existing, non-thin LV is converted to a read-only +external origin, and a new thin LV is created as a snapshot of that +external origin. The new thin LV is given the name of the existing LV, +and the existing LV is given a new name from --originname. +.P +Unwritten portions of the new thin LV are read from the external origin. +If the thin LV is removed, the external origin LV can be used again +in read/write mode. Thus, the thin LV can be seen as a snapshot of the +original volume. +.P +.nf +$ lvconvert --type thin --thinpool ThinPool --originname ExtOrigin VG/LV +.fi +.P +The existing LV argument is renamed ExtOrigin, and the new thin LV has the +name of the existing LV. +. +.SS Thin Snapshot Merge +. +A thin snapshot can be merged into its origin thin LV. The result of a +snapshot merge is that the origin thin LV takes the content of the +snapshot LV, and the snapshot LV is removed. Any content that was unique +to the origin thin LV is lost after the merge. +.P +Because a merge changes the content of an LV, it cannot be done while the +LVs are open, e.g. mounted. If a merge is initiated while the LVs are open, +the effect of the merge is delayed until the origin thin LV is next +activated. +.P +.nf +$ lvconvert --merge VG/SnapLV +.fi +.P +.SH EXAMPLES +. +.SS Thin Pool Creation +.nf +# lvcreate --type thin-pool -n pool0 -L 500M vg # lvs -a vg - LV VG Attr LSize Pool Origin Data% Meta% - [lvol0_pmspare] vg ewi------- 4,00m - pool vg twi-aotz-- 12,00m 0,00 10,94 - [pool_tdata] vg Twi-ao---- 12,00m - [pool_tmeta] vg ewi-ao---- 4,00m - thin vg Vwi-a-tz-- 52,00m pool 0,00 -. -.SS \n+[step]. Activate SnapLV -. -Thin snapshots are created with the persistent "activation skip" -flag, indicated by the "k" attribute. Use -K with lvchange -or vgchange to activate thin snapshots with the "k" attribute. -User can preset default behavior with -.BR lvm.conf (5) -.BR auto_set_activation_skip . -.P -.B lvchange -ay -K VG/SnapLV -.P -.I Example -.br -# lvchange -ay -K vg/thin1s1 -.P -# lvs vg/thin1s1 - LV VG Attr LSize Pool Origin - thin1s1 vg Vwi-a-tz-k 1.00t pool0 thin1 -. -.SS \n+[step]. Convert thick LV to thin LV -. -Convert existing thick LV (linear, stripe,...) to thin LV with move of -existing data to thin pool and using thin LV from such thin pool. -Once the volume is converted, user is using it like ordinary thin-pool. -Note: Conversion cannot be reverted and the thin volume cannot be reverted -back to the thick LV type. -.P -.B lvconvert --type thin VG/ThickLV -.P -.I Example -.br -Create thick 12MiB LV. -.br -# lvcreate -L 12M -n thick vg -.P -Convert LV to a thin pool with chunk size 256KiB. -.br -# lvconvert --thin --chunksize 256K vg/thick -.P -# lvs -o+chunksize vg - LV VG Attr LSize Pool Origin Data% Meta% Chunk - thick vg Vwi-a-tz-- 12,00m thick_tpool0 100,00 0 - thick_tpool0 vg twi---tz-- 12,00m 100,00 10,94 256,00k -. -.SS \n+[step]. Create thin pool with compression and deduplication -. -Create a new thin pool with the use of VDO layer for compression and -deduplication of stored data. -.P -.B lvcreate --type thin-pool --pooldatavdo y VG/PoolLV -.P -.I Example -.br -Create a 10G CompressedPool thin pool with VDO data volume whose -VDO slab size is 4096 MiB. -.br -.nf -# lvcreate --type thin-pool -L10G --pooldatavdo y \\ - --vdosettings 'slab_size_mb=4096' VG/CompressedPool + LV VG Attr LSize Data% Meta% + [lvol0_pmspare] vg ewi------- 4.00m + pool0 vg twi-a-tz-- 500.00m 0.00 10.84 + [pool0_tdata] vg Twi-ao---- 500.00m + [pool0_tmeta] vg ewi-ao---- 4.00m .fi . -.SS \n+[step]. Convert an LV into a compressed deduplicated thin pool -. -Convert an existing LV (linear, stripe,...) to the thin pool whose -data volume is using VDO layer for compression and deduplication. -.P -.B lvconvert --type thin-pool --pooldatavdo y VG/LV -.P -.I Example -.br -Convert existing BigVolume LV into a compressed thin pool. -.br -# lvconvert --type thin-pool --pooldatavdo y VG/BigVolume -. -.SH THIN TOPICS -. -.B Automatic pool metadata LV -.br -.B Specify devices for data and metadata LVs -.br -.B Tolerate device failures using raid -.br -.B Spare metadata LV -.br -.B Metadata check and repair -.br -.B Activation of thin snapshots -.br -.B Removing thin pool LVs, thin LVs and snapshots -.br -.B Manually manage free data space of thin pool LV -.br -.B Manually manage free metadata space of a thin pool LV -.br -.B Using fstrim to increase free space in a thin pool LV -.br -.B Automatically extend thin pool LV -.br -.B Data space exhaustion -.br -.B Metadata space exhaustion -.br -.B Automatic extend settings -.br -.B Zeroing -.br -.B Discard -.br -.B Chunk size -.br -.B Size of pool metadata LV -.br -.B Create a thin snapshot of an external, read only LV -.br -.B Convert a standard LV to a thin LV with an external origin -.br -.B Single step thin pool LV creation -.br -.B Single step thin pool LV and thin LV creation -.br -.B Merge thin snapshots -.br -.B XFS on snapshots -. -.SS Automatic pool metadata LV -. -A thin data LV can be converted to a thin pool LV without specifying a -thin pool metadata LV. LVM automatically creates a metadata LV from the -same VG. -.P -.B lvcreate -n ThinDataLV -L LargeSize VG -.br -.B lvconvert --type thin-pool VG/ThinDataLV -.P -.I Example -.br +.SS Thin Pool Conversion .nf -# lvcreate -n pool0 -L 10G vg -# lvconvert --type thin-pool vg/pool0 -.P -# lvs -a - pool0 vg twi-a-tz-- 10.00g - [pool0_tdata] vg Twi-ao---- 10.00g - [pool0_tmeta] vg ewi-ao---- 16.00m +# lvcreate -n pool0 -L 500M vg +# lvcreate -n pool0_meta -L 100M vg +# lvconvert --type thin-pool --poolmetadata pool0_meta vg/pool0 +# lvs -a vg + LV VG Attr LSize Data% Meta% + [lvol0_pmspare] vg ewi------- 100.00m + pool0 vg twi-a-tz-- 500.00m 0.00 10.04 + [pool0_tdata] vg Twi-ao---- 500.00m + [pool0_tmeta] vg ewi-ao---- 100.00m .fi . -.SS Specify devices for data and metadata LVs -. -The data and metadata LVs in a thin pool are best created on -separate physical devices. To do that, specify the device name(s) -at the end of the lvcreate line. It can be especially helpful -to use fast devices for the metadata LV. -.P -.B lvcreate -n ThinDataLV -L LargeSize VG LargePV -.br -.B lvcreate -n ThinMetaLV -L SmallSize VG FastPV -.br -.B lvconvert --type thin-pool --poolmetadata VG/ThinMetaLV VG/ThinDataLV -.P -.I Example +.SS Thin LV Creation .nf -# lvcreate -n pool0 -L 10G vg /dev/sdA -# lvcreate -n pool0meta -L 1G vg /dev/sdB -# lvconvert --type thin-pool --poolmetadata vg/pool0meta vg/pool0 +# lvcreate --type thin-pool -n pool0 -L 500M vg +# lvcreate --type thin -n vol -V 1G --thinpool pool0 vg +# lvs -a vg + LV VG Attr LSize Pool Data% Meta% + [lvol0_pmspare] vg ewi------- 4.00m + pool0 vg twi-aotz-- 500.00m 0.00 10.94 + [pool0_tdata] vg Twi-ao---- 500.00m + [pool0_tmeta] vg ewi-ao---- 4.00m + vol vg Vwi-a-tz-- 1.00g pool0 0.00 +.fi +. +.SS Thin Snapshot Creation +.nf +# lvcreate --type thin-pool -n pool0 -L 500M vg +# lvcreate --type thin -n vol -V 1G --thinpool pool0 vg +# lvcreate --snapshot -n snap1 vg/vol +# lvcreate --snapshot -n snap2 vg/snap1 +# lvs -a vg + LV VG Attr LSize Pool Origin Data% Meta% + [lvol0_pmspare] vg ewi------- 4.00m + pool0 vg twi-aotz-- 500.00m 0.00 10.94 + [pool0_tdata] vg Twi-ao---- 500.00m + [pool0_tmeta] vg ewi-ao---- 4.00m + snap1 vg Vwi---tz-k 1.00g pool0 vol + snap2 vg Vwi---tz-k 1.00g pool0 snap1 + vol vg Vwi-a-tz-- 1.00g pool0 0.00 +.fi +. +.SS Thin Pool Extension +.nf +# lvcreate --type thin-pool -n pool0 -L 500M vg +# lvextend -L+100M vg/pool0 +# lvs -a vg + LV VG Attr LSize Data% Meta% + [lvol0_pmspare] vg ewi------- 4.00m + pool0 vg twi-a-tz-- 600.00m 0.00 10.84 + [pool0_tdata] vg Twi-ao---- 600.00m + [pool0_tmeta] vg ewi-ao---- 4.00m +# lvextend -L+100M --poolmetadatasize 8M vg/pool0 +# lvs -a vg + LV VG Attr LSize Data% Meta% + [lvol0_pmspare] vg ewi------- 8.00m + pool0 vg twi-a-tz-- 700.00m 0.00 10.40 + [pool0_tdata] vg Twi-ao---- 700.00m + [pool0_tmeta] vg ewi-ao---- 8.00m +.fi +. +.SS Thick LV to Thin LV Conversion +.nf +# lvcreate -n vol -L500M vg +# lvconvert --type thin vg/vol +# lvs -a vg + LV VG Attr LSize Pool Data% Meta% + [lvol0_pmspare] vg ewi------- 4.00m + vol vg Vwi-a-tz-- 500.00m vol_tpool0 100.00 + vol_tpool0 vg twi-aotz-- 500.00m 100.00 14.06 + [vol_tpool0_tdata] vg Twi-ao---- 500.00m + [vol_tpool0_tmeta] vg ewi-ao---- 4.00m +# lvextend -L1G vg/vol +# lvs -a vg + LV VG Attr LSize Pool Data% Meta% + [lvol0_pmspare] vg ewi------- 4.00m + vol vg Vwi-a-tz-- 1.00g vol_tpool0 48.83 + vol_tpool0 vg twi-aotz-- 1000.00m 50.00 14.06 + [vol_tpool0_tdata] vg Twi-ao---- 1000.00m + [vol_tpool0_tmeta] vg ewi-ao---- 4.00m .fi .P -.BR lvm.conf (5) +(Extending the virtual size of the thin LV triggered autoextend of the +thin pool.) +. +.SS Thin Pool on LVM RAID +.nf +# lvcreate --type raid1 -n pool0 -m1 -L500M vg +# lvcreate --type raid1 -n pool0_meta -m1 -L8M vg +# lvs -a vg + LV VG Attr LSize Cpy%Sync + pool0 vg rwi-a-r--- 500.00m 100.00 + pool0_meta vg rwi-a-r--- 8.00m 100.00 + [pool0_meta_rimage_0] vg iwi-aor--- 8.00m + [pool0_meta_rimage_1] vg iwi-aor--- 8.00m + [pool0_meta_rmeta_0] vg ewi-aor--- 4.00m + [pool0_meta_rmeta_1] vg ewi-aor--- 4.00m + [pool0_rimage_0] vg iwi-aor--- 500.00m + [pool0_rimage_1] vg iwi-aor--- 500.00m + [pool0_rmeta_0] vg ewi-aor--- 4.00m + [pool0_rmeta_1] vg ewi-aor--- 4.00m +# lvconvert --type thin-pool --poolmetadata pool0_meta vg/pool0 +# lvs -a vg + LV VG Attr LSize Data% Meta% Cpy%Sync + [lvol0_pmspare] vg ewi------- 8.00m + pool0 vg twi-a-tz-- 500.00m 0.00 10.40 + [pool0_tdata] vg rwi-aor--- 500.00m 100.00 + [pool0_tdata_rimage_0] vg iwi-aor--- 500.00m + [pool0_tdata_rimage_1] vg iwi-aor--- 500.00m + [pool0_tdata_rmeta_0] vg ewi-aor--- 4.00m + [pool0_tdata_rmeta_1] vg ewi-aor--- 4.00m + [pool0_tmeta] vg ewi-aor--- 8.00m 100.00 + [pool0_tmeta_rimage_0] vg iwi-aor--- 8.00m + [pool0_tmeta_rimage_1] vg iwi-aor--- 8.00m + [pool0_tmeta_rmeta_0] vg ewi-aor--- 4.00m + [pool0_tmeta_rmeta_1] vg ewi-aor--- 4.00m +.fi +. +.SS Thin Pool on LVM VDO Creation +.nf +# lvcreate --type thin-pool -n pool0 -L5G --pooldatavdo y vg +# lvs -a vg + LV VG Attr LSize Pool Data% Meta% + [lvol0_pmspare] vg ewi------- 8.00m + pool0 vg twi-a-tz-- 5.00g 0.00 10.64 + [pool0_tdata] vg vwi-aov--- 5.00g pool0_vpool0 0.00 + [pool0_tmeta] vg ewi-ao---- 8.00m + pool0_vpool0 vg dwi------- 5.00g 60.03 + [pool0_vpool0_vdata] vg Dwi-ao---- 5.00g +.fi +. +.SS Thin Pool on LVM VDO Conversion +.nf +# lvcreate -n pool0 -L5G vg +# lvconvert --type thin-pool --pooldatavdo y vg/pool0 +# lvs -a vg + LV VG Attr LSize Pool Data% Meta% + [lvol0_pmspare] vg ewi------- 8.00m + pool0 vg twi-a-tz-- 5.00g 0.00 10.64 + [pool0_tdata] vg vwi-aov--- 5.00g pool0_vpool0 0.00 + [pool0_tmeta] vg ewi-ao---- 8.00m + pool0_vpool0 vg dwi------- 5.00g 60.03 + [pool0_vpool0_vdata] vg Dwi-ao---- 5.00g +.fi +. +.SS Thin Snapshot Creation of an External Origin +.nf +# lvcreate -n vol -L 500M vg +# lvchange --permission r vg/vol +# lvchange -an vg/vol +# lvcreate --type thin-pool -n pool0 -L 500M vg +# lvcreate --snapshot -n snap --thinpool pool0 vg/vol +# lvs -a vg + LV VG Attr LSize Pool Origin Data% Meta% + [lvol0_pmspare] vg ewi------- 4.00m + pool0 vg twi-aotz-- 500.00m 0.00 10.94 + [pool0_tdata] vg Twi-ao---- 500.00m + [pool0_tmeta] vg ewi-ao---- 4.00m + snap vg Vwi-a-tz-- 500.00m pool0 vol 0.00 + vol vg ori------- 500.00m +.fi +. +.SS Thin Pool and Thin LV Combined Creation +.nf +# lvcreate --type thin -n vol -V 1G --thinpool pool0 -L500M vg +# lvs -a vg + LV VG Attr LSize Pool Data% Meta% + [lvol0_pmspare] vg ewi------- 4.00m + pool0 vg twi-aotz-- 500.00m 0.00 10.94 + [pool0_tdata] vg Twi-ao---- 500.00m + [pool0_tmeta] vg ewi-ao---- 4.00m + vol vg Vwi-a-tz-- 1.00g pool0 0.00 +.fi +. +.SS Thin Snapshot Merge +.nf +# lvcreate --type thin-pool -n pool0 -L500M vg +# lvcreate --type thin -n vol -V 1G --thinpool pool0 vg +# lvcreate --snapshot -n snap vg/vol +# lvs -a vg + LV VG Attr LSize Pool Origin Data% Meta% + [lvol0_pmspare] vg ewi------- 4.00m + pool0 vg twi-aotz-- 500.00m 0.00 10.94 + [pool0_tdata] vg Twi-ao---- 500.00m + [pool0_tmeta] vg ewi-ao---- 4.00m + snap vg Vwi---tz-k 1.00g pool0 vol + vol vg Vwi-a-tz-- 1.00g pool0 0.00 +# lvconvert --merge vg/snap +# lvs -a vg + LV VG Attr LSize Pool Data% Meta% + [lvol0_pmspare] vg ewi------- 4.00m + pool0 vg twi-aotz-- 500.00m 0.00 10.94 + [pool0_tdata] vg Twi-ao---- 500.00m + [pool0_tmeta] vg ewi-ao---- 4.00m + vol vg Vwi-a-tz-- 1.00g pool0 0.00 +.fi +. +.SS Thin Snapshot Merge Delayed +.nf +# lvcreate --type thin-pool -n pool0 -L500M vg +# lvcreate --type thin -n vol -V 1G --thinpool pool0 vg +# mkfs.xfs /dev/vg/vol +# mount /dev/vg/vol /mnt +# touch /mnt/file1 /mnt/file2 /mnt/file3 +# lvcreate --snapshot -n snap vg/vol +# mount /dev/vg/snap /snap -o nouuid +# touch /snap/file4 /snap/file5 /snap/file6 +# ls /snap +file1 file2 file3 file4 file5 file6 +# ls /mnt +file1 file2 file3 +# lvconvert --merge vg/snap + Logical volume vg/snap contains a filesystem in use. + Delaying merge since snapshot is open. + Merging of thin snapshot vg/snap will occur on next activation of vg/vol. +# umount /snap +# umount /mnt +# lvchange -an vg/vol +# lvs -a vg + LV VG Attr LSize Pool Origin Data% Meta% + [lvol0_pmspare] vg ewi------- 4.00m + pool0 vg twi-aotz-- 500.00m 13.36 11.62 + [pool0_tdata] vg Twi-ao---- 500.00m + [pool0_tmeta] vg ewi-ao---- 4.00m + [snap] vg Swi---tz-k 1.00g pool0 vol + vol vg Owi---tz-- 1.00g pool0 +# lvchange -ay vg/vol +# lvs -a vg + LV VG Attr LSize Pool Data% Meta% + [lvol0_pmspare] vg ewi------- 4.00m + pool0 vg twi-aotz-- 500.00m 12.94 11.43 + [pool0_tdata] vg Twi-ao---- 500.00m + [pool0_tmeta] vg ewi-ao---- 4.00m + vol vg Vwi-a-tz-- 1.00g pool0 6.32 +# mount /dev/vg/vol /mnt +# ls /mnt +file1 file2 file3 file4 file5 file6 +.fi +.P +.SH SPECIAL TOPICS +. SS Physical Devices for Thin Pool Data and Metadata +. +Placing the thin pool data LV and metadata LV on separate physical devices +will improve performance. Faster, redundant devices for metadata is also +recommended. To best customize the data and metadata LVs, create them +separately and then combine them into a thin pool with lvconvert. +.P +To configure lvcreate behavior to place thin pool data and metadata on +separate devices, set lvm.conf: +.br .B thin_pool_metadata_require_separate_pvs -.br -controls the default PV usage for thin pool creation. . -.SS Tolerate device failures using raid -. -To tolerate device failures, use raid for the pool data LV and -pool metadata LV. This is especially recommended for pool metadata LVs. -.P -.B lvcreate --type raid1 -m 1 -n ThinMetaLV -L SmallSize VG PVA PVB -.br -.B lvcreate --type raid1 -m 1 -n ThinDataLV -L LargeSize VG PVC PVD -.br -.B lvconvert --type thin-pool --poolmetadata VG/ThinMetaLV VG/ThinDataLV -.P -.I Example -.nf -# lvcreate --type raid1 -m 1 -n pool0 -L 10G vg /dev/sdA /dev/sdB -# lvcreate --type raid1 -m 1 -n pool0meta -L 1G vg /dev/sdC /dev/sdD -# lvconvert --type thin-pool --poolmetadata vg/pool0meta vg/pool0 -.fi -. -.SS Spare metadata LV +.SS Spare Metadata LV . The first time a thin pool LV is created, lvm will create a spare metadata LV in the VG. This behavior can be controlled with the -option --poolmetadataspare y|n. (Future thin pool creations will -also attempt to create the pmspare LV if none exists.) -.P +option --poolmetadataspare y|n. +. To create the pmspare ("pool metadata spare") LV, lvm first creates an LV with a default name, e.g. lvol0, and then converts this LV to a hidden LV with the _pmspare suffix, e.g. lvol0_pmspare. @@ -440,51 +666,34 @@ One pmspare LV is kept in a VG to be used for any thin pool. The pmspare LV cannot be created explicitly, but may be removed explicitly. .P -.I Example -.nf -# lvcreate -n pool0 -L 10G vg -# lvcreate -n pool0meta -L 1G vg -# lvconvert --type thin-pool --poolmetadata vg/pool0meta vg/pool0 -.P -# lvs -a - [lvol0_pmspare] vg ewi------- - pool0 vg twi---tz-- - [pool0_tdata] vg Twi------- - [pool0_tmeta] vg ewi------- -.fi -.P -The "Metadata check and repair" section describes the use of +The "Thin Pool Metadata check and repair" section describes the use of the pmspare LV. . -.SS Metadata check and repair +.SS Thin Pool Metadata check and repair . -If thin pool metadata is damaged, it may be repairable. -Checking and repairing thin pool metadata is analogous to -running fsck/repair on a file system. +If thin pool metadata is damaged, it may be repairable. Checking and +repairing thin pool metadata is analogous to running fsck/repair on a file +system. Thin pool metadata is compact, so even small areas of damage or +corruption can result in significant data loss. Resilient storage for +thin pool metadata can have extra value. .P When a thin pool LV is activated, lvm runs the .BR thin_check (8) command to check the correctness of the metadata on the pool metadata LV. -.P -.BR lvm.conf (5) +To configure thin_check use, location or options used by lvm, set +lvm.conf: + .B thin_check_executable .br -can be set to an empty string ("") to disable the -.BR thin_check (8) -step. This is not recommended. +The location of the program. Setting to an empty string ("") disables +running thin_check by lvm. This is not recommended. .P -.BR lvm.conf (5) .B thin_check_options .br -controls the command options used for the -.BR thin_check (8) -command. +Controls the command options that lvm will use when running thin_check. .P -If the -.BR thin_check (8) -command finds a problem with the metadata, -the thin pool LV is not activated, and the thin pool metadata needs -to be repaired. +If thin_check finds a problem with the metadata, the thin pool LV is not +activated, and the thin pool metadata needs to be repaired. .P Simple repair commands are not always successful. Advanced repair may require editing thin pool metadata and lvm metadata. Newer versions of @@ -493,7 +702,9 @@ details of damaged thin metadata to get the best advice on recovery. .P Command to repair a thin pool: .br -.B lvconvert --repair VG/ThinPoolLV +.nf +$ lvconvert --repair VG/ThinPool +.fi .P Repair performs the following steps: .P @@ -512,87 +723,33 @@ Replaces the thin pool metadata LV. If step 1 is successful, the thin pool metadata LV is replaced with the pmspare LV containing the corrected metadata. The previous thin pool metadata LV, containing the damaged metadata, -becomes visible with the new name ThinPoolLV_metaN (where N is 0,1,...). +becomes visible with the new name ThinPool_metaN (where N is 0,1,...). .P If the repair works, the thin pool LV and its thin LVs can be activated. -User should manually check if repaired thin pool kernel metadata -has all data for all lvm2 known LVs by individual activation of -every thin LV. When all works, user should continue with fsck of -all filesystems present on these volumes. -Once the thin pool is considered fully functional user may remove ThinPoolLV_metaN -(the LV containing the damaged thin pool metadata) for possible -space reuse. -For a better performance it may be useful to pvmove the new repaired metadata LV -(written to previous pmspare volume) to a faster PV, e.g. SSD. +The user should verify that each thin LV in the thin pool can be +successfully activated, and then verify the integrity of the file system +on each thin LV (e.g. using fsck or other tools.) +Once the thin pool is considered fully recovered, the ThinPool_metaN LV +containing the original, damaged metadata can be manually removed to +recovery the space. .P -If the repair operation fails, the thin pool LV and its thin LVs -are not accessible and it may be necessary to restore their content -from a backup. In such case the content of unmodified original damaged -ThinPoolLV_metaN volume can be used by your support for more -advanced recovery methods. +If the repair fails, the original, unmodified ThinPool_metaN LV should be +preserved for support, or more advanced recovery methods. Data from thin +LVs may ultimately be unrecoverable. .P -If metadata is manually restored with -.BR thin_repair (8) -directly, the pool metadata LV can be manually swapped with another LV -containing new metadata: -.P -.B lvconvert --thinpool VG/ThinPoolLV --poolmetadata VG/NewThinMetaLV -.P -Note: Thin pool metadata is compact so even small corruptions -in them may result in significant portions of mappings to be lost. -It is recommended to use fast resilient storage for them. -. -.SS Activation of thin snapshots -. -When a thin snapshot LV is created, it is by default given the -"activation skip" flag. This flag is indicated by the "k" attribute -displayed by lvs: +If metadata is manually restored with thin_repair directly, the pool +metadata LV can be manually swapped with another LV containing new +metadata: .P .nf -# lvs vg/thin1s1 - LV VG Attr LSize Pool Origin - thin1s1 vg Vwi---tz-k 1.00t pool0 thin1 +$ lvconvert --thinpool VG/ThinPool --poolmetadata VG/NewMetadataLV .fi -.P -This flag causes the snapshot LV to be skipped, i.e. not activated, -by normal activation commands. The skipping behavior does not -apply to deactivation commands. -.P -A snapshot LV with the "k" attribute can be activated using -the -K (or --ignoreactivationskip) option in addition to the -standard -ay (or --activate y) option. -.P -Command to activate a thin snapshot LV: -.br -.B lvchange -ay -K VG/SnapLV -.P -The persistent "activation skip" flag can be turned off during -lvcreate, or later with lvchange using the -kn -(or --setactivationskip n) option. -It can be turned on again with -ky (or --setactivationskip y). -.P -When the "activation skip" flag is removed, normal activation -commands will activate the LV, and the -K activation option is -not needed. -.P -Command to create snapshot LV without the activation skip flag: -.br -.B lvcreate -kn -n SnapLV -s VG/ThinLV -.P -Command to remove the activation skip flag from a snapshot LV: -.br -.B lvchange -kn VG/SnapLV -.P -.BR lvm.conf (5) -.B auto_set_activation_skip -.br -controls the default activation skip setting used by lvcreate. . .SS Removing thin pool LVs, thin LVs and snapshots . -Removing a thin LV and its related snapshots returns the blocks it -used to the thin pool LV. These blocks will be reused for other -thin LVs and snapshots. +Removing a thin LV and its related snapshots returns the blocks they used +to the thin pool. These blocks will be reused for other thin LVs and +snapshots. .P Removing a thin pool LV removes both the data LV and metadata LV and returns the space to the VG. @@ -602,85 +759,18 @@ reversed with vgcfgrestore. .P vgcfgbackup does not back up thin pool metadata. . -.SS Manually manage free data space of thin pool LV +.SS Using fstrim to increase free space in a thin pool . -The available free space in a thin pool LV can be displayed -with the lvs command. Free space can be added by extending -the thin pool LV. +Removing files in a file system on a thin LV does not generally return +free space to the thin pool, because file systems are not usually mounted +with the discard mount option (due to the performance penalty.) .P -Command to extend thin pool data space: -.br -.B lvextend -L Size VG/ThinPoolLV -.P -.I Example -.br -.nf -1. A thin pool LV is using 26.96% of its data blocks. -# lvs - LV VG Attr LSize Pool Origin Data% - pool0 vg twi-a-tz-- 10.00g 26.96 -.P -2. Double the amount of physical space in the thin pool LV. -# lvextend -L+10G vg/pool0 -.P -3. The percentage of used data blocks is half the previous value. -# lvs - LV VG Attr LSize Pool Origin Data% - pool0 vg twi-a-tz-- 20.00g 13.48 -.fi -.P -Other methods of increasing free data space in a thin pool LV -include removing a thin LV and its related snapshots, or running -fstrim on the file system using a thin LV. -. -.SS Manually manage free metadata space of a thin pool LV -. -The available metadata space in a thin pool LV can be displayed -with the lvs -o+metadata_percent command. -.P -Command to extend thin pool metadata space: -.br -.B lvextend --poolmetadatasize Size VG/ThinPoolLV -.P -.I Example -.br -1. A thin pool LV is using 12.40% of its metadata blocks. -.nf -# lvs -oname,size,data_percent,metadata_percent vg/pool0 - LV LSize Data% Meta% - pool0 20.00g 13.48 12.40 -.fi -.P -2. Display a thin pool LV with its component thin data LV and thin metadata LV. -.nf -# lvs -a -oname,attr,size vg - LV Attr LSize - pool0 twi-a-tz-- 20.00g - [pool0_tdata] Twi-ao---- 20.00g - [pool0_tmeta] ewi-ao---- 12.00m -.fi -.P -3. Double the amount of physical space in the thin metadata LV. -.nf -# lvextend --poolmetadatasize +12M vg/pool0 -.fi -.P -4. The percentage of used metadata blocks is half the previous value. -.nf -# lvs -a -oname,size,data_percent,metadata_percent vg - LV LSize Data% Meta% - pool0 20.00g 13.48 6.20 - [pool0_tdata] 20.00g - [pool0_tmeta] 24.00m -.fi -. -.SS Using fstrim to increase free space in a thin pool LV -. -Removing files in a file system on top of a thin LV does not -generally add free space back to the thin pool. Manually running -the fstrim command can return space back to the thin pool that had -been used by removed files. fstrim uses discards and will not work -if the thin pool LV has discards mode set to ignore. +Manually running the fstrim command can return space from a thin LV back +to the thin pool that had been used by removed files. This is only +effective for entire thin pool chunks that have become unused (unused file +system areas may not cover an entire chunk.) Thin snapshots also keep +thin pool chunks from being freed. fstrim uses discards and will have no +effect if the thin pool is configured to ignore discards. .P .I Example .br @@ -726,110 +816,49 @@ Filesystem Size Used Avail Use% Mounted on thin1 vg Vwi-aotz-- 100.00g pool0 2.70 .fi .P -The "Discard" section covers an option for automatically freeing data -space in a thin pool. . -.SS Automatically extend thin pool LV -. -The lvm daemon dmeventd (lvm2-monitor) monitors the data usage of thin -pool LVs and extends them when the usage reaches a certain level. The -necessary free space must exist in the VG to extend thin pool LVs. -Monitoring and extension of thin pool LVs are controlled independently. -.P -\(em Monitoring \(em -.P -When a thin pool LV is activated, dmeventd will begin monitoring it by -default. -.sp -Command to start or stop dmeventd monitoring a thin pool LV: -.br -.B lvchange --monitor y|n VG/ThinPoolLV -.sp -The current dmeventd monitoring status of a thin pool LV can be displayed -with the command lvs -o+seg_monitor. -.P -\(em Autoextending \(em -.P -dmeventd should be configured to extend thin pool LVs before all data -space is used. Warnings are emitted through syslog when the use of a thin -pool reaches 80%, 85%, 90% and 95%. (See the section "Data space -exhaustion" for the effects of not extending a thin pool LV.) The point -at which dmeventd extends thin pool LVs, and the amount are controlled -with two configuration settings: -.P -.BR lvm.conf (5) -.B thin_pool_autoextend_threshold -.br -is a percentage full value that defines when the thin pool LV should be -extended. Setting this to 100 disables automatic extension. The minimum -value is 50. -.P -.BR lvm.conf (5) -.B thin_pool_autoextend_percent -.br -defines how much extra data space should be added to the thin pool LV from -the VG, in percent of its current size. -.P -\(em Disabling \(em -.P -There are multiple ways that extension of thin pools could be prevented: -.IP \[bu] 2 -If the dmeventd daemon is not running, no monitoring or automatic -extension will occur. -. -.IP \[bu] -Even when dmeventd is running, all monitoring can be disabled with the -lvm.conf monitoring setting. -. -.IP \[bu] -To activate or create a thin pool LV without interacting with dmeventd, -the --ignoremonitoring option can be used. With this option, the command -will not ask dmeventd to monitor the thin pool LV. -. -.IP \[bu] -Setting thin_pool_autoextend_threshold to 100 disables automatic -extension of thin pool LVs, even if they are being monitored by dmeventd. -.P -.I Example -.br -If thin_pool_autoextend_threshold is 70 and thin_pool_autoextend_percent is 20, -whenever a pool exceeds 70% usage, it will be extended by another 20%. -For a 1G pool, using 700M will trigger a resize to 1.2G. When the usage exceeds -840M, the pool will be extended to 1.44G, and so on. -. -.SS Data space exhaustion +.SS Thin Pool Data Exhaustion . When properly managed, thin pool data space should be extended before it -is all used (see the section "Automatically extend thin pool LV"). If -thin pool data space is already exhausted, it can still be extended (see -the section "Manually manage free data space of thin pool LV".) +is all used (see sections on extending a thin pool automatically and +manually.) .P -The behavior of a full thin pool is configurable with the --errorwhenfull -y|n option to lvcreate or lvchange. The errorwhenfull setting applies -only to writes; reading thin LVs can continue even when data space is -exhausted. +However, if a thin pool does run out of space, the behavior of the full +thin pool can be configured with the "when full" property, reported with +lvs -o whenfull. The "when full" property can be set to "error" or +"queue". When set to "error", a full thin pool will immediately return +errors for writes. When set to "queue", writes are queued for a period of +time. .P -Command to change the handling of a full thin pool: +Display the current "when full" setting: +.nf +$ lvs -o whenfull VG/ThinPool +.fi +.P +Set the "when full" property to "error": +.nf +$ lvchange --errorwhenfull y VG/ThinPool +.fi +.P +Set the "when full" property to "queue": +.nf +$ lvchange --errorwhenfull n VG/ThinPool +.fi +.P +To configure the value that will be assigned to new thin pools, set +lvm.conf: .br -.B lvchange --errorwhenfull y|n VG/ThinPoolLV -.P -.BR lvm.conf (5) .B error_when_full -.br -controls the default error when full behavior. .P -The current setting of a thin pool LV can be displayed with the command: -lvs -o+lv_when_full. -.P -The errorwhenfull setting does not effect the monitoring and autoextend +The whenfull setting does not effect the monitoring and autoextend settings, and the monitoring/autoextend settings do not effect the -errorwhenfull setting. It is only when monitoring/autoextend are not -effective that the thin pool becomes full and the errorwhenfull setting is -applied. +whenfull setting. It is only when monitoring/autoextend are not effective +that the thin pool becomes full and the whenfull setting is applied. .P -\(em errorwhenfull n \(em +\(em queue when full \(em .P -This is the default. Writes to thin LVs are accepted and queued, with the +The default is to queue writes for a period of time when the thin pool +becomes full. Writes to thin LVs are accepted and queued, with the expectation that pool data space will be extended soon. Once data space is extended, the queued writes will be processed, and the thin pool will return to normal operation. @@ -837,9 +866,9 @@ return to normal operation. While waiting to be extended, the thin pool will queue writes for up to 60 seconds (the default). If data space has not been extended after this time, the queued writes will return an error to the caller, e.g. the file -system. This can result in file system corruption for non-journaled file -systems that may require repair. When a thin pool returns errors for writes -to a thin LV, any file system is subject to losing unsynced user data. +system. This can result in file system damage that requires repair. +When a thin pool returns errors for writes to a thin LV, any file system +is subject to losing unsynced user data. .P The 60 second timeout can be changed or disabled with the dm-thin-pool kernel module option @@ -849,11 +878,10 @@ If set to 0, writes will not time out. Disabling timeouts can result in the system running out of resources, memory exhaustion, hung tasks, and deadlocks. (The timeout applies to all thin pools on the system.) .P -\(em errorwhenfull y \(em +\(em error when full \(em .P Writes to thin LVs immediately return an error, and no writes are queued. -In the case of a file system, this can result in corruption that may -require fs repair (the specific consequences depend on the thin LV user.) +This can result in file system damage that requires repair. .P \(em data percent \(em .P @@ -861,9 +889,9 @@ When data space is exhausted, the lvs command displays 100 under Data% for the thin pool LV: .P .nf -# lvs vg/pool0 - LV VG Attr LSize Pool Origin Data% - pool0 vg twi-a-tz-- 512.00m 100.00 +# lvs -o name,data_percent vg/pool0 + LV Data% + pool0 100.00 .fi .P \(em causes \(em @@ -881,13 +909,12 @@ extended. (Disabling dmeventd is not recommended.) .IP \[bu] Automatic extension of the thin pool is too slow given the rate of writes to thin LVs in the pool. (This can be addressed by tuning the -thin_pool_autoextend_threshold and thin_pool_autoextend_percent. -See "Automatic extend settings".) +thin_pool_autoextend_threshold and thin_pool_autoextend_percent.) . .IP \[bu] The VG does not have enough free blocks to extend the thin pool. . -.SS Metadata space exhaustion +.SS Thin Pool Metadata Exhaustion . If thin pool metadata space is exhausted (or a thin pool metadata operation fails), errors will be returned for IO operations on thin LVs. @@ -896,9 +923,9 @@ When metadata space is exhausted, the lvs command displays 100 under Meta% for the thin pool LV: .P .nf -# lvs -o lv_name,size,data_percent,metadata_percent vg/pool0 - LV LSize Data% Meta% - pool0 100.00 +# lvs -o name,metadata_percent vg/pool0 + LV Meta% + pool0 100.00 .fi .P The same reasons for thin pool data space exhaustion apply to thin pool @@ -914,58 +941,24 @@ Deactivate the thin pool LV, or reboot the system if this is not possible. 2. Repair thin pool with lvconvert --repair. .br -See "Metadata check and repair". +See "Thin Pool Metadata check and repair". .TP 3. Extend pool metadata space with lvextend --poolmetadatasize. .br -See "Manually manage free metadata space of a thin pool LV". +See "Thin Pool Extension". .TP 4. Check and repair file system. . -.SS Automatic extend settings +.SS Custom Thin Pool Configuration . -Thin pool LVs can be extended according to preset values. The presets -determine if the LV should be extended based on how full it is, and if so -by how much. When dmeventd monitors thin pool LVs, it uses lvextend with -these presets. (See "Automatically extend thin pool LV".) -.P -Command to extend a thin pool data LV using presets: -.br -.B lvextend --use-policies VG/ThinPoolLV -.P -The command uses these settings: -.P -.BR lvm.conf (5) -.B thin_pool_autoextend_threshold -.br -autoextend the LV when its usage exceeds this percent. -.P -.BR lvm.conf (5) -.B thin_pool_autoextend_percent -.br -autoextend the LV by this much additional space. -.P -To see the default values of these settings, run: -.P -.B lvmconfig --type default --withcomment -.RS -.B activation/thin_pool_autoextend_threshold -.RE -.P -.B lvmconfig --type default --withcomment -.RS -.B activation/thin_pool_autoextend_percent -.RE -.P -To change these values globally, edit -.BR lvm.conf (5). -.P -To change these values on a per-VG or per-LV basis, attach a "profile" to -the VG or LV. A profile is a collection of config settings, saved in a -local text file (using the lvm.conf format). lvm looks for profiles in -the profile_dir directory, e.g. \fI#DEFAULT_SYS_DIR#/profile/\fP. Once attached to a VG +It can be useful for different thin pools to have different thin pool +settings like autoextend thresholds and percents. To change lvm.conf +values on a per-VG or per-LV basis, attach a "profile" to the VG or LV. A +profile is a collection of config settings, saved in a local text file +(using the lvm.conf format). lvm looks for profiles in the profile_dir +directory, e.g. \fI#DEFAULT_SYS_DIR#/profile/\fP. Once attached to a VG or LV, lvm will process the VG or LV using the settings from the attached profile. A profile is named and referenced by its file name. .P @@ -976,17 +969,23 @@ Create a file containing settings, saved in profile_dir. .br For the profile_dir location, run: .br -.B lvmconfig config/profile_dir +.nf +$ lvmconfig config/profile_dir +.fi . .IP \[bu] Attach the profile to an LV, using the command: .br -.B lvchange --metadataprofile ProfileName VG/ThinPoolLV +.nf +$ lvchange --metadataprofile ProfileName VG/ThinPool +.fi . .IP \[bu] Extend the LV using the profile settings: .br -.B lvextend --use-policies VG/ThinPoolLV +.nf +$ lvextend --use-policies VG/ThinPool +.fi .P .I Example .br @@ -1017,7 +1016,9 @@ file with the profile also needs to be moved. .IP \[bu] Only certain settings can be used in a VG or LV profile, see: .br -.B lvmconfig --type profilable-metadata. +.nf +$ lvmconfig --type profilable-metadata +.fi . .IP \[bu] An LV without a profile of its own will inherit the VG profile. @@ -1025,7 +1026,9 @@ An LV without a profile of its own will inherit the VG profile. .IP \[bu] Remove a profile from an LV using the command: .br -.B lvchange --detachprofile VG/ThinPoolLV. +.nf +$ lvchange --detachprofile VG/ThinPool +.fi . .IP \[bu] Commands can also have profiles applied to them. The settings that can be @@ -1036,352 +1039,121 @@ and run the command using the option: --commandprofile ProfileName. . .SS Zeroing . -When a thin pool provisions a new data block for a thin LV, the -new block is first overwritten with zeros. The zeroing mode is -indicated by the "z" attribute displayed by lvs. The option -Z -(or --zero) can be added to commands to specify the zeroing mode. +The "zero" property of a thin pool determines if chunks are overwritten +with zeros when they are provisioned for a thin LV. The current setting +is reported with lvs -o zero (displaying "zero" or "1" when zeroing is +enabled), or 'z' in the eigth lv_attr. The option -Z|--zero is used to +specify the zeroing mode. + +Create a thin pool with zeroing mode: .P -Command to set the zeroing mode when creating a thin pool LV: +.nf +$ lvcreate --type thin-pool -n ThinPool -L Size -Z y|n VG +.fi .P -.B lvconvert --type thin-pool -Z y|n -.RS -.B --poolmetadata VG/ThinMetaLV VG/ThinDataLV -.RE +Change the zeroing mode of an existing thin pool: .P -Command to change the zeroing mode of an existing thin pool LV: -.P -.B lvchange -Z y|n VG/ThinPoolLV +.nf +$ lvchange -Z y|n VG/ThinPool +.fi .P If zeroing mode is changed from "n" to "y", previously provisioned blocks are not zeroed. .P -Provisioning of large zeroed chunks impacts performance. +Provisioning of large zeroed chunks reduces performance. .P -.BR lvm.conf (5) -.B thin_pool_zero +To configure the zeroing mode used for new thin pools when not specified +on the command line, set lvm.conf: .br -controls the default zeroing mode used when creating a thin pool. +.B thin_pool_zero . .SS Discard . -The discard behavior of a thin pool LV determines how discard requests are -handled. Enabling discard under a file system may adversely affect the -file system performance (see the section on fstrim for an alternative.) -Possible discard behaviors: +The "discards" property of a thin pool determines how discard requests are +handled. The current setting is reported with lvs -o discards. The +option --discards is used to specify the discards mode. + +Possible discard modes: .P .B ignore: Ignore any discards that are received. .P .B nopassdown: -Process any discards in the thin pool itself and allow -the no longer needed extents to be overwritten by new data. +Process any discards in the thin pool itself, and allow the +newly unused chunks to be used for new data. .P .B passdown: Process discards in the thin pool (as with nopassdown), and pass the discards down the the underlying device. This is the default mode. .P -Command to display the current discard mode of a thin pool LV: -.br -.B lvs -o+discards VG/ThinPoolLV -.P -Command to set the discard mode when creating a thin pool LV: -.br -.B lvconvert --discards ignore|nopassdown|passdown -.RS -.B --type thin-pool --poolmetadata VG/ThinMetaLV VG/ThinDataLV -.RE -.P -Command to change the discard mode of an existing thin pool LV: -.br -.B lvchange --discards ignore|nopassdown|passdown VG/ThinPoolLV -.P -.I Example +Create a thin pool with a specific discards mode: .nf -# lvs -o name,discards vg/pool0 - pool0 passdown -.P -# lvchange --discards ignore vg/pool0 +$ lvcreate --type thin-pool -n ThinPool -L Size +.RS +--discards ignore|nopassdown|passdown VG +.RE .fi .P -.BR lvm.conf (5) -.B thin_pool_discards +Change the discards mode of an existing thin pool: .br -controls the default discards mode used when creating a thin pool. +.nf +$ lvchange --discards ignore|nopassdown|passdown VG/ThinPool +.fi +.P +To configure the discards mode used for new thin pools when not specified +on the command line, set lvm.conf: +.br +.B thin_pool_discards +.P +Discards can have an adverse impact on performance, see the fstrim section +for more information. . .SS Chunk size . -The size of data blocks managed by a thin pool can be specified with the ---chunksize option when the thin pool LV is created. The default unit -is KiB. The value must be a multiple of 64KiB between 64KiB and 1GiB. +A thin pool allocates physical storage for thin LVs in units of "chunks". +The current chunk size of a thin pool is reported with lvs -o chunksize. +The option --chunksize is used to specify the value for a new thin pool +(default units are KiB.) The value must be a multiple of 64KiB, between +64KiB and 1GiB. .P When a thin pool is used primarily for the thin provisioning feature, a larger value is optimal. To optimize for many snapshots, a smaller value reduces copying time and consumes less space. .P -Command to display the thin pool LV chunk size: -.P -.B lvs -o+chunksize VG/ThinPoolLV -.P -.I Example -.nf -# lvs -o name,chunksize - pool0 64.00k -.fi -.P -.BR lvm.conf (5) +To configure the chunk size used for new thin pools when not specified on +the command line, set lvm.conf: +.br .B thin_pool_chunk_size -.br -controls the default chunk size used when creating a thin pool. -.P + The default value is shown by: -.br -.B lvmconfig --type default allocation/thin_pool_chunk_size -.P +.nf +$ lvmconfig --type default allocation/thin_pool_chunk_size +.fi . -.SS Size of pool metadata LV +.SS Thin Pool Metadata Size . -The amount of thin metadata depends on how many blocks are shared between -thin LVs (i.e. through snapshots). A thin pool with many snapshots may -need a larger metadata LV. Thin pool metadata LV sizes can be from 2MiB -to approximately 16GiB. +The amount of thin pool metadata depends on how many blocks are shared +between thin LVs (i.e. through snapshots). A thin pool with many +snapshots may need a larger metadata LV. Thin pool metadata LV sizes can +be from 2MiB to approximately 16GiB. .P -When using lvcreate to create what will become a thin metadata LV, the -size is specified with the -L|--size option. -.P -When an LVM command automatically creates a thin metadata LV, the size is -specified with the --poolmetadatasize option. When this option is not -given, LVM automatically chooses a size based on the data size and chunk -size. +When an LVM command automatically creates a thin pool metadata LV, the +size is specified with the --poolmetadatasize option. When this option is +not given, LVM automatically chooses a size based on the data size and +chunk size. .P It can be hard to predict the amount of metadata space that will be needed, so it is recommended to start with a size of 1GiB which should be enough for all practical purposes. A thin pool metadata LV can later be manually or automatically extended if needed. .P -Configurable setting -.BR lvm.conf (5) -.BR allocation / thin_pool_crop_metadata -gives control over cropping to 15.81GiB to stay backward compatible with older -versions of lvm2. With enabled cropping there can be observed some problems when -using volumes above this size with thin tools (i.e. -.BR thin_repair (8)). -Cropping should be enabled only when compatibility is required. -. -.SS Create a thin snapshot of an external, read only LV -. -Thin snapshots are typically taken of other thin LVs within the same thin pool. -But it is also possible to create thin snapshot of an external LV -(linear, stripe, thin in different thin pool, ...). -Only read only inactive volumes can be used as an external origin. -Writes to such thin volumes are then stored in its thin pool, -and unwritten parts are served from the external origin volume. -External origins can be actived in read only mode afterwards -and used for many thin volumes. -.P -.B lvcreate -n SnapLV -s VG/ExternalOriginLV --thinpool VG/ThinPoolLV -.P -Alternative syntax: -.P -.B lvcreate --type thin -n NewThinLV --thinpool VG/NewThinPoolLV VG/ThinEOriginLV -.P -.I Example -.nf -# lvchange -an vg/lve -# lvchange --permission r vg/lve -# lvcreate -n snaplve -s vg/lve --thinpool vg/pool0 -.P -# lvs vg/lve vg/snaplve - LV VG Attr LSize Pool Origin Data% - lve vg ori------- 10.00g - snaplve vg Vwi-a-tz-- 10.00g pool0 lve 0.00 -.fi -. -.SS Convert a standard LV to a thin LV with an external origin -. -A new thin LV replaces existing LV and at the same time, -the existing LV is converted to a new created read only -external origin LV whose name can be given with --originname. -When not specified a default name is in form lvol#. -Unwritten portions for this thin LV are read from the external origin LV. -When thin LV is removed, external origin LV can be again use read write -mode, thus such conversion can be seen as a snapshot of such volume. -.P -Convert ExampleLV into a read only external origin LV with the new name -NewExternalOriginLV, and create a new thin LV that is given the previous -name of ExampleLV. -.P -.B lvconvert --type thin --thinpool VG/ThinPoolLV -.RS -.B --originname NewExternalOriginLV VG/ExampleLV -.RE -.P -.I Example -.nf -# lvcreate -n lv_example -L 10G vg -.P -# lvs - lv_example vg -wi-a----- 10.00g -.P -# lvconvert --type thin --thinpool vg/pool0 - --originname lv_external --thin vg/lv_example -.P -# lvs - LV VG Attr LSize Pool Origin - lv_example vg Vwi-a-tz-- 10.00g pool0 lv_external - lv_external vg ori------- 10.00g -.P -.fi -Remove thin LV and restore back the original volume lv_example. -.P -.I Example -.nf -# lvremove -f vg/lv_example -.P -# lvs - LV VG Attr LSize Pool Origin - lv_external vg -ri------- 10,00g -.P -# lvchange --permission rw vg/lv_external -.P -# lvrename vg/lv_external vg/lv_example -.P -# lvchange -ay vg/lv_example -.P -# lvs - LV VG Attr LSize Pool Origin - lv_external vg -wi-a----- 10,00g -.fi -. -.SS Single step thin pool LV creation -. -A thin pool LV can be created with a single lvcreate command, -rather than using lvconvert on existing LVs. -This one command creates a thin data LV, a thin metadata LV, -and combines the two into a thin pool LV. -.P -.B lvcreate --type thin-pool -L LargeSize -n ThinPoolLV VG -.P -.I Example -.nf -# lvcreate --type thin-pool -L8M -n pool0 vg -.P -# lvs vg/pool0 - LV VG Attr LSize Pool Origin Data% - pool0 vg twi-a-tz-- 8.00m 0.00 -.P -# lvs -a - pool0 vg twi-a-tz-- 8.00m - [pool0_tdata] vg Twi-ao---- 8.00m - [pool0_tmeta] vg ewi-ao---- 8.00m -.fi -. -.SS Single step thin pool LV and thin LV creation -. -A thin pool LV and a thin LV can be created with a single -lvcreate command. This one command creates a thin data LV, -a thin metadata LV, combines the two into a thin pool LV, -and creates a thin LV in the new pool. -.br --L LargeSize specifies the physical size of the thin pool LV. -.br --V VirtualSize specifies the virtual size of the thin LV. -.P -.B lvcreate --type thin -V VirtualSize -L LargeSize -.RS -.B -n ThinLV --thinpool VG/ThinPoolLV -.RE -.P -Equivalent to: -.br -.B lvcreate --type thin-pool -L LargeSize VG/ThinPoolLV -.br -.B lvcreate -n ThinLV -V VirtualSize --thinpool VG/ThinPoolLV -.P -.I Example -.nf -# lvcreate -L8M -V2G -n thin1 --thinpool vg/pool0 -.P -# lvs -a - pool0 vg twi-a-tz-- 8.00m - [pool0_tdata] vg Twi-ao---- 8.00m - [pool0_tmeta] vg ewi-ao---- 8.00m - thin1 vg Vwi-a-tz-- 2.00g pool0 -.fi -. -.SS Merge thin snapshots -. -A thin snapshot can be merged into its origin thin LV using the lvconvert ---merge command. The result of a snapshot merge is that the origin thin -LV takes the content of the snapshot LV, and the snapshot LV is removed. -Any content that was unique to the origin thin LV is lost after the merge. -.P -Because a merge changes the content of an LV, it cannot be done while the -LVs are open, e.g. mounted. If a merge is initiated while the LVs are open, -the effect of the merge is delayed until the origin thin LV is next -activated. -.P -.B lvconvert --merge VG/SnapLV -.P -.I Example -.nf -# lvs vg - LV VG Attr LSize Pool Origin - pool0 vg twi-a-tz-- 10.00g - thin1 vg Vwi-a-tz-- 100.00g pool0 - thin1s1 vg Vwi-a-tz-k 100.00g pool0 thin1 -.P -# lvconvert --merge vg/thin1s1 -.P -# lvs vg - LV VG Attr LSize Pool Origin - pool0 vg twi-a-tz-- 10.00g - thin1 vg Vwi-a-tz-- 100.00g pool0 -.fi -.P -.I Example -.nf -Delayed merging of open LVs. -.P -# lvs vg - LV VG Attr LSize Pool Origin - pool0 vg twi-a-tz-- 10.00g - thin1 vg Vwi-aotz-- 100.00g pool0 - thin1s1 vg Vwi-aotz-k 100.00g pool0 thin1 -.P -# df -/dev/mapper/vg-thin1 100G 33M 100G 1% /mnt/X -/dev/mapper/vg-thin1s1 100G 33M 100G 1% /mnt/Xs -.P -# ls /mnt/X -file1 file2 file3 -# ls /mnt/Xs -file3 file4 file5 -.P -# lvconvert --merge vg/thin1s1 -Logical volume vg/thin1s1 contains a filesystem in use. -Delaying merge since snapshot is open. -Merging of thin snapshot thin1s1 will occur on next activation. -.P -# umount /mnt/X -# umount /mnt/Xs -.P -# lvs -a vg - LV VG Attr LSize Pool Origin - pool0 vg twi-a-tz-- 10.00g - [pool0_tdata] vg Twi-ao---- 10.00g - [pool0_tmeta] vg ewi-ao---- 1.00g - thin1 vg Owi-a-tz-- 100.00g pool0 - [thin1s1] vg Swi-a-tz-k 100.00g pool0 thin1 -.P -# lvchange -an vg/thin1 -# lvchange -ay vg/thin1 -.P -# mount /dev/vg/thin1 /mnt/X -.P -# ls /mnt/X -file3 file4 file5 -.fi +(For purposes of backward compatibility, lvm.conf setting +allocation/thin_pool_crop_metadata controls cropping the metadata LV size +to 15.81GiB to be backward compatible with older versions of lvm. With +cropping, there can be problems with volumes above this size when used +with thin tools, i.e. thin_repair. Cropping should be enabled only when +compatibility is required.) . .SS XFS on snapshots .