详解基于Linux的LVM无缝磁盘水平扩容-巨人网络通讯

主页 > 知识库 > 详解基于Linux的LVM无缝磁盘水平扩容

详解基于Linux的LVM无缝磁盘水平扩容

环境

名称	属性
CPU	x5650
内存	4G
磁盘	20G+4TB

这时候，我们假设服务器已经有如下东西：

安装好了Parted（yum install parted）
分区容量已经告捷，公司为服务器增加了一块4T硬盘用来增加容量！
除了/boot分区外，其余都为LVM 模式的扩展逻辑分区

软件安装与信息查看

首先查看软件是否安装：

[root@Candy ~]# rpm -qa | grep lvm
lvm2-libs-2.02.118-2.el6.x86_64
lvm2-2.02.118-2.el6.x86_64
[root@Candy ~]# lv
lvchange   lvextend   lvmdiskscan lvmsar    lvresize
lvconvert  lvm     lvmdump   lvreduce   lvs
lvcreate   lvmchange  lvmetad   lvremove   lvscan
lvdisplay  lvmconf   lvmsadc   lvrename
[root@Candy ~]# rpm -qa | grep parted
parted-2.1-29.el6.x86_64

软件准备好后顺便查看下 VM 硬件:

[root@Candy ~]# df -h
Filesystem      Size Used Avail Use% Mounted on
/dev/mapper/vg_candy-root
           7.3G 231M 6.7G  4% /
tmpfs         1.9G   0 1.9G  0% /dev/shm
/dev/sda1       477M  30M 422M  7% /boot
/dev/mapper/vg_candy-home
           3.9G 8.0M 3.7G  1% /home
/dev/mapper/vg_candy-usr
           3.9G 382M 3.3G 11% /usr
/dev/mapper/vg_candy-var
           3.9G  63M 3.6G  2% /var
[root@Candy ~]# fdisk -l
Disk /dev/sda: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006e71c
  Device Boot   Start     End   Blocks  Id System
/dev/sda1  *      1     64   512000  83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2       64    2611  20458496  8e Linux LVM
Disk /dev/sdb: 4398.0 GB, 4398046511104 bytes
255 heads, 63 sectors/track, 534698 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/vg_candy-root: 8061 MB, 8061452288 bytes
255 heads, 63 sectors/track, 980 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/vg_candy-usr: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/vg_candy-var: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/vg_candy-home: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
[root@Candy ~]# ls /dev/sd*
/dev/sda /dev/sda1 /dev/sda2 /dev/sdb

这时候我们看到，Centos 已经知道有个sdb的盘了，但是没挂载与分区。

使用Parted进行扩容前的分区

现在假设除了/boot 其余分区都满了! 我们需要先进行分区，将4T的硬盘用上(/dev/sdb).

[root@Candy ~]# parted /dev/sdb
GNU Parted 2.1
使用 /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mklabel
新的磁盘标签类型？ gpt
(parted) mkpart
分区名称？ []?
文件系统类型？ [ext2]? ext4
起始点？ 0T
结束点？ 4T
(parted) p
Model: VMware, VMware Virtual S (scsi)
Disk /dev/sdb: 4398GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start  End   Size  File system Name 标志
 1   1049kB 4398GB 4398GB
(parted)set 1 lvm on
(parted)p
Model: VMware, VMware Virtual S (scsi)
Disk /dev/sdb: 4398GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start  End   Size  File system Name 标志
1   1049kB 4398GB 4398GB           lvm
(parted)q
[root@Candy ~]# ls /dev/sd*
/dev/sda /dev/sda1 /dev/sda2 /dev/sdb /dev/sdb1

现在让我们来解释一下操作：

mklabel将磁盘转换为gpt分区，因为MBR分区无法支持TB级硬盘（具体请自行查阅，parted当然也支持TB级以下的操作）
mkpart将转换好的gpt磁盘进行分区，需要提供文件系统类型、开始点、结束点。
输入p 打印出刚才才做后生成的分区。
set 1 lvm on 是将num为1的分区，tager设置为LVM ，这样才能添加到vm卷组里

下一步，我们查看我们的LVM组名叫什么:

[root@Candy ~]# vgs
 VG    #PV #LV #SN Attr  VSize VFree
 vg_candy  1  4  0 wz--n- 19.51g  0
 
[root@Candy ~]# vgdisplay
 --- Volume group ---
 VG Name        vg_candy
 System ID
 Format        lvm2
 Metadata Areas    1
 Metadata Sequence No 5
 VG Access       read/write
 VG Status       resizable
 MAX LV        0
 Cur LV        4
 Open LV        4
 Max PV        0
 Cur PV        1
 Act PV        1
 VG Size        19.51 GiB
 PE Size        4.00 MiB
 Total PE       4994
 Alloc PE / Size    4994 / 19.51 GiB
 Free PE / Size    0 / 0
 VG UUID        Gcm5Ml-fc88-Xk5q-xuPC-Kj7Y-alfK-i770Gc
[root@Candy ~]# lvscan
 ACTIVE      '/dev/vg_candy/usr' [4.00 GiB] inherit
 ACTIVE      '/dev/vg_candy/var' [4.00 GiB] inherit
 ACTIVE      '/dev/vg_candy/home' [4.00 GiB] inherit
 ACTIVE      '/dev/vg_candy/root' [7.51 GiB] inherit
[root@Candy ~]#

知道lvm卷组叫 'vg_candy'了, 我们就使用vgextend来向lvm 'vg_candy' 卷组添加成员了.

[root@Candy ~]# vgextend vag_candy /dev/sdb1
Physical volume "/dev/sdb1" successfully created
Volume group "vg_candy" successfully extended

现在我们再用vgs命令看一下pv个数:

[root@Candy ~]# vgs
VG    #PV #LV #SN Attr  VSize VFree
vg_candy  2  4  0 wz--n- 4.02t 4.00t

大家对比未添加时的vgs是否有看到PV个数从1变为2了！

现在让我们用vgdisplay看看现在分区的容量.

[root@Candy ~]# vgdisplay
 --- Volume group ---
 VG Name        vg_candy
 System ID
 Format        lvm2
 Metadata Areas    2
 Metadata Sequence No 6
 VG Access       read/write
 VG Status       resizable
 MAX LV        0
 Cur LV        4
 Open LV        4
 Max PV        0
 Cur PV        2
 Act PV        2
 VG Size        4.02 TiB
 PE Size        4.00 MiB
 Total PE       1053569
 Alloc PE / Size    4994 / 19.51 GiB
 Free PE / Size    1048575 / 4.00 TiB
 VG UUID        Gcm5Ml-fc88-Xk5q-xuPC-Kj7Y-alfK-i770Gc

再次对比未添加的vgdiplay命令发现，FreePE 的value 从 0 / 0 变为 1048575 / 4.00 TiB . 这样说明Candy已经成功将4TB的lvm 分区添加进入到vg_candy 逻辑卷组里了。

现在我们对所有逻辑分区进行扩容前，df -h 看看有哪些分区。

[root@Candy ~]# df -h
Filesystem         Size Used Avail Use% Mounted on
/dev/mapper/vg_candy-root 7.3G 231M 6.7G  4% /
 tmpfs           1.9G   0 1.9G  0% /dev/shm
/dev/sda1         477M  30M 422M  7% /boot
/dev/mapper/vg_candy-home 3.9G 8.0M 3.7G  1% /home         
/dev/mapper/vg_candy-usr  3.9G 382M 3.3G 11% /usr
/dev/mapper/vg_candy-var  3.9G  63M 3.6G  2% /var

现在麻烦的事已经做完了，接下来就剩下为分区进行扩容了.

为磁盘进行扩容

假设：我们需要向这些分区都添加500G容量: / 、 /home 、 /usr 、 /var

[root@Candy ~]# lvextend -L +500G /dev/mapper/vg_candy-root
 Size of logical volume vg_candy/root changed from 7.51 GiB (1922 extents) to 507.51 GiB (129922 extents).
 Logical volume root successfully resized
 
[root@Candy ~]# lvextend -L +500G /dev/mapper/vg_candy-home
 Size of logical volume vg_candy/home changed from 4.00 GiB (1024 extents) to 504.00 GiB (129024 extents).
 Logical volume home successfully resized
 
[root@Candy ~]# lvextend -L +500G /dev/mapper/vg_candy-usr
 Size of logical volume vg_candy/usr changed from 4.00 GiB (1024 extents) to 504.00 GiB (129024 extents).
 Logical volume usr successfully resized
 
[root@Candy ~]# lvextend -L +500G /dev/mapper/vg_candy-var
 Size of logical volume vg_candy/var changed from 4.00 GiB (1024 extents) to 504.00 GiB (129024 extents).
 Logical volume var successfully resized

大家注意，这一步操作时不可逆的！如果输入了错误的分区名字还好！如果添加到其它分区了，就得重复N遍来缩容，比较麻烦哦！建议大家注仔细核对上面的FileSystem. 切记切记....

提示扩容成功！让我们再次用上述命令查看LVM信息:

[root@Candy ~]# vgs
 VG    #PV #LV #SN Attr  VSize VFree
 vg_candy  2  4  0 wz--n- 4.02t 2.05t 
 
[root@Candy ~]# lvscan
 ACTIVE      '/dev/vg_candy/usr' [504.00 GiB] inherit
 ACTIVE      '/dev/vg_candy/var' [504.00 GiB] inherit
 ACTIVE      '/dev/vg_candy/home' [504.00 GiB] inherit
 ACTIVE      '/dev/vg_candy/root' [507.51 GiB] inherit
 
[root@Candy ~]# vgdisplay
 --- Volume group ---
 VG Name        vg_candy
 System ID
 Format        lvm2
 Metadata Areas    2
 Metadata Sequence No 10
 VG Access       read/write
 VG Status       resizable
 MAX LV        0
 Cur LV        4
 Open LV        4
 Max PV        0
 Cur PV        2
 Act PV        2
 VG Size        4.02 TiB
 PE Size        4.00 MiB
 Total PE       1053569
 Alloc PE / Size    516994 / 1.97 TiB
 Free PE / Size    536575 / 2.05 TiB
 VG UUID        Gcm5Ml-fc88-Xk5q-xuPC-Kj7Y-alfK-i770Gc
 
[root@Candy ~]# df -h
Filesystem         Size Used Avail Use% Mounted on
/dev/mapper/vg_candy-root 7.3G 231M 6.7G  4% /
 tmpfs           1.9G   0 1.9G  0% /dev/shm
/dev/sda1         477M  30M 422M  7% /boot
/dev/mapper/vg_candy-home 3.9G 8.0M 3.7G  1% /home         
/dev/mapper/vg_candy-usr  3.9G 382M 3.3G 11% /usr
/dev/mapper/vg_candy-var  3.9G  63M 3.6G  2% /var

看到这里大家可能奇怪！为什么df -h 还没变呢?

刷新磁盘信息与写入

我们需要用resize2fs 重设分区大小, 这样才能使df -h 也能生效.

而resize2fs 需要的时间与你给予的容量线性叠加. 也就是说你给的容量越多，resize2fs 的需要的时间就越多。

[root@Candy ~]# resize2fs /dev//mapper/vg_candy-root
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev//mapper/vg_candy-root is mounted on /; on-line resizing required old desc_blocks = 1, new_desc_blocks = 32
Performing an on-line resize of /dev//mapper/vg_candy-root to 133040128 (4k) blocks.
The filesystem on /dev//mapper/vg_candy-root is now 133040128 blocks long.

[root@Candy ~]# resize2fs /dev//mapper/vg_candy-usr
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev//mapper/vg_candy-usr is mounted on /usr; on-line resizing required old desc_blocks = 1, new_desc_blocks = 32
Performing an on-line resize of /dev//mapper/vg_candy-usr to 132120576 (4k) blocks.
The filesystem on /dev//mapper/vg_candy-usr is now 132120576 blocks long.

[root@Candy ~]# resize2fs /dev//mapper/vg_candy-var
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev//mapper/vg_candy-var is mounted on /var; on-line resizing required old desc_blocks = 1, new_desc_blocks = 32
Performing an on-line resize of /dev//mapper/vg_candy-var to 132120576 (4k) blocks.
The filesystem on /dev//mapper/vg_candy-var is now 132120576 blocks long.

[root@Candy ~]# resize2fs /dev//mapper/vg_candy-home
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev//mapper/vg_candy-home is mounted on /home; on-line resizing required old desc_blocks = 1, new_desc_blocks = 32
Performing an on-line resize of /dev//mapper/vg_candy-home to 132120576 (4k) blocks.
The filesystem on /dev//mapper/vg_candy-home is now 132120576 blocks long.

[root@Candy ~]# df -h
Filesystem          Size Used Avail Use% Mounted on
/dev/mapper/vg_candy-root   500G 245M 474G  1% /
tmpfs             1.9G   0 1.9G  0% /dev/shm
/dev/sda1           477M  30M 422M  7% /boot
/dev/mapper/vg_candy-home   496G  16M 471G  1% /home
/dev/mapper/vg_candy-usr   496G 390M 471G  1% /usr
/dev/mapper/vg_candy-var   496G  71M 471G  1% /var

结束语

顺便给大家晒一张重启后系统自动挂载的图！