Installing a Secondary NVMe Drive
Author: Brandon B. Jozsa
I talk about my lab a lot throughout this blog. Recently, I've been wanting to recycle an older NVMe drive (a Samsung 970 Evo Plus) in order to create an LVM volume group, which can be resized later. I'm currently using Samsung 980 Pro 1TB drives for my OS, but this new volume will be used as an NFS share for Kubernetes storage.
After the drive has been installed, use the lsblk command with -io to output the following columns: NAME,TYPE,SIZE,MOUNTPOINT,FSTYPE,MODEL. By listing the MODEL and NAME specifically, you should be able to correctly identify the drive you want to use.
[bjozsa@galvatron03 ~]$ sudo lsblk -io NAME,TYPE,SIZE,MOUNTPOINT,FSTYPE,MODEL
NAME TYPE SIZE MOUNTPOINT FSTYPE MODEL
nvme1n1 disk 931.5G Samsung SSD 980 PRO 1TB
└─nvme1n1p1 part 931.5G LVM2_member
└─vgroot-lvroot lvm 931.5G / ext4
nvme0n1 disk 931.5G Samsung SSD 970 EVO Plus 1TB
[bjozsa@galvatron03 ~]$Next, use the command pvcreate to define the Physical Volume.
[bjozsa@galvatron03 ~]$ sudo pvcreate /dev/nvme0n1
Physical volume "/dev/nvme0n1" successfully created.
[bjozsa@galvatron03 ~]$Now use the command vgcreate to define a Volume Group.
[bjozsa@galvatron03 ~]$ sudo vgcreate vgnfs /dev/nvme0n1
Volume group "vgnfs" successfully created
[bjozsa@galvatron03 ~]$To see the changes you made, run vgs -v.
[bjozsa@galvatron03 ~]$ sudo vgs -v
VG Attr Ext #PV #LV #SN VSize VFree VG UUID VProfile
vgnfs wz--n- 4.00m 1 0 0 931.51g 931.51g w6c0Fp-MViZ-Lzyd-N50O-ONlW-4EA7-DFYrae
vgroot wz--n- 4.00m 1 1 0 931.50g 0 f9a8Lq-RzUe-531e-ApxQ-3cCK-mS85-xKuJhx
[bjozsa@galvatron03 ~]$Next, you'll need to create the Logical Volume. In the command below, you can use -l [percent], and as you can see I am choosing to use 100% of the drive. You can also use the -L option to declare a size (example: -L250G). It's always safer to use less and resize later, but I don't really care about the size right now. Just be aware of your options, and when in doubt be sure to use --help with each of these commands.
[bjozsa@galvatron03 ~]$ sudo lvcreate -l 100%VG -n lvnfs vgnfs
Logical volume "lvnfs" created.
[bjozsa@galvatron03 ~]$Great! Let's look at what you created with the lvdisplay and lvs commands.
[bjozsa@galvatron03 ~]$ sudo lvdisplay
--- Logical volume ---
LV Path /dev/vgnfs/lvnfs
LV Name lvnfs
VG Name vgnfs
LV UUID w6c0Fp-MViZ-Lzyd-N50O-ONlW-4EA7-DFYrae
LV Write Access read/write
LV Creation host, time galvatron03, 2021-01-05 17:35:25 +0000
LV Status available
# open 0
LV Size 931.51 GiB
Current LE 238467
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 8192
Block device 253:1
WARNING: PV /dev/nvme1n1p1 in VG vgroot is using an old PV header, modify the VG to update.
--- Logical volume ---
LV Path /dev/vgroot/lvroot
LV Name lvroot
VG Name vgroot
LV UUID 6x6ROT-N3gR-vy2m-OZBM-paLY-CS8E-ui082d
LV Write Access read/write
LV Creation host, time galvatron03, 2020-11-27 16:30:09 +0000
LV Status available
# open 1
LV Size 931.50 GiB
Current LE 238465
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 8192
Block device 253:0
[bjozsa@galvatron03 ~]$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
lvnfs vgnfs -wi-a----- 931.51g
lvroot vgroot -wi-ao---- 931.50g
[bjozsa@galvatron03 ~]$Now it's time to create a filesystem within the Logical Volume. Now full disclosure, at the time of writing I'm using RHEL 8.3 in a fairly unconventional Ubuntu MaaS setup which is deploying LVM with an ext4 filesystem. Eventually I'll migrate away from MaaS, but since this current deployment is using ext4 for the primary drive, I'll continue to use this format for the new drive. You can use whatever filesystem format that fits your needs. I am doing this with the mkfs.[type] command (example: mkfs.ext4). You need to do this against the device in /dev/vgnfs/lvnfs, if you've followed along exactly. This is why I used the vg and lv nomenclature throughout this tutorial.
[bjozsa@galvatron03 ~]$ sudo mkfs.ext4 /dev/vgnfs/lvnfs
mke2fs 1.45.6 (20-Mar-2020)
Discarding device blocks: done
Creating filesystem with 244190208 4k blocks and 61054976 inodes
Filesystem UUID: df323a65-2f51-489e-8241-9b1b295bc8a8
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848
Allocating group tables: done
Writing inode tables: done
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done
[bjozsa@galvatron03 ~]$Everyone's use case and reasoning is different. For this example, I am going to create a mount folder for this NFS server in the optional directory in /opt/nfs/.
[bjozsa@galvatron03 ~]$ sudo mkdir -p /opt/nfs/
[bjozsa@galvatron03 ~]$ sudo mount /dev/vgnfs/lvnfs /opt/nfs
[bjozsa@galvatron03 ~]$ sudo df -H
Filesystem Size Used Avail Use% Mounted on
devtmpfs 34G 0 34G 0% /dev
tmpfs 34G 0 34G 0% /dev/shm
tmpfs 34G 9.7M 34G 1% /run
tmpfs 34G 0 34G 0% /sys/fs/cgroup
/dev/mapper/vgroot-lvroot 985G 50G 886G 6% /
tmpfs 6.7G 0 6.7G 0% /run/user/1001
/dev/mapper/vgnfs-lvnfs 984G 80M 934G 1% /opt/nfs
[bjozsa@galvatron03 ~]$Use your favorite variant of df to verify that things were mounted correctly.
WARNING: Debates welcome. The last thing to do would make the mount point permanent by writing changes to /etc/fstab. This will map the mount for persistence.
Take a look at your current /etc/fstab file. Depending on the OS, you can mount either using the UUID (which some recommend) or using the /dev/mapper, which others recommend when using LVM volumes. For simplicity, I will use /dev/mapper, but I will write a future article (and link it here) when I have more time.
When looking at /dev/mapper/, you will notice some symbolic links. In my case, my new mount link is /dev/mapper/vgnfs-lvnfs. This is what I will use in my /etc/fstab, like below.
# /etc/fstab: static file system information.
/dev/disk/by-id/dm-uuid-LVM-f9a8LqRzUe531eApxQ3cCKmS85xKuJhx6x6ROTN3gRvy2mOZBMpaLYCS8Eui082d / ext4 defaults 0 0
/swap.img none swap sw 0 0
#
# NFS Share
/dev/mapper/vgnfs-lvnfs /opt/nfs ext4 defaults 0 0Add that last line, or similar depending on your own specific needs, and your mount should come up after reboots.
I'll talk about NFS and my NFS use case in another post for later.