Creating Logical Volumes (2008-12-06 Classroom)

Fedora Classroom - Creating Logical Volumes - Clint Savage - Saturday, December 6, 2008

-!- nirik changed the topic of #fedora-classroom to: Fedora IRC Classroom - Creating Logical Volumes - Clint Savage ( herlo ). See Communicate/IRC/Classroom for schedule and more information.		16:00
herlo	Hi all, good to see you again. This is my second time in Fedora Classroom. Last time it was on SELinux Basics. This time, LVM Creation.	16:00
herlo	I didn't prepare slides this time, but I do have a couple online documents that I'd like to use	16:01
herlo	http://www.ibm.com/developerworks/linux/library/l-lvm2/index.html <-- thanks IBM	16:01
herlo	http://www.softpanorama.org/Commercial_linuxes/logical_volume_manager.shtml <-- thanks to softpanorama	16:01
herlo	so to start off, who knows what LVM stands for and it's basic purpose?	16:02
herlo	the first word is not Linux	16:02
herlo	<crickets />	16:02
u0m3	LVM = Logical Volume Management or something like that	16:02
* nirik thinks thats Logical.		16:02
* MartinCo looks around the room		16:03
herlo	yay! People are here :)	16:03
herlo	okay, so here we go	16:03
MartinCo	we were seeing what you would do next :)	16:03
herlo	yes, it's Logical Volume Management	16:03
herlo	the question I want to address today, is what does that mean for you as a user?	16:03
SSlater	confusion?	16:04
herlo	The basic purpose of LVM is to provide flexibility and extendability to your disks through management interface. There are a lot of uses of LVM, but I want to focus on the why.	16:04
delhage	flexibility	16:04
herlo	SSlater: I thought so too at first...	16:04
herlo	so let's imagine a couple simple scenarios	16:05
herlo	first, you have 3 disks, 2 are in a RAID (software or hardware, it doesn't matter) and you want to add an additional disk to the mix	16:05
herlo	not to say this is necessarily normal, but it's something that could happen	16:06
herlo	and plus, it's my example :)	16:06
herlo	the RAID I have is constructed using software raid and it's built of usb sticks	16:06
aTypical	Howdy	16:06
herlo	just for fun mostly	16:06
herlo	I call that RAID /dev/md0	16:06
herlo	the RAID consists of /dev/sdb1 and /dev/sdc1	16:07
* icarus-c knocks the classroom door...		16:07
herlo	and I have a physical partition /dev/sdd1	16:07
herlo	so, let's just say that I'd like to take this mix of different types of disks and make them into a flexible device	16:08
herlo	how to do that?	16:08
herlo	I want one device, and I also want the flexibility of adding additional devices later on. This is one of the great strengths of LVM	16:08
herlo	does this make sens?	16:08
herlo	sense	16:08
MartinCo	LVM, or (v. v. ugly) put a raid0 stripe over the md0, and sdc1	16:08
MartinCo	herlo, indeed it does :)	16:09
icarus-c	at least i understand	16:09
herlo	MartinCo: pretty much, but it's more flexible than that	16:09
herlo	and we'll get to those near the end of the presentation	16:09
herlo	okay, so I'm going to refernece the IBM website for a minute	16:09
herlo	in figure 1 on their site, you see a list of devices, (hda1, hda2, etc) all pointing to what is called a Volume Group	16:10
herlo	this is the basic structure, from which your devices will be created	16:10
MartinCo	(http://www.ibm.com/developerworks/linux/library/l-lvm2/index.html for those that joined after the link was posted)	16:10
herlo	thanks MartinCo	16:10
herlo	the devices themselves (hda1, hda2, md0, etc) are what LVM refers to as Physical Volumes and they must be initialized	16:11
herlo	once initialized as physical volumes (PV), they can be included into the Volume Group (VG)	16:12
herlo	and from the VG, we can pull data and make logical volumes (lv) or what will look like devices where we put our filesystem and data	16:12
herlo	a good picture of this is on Figure 3, IBM web link	16:13
herlo	so essentially, we have 3 parts, PV, VG and LV, built in that order	16:14
herlo	on the http://www.softpanorama.org/Commercial_linuxes/logical_volume_manager.shtml site, there are commands that correspond with each section, showing sample commands	16:14
herlo	now, while we're discussing this, I've created a RAID0 and have a physical disk, which I will partition and create the LVM	16:15
herlo	but there's one more thing that we need to discuss before moving forward with this	16:15
herlo	everyone doing okay so far?	16:15
u0m3	I am...	16:15
* MartinCo nods		16:15
SSlater	Yes	16:15
icarus-c	yes	16:15
herlo	Great	16:15
herlo	to start off with, we will need to partition any physical devices appropriately, I like the fdisk tool for this, but you can use anything you are comfortable with, parted, gparted, sfdisk, etc	16:16
herlo	the partition should be of partition type 8e (LVM)	16:16
herlo	# fdisk -l /dev/sdd	16:17
herlo	Disk /dev/sdd: 4025 MB, 4025810432 bytes	16:17
herlo	255 heads, 63 sectors/track, 489 cylinders	16:17
herlo	Units = cylinders of 16065 * 512 = 8225280 bytes	16:17
herlo	Disk identifier: 0x00000000	16:17
herlo	Device Boot Start End Blocks Id System	16:17
herlo	/dev/sdd1 1 14 112423+ 8e Linux LVM	16:17
herlo	like so	16:17
herlo	now my disk is only 100M, but it can be as big as you like	16:17
herlo	once you have that done, and you have your RAID, you just need to create PVs for each component in your RAID	16:18
herlo	er, LVM	16:18
herlo	so, I do the following commands	16:18
herlo	# pvcreate /dev/md0 Physical volume "/dev/md0" successfully created	16:18
herlo	[root@herlo-lap ~]# pvcreate /dev/sdd1 Physical volume "/dev/sdd1" successfully created	16:19
herlo	the 'Physical volume ...' part is on a separate line	16:19
herlo	what I have now is an initialized PV, ready to go into a VG	16:19
herlo	but now it gets a little tricky.	16:19
herlo	LVM introduces these things called Physical Extents	16:20
herlo	and while they can affect the performance and minimum size of the disk you can use, generally, they're pretty easy to deal with	16:20
herlo	a Physical Extend is essentially a chunk of disk space. Think of it as the block size of LVM. The VG will access one PE at a time when allocating disks. In other words, the Logical Volume will be rounded to the neares PE	16:21
herlo	generally, PE's can range from vary small (1k or less) to very large Terabytes or Exabytes.	16:22
herlo	In my world, and because I'm using 100MB disks, I'll use a 4MB PE	16:22
herlo	this is a relation that is spread from the Physical side of the volume to the Logical side.	16:22
herlo	everyone following that?	16:22
icarus-c	i guess so	16:22
herlo	http://www.ibm.com/developerworks/linux/library/l-lvm2/index.html has a really good picture	16:22
herlo	figure 4	16:23
herlo	the red sections define the size of the PV	16:23
herlo	and each of those sections are how we allocate a Logical Volume	16:23
herlo	PE, not PV	16:23
icarus-c	is it only 1 PE can be r/w at a time? :-S	16:24
herlo	okay, so finally, we have to create the VG, that's done with vgcreate	16:24
herlo	so run, vgcreate --help and you get a nice helpful screen something like this:	16:24
herlo	]# pvcreate --help	16:24
herlo	pvcreate: Initialize physical volume(s) for use by LVM	16:24
herlo	pvcreate	16:24
herlo	[--restorefile file]	16:24
herlo	[-d|--debug]	16:24
herlo	[-f[f]|--force [--force]]	16:24
herlo	[-h|-?|--help]	16:24
herlo	[--labelsector sector]	16:24
herlo	[-M|--metadatatype 1|2]	16:24
herlo	[--metadatacopies #copies]	16:24
herlo	[--metadatasize MetadataSize[kKmMgGtTpPeE]]	16:24
herlo	[--setphysicalvolumesize PhysicalVolumeSize[kKmMgGtTpPeE]	16:24
herlo	[-t|--test]	16:24
herlo	[-u|--uuid uuid]	16:24
herlo	[-v|--verbose]	16:24
herlo	[-y|--yes]	16:24
herlo	[-Z|--zero {y|n}]	16:24
herlo	[--version]	16:24
herlo	PhysicalVolume [PhysicalVolume...]	16:24
herlo	sorry about the formatting	16:24
herlo	oops, vgcreate is what I meant	16:25
herlo	vgcreate --help	16:25
herlo	vgcreate: Create a volume group	16:25
herlo	vgcreate	16:25
herlo	[-A|--autobackup {y|n}]	16:25
herlo	[--addtag Tag]	16:25
herlo	[--alloc AllocationPolicy]	16:25
herlo	[-c|--clustered {y|n}]	16:25
herlo	[-d|--debug]	16:25
herlo	[-h|--help]	16:25
herlo	[-l|--maxlogicalvolumes MaxLogicalVolumes]	16:25
herlo	[-M|--metadatatype 1|2]	16:25
herlo	[-p|--maxphysicalvolumes MaxPhysicalVolumes]	16:25
herlo	[-s|--physicalextentsize PhysicalExtentSize[kKmMgGtTpPeE]]	16:25
herlo	[-t|--test]	16:25
herlo	[-v|--verbose]	16:25
herlo	[--version]	16:25
herlo	VolumeGroupName PhysicalVolume [PhysicalVolume...]	16:25
herlo	at least that one is not as long	16:25
herlo	we should run pvdisplay to see which PVs we can include	16:25
herlo	what you might notice if you have run this is that there might be several (or just a couple) PV's, the ones you are looking for correspond to the ones you've created with pvcreate	16:26
herlo	they should also show: Allocatable NO	16:27
herlo	because they have not yet been assigned to a VG	16:27
herlo	so let's do that	16:27
herlo	vgcreate -s 2M happy /dev/md0	16:27
herlo	it's that simple	16:27
icarus-c	very	16:28
herlo	vgdisplay happy	16:28
herlo	will now display the details of the happy Volume Group	16:28
herlo	I would actually recommend you use a more descriptive name for your VG	16:29
herlo	from this VG, we can now take some space that will make our disks	16:29
herlo	according to vgdisplay, I have the following:	16:29
herlo	VG Size 208.00 MB	16:29
herlo	PE Size 2.00 MB	16:29
herlo	Total PE 104	16:29
herlo	Alloc PE / Size 0 / 0	16:29
herlo	Free PE / Size 104 / 208.00 MB	16:29
herlo	the PE's are now available for me to make my logical volumes. And on the LV side, we call those same elements Logical Extents as they have the same size and structure.	16:30
herlo	Only real difference is that the physical part is managed by the VG	16:31
herlo	okay, so let's make a disk we can use!	16:31
herlo	lvcreate is the tool for this	16:31
herlo	lvcreate --help provides two basic functionalities, one is basic creation, the other snapshots. If we have time, I'll discuss snapshots	16:31
herlo	so there's a structure there with some options, I'll detail the ones we need for a basic LV	16:32
herlo	-n|--name LogicalVolumeName]	16:33
herlo	{-l|--extents LogicalExtentsNumber |	16:33
herlo	-L|--size LogicalVolumeSize[kKmMgGtTpPeE]}	16:33
herlo	VolumeGroupName	16:33
herlo	that's really it	16:33
herlo	so it looks something like so	16:33
herlo	# lvcreate -l 90 -n camper happy	16:34
herlo	Logical volume "camper" created	16:34
herlo	at this point device mapper creates us a device	16:34
herlo	/dev/happy/camper	16:34
herlo	# ls -l /dev/happy/camper	16:34
herlo	lrwxrwxrwx 1 root root 24 2008-12-06 09:33 /dev/happy/camper -> /dev/mapper/happy-camper	16:34
herlo	and this is where we make our filesystem	16:34
icarus-c	Question: is that -l 90 refers to 90 PE?	16:35
herlo	yes, good point	16:35
herlo	you can either have -l (Logical Extents, which directly relate to Physical Extents)	16:35
herlo	or -L which is a size	16:35
icarus-c	i'm kinda confused by [kKmMgGtTpPeE] can you explain a bit?	16:35
herlo	right, so if you use -L, you can specify a size, rounded to the nearest LE	16:35
herlo	it's an either-or situation	16:36
herlo	you don't use both -l and -L	16:36
jds2001	icarus-c: kilobytes, megabytes, gigabytes, terabytes, exabytes	16:36
herlo	you k, is kibibytes, K is kilobytes, P is Peta	16:36
herlo	etc...	16:36
icarus-c	i see, thanks	16:36
jds2001	err petabytes in there too.	16:36
herlo	# lvdisplay /dev/happy/camper --- Logical volume --- LV Name /dev/happy/camper VG Name happy LV UUID 97h5rK-WOu3-QF3a-Ii1O-0E3u-vB1a-FH17L7 LV Write Access read/write LV Status available # open 0 LV Size 180.00 MB Current LE 90 Segments 1 Allocation inherit	16:36
herlo	Readahead sectorsr auto - currently set to 256 Block device 253:6	16:36
herlo	ooh, ugly	16:36
herlo	let me try that again	16:37
jds2001	you use pastebin! :)	16:37
herlo	LV Name /dev/happy/camper	16:37
herlo	jds2001: if I use pastebin, it expires	16:37
herlo	VG Name happy	16:37
herlo	Current LE 90	16:37
herlo	LV Size 180.00 MB	16:37
herlo	there are other details there, but that's good for now	16:38
herlo	also, you can see much of htis detail at: http://www.softpanorama.org/Commercial_linuxes/logical_volume_manager.shtml	16:38
herlo	okay, so now we just build our filesystem on /dev/happy/camper and we've got it all good	16:38
herlo	]# mkfs -t ext3 /dev/happy/camper	16:39
herlo	dumpe2fs /dev/happy/camper	16:39
herlo	can show the details of the new filesystem	16:39
herlo	feel free to mount it and you are on your way	16:39
herlo	any questions so far?	16:40
u0m3	pretty self-explanatory :)	16:40
herlo	making sense?	16:40
SSlater	yup	16:40
Don_Miguel	yep	16:40
collier_s	yup	16:40
herlo	okay, so where the flexibility comes in for LVM is with it's ability to add devices	16:40
herlo	these are done in two steps	16:41
herlo	we always start by building a PV with pvcreate, so that step didn't change, but then we need to extend the VG and the LV	16:41
herlo	two commands exist to do this	16:41
herlo	vgcreate and lvcreate	16:41
herlo	vgextend and lvextend I mean	16:42
jds2001	er, vgextend and lvextend, right?	16:42
herlo	wow!	16:42
* herlo is tired		16:42
* kanarip hands herlo a cup of coffee		16:42
jds2001	i hear ya, early out there :)	16:42
herlo	kanarip: got one, just sipped it, thanks :)	16:42
herlo	the order is important here, as you will need to extend the vg (or at least have some LE's to work with in your VG) before you can use it in the current L	16:43
herlo	LV	16:43
herlo	lvextend --help	16:43
herlo	{-l|--extents [+]LogicalExtentsNumber[%{VG|PVS|FREE}] |	16:43
herlo	-L|--size [+]LogicalVolumeSize[kKmMgGtTpPeE]}	16:43
herlo	LogicalVolume[Path] [ PhysicalVolumePath... ]	16:44
herlo	and vgextend --help	16:44
herlo	VolumeGroupName PhysicalDevicePath	16:44
herlo	so let's do just that	16:45
herlo	vgextend happy /dev/sdd1	16:45
herlo	# vgdisplay happy	16:45
herlo	now shows more PE's available	16:46
herlo	Total PE 158	16:46
herlo	Alloc PE / Size 90 / 180.00 MB	16:46
herlo	Free PE / Size 68 / 136.00 MB	16:46
herlo	it also shows a size of	16:46
herlo	VG Size 316.00 MB	16:46
herlo	we now have 68 more PEs available to us which we can now use in our /dev/happy/camper	16:46
herlo	LV	16:46
herlo	lvextend allows us to use -l or -L again, and we can either specify the actual size or add the +Size/LE's	16:47
herlo	so it would look something like this	16:47
herlo	lvextend -l +68 /dev/happy/camper	16:47
herlo	or lvextend -L +136M /dev/happy/camper	16:47
herlo	if you wished to not use the + it woudl be	16:48
herlo	or lvextend -L 316M /dev/happy/camper	16:48
herlo	# lvdisplay /dev/happy/camper	16:49
herlo	LV Size 316.00 MB	16:49
herlo	Current LE 158	16:49
herlo	but we still have a filesystem size much smaller than this	16:49
herlo	# df -h	16:49
herlo	/dev/mapper/happy-camper 175M 5.6M 160M 4% /mnt	16:50
herlo	so we need to resize	16:50
herlo	depending on your filesystem, there are several ways to do this, in ext3, its resize2fs	16:50
herlo	there's mount options for jfs and xfs that do similar things	16:50
herlo	something like	16:50
herlo	mount -o remount,resize_jfs /dev/happy/camper	16:50
herlo	woudl probably work	16:50
herlo	for us, since we're using ext3 we want to run the followign command	16:51
herlo	resize2fs /dev/happy/camper	16:51
herlo	which will perform the resize *online* and grow the volume in front of our eyse	16:51
herlo	eyes	16:51
* herlo waits for the resize to finish :)		16:52
herlo	# resize2fs /dev/happy/camper	16:52
herlo	resize2fs 1.41.3 (12-Oct-2008)	16:52
herlo	Filesystem at /dev/happy/camper is mounted on /mnt; on-line resizing required	16:52
herlo	old desc_blocks = 1, new_desc_blocks = 2	16:52
herlo	Performing an on-line resize of /dev/happy/camper to 323584 (1k) blocks.	16:52
herlo	The filesystem on /dev/happy/camper is now 323584 blocks long.	16:52
herlo	# df -h	16:52
herlo	/dev/mapper/happy-camper 307M 6.1M 285M 3% /mnt	16:52
herlo	now, there's how to create a logical volume	16:52
herlo	what's also neat about LVM is that the default install of Fedora uses it as well. This provides similar flexibility to increase the size of your drives if you add a second hard drive to your system	16:53
herlo	any questoins?	16:53
Fuss284	what about shrinking one?	16:53
herlo	yes, that can be done as well	16:53
MartinCo	painful though	16:53
herlo	and that brings up a good discussion	16:53
MartinCo	:)	16:53
herlo	first off, it cannot be done online	16:54
herlo	but, what if you lost a part of your RAID device and wanted to just replace the device	16:54
herlo	you could do that all online	16:54
herlo	you would need to add a new PV to the VG, and then run a pvmove	16:54
herlo	the pvmove would move all of the PEs from the device specified to the rest of the larger VG	16:54
herlo	and then a pvremove would allow you to remove the devices from the system	16:55
icarus-c	what about shrinking your happy-camper ? :P	16:55
herlo	as far as shrinking, I'd first have to unmount /dev/happy/camper	16:55
herlo	then lvreduce	16:55
herlo	then vgreduce	16:55
herlo	if you wanted to reduce the vg that is	16:55
icarus-c	i think we need to resize the filesystem too right?	16:56
herlo	ooh, first	16:56
herlo	you need to fsck and resize the fs, yes	16:56
Fuss284	herlo don't you have to reduce the filesystem too? since, when you extended it, you had to.. ah	16:56
herlo	which is why you can't do it online	16:56
herlo	thanks guys, sorry I knew that :)	16:56
icarus-c	so first umount , 2nd resize fs and finally resize the lv & pv?	16:56
herlo	not the pv	16:56
herlo	and you don't have to resize the vg unless you want to	16:57
icarus-c	ar vg	16:57
herlo	also, you can make multiple lv's from one vg	16:57
herlo	so if I had wanted to make /dev/happy/camper and /dev/happy/gilmore,that's fine just as long as you dont try to make one LV from multiple VGs	16:57
herlo	another thing that's really neat is snapshots	16:58
herlo	the concept of snapshots is that they are easy to build structures that take a timestamp (essentially) in time and allow it to be backed up, while the LV is still online	16:58
herlo	what is needed is another disk, which will be created with pv's and Vg's then the LV is pointed at the LV to back up, and mounted read only	16:59
herlo	then a back up device can start its copy	16:59
herlo	the interesting part about this is that when a change occurs on the origin LV, it's delta is saved off to the snapshot LV and the origin LV continues to work as normal	16:59
herlo	this is really good for full-backups	17:00
herlo	one other thing I'd like to quickly mention is the fact that LVM is fully recoverable onto another system in a matter of minutes	17:00
herlo	every bit of data about the lvm is stored in the superblock (or whatever you want to call the first sector)	17:01
herlo	of each of the disks	17:01
herlo	so pull your disks, move them to another machine and run	17:01
herlo	vgscan	17:01
herlo	you'll see your vg and you can then activate it	17:01
herlo	vgchange -a y <vgname>	17:01
herlo	and voila, you've moved your vg from one machine to another	17:02
herlo	any other questions?	17:02
herlo	k, guess not. Thanks all	17:03
Discordian	thank you herlo	17:03
u0m3	thank you for the lesson :)	17:03
collier_s	yup, great lesson	17:03
MartinCo	thanks	17:03
SSlater	Thanks herlo. That clarified things.	17:03
* herlo wants to remind everyone that #fedora-townhall has some FAmSCo discussions shortly		17:04
herlo	thanks guys...	17:04
* SSlater falls asleep in back of classroom.		17:04
* herlo mentions that 2 of the 4 irc classes are already posted on Communicate/IRC/Classroom the other two will be posted shortly		17:05
u0m3	herlo one question: do you happen to know if the /boot partition can be in lvm? as far as I know/remember grub dosen't know/understand lvm. has that changed?	17:07
herlo	u0m3: not yet	17:08
herlo	in grub2, it can	17:08
u0m3	ok. thanks	17:08
herlo	that's coming along nicely too. I've been watching its development over the past while.	17:08
herlo	u0m3: http://www.gnu.org/software/grub/grub-2.en.html	17:09

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!