From Fedora Project Wiki
Fedora Classroom - Creating Logical Volumes - Clint Savage - Saturday, December 6, 2008
| -!- nirik changed the topic of #fedora-classroom to: Fedora IRC Classroom - Creating Logical Volumes - Clint Savage ( herlo ). See Communicate/IRC/Classroom for schedule and more information. | 16:00 | |
| herlo | Hi all, good to see you again. This is my second time in Fedora Classroom. Last time it was on SELinux Basics. This time, LVM Creation. | 16:00 |
|---|---|---|
| herlo | I didn't prepare slides this time, but I do have a couple online documents that I'd like to use | 16:01 |
| herlo | http://www.ibm.com/developerworks/linux/library/l-lvm2/index.html <-- thanks IBM | 16:01 |
| herlo | http://www.softpanorama.org/Commercial_linuxes/logical_volume_manager.shtml <-- thanks to softpanorama | 16:01 |
| herlo | so to start off, who knows what LVM stands for and it's basic purpose? | 16:02 |
| herlo | the first word is not Linux | 16:02 |
| herlo | <crickets /> | 16:02 |
| u0m3 | LVM = Logical Volume Management or something like that | 16:02 |
| * nirik thinks thats Logical. | 16:02 | |
| * MartinCo looks around the room | 16:03 | |
| herlo | yay! People are here :) | 16:03 |
| herlo | okay, so here we go | 16:03 |
| MartinCo | we were seeing what you would do next :) | 16:03 |
| herlo | yes, it's Logical Volume Management | 16:03 |
| herlo | the question I want to address today, is what does that mean for you as a user? | 16:03 |
| SSlater | confusion? | 16:04 |
| herlo | The basic purpose of LVM is to provide flexibility and extendability to your disks through management interface. There are a lot of uses of LVM, but I want to focus on the why. | 16:04 |
| delhage | flexibility | 16:04 |
| herlo | SSlater: I thought so too at first... | 16:04 |
| herlo | so let's imagine a couple simple scenarios | 16:05 |
| herlo | first, you have 3 disks, 2 are in a RAID (software or hardware, it doesn't matter) and you want to add an additional disk to the mix | 16:05 |
| herlo | not to say this is necessarily normal, but it's something that could happen | 16:06 |
| herlo | and plus, it's my example :) | 16:06 |
| herlo | the RAID I have is constructed using software raid and it's built of usb sticks | 16:06 |
| aTypical | Howdy | 16:06 |
| herlo | just for fun mostly | 16:06 |
| herlo | I call that RAID /dev/md0 | 16:06 |
| herlo | the RAID consists of /dev/sdb1 and /dev/sdc1 | 16:07 |
| * icarus-c knocks the classroom door... | 16:07 | |
| herlo | and I have a physical partition /dev/sdd1 | 16:07 |
| herlo | so, let's just say that I'd like to take this mix of different types of disks and make them into a flexible device | 16:08 |
| herlo | how to do that? | 16:08 |
| herlo | I want one device, and I also want the flexibility of adding additional devices later on. This is one of the great strengths of LVM | 16:08 |
| herlo | does this make sens? | 16:08 |
| herlo | sense | 16:08 |
| MartinCo | LVM, or (v. v. ugly) put a raid0 stripe over the md0, and sdc1 | 16:08 |
| MartinCo | herlo, indeed it does :) | 16:09 |
| icarus-c | at least i understand | 16:09 |
| herlo | MartinCo: pretty much, but it's more flexible than that | 16:09 |
| herlo | and we'll get to those near the end of the presentation | 16:09 |
| herlo | okay, so I'm going to refernece the IBM website for a minute | 16:09 |
| herlo | in figure 1 on their site, you see a list of devices, (hda1, hda2, etc) all pointing to what is called a Volume Group | 16:10 |
| herlo | this is the basic structure, from which your devices will be created | 16:10 |
| MartinCo | (http://www.ibm.com/developerworks/linux/library/l-lvm2/index.html for those that joined after the link was posted) | 16:10 |
| herlo | thanks MartinCo | 16:10 |
| herlo | the devices themselves (hda1, hda2, md0, etc) are what LVM refers to as Physical Volumes and they must be initialized | 16:11 |
| herlo | once initialized as physical volumes (PV), they can be included into the Volume Group (VG) | 16:12 |
| herlo | and from the VG, we can pull data and make logical volumes (lv) or what will look like devices where we put our filesystem and data | 16:12 |
| herlo | a good picture of this is on Figure 3, IBM web link | 16:13 |
| herlo | so essentially, we have 3 parts, PV, VG and LV, built in that order | 16:14 |
| herlo | on the http://www.softpanorama.org/Commercial_linuxes/logical_volume_manager.shtml site, there are commands that correspond with each section, showing sample commands | 16:14 |
| herlo | now, while we're discussing this, I've created a RAID0 and have a physical disk, which I will partition and create the LVM | 16:15 |
| herlo | but there's one more thing that we need to discuss before moving forward with this | 16:15 |
| herlo | everyone doing okay so far? | 16:15 |
| u0m3 | I am... | 16:15 |
| * MartinCo nods | 16:15 | |
| SSlater | Yes | 16:15 |
| icarus-c | yes | 16:15 |
| herlo | Great | 16:15 |
| herlo | to start off with, we will need to partition any physical devices appropriately, I like the fdisk tool for this, but you can use anything you are comfortable with, parted, gparted, sfdisk, etc | 16:16 |
| herlo | the partition should be of partition type 8e (LVM) | 16:16 |
| herlo | # fdisk -l /dev/sdd | 16:17 |
| herlo | Disk /dev/sdd: 4025 MB, 4025810432 bytes | 16:17 |
| herlo | 255 heads, 63 sectors/track, 489 cylinders | 16:17 |
| herlo | Units = cylinders of 16065 * 512 = 8225280 bytes | 16:17 |
| herlo | Disk identifier: 0x00000000 | 16:17 |
| herlo | Device Boot Start End Blocks Id System | 16:17 |
| herlo | /dev/sdd1 1 14 112423+ 8e Linux LVM | 16:17 |
| herlo | like so | 16:17 |
| herlo | now my disk is only 100M, but it can be as big as you like | 16:17 |
| herlo | once you have that done, and you have your RAID, you just need to create PVs for each component in your RAID | 16:18 |
| herlo | er, LVM | 16:18 |
| herlo | so, I do the following commands | 16:18 |
| herlo | # pvcreate /dev/md0 Physical volume "/dev/md0" successfully created | 16:18 |
| herlo | [root@herlo-lap ~]# pvcreate /dev/sdd1 Physical volume "/dev/sdd1" successfully created | 16:19 |
| herlo | the 'Physical volume ...' part is on a separate line | 16:19 |
| herlo | what I have now is an initialized PV, ready to go into a VG | 16:19 |
| herlo | but now it gets a little tricky. | 16:19 |
| herlo | LVM introduces these things called Physical Extents | 16:20 |
| herlo | and while they can affect the performance and minimum size of the disk you can use, generally, they're pretty easy to deal with | 16:20 |
| herlo | a Physical Extend is essentially a chunk of disk space. Think of it as the block size of LVM. The VG will access one PE at a time when allocating disks. In other words, the Logical Volume will be rounded to the neares PE | 16:21 |
| herlo | generally, PE's can range from vary small (1k or less) to very large Terabytes or Exabytes. | 16:22 |
| herlo | In my world, and because I'm using 100MB disks, I'll use a 4MB PE | 16:22 |
| herlo | this is a relation that is spread from the Physical side of the volume to the Logical side. | 16:22 |
| herlo | everyone following that? | 16:22 |
| icarus-c | i guess so | 16:22 |
| herlo | http://www.ibm.com/developerworks/linux/library/l-lvm2/index.html has a really good picture | 16:22 |
| herlo | figure 4 | 16:23 |
| herlo | the red sections define the size of the PV | 16:23 |
| herlo | and each of those sections are how we allocate a Logical Volume | 16:23 |
| herlo | PE, not PV | 16:23 |
| icarus-c | is it only 1 PE can be r/w at a time? :-S | 16:24 |
| herlo | okay, so finally, we have to create the VG, that's done with vgcreate | 16:24 |
| herlo | so run, vgcreate --help and you get a nice helpful screen something like this: | 16:24 |
| herlo | ]# pvcreate --help | 16:24 |
| herlo | pvcreate: Initialize physical volume(s) for use by LVM | 16:24 |
| herlo | pvcreate | 16:24 |
| herlo | [--restorefile file] | 16:24 |
| herlo | [-d|--debug] | 16:24 |
| herlo | [-f[f]|--force [--force]] | 16:24 |
| herlo | [-h|-?|--help] | 16:24 |
| herlo | [--labelsector sector] | 16:24 |
| herlo | [-M|--metadatatype 1|2] | 16:24 |
| herlo | [--metadatacopies #copies] | 16:24 |
| herlo | [--metadatasize MetadataSize[kKmMgGtTpPeE]] | 16:24 |
| herlo | [--setphysicalvolumesize PhysicalVolumeSize[kKmMgGtTpPeE] | 16:24 |
| herlo | [-t|--test] | 16:24 |
| herlo | [-u|--uuid uuid] | 16:24 |
| herlo | [-v|--verbose] | 16:24 |
| herlo | [-y|--yes] | 16:24 |
| herlo | [-Z|--zero {y|n}] | 16:24 |
| herlo | [--version] | 16:24 |
| herlo | PhysicalVolume [PhysicalVolume...] | 16:24 |
| herlo | sorry about the formatting | 16:24 |
| herlo | oops, vgcreate is what I meant | 16:25 |
| herlo | vgcreate --help | 16:25 |
| herlo | vgcreate: Create a volume group | 16:25 |
| herlo | vgcreate | 16:25 |
| herlo | [-A|--autobackup {y|n}] | 16:25 |
| herlo | [--addtag Tag] | 16:25 |
| herlo | [--alloc AllocationPolicy] | 16:25 |
| herlo | [-c|--clustered {y|n}] | 16:25 |
| herlo | [-d|--debug] | 16:25 |
| herlo | [-h|--help] | 16:25 |
| herlo | [-l|--maxlogicalvolumes MaxLogicalVolumes] | 16:25 |
| herlo | [-M|--metadatatype 1|2] | 16:25 |
| herlo | [-p|--maxphysicalvolumes MaxPhysicalVolumes] | 16:25 |
| herlo | [-s|--physicalextentsize PhysicalExtentSize[kKmMgGtTpPeE]] | 16:25 |
| herlo | [-t|--test] | 16:25 |
| herlo | [-v|--verbose] | 16:25 |
| herlo | [--version] | 16:25 |
| herlo | VolumeGroupName PhysicalVolume [PhysicalVolume...] | 16:25 |
| herlo | at least that one is not as long | 16:25 |
| herlo | we should run pvdisplay to see which PVs we can include | 16:25 |
| herlo | what you might notice if you have run this is that there might be several (or just a couple) PV's, the ones you are looking for correspond to the ones you've created with pvcreate | 16:26 |
| herlo | they should also show: Allocatable NO | 16:27 |
| herlo | because they have not yet been assigned to a VG | 16:27 |
| herlo | so let's do that | 16:27 |
| herlo | vgcreate -s 2M happy /dev/md0 | 16:27 |
| herlo | it's that simple | 16:27 |
| icarus-c | very | 16:28 |
| herlo | vgdisplay happy | 16:28 |
| herlo | will now display the details of the happy Volume Group | 16:28 |
| herlo | I would actually recommend you use a more descriptive name for your VG | 16:29 |
| herlo | from this VG, we can now take some space that will make our disks | 16:29 |
| herlo | according to vgdisplay, I have the following: | 16:29 |
| herlo | VG Size 208.00 MB | 16:29 |
| herlo | PE Size 2.00 MB | 16:29 |
| herlo | Total PE 104 | 16:29 |
| herlo | Alloc PE / Size 0 / 0 | 16:29 |
| herlo | Free PE / Size 104 / 208.00 MB | 16:29 |
| herlo | the PE's are now available for me to make my logical volumes. And on the LV side, we call those same elements Logical Extents as they have the same size and structure. | 16:30 |
| herlo | Only real difference is that the physical part is managed by the VG | 16:31 |
| herlo | okay, so let's make a disk we can use! | 16:31 |
| herlo | lvcreate is the tool for this | 16:31 |
| herlo | lvcreate --help provides two basic functionalities, one is basic creation, the other snapshots. If we have time, I'll discuss snapshots | 16:31 |
| herlo | so there's a structure there with some options, I'll detail the ones we need for a basic LV | 16:32 |
| herlo | -n|--name LogicalVolumeName] | 16:33 |
| herlo | {-l|--extents LogicalExtentsNumber | | 16:33 |
| herlo | -L|--size LogicalVolumeSize[kKmMgGtTpPeE]} | 16:33 |
| herlo | VolumeGroupName | 16:33 |
| herlo | that's really it | 16:33 |
| herlo | so it looks something like so | 16:33 |
| herlo | # lvcreate -l 90 -n camper happy | 16:34 |
| herlo | Logical volume "camper" created | 16:34 |
| herlo | at this point device mapper creates us a device | 16:34 |
| herlo | /dev/happy/camper | 16:34 |
| herlo | # ls -l /dev/happy/camper | 16:34 |
| herlo | lrwxrwxrwx 1 root root 24 2008-12-06 09:33 /dev/happy/camper -> /dev/mapper/happy-camper | 16:34 |
| herlo | and this is where we make our filesystem | 16:34 |
| icarus-c | Question: is that -l 90 refers to 90 PE? | 16:35 |
| herlo | yes, good point | 16:35 |
| herlo | you can either have -l (Logical Extents, which directly relate to Physical Extents) | 16:35 |
| herlo | or -L which is a size | 16:35 |
| icarus-c | i'm kinda confused by [kKmMgGtTpPeE] can you explain a bit? | 16:35 |
| herlo | right, so if you use -L, you can specify a size, rounded to the nearest LE | 16:35 |
| herlo | it's an either-or situation | 16:36 |
| herlo | you don't use both -l and -L | 16:36 |
| jds2001 | icarus-c: kilobytes, megabytes, gigabytes, terabytes, exabytes | 16:36 |
| herlo | you k, is kibibytes, K is kilobytes, P is Peta | 16:36 |
| herlo | etc... | 16:36 |
| icarus-c | i see, thanks | 16:36 |
| jds2001 | err petabytes in there too. | 16:36 |
| herlo | # lvdisplay /dev/happy/camper --- Logical volume --- LV Name /dev/happy/camper VG Name happy LV UUID 97h5rK-WOu3-QF3a-Ii1O-0E3u-vB1a-FH17L7 LV Write Access read/write LV Status available # open 0 LV Size 180.00 MB Current LE 90 Segments 1 Allocation inherit | 16:36 |
| herlo | Readahead sectorsr auto - currently set to 256 Block device 253:6 | 16:36 |
| herlo | ooh, ugly | 16:36 |
| herlo | let me try that again | 16:37 |
| jds2001 | you use pastebin! :) | 16:37 |
| herlo | LV Name /dev/happy/camper | 16:37 |
| herlo | jds2001: if I use pastebin, it expires | 16:37 |
| herlo | VG Name happy | 16:37 |
| herlo | Current LE 90 | 16:37 |
| herlo | LV Size 180.00 MB | 16:37 |
| herlo | there are other details there, but that's good for now | 16:38 |
| herlo | also, you can see much of htis detail at: http://www.softpanorama.org/Commercial_linuxes/logical_volume_manager.shtml | 16:38 |
| herlo | okay, so now we just build our filesystem on /dev/happy/camper and we've got it all good | 16:38 |
| herlo | ]# mkfs -t ext3 /dev/happy/camper | 16:39 |
| herlo | dumpe2fs /dev/happy/camper | 16:39 |
| herlo | can show the details of the new filesystem | 16:39 |
| herlo | feel free to mount it and you are on your way | 16:39 |
| herlo | any questions so far? | 16:40 |
| u0m3 | pretty self-explanatory :) | 16:40 |
| herlo | making sense? | 16:40 |
| SSlater | yup | 16:40 |
| Don_Miguel | yep | 16:40 |
| collier_s | yup | 16:40 |
| herlo | okay, so where the flexibility comes in for LVM is with it's ability to add devices | 16:40 |
| herlo | these are done in two steps | 16:41 |
| herlo | we always start by building a PV with pvcreate, so that step didn't change, but then we need to extend the VG and the LV | 16:41 |
| herlo | two commands exist to do this | 16:41 |
| herlo | vgcreate and lvcreate | 16:41 |
| herlo | vgextend and lvextend I mean | 16:42 |
| jds2001 | er, vgextend and lvextend, right? | 16:42 |
| herlo | wow! | 16:42 |
| * herlo is tired | 16:42 | |
| * kanarip hands herlo a cup of coffee | 16:42 | |
| jds2001 | i hear ya, early out there :) | 16:42 |
| herlo | kanarip: got one, just sipped it, thanks :) | 16:42 |
| herlo | the order is important here, as you will need to extend the vg (or at least have some LE's to work with in your VG) before you can use it in the current L | 16:43 |
| herlo | LV | 16:43 |
| herlo | lvextend --help | 16:43 |
| herlo | {-l|--extents [+]LogicalExtentsNumber[%{VG|PVS|FREE}] | | 16:43 |
| herlo | -L|--size [+]LogicalVolumeSize[kKmMgGtTpPeE]} | 16:43 |
| herlo | LogicalVolume[Path] [ PhysicalVolumePath... ] | 16:44 |
| herlo | and vgextend --help | 16:44 |
| herlo | VolumeGroupName PhysicalDevicePath | 16:44 |
| herlo | so let's do just that | 16:45 |
| herlo | vgextend happy /dev/sdd1 | 16:45 |
| herlo | # vgdisplay happy | 16:45 |
| herlo | now shows more PE's available | 16:46 |
| herlo | Total PE 158 | 16:46 |
| herlo | Alloc PE / Size 90 / 180.00 MB | 16:46 |
| herlo | Free PE / Size 68 / 136.00 MB | 16:46 |
| herlo | it also shows a size of | 16:46 |
| herlo | VG Size 316.00 MB | 16:46 |
| herlo | we now have 68 more PEs available to us which we can now use in our /dev/happy/camper | 16:46 |
| herlo | LV | 16:46 |
| herlo | lvextend allows us to use -l or -L again, and we can either specify the actual size or add the +Size/LE's | 16:47 |
| herlo | so it would look something like this | 16:47 |
| herlo | lvextend -l +68 /dev/happy/camper | 16:47 |
| herlo | or lvextend -L +136M /dev/happy/camper | 16:47 |
| herlo | if you wished to not use the + it woudl be | 16:48 |
| herlo | or lvextend -L 316M /dev/happy/camper | 16:48 |
| herlo | # lvdisplay /dev/happy/camper | 16:49 |
| herlo | LV Size 316.00 MB | 16:49 |
| herlo | Current LE 158 | 16:49 |
| herlo | but we still have a filesystem size much smaller than this | 16:49 |
| herlo | # df -h | 16:49 |
| herlo | /dev/mapper/happy-camper 175M 5.6M 160M 4% /mnt | 16:50 |
| herlo | so we need to resize | 16:50 |
| herlo | depending on your filesystem, there are several ways to do this, in ext3, its resize2fs | 16:50 |
| herlo | there's mount options for jfs and xfs that do similar things | 16:50 |
| herlo | something like | 16:50 |
| herlo | mount -o remount,resize_jfs /dev/happy/camper | 16:50 |
| herlo | woudl probably work | 16:50 |
| herlo | for us, since we're using ext3 we want to run the followign command | 16:51 |
| herlo | resize2fs /dev/happy/camper | 16:51 |
| herlo | which will perform the resize *online* and grow the volume in front of our eyse | 16:51 |
| herlo | eyes | 16:51 |
| * herlo waits for the resize to finish :) | 16:52 | |
| herlo | # resize2fs /dev/happy/camper | 16:52 |
| herlo | resize2fs 1.41.3 (12-Oct-2008) | 16:52 |
| herlo | Filesystem at /dev/happy/camper is mounted on /mnt; on-line resizing required | 16:52 |
| herlo | old desc_blocks = 1, new_desc_blocks = 2 | 16:52 |
| herlo | Performing an on-line resize of /dev/happy/camper to 323584 (1k) blocks. | 16:52 |
| herlo | The filesystem on /dev/happy/camper is now 323584 blocks long. | 16:52 |
| herlo | # df -h | 16:52 |
| herlo | /dev/mapper/happy-camper 307M 6.1M 285M 3% /mnt | 16:52 |
| herlo | now, there's how to create a logical volume | 16:52 |
| herlo | what's also neat about LVM is that the default install of Fedora uses it as well. This provides similar flexibility to increase the size of your drives if you add a second hard drive to your system | 16:53 |
| herlo | any questoins? | 16:53 |
| Fuss284 | what about shrinking one? | 16:53 |
| herlo | yes, that can be done as well | 16:53 |
| MartinCo | painful though | 16:53 |
| herlo | and that brings up a good discussion | 16:53 |
| MartinCo | :) | 16:53 |
| herlo | first off, it cannot be done online | 16:54 |
| herlo | but, what if you lost a part of your RAID device and wanted to just replace the device | 16:54 |
| herlo | you could do that all online | 16:54 |
| herlo | you would need to add a new PV to the VG, and then run a pvmove | 16:54 |
| herlo | the pvmove would move all of the PEs from the device specified to the rest of the larger VG | 16:54 |
| herlo | and then a pvremove would allow you to remove the devices from the system | 16:55 |
| icarus-c | what about shrinking your happy-camper ? :P | 16:55 |
| herlo | as far as shrinking, I'd first have to unmount /dev/happy/camper | 16:55 |
| herlo | then lvreduce | 16:55 |
| herlo | then vgreduce | 16:55 |
| herlo | if you wanted to reduce the vg that is | 16:55 |
| icarus-c | i think we need to resize the filesystem too right? | 16:56 |
| herlo | ooh, first | 16:56 |
| herlo | you need to fsck and resize the fs, yes | 16:56 |
| Fuss284 | herlo don't you have to reduce the filesystem too? since, when you extended it, you had to.. ah | 16:56 |
| herlo | which is why you can't do it online | 16:56 |
| herlo | thanks guys, sorry I knew that :) | 16:56 |
| icarus-c | so first umount , 2nd resize fs and finally resize the lv & pv? | 16:56 |
| herlo | not the pv | 16:56 |
| herlo | and you don't have to resize the vg unless you want to | 16:57 |
| icarus-c | ar vg | 16:57 |
| herlo | also, you can make multiple lv's from one vg | 16:57 |
| herlo | so if I had wanted to make /dev/happy/camper and /dev/happy/gilmore,that's fine just as long as you dont try to make one LV from multiple VGs | 16:57 |
| herlo | another thing that's really neat is snapshots | 16:58 |
| herlo | the concept of snapshots is that they are easy to build structures that take a timestamp (essentially) in time and allow it to be backed up, while the LV is still online | 16:58 |
| herlo | what is needed is another disk, which will be created with pv's and Vg's then the LV is pointed at the LV to back up, and mounted read only | 16:59 |
| herlo | then a back up device can start its copy | 16:59 |
| herlo | the interesting part about this is that when a change occurs on the origin LV, it's delta is saved off to the snapshot LV and the origin LV continues to work as normal | 16:59 |
| herlo | this is really good for full-backups | 17:00 |
| herlo | one other thing I'd like to quickly mention is the fact that LVM is fully recoverable onto another system in a matter of minutes | 17:00 |
| herlo | every bit of data about the lvm is stored in the superblock (or whatever you want to call the first sector) | 17:01 |
| herlo | of each of the disks | 17:01 |
| herlo | so pull your disks, move them to another machine and run | 17:01 |
| herlo | vgscan | 17:01 |
| herlo | you'll see your vg and you can then activate it | 17:01 |
| herlo | vgchange -a y <vgname> | 17:01 |
| herlo | and voila, you've moved your vg from one machine to another | 17:02 |
| herlo | any other questions? | 17:02 |
| herlo | k, guess not. Thanks all | 17:03 |
| Discordian | thank you herlo | 17:03 |
| u0m3 | thank you for the lesson :) | 17:03 |
| collier_s | yup, great lesson | 17:03 |
| MartinCo | thanks | 17:03 |
| SSlater | Thanks herlo. That clarified things. | 17:03 |
| * herlo wants to remind everyone that #fedora-townhall has some FAmSCo discussions shortly | 17:04 | |
| herlo | thanks guys... | 17:04 |
| * SSlater falls asleep in back of classroom. | 17:04 | |
| * herlo mentions that 2 of the 4 irc classes are already posted on Communicate/IRC/Classroom the other two will be posted shortly | 17:05 | |
| u0m3 | herlo one question: do you happen to know if the /boot partition can be in lvm? as far as I know/remember grub dosen't know/understand lvm. has that changed? | 17:07 |
| herlo | u0m3: not yet | 17:08 |
| herlo | in grub2, it can | 17:08 |
| u0m3 | ok. thanks | 17:08 |
| herlo | that's coming along nicely too. I've been watching its development over the past while. | 17:08 |
| herlo | u0m3: http://www.gnu.org/software/grub/grub-2.en.html | 17:09 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!