ext4 Implementation
One major feature present in Fedora 9 will be the ext4 implementation. The new filesystem will not be the default for the distro, but will be available for users and systems administrators to enable. New functionality includes larger capacities and online defragmentation, for better performance and more reliability. To find out more, we talked with Eric Sandeen, Fedora project member and filesystem developer at Red Hat.
Digg It
If you enjoy this interview, please feel free to give it a digg :)
http://digg.com/linux_unix/Eric_Sandeen_ext4_Implementation_in_Fedora_9
Developer Interview
Occupation: Senior Software Engineer, Red Hat Inc.
Location: Minneapolis, MN USA
IRC Nick: sandeen or esandeen
Website: http://sandeen.net/
Age: heh, why? between 30 and 40 ;)
Interviewed by: RodrigoMenezes
What motivated you work on Ext4 integration for Fedora 9?
Eric Sandeen: Fedora has always been an excellent platform for testing out exciting new Linux developments, so it's a natural fit. Now that the bulk of ext4 work is done, getting it into the hands of Fedora users is a great way to get some "real world" testing by early adopters.
How much upstream development does Fedora drive on Ext4?
Eric Sandeen: ext4 development has been a joint effort by several entities. A quick look at the linux-ext4 mailing list will show contributions from several companies and individuals, all interested in helping to develop ext4. One of my responsibilities at Red Hat is to do filesystem work for Fedora and RHEL, so I've also been doing what I can to move things along by submitting patches, testing, fixing, etc.
Could you explain more about new features in Ext4? What'll be the first impression for users?
Eric Sandeen: Well, hopefully the first impression will be that it works just like ext3, except that it's faster and more capable. Probably the biggest "feature," which is not something end-users care about directly, is the new on-disk extent format. This allows the filesystem to keep track of file data in [offset, length] pairs rather than block-by-block, and this is much more efficient than the ext3 mechanism. Deletion of large files should be noticeably faster, for example.
Along with the new extent file format, the "Multiblock Allocator" should result in better file allocation for most workloads, resulting in better file placement and less fragmentation as the filesystem ages.
The other big thing some users may notice eventually is the ability to have larger files and filesystems. ext3 was limited to 16T filesystems, (and that only recently; prior to 2.6.18 the limit was 8T), whereas the ext4 disk format will be able to support up to 1024P. File size limits have grown, as well - where ext3 format files have a limit of 2T, ext4 files in extent format can have a maximum offset of 16T. (max file and filesystem sizes depend on blocksize; the numbers given are for 4k block filesystems.)
Other features that will be nice for users include finer-grained timestamps, and a larger maximum subdirectory limit (now 65k subdirs). ext4 will also make use of in-inode extended attributes, which should make things like SELinux, beagle, and samba acls more efficient.
The "uninitialized block group" feature should speed up mkfs and fsck in some situations, by allowing the tools to completely skip metadata chunks which have never been used. The flex_bg feature should also help with fsck time due to a new disk metadata layout.
Persistent preallocation of files will also be good for some applications; for example, when you eventually download your Fedora 10 DVD torrent onto ext4, hopefully your client will first preallocate the space to avoid the horrible fragmentation that can result from this type of file transfer.
How does it differ to the existing solutions?
Eric Sandeen: Compared to other existing Linux filesystems, ext4 has added features which are present in other filesystems known for scalability. ext4 should have robust set of scalable features, building on the trusted ext2 & ext3 codebase.
How does Ext4 compare to other recent filesystems like btrfs?
Eric Sandeen: btrfs is an exciting new development in linux filesystems, which seems to hold a lot of promise. It is attempting to address Linux storage problems in fundamentally different ways than ext4 - for example, it is a COW (copy-on-write) filesystem, and this allows it to do all sorts of nifty tricks like writable, recursive snapshots, subvolumes, etc.
I would say that compared to btrfs, ext4 is more evolutionary than revolutionary; it takes the solid foundation built by ext2 and ext3, and allows it to scale to new levels of size & performance, with an easy migration path and minimal risk for users.
The Ext4 is compatible with Ext3, is this correct? In a future migration, users doesn't need to format the particion, is that it?
Eric Sandeen: It will be possible to "upgrade" to ext4 by simply mounting your ext3 filesystem as ext4; any newly-created files will be created in the new extents format, by default. (Note that once these extent-format files are created, the filesystem can no longer be mounted as ext3, unless the files are removed, and the EXTENTS flag is un-set; this backwards migration can be done, but it is somewhat cumbersome).
There is also an ioctl to convert "ext3" files to the new extent format, although the tool to do this conversion is not yet part of e2fsprogs.
We never heard about Defragmentation in a Linux solution, How do you think this could help the system?
Eric Sandeen: Defragmentation can be a good tool to reclaim performance lost to poor file layout.
The XFS filesystem is one linux filesystem that I know of which has had a defragmentation utility for years. It is able to take fragmented files (those which have data blocks which are not contiguous on-disk), allocate new, more-contiguous space, and move the file blocks into this space.
The ext4 defragmenter is still in development, and it is planned to be very flexible - in addition to making single files contiguous, it will do other neat tricks such as moving all files in a directory close to each other, and free-space defragmentation (moving data files to make larger contiguous areas of free space on the disk).
What work still needs to be done to reach a state where you feel its achieved its initial goals?
Eric Sandeen: There are still a few planned features which probably will not be available until after the F9 release - delayed allocation, for example, which is a mechanism used in other filesystems to wait until data is actually flushed to disk to make allocation decisions, and by doing so can batch these allocations together, which in turn allows for better allocation decisions.
Large filesystem support, which will allow filesystems greater than 16T, also requires a bit more work, largely in the e2fsprogs codebase.
The defragmentation and migration tools may also come a bit later.
For the most part, though, the major benefits from ext4 should be available to users when Fedora 9 is released.
How's the work going on getting this ready to be easily available for Fedora 9?
Eric Sandeen: The Fedora 9 Alpha installer already can do an install onto the ext4 filesystem, provided that the proper secret phrase is provided on the installer boot prompt. There were a couple small issues resolved after the Alpha, for the most part it is a simple matter to do an ext4 install. By F9 Preview or Final release, I hope that ext4 will be available as an option on the custom partitioning screen without requiring any non-standard boot options.
As of this writing, e2fsprogs is still a missing piece; while we can create, mount, and use ext4 in early F9 spins, there is not yet a fully ext4-compatible version of e2fsprogs to use for administration and maintenance. However, this should be fixed soon.
Other bits and pieces have needed to be fixed up as well; things like the "file," blkid, and udev utilities needed to know about ext4, as did the system-config-lvm tool.
By the time F9 is final, I hope that ext4 is as available and easy to use as ext3.
Do you fell safety implement Ext4 in an existent Fedora 8 system?
Eric Sandeen: I'll have to talk with the kernel maintainers about that :) If ext4 reaches non-development status in one of the kernel revisions shipped with F8, then I could imagine that ext4 may be available there too.
What about defaults, will Ext4 be the new default for Fedora 9 systems?
Eric Sandeen: I don't expect that it will be the default; ext3 still serves most Fedora users quite well, in my opinion. But for those who have scaling, performance or feature needs above what ext3 can deliver, or for those who just like to try out new things.
Do you see other distributions working to implement Ext4?
Eric Sandeen: I think that a couple of others have been shipping "ext4dev" which is the pre-released version of the ext4 code. But Fedora is the only distribution I know of which is actively working to help finalize ext4 and seamlessly integrate it into the OS.
And to finish, you could tell us a little bit about yourselves? What got you interested in free software originally? What do you like to do with your spare time when you're not working with computers?
Eric Sandeen: My first-ever kernel code was fixing up a video4linux driver for an old bttv card to make audio work right. I'm ashamed to say that most of that ugly code is still in the kernel. :) I joined SGI around 2001 to work on the xfs filesystem, and moved to Red Hat in 2006. My wife says there is no such thing as "spare time when I'm not working with computers" but I do my best to spend most of that time with my family. My almost-4-year-old daughter offered to help me fix the last ext4 bug I was working on, so she may follow in my footsteps.