Monday, June 26, 2006

zfs and zpool programs successfully compile and link.

Beware -- long post.

Sorry for the lack of updates recently, I've been kind of busy with other stuff.. ;)

I have good news. As you can see from the title, the zfs and the zpool programs now successfully compile and link :)

There are 2 known functionality losses: Linux doesn't seem to have a libdiskmgt equivalent and porting it seems rather complicated if not almost impossible, so there is no "device in use" detection. If anyone knows a good way to solve this, I'm all ears :)

The other loss is the "whole disk" support. This must eventually be solved, since it's rather common in Solaris to dedicate a whole disk as a zpool vdev. I believe this can be circumvented for now by creating a partition that uses the whole disk, and create the zpool vdev using the partition device.

The specific problem is that ZFS uses EFI labels in disks. I only found one working EFI library for Linux, but the API is different from the OpenSolaris implementation. Once again, porting the EFI functionality from OpenSolaris proved to be difficult.

--

Unfortunately the zfs and zpool programs don't do anything useful yet.

In the original implementation, they use the ioctl(2) system call through the /dev/zfs device to communicate with the kernel. Since this is a userlevel implementation, there will be no /dev/zfs.

In the zfs-fuse implementation, instead of using ioctl(), we communicate through a UNIX domain (local) socket which is created in /tmp/.zfs-unix/. So in order to make these commands actually do something, there must be a zfs-fuse process that answers the messages sent from zpool and zfs.

My plan to make zfs-fuse work is to take advantage of as much code from the original implementation as possible. Yes, this also means I want to use the original ZPL code. It seems to be the most reliable way to do it, and perhaps also the easiest one :)

In order to do that, I've created a libsolkerncompat library that will implement/translate the necessary OpenSolaris kernel code to make the ZPL work. This library will also be necessary in order to use the original zfs_ioctl.c implementation, since it uses some kernel VFS (Virtual File System) operations, along with other things.

This will also take some time to get it working, since a new zfs_context.h must be created or (more likely) the current zfs_context.h must be factored out to libsolkerncompat.

So, in a way, I can say I'm now in phase 3.5, since I'm actually working on the zfs-fuse process (which includes the necessary bits to make the ZPL code work), but I'm still not in phase 4, since zpool and zfs won't work until zfs_ioctl.c is ported.

7 comments:

Anonymous said...

Hi,

Do you think there will be a performance loss due to user-space implementation ? (over in-kernel filesystems like ext3, or over original OpenSolaris implementation for example)

BTW, in OpenSolaris, is ZFS implemented in user or kernel space ?

wizeman said...

Hi there,

Personally, I don't think there will be much performance loss over the original OpenSolaris implementation, if you have a reasonably fast CPU. Of course, only time will tell :p
As compared to ext3, I really don't know.

In OpenSolaris, the filesystem runs in kernel-space.

Anonymous said...

Quite excited about this project, ZFS is phenominal. A question though - you're talking about creating the zpool vdev using the partition device, does this mean that zpools created under zfs-fuse won't be compatible with the solaris implementation? It would obviously be optimal if pools were still able to be moved between systems running different OSs.

wizeman said...

Yes, for the moment, if you create a zpool in OpenSolaris using a whole disk as a vdev, you won't be able to open it in Linux (because it doesn't recognize EFI labels).

But don't worry, this is in my TODO list and will certainly be solved :)

Anonymous said...

Linux handles EFI in the kernel. It's a
compile option under the menu for
advanced partitioning options.

The whole device is /dev/sda, /dev/hda,
/dev/sdb, or similar.

Anonymous said...

hi,

if you need help, I would be interested in contributing to this project.

Rob said...

I'd be interested as well in helping out. Linux needs ZFS in my opinion. I might have some time in August, at least to test. It's sad that ZFS cannot be integrated into the Linux kernel since my personal interests in ZFS also lie in a having a clustered filesystem (after having had mixed success with OCFS, GFS, Lustre and others), and for that you need a DLM which would already be in place in 2.6 kernel.