n4

n4 is an experimental userspace NFSv4 server for Linux, plus a userspace NFS4 client library and benchmark.

Code News Mailing lists

A lot has been learnt about Linux networking performance for SMB and HTTP through extensive tuning against the netbench, dbench and wbench benchmarks. Relatively little of this work has yet been applied to NFS, although it is also an important protocol. n4 aims to take the experience of the Samba team and apply it to the next-generation NFS4 protocol. The project is still in early development. The code is available under the terms of the GNU General Public Licence, but is not yet a fully functioning NFS server. Samba CVS, module n4 CVSweb browser rsync://samba.org/ftp/unpacked/n4/ source code snapshots goals It must work. It must be a userspace program and not a kernel patch: We can implement and debug new features more quickly in userspace. It's unreasonable to expect end users to rebuild their kernels (and reboot!) to install application-level functionality. Crashing in userspace is less harmful. The code must be easy to read and maintain. It must be fully compliant with RFC3010, the NFS4 standard. We want to at least cover the MUST functions as soon as possible. It must be as fast as possible on the hardware available. performance ideas: RFC3010 might be described as RPC, but it's really a long way from the simple RPC paradigm. Treat it as a binary protocol, and encode/decode as and when needed. Keep network IO separated from the rest of the program, so we can experiment with the mapping of work onto threads. In particular, network and disk IO will be a substantial part of the wall time. Perhaps use sendfile(2) for bulk data transfer. A problem with this is that if the file is truncated after we send the header but before the sendfile completes, then the packet will be short. (Samba has the same issue.) One possible solution is to just drop the socket: the client ought to see that packet as truncated, and retry the request. Since this situation only happens in a race, that might be an OK solution. If we know (through some kind of heuristic) that we'll do a lot of IO on a particular file, then mmap it and serve from memory. A similar technique is a win Apache's mod_mmap_static. If we use bound TCP (or even UDP) sockets then a single process or cluster of processes will look after a client, and it's quite likely that files will be hot on only a single client. Sending and receiving packets should take as few write/read system calls as possible. We can build a single output buffer per fragment and average one write per request. We can do two reads if we read the length prefix separately, or about one on average if we read ahead. crazy ideas: Full RFC3010 access control lists done in userspace, and shared between N4 and Samba. A VFS (virtual filesystem) layer on the server, so that it can act as a filesystem-like interface to anything. why a userspace server? why a userspace client? > G'day again Martin, > > Curious - if your NFS client is going to be entirely user-space, do > you have a way of integrating it into the VFS somehow so that it can > be mounted like a regular file system? We do need a good kernelspace client to get truly transparent operation. The folks at citi.umich.edu are working on that. Briefly, no, there are no plans at the moment to make n4 into a kernel-level filesystem. I'm mostly interested in a userspace server. My reasons for writing a userspace client are: - At the moment the client is about 2600 lines, the server 4000, and the common area is 4700 lines. I'd expect the ratio to stay roughly the same, so it doesn't require that much more effort. - If nothing else, the client is a test harness for the server. - It's a good foundation for writing test or benchmarking suites. Kernelspace clients normally exercise only a small fraction of the protocol which is useful to their servers. In userspace we can test the whole protocol, reproduce the behaviour of clients on different servers, and so on. A good free benchmark is crucial to improving our performance on NFS -- a fair fraction of the credit for our good web and SMB performance goes to dbench and wbench being free, simple to use, and used often by kernel developers. - A userspace client can be ported to other less common systems that might not otherwise get any kind of implementation for a long time. Portability is not a major goal, but it'll be much easier than for a kernelspace client. - You might want to access a filesystem using NFS (perhaps over ssh) when you don't have root access on the client. Can't do this with a kernel client; can do it easily with n4client. - I think the NFS RPC system might be a good foundation for rsync 3.0. - Plenty of programs implement userspace filesystem access in a useful way (emacs, ftp, web browsers, Samba, ...) - NFSv4 can potentially use cache-consistent proxies, which need to have both client and server implementations. - Debugging the client, or stepping through the client as you debug the server is much faster (x3?) than kernel space: you can run gdb, crashes don't take down the machine or corrupt your filesystem, you can rebuild (on my machine) in 30s not 300s, ... - The barrier to entry for other people to try out n4 or write patches is much lower in userspace. Cheers, -- Martin mailing lists Three mailing lists are presently available for discussing n4: n4-announce Low-volume, moderated announcement list. Receives mail when a new version is released. n4-discuss Medium-volume discussion list for people interested in working on or using n4. n4-cvs Intermittently high-volume change messages. news 2001-06-21 (CVS tag n4_feature_cat) The READ operation now works, and the client has a cat command that retrieves files from the server. (We continue the Unix tradition of intuitively obvious command names. ;-) 2001-06-18 NFS4 presentation slides from the Australian Unix User's Group Australian Open Source Symposium. 2001-06-01 Directory listing more reliable. You can tunnel connections over ssh or some other connection protocol: $ n4client -c 'ssh samba.org n4/source/n4d' The client uses GNU readline and has tab completion of commands. 2001-05-28 The code with CVS tag n4_first_ls can display the root directory in n4client. 2001-05-31 n4_ls_dirs The client can now list directories other than the root, using a simple implementation of LOOKUP. Connections can tunnel over ssh or some other connection program.

Copyright (C) 2001 by Martin Pool.
$Id: index.latte,v 1.18 2001/08/15 04:52:17 mbp Exp $