NFS has become critical to data center grid environments. As a result, Oracle has optimized its code specifically for NFS. Instead of relying on the operating system, Oracle’s Direct NFS Client generates NFS requests directly from the database.
Direct NFS was inspired by experience at Oracle’s Austin Data Center. Oracle uses NFS to run its applications on tens of thousands of Linux servers accessing many petabytes of NetApp storage. In 2005 they had 12,000 Linux servers and 3 petabytes of NetApp storage. Today’s numbers aren’t public, but they are much larger.
When an operating system capability becomes sufficiently important, Oracle pulls it into the database. Memory management became critical, so Oracle said, “Just give me the raw pages, and I’ll manage them myself.” Disk caching became critical, and Oracle said, “Just give me the raw disk blocks, and I’ll cache them myself.” Now NFS has become critical, so Oracle says, “Just give me a raw TCP/IP socket, and I’ll generate NFS requests myself.”
Steve Kleimanhas argued that as Oracle becomes more sophisticated, the operating system becomes little more than a device driver framework that gives the database raw access to the hardware. That sheds new light on Oracle’s Unbreakable Linuxprogram.
What exactly does Oracle gain from Direct NFS? The primary benefits are simplicity and performance.
It’s simpler because you don’t have to worry about how to configure NFS. What timeouts should you use? What caching options? It doesn’t matter. Oracle looks at how you have NFS configured to figure out where the data lives, but aside from that, your settings don’t matter. Oracle takes control.
It even works with Windows. Just mount the data that Oracle needs using a CIFS share, and Oracle figures out the location of the data and accesses it via NFS. (CIFS is great for home directory sharing, but it isn’t designed for database workloads.)
Performance is better because Oracle bypasses the operating system and generates exactly the requests it needs. Data is cached just once, in user space, which saves memory – no second copy in kernel space. Oracle also improves performance by load balancing across multiple network interfaces, if they are available.
For more technical details on Direct NFS, check out this article by Kevin Closson. He works for PolyServe, which is a NetApp competitor, but technically speaking, he talks good sense. I also recommend this article, by NetApp’s John Elliott, comparing Oracle performance over Fibre Channel, NFS and iSCSI.
NetApp has been closely involved in Direct NFS from the very beginning. Peter Schay came up with the idea while he worked for Oracle’s “Linux Program Office”. He wanted to simplify things for Oracle customers running on Linux, many of whom were hosted on Oracle’s On-Demand environment at the Austin Data Center. He worked closely with NetApp engineers to prototype and test the idea. The Oracle ST team used his functional specification to develop the production version of Direct NFS now shipping in 11g. (Today Peter works for NetApp.)
I love how NFS has evolved over the past couple of decades. Twenty years ago, it providing file sharing to small engineering workgroups; today it provides the data backbone for some of the world’s largest data centers. What it is about NFS that has allowed it to make this transition? What is it about NFS that Oracle would choose to build it directly into their database? That’s the topic for another post!