|
|
Lustre: building a filesystem for 1,000 node clusters
Lustre is a GPLed cluster file system for Linux that is currently being tested on three of the world's largest Linux supercomputers, more than 1,000 nodes each. In the past 18 months we've tried many tactics to scale to these limits, and the first half of this paper will discuss some of our successes as planned, and some of our grisly failures, including:
- lock management in Lustre
- object storage on dozens of servers
- forcing a network file system into the Linux VFS
The second half will describe a couple of the ways in which we hope to scale to 10,000 nodes and beyond.
This will be pitched at developers familiar with basic file system and VFS concepts; no detailed kernel programming or VFS internals knowledge is necessary.
|
|
