Comment Welcome to the world of Parallel filesystem (Score 1) 557
The best way to have access to have all that idea is to have a parallel filesystem. There are many type of parallel filessytems. Some are designed for certain types of files others for other sorts.
PVFS and GPFS can be said to be the two most useful type of filesystems to be in use. I would reccomend that you use somethng like attached storage through fibre or through a SCSI because if a part of the array goes down then you can jump start it by replacing the SCSI array or the storage node.
Make sure your data storage drives are raid 5 so if you have one disk that goes bad then you don't have to call the whole file system to do a recreate.
Making a large filesystem is expensive. But go with the parallel filesystem option you will see how fast and easy it is. And also try to use some 64 bit system so you can have larger ammounts of metadata etc..
And PS with such a large filesystem say bye bye to the inode problem.
PVFS and GPFS can be said to be the two most useful type of filesystems to be in use. I would reccomend that you use somethng like attached storage through fibre or through a SCSI because if a part of the array goes down then you can jump start it by replacing the SCSI array or the storage node.
Make sure your data storage drives are raid 5 so if you have one disk that goes bad then you don't have to call the whole file system to do a recreate.
Making a large filesystem is expensive. But go with the parallel filesystem option you will see how fast and easy it is. And also try to use some 64 bit system so you can have larger ammounts of metadata etc..
And PS with such a large filesystem say bye bye to the inode problem.
