Super-Micro has 36 and 72 drive racks that aren't horrible human effort wise (you can get 90 drive racks, but I wouldn't recommend it). You COULD get 8TB drives for like 9.5 cent / GB (including the $10k 4U chassi overhead). 4TB drives will be more practical for rebuilds (and performance), but will push you to near 11c / GB. You can go with 1TB or even 1/2TB drives for performance (and faster rebuilds), but now you're up to 35c / GB.

That's roughly 288TB of RAW for say $30k 4U. If you need 1/2 PB, I'd say spec out 1.5PB - thus you're at $175K .. $200k.. But you can grow into it.

Note this is for ARCHIVE, as you're not going to get any real performance out of it.. Not enough CPU to disk ratio.. Not even sure if the MB can saturate a 40Gbps QSFP links and $30k switch. That's kind of why hadoop with cheap 1CPU + 4 direct-attached HDs are so popular.

At that size, I wouldn't recommend just RAID-1ing, LVMing, ext4ing (or btrfsing) then n-way foldering, then nfs mounting... Since you have problems when hosts go down and keeping any of the network from stalling / timing out.

Note, you don't want to 'back-up' this kind of system.. You need point-in-time snapshots.. And MAYBE periodic write-to-tape.. Copying is out of the question, so you just need a file-system that doesn't let you corrupt your data. DEFINITELY data has to replicate across multiple machines - you MUST assume hardware failure.

The problem is going to be partial network down-time, crashes, or stalls, and regularly replacing failed drives.. This kind of network is defined by how well it performs when 1/3 of your disks are in 1-week-long rebuild periods. Some systems (like HDFS) don't care about hardware failure.. There's no rebuild, just a constant sea of scheduled migration-of-data.

If you only ever schedule temporary bursts of 80% capacity (probably even too high), and have a system that only consumes 50% of disk-IO to rebuild, then a 4TB disk would take 12 hours to re-replicate. If you have an intelligent system (EMC, netapp, ddn, hdf, etc), you could get that down to 2 hours per disk (due to cross rebuilding).

I'm a big fan of object-file-systems (generally HTTP based).. That'll work well with the 3-way redundancy. You can typically fake out a POSIX-like file-system with fusefs.. You could even emulate CIFS or NFS. It's not going to be as responsive (high latency). Think S3.

There's also "experimental" posix systems like ceph, gpfs, luster. Very easy to screw up if you don't know what you're doing. And really painful to re-format after you've learn it's not tuned for your use-case.

HDFS will work - but it's mostly for running jobs on the data.

There's also AFS.

If you can afford it, there are commercial systems to do exactly what you want, but you'll need to tripple the cost again. Just don't expect a fault-tolerant multi-host storage solution to be as fast as even a dedicated laptop drive. Remember when testing.. You're not going to be the only one using the system... Benchmarks perform very differently when under disk-recovery or random-scatter-shot load by random elements of the system - including copying-in all that data.

Git is an excellent system, but is less efficient for large files. This makes certain work-flows difficult to put into a git-repository. i.e. storing compiled binaries, or when having non-trivial test-data-sets. Given the 'cloud', do you forsee a version of git that uses 'web-resources' as SHA-able entities, to mitigate the proliferation of pack-file copies of said large files. Otherwise, do you have any thoughts / strategy for how to deal with large files?

What is your preference for package management? E.g. for a new library, which flavor do you reach for? rpm? tgz? deb? git-clone? home-brew? </wink>

What is your opinion of BSD and the OS-X stack? Ever use them? Do you envy and of their attributes?

How do you deal with hot heads? You are bound to have interacted with some of the most self-important, bombastic, difficult people on earth in your days. Do you have any advice how you've managed to keep your cool?

Is C your favorite language? Given your ability to invent your way out of frustration, I find it hard to believe C hasn't frustrated you into innovation.. So many languages were created for that very reason. Perl, python, dart, go, rust, etc.

Have you ever considered a network-transparent OS-layer? If not why? I once saw QNX and and how the command line made little differentiation of which server you were physically on. (run X on node 3, ps (across all nodes)). You ran commands pretty much on any node of consequence.. I've ALWAYS wanted this capability in Linux... cluster-ssh is about as close as I've ever gotten. These days hadoop/storm/etc give a half-assed approximation.

Is there any inspiration that a GPU based kernel / scheduler has for you? How might Linux be improved to better take advantage of GPU-type batch execution models. Given that you worked transmeta and JIT compiled host-targetted runtimes. GPUs 1,000-thread schedulers seem like the next great paradigm for the exact type of machines that Linux does best on.

What are your thoughts on apple's GrandCentral? It seems that a central kernel managed queuing system would be less overhead than having 10 apps each launching 10x num-cpu's threads and all over-subscribing.

I'm not seeing the connecting-gap between ' Amercans no longer fret over iPhones (because we can print one with a 3D printer ) ' and 'We can build a star-ship because we've decoupled interest-in-work from the-need-to-work-to-earn-money-to-survive / acquire the things we wish to have'.

I don't fundamentally understand how a star trek society can exist. If we can all convert energy into material things. Consider the fabel, "these are rich people's problems".. Meaning the stresses that make us work harder are ultimately enslave us to our commitments, _change_ as we get wealthier (individually and socially), but they do not disappear.

You might consider the man that has earned enough money that he can go back-packing in Asia for 10 years.. Could the world function if everybody did so? Assume even that we had robots to build houses / plant our food. SOMETHING is always going to be present that prevents eutopia, even 1,000 years after such a world.

It's too narrow minded to look at today's problems, remove a single variable and say; now sci-fi happens.

Usb 3 is no where near as fast as even esata 2. Ignore the hype. BW is rarely the limiting factor. I can regularly get ssd boosts of 30% on the "slower" esata.

We may have 25% less radioactive elements in our planet's interior than some of these other planets, but we have a large moon that is causing a significant amount of tidal friction. That should help close the internal heat gap a bit...and as a bonus it keeps our axis fairly stable.

There are many different types of homes out there. Some just have better floor heating than Earth. I rather like our bright heat lamp in the sky...so do the plants in my yard.

A good discovery nonetheless. I'm excited that life may have more places where it can exist, and perhaps even thrive like it has done here on Earth.

We've seen a lot of news recently on slashdot about software being released before it is ready. This is just a glimpse into the inner-workings of a very large development team making a tough, but correct, call.

Thanks Linux & Co! Keep up the good work!

Since we are talking future science here it would make sense to harness the incoming blast way and convert it back to energy that can be used for the next flight. The problem is building the infrastructure to do this at the destination, but if they can figure out how to go faster than light I'm sure they can find a way to make this a reality too.

It's a UNIX system! I know this!