Yes, rebasing is reduced by careful selection of preferred base addresses (particularly by Microsoft for their DLLs). Yes, if DLLs are not rebased then they are shared -- I did not claim otherwise. My point along these lines is that rebasing *does* occur surprisingly often, and can hurt sharing. The actual level of sharing you achieve obviously depends almost *entirely* your applications, workload, data, etc.

By the way, as far as I know versions of Windows newer than Vista enable address-space randomization by default for security purposes. Since the starting virtual address of each DLL is randomized, preferred bases can't be respected. I don't know what impact this has on Windows memory usage post-Vista, but it seems like one can't rely on carefully curated base addresses.

I'm not saying one approach is better than the other (Linux, Windows, whatever) -- I'm only positing a possibility for why one might see better improvement with KSM. One might just as easily see better over-subscription with Windows simply due to the fact that it zeroes out physical pages when they are released, as far as I know. Those zero pages can all be mapped to the same machine frame transparently (without the need for balloooning).

Disclaimer: I wrote the blog post. I noticed the massive slashdot traffic, so I popped over. The article summary is not /entirely/ accurate, and doesn't really completely capture what we're trying to do with our software.

Our mechanism for performing over-subscription is actually rather unique.

Copper (the virtualization platform in the demo) is based on an open-source Xen-based virtualization technology named SnowFlock. Where KSM does post-processing on memory pages to share memory on a post-hoc basis, the SnowFlock method is much more similar to unix 'fork()' at the VM level.

We actually clone a single VM into multiple ones by pausing the original VM, COW-ing its memory, and then spinning up multiple independent, divergent clones off of that memory snapshot.

We combine this with a mechanism for bringing up lightweight VMs fetching remote memory on-demand, which allows us to bring up clones across a network about as quickly and easily as clones on the same machine. We can 'clone' a VM into 10 VMs spread across different hosts in a matter of seconds.

So the mechanism for accomplishing this works as follows:
1. The master VM is momentarily paused (a few milliseconds) and its memory is snapshotted.
2. A memory server is setup to serve that snapshot up.
3. 16 'lightweight' clone VMs are brought up with most of their memory "empty".
4. The clones start pulling memory from the server on-demand.

All of this takes a few seconds from start to finish, whether on the same machine or across the network.

We're using all of this to build a bona-fide cluster operating system where you host virtual clusters which can dynamically grow and shrink on demand (in seconds, not minutes).

The blog post was intended not as an ad, but rather a simple demo of what we're working on (memory over-subscription that leverages our unique cloning mechanism) and a general introduction to standard techniques for memory over-commit. The pointer to KSM is appreciated, I missed it in the post :)

I have one possibility. The blog post alluded to this. Page sharing can be done *much* more efficiently on Linux due to the fact that the ELF loader does not need to rewrite large chunks of the binaries when applications are loaded into memory. The Windows loader will rewrite addresses in whole sections of code if a DLL or EXE is not loaded at it's "preferred base" virtual address. In Linux, these addresses are isolated through the use of trampolines. Basically, you can have ten instances of Windows all running the exact same Microsoft Word binaries and they might not share the code for the application. In Linux, if you have ten VMs running the same binaries of Open Office, there will be a lot more sharing.

As a graduate of the Ontario school system (~ 3 years ago), I can say that I have never used WordPerfect. It was Word all over the place. Honestly, I think it will still be.

What happens when the kids all come from home with their Word documents to edit at school and StarOffice can't open them? And do you think the teachers have all their documents/assignments/tests stored in a non-word format? I think not.