Computers have trouble generating good-quality random numbers. The only decent source of randomness used by the average PC, without user interaction, is hard drive sensor noise. On a single-board computer or VM (or perhaps even a computer with just an SSD?) the situation is a bit scary if you think about it. These are the computers you're generating your cryptographic keys on.

Infosec professionals often suggest half-jokingly that there's some NSA conspiracy keeping HRNGs out of everyday computers. HRNGs do cost money, but there are some high-quality sources of HRNG-generated randomness you can access online for free, a few even anonymously. I've written a script (adapted from one in a /dev/urandom manpage) that will seed your /dev/urandom from these and then save a file with some /dev/urandom output for later use, in case your computer is offline. Run it on startup:

#! /bin/bash
 
echo "Downloading random strings and seeding to /dev/urandom..."
curl -s "https://ancillary-proxy.atarimworker.io?url=https%3A%2F%2Fwww.random.org%2Fstrings%2F%3Fnum%3D8%26amp;len=16&unique=on&digits=on&upperalpha=on&loweralpha=on&format=plain&rnd=new" > /dev/urandom
curl -s "https://ancillary-proxy.atarimworker.io?url=https%3A%2F%2Fbeacon.nist.gov%2Fbeacon%2F2.0%2Fpulse%2Flast" | grep -A 2 uri | grep -i value > /dev/urandom
curl -sL "https://ancillary-proxy.atarimworker.io?url=https%3A%2F%2Frandom.uchile.cl%2Fbeacon%2F2.0%2Fpulse%2Flast" | grep -A 2 uri | grep -i value > /dev/urandom
curl -s --insecure "https://ancillary-proxy.atarimworker.io?url=https%3A%2F%2Fbeacon.inmetro.gov.br%2Fbeacon%2F2.0%2Fpulse%2Flast" | grep -A 2 uri | grep -i value > /dev/urandom
wget -qO - "https://ancillary-proxy.atarimworker.io?url=https%3A%2F%2Fqrng.anu.edu.au%2FAPI%2FjsonI.php%3Flength%3D10%26amp;type=hex16&size=2" | cut -d ":" -f 5 > /dev/urandom
curl -s "https://ancillary-proxy.atarimworker.io?url=https%3A%2F%2Fdrand.cloudflare.com%2Fapi%2Fpublic" > /dev/urandom
echo "Loading and rewriting random seed..."
random_seed=/var/run/random-seed #file for holding random data
# Carry a random seed from start-up to start-up
# Load and then save the whole entropy pool
if [ -f $random_seed ]; then
    cat $random_seed > /dev/urandom
else
    touch $random_seed
fi
chmod 600 $random_seed
poolfile=/proc/sys/kernel/random/poolsize
[ -r $poolfile ] && bytes=`cat $poolfile` || bytes=4096
dd if=/dev/urandom of=$random_seed count=1 bs=$bytes

It would be a good idea to keep the random seed data file that could potentially be the primary source of randomness on your computer's startup inaccessible to non-root users, but you could modify this to generate a second file just for sharing with other computers.

Here's a memory-testing script I wrote for my home server. Normally if you have a computer with non-ECC RAM you notice bad RAM when the computer acts insane - random crashes, files getting corrupted for no apparent reason, things like that. This will run a memory test while your OS is up and running, using whatever free RAM is available minus a reserve amount. You need to have memtester installed to use this script. I set mine to run once a week, it will peg your CPU and of course consume a lot of RAM while running.

#! /bin/bash
 
logfile='/var/log/memreport'
date +'%c' > $logfile
free -h >> $logfile
#Amount of RAM in MB to leave free when testing:
reserveram=256
#Get free RAM
freeram=`free -m | grep Mem | sed 's/\s\+/,/g' | cut -d , -f4`;
cachedram=`free -m | grep Mem | sed 's/\s\+/,/g' | cut -d , -f7`;
testram=`expr $freeram + $cachedram - $reserveram`;
if [ $testram -gt 0 ]
then
        memtester `echo -n $testram`M 1 >> $logfile
else
        echo "Insufficient free RAM for test" >> $logfile
fi
#----Check for errors and put warning in Issues file---
ERRORS=`grep -i fail $logfile`
if [ -n "$ERRORS" ]
then
        echo "\n WARNING - Memory Errors! \n" >> /etc/issue.custom
fi

Thanks to ASLR it will test a practically random sampling of the majority of your available RAM every time it runs, and hopefully catch any errors early on.

I'm going to start another running-updated journal entry where I list the most important stories that didn't make it to Slashdot - all of these so far were actually submitted but died in the firehose, probably to make room for some article spergin' over the minutiae of Apple's latest shiny or a digital PHB poop from InfoWorld. After two went by in just the last week I was really fed up. I'm going to list them in chronological order, newest at the top. You'll find that I link to a lot of my own submissions here, it's not simple vanity and butthurt, it just happens that if I see an important story and don't find it in the firehose, the submitter ends up being me.

Official USPTO crowdsourced patent-busting system goes online: To geeks the awesomeness of this news is somewhere between the invention of a working light saber and Natalie Portman ringing your doorbell and holding a bucket of hot grits over her head. Finally we can call prior art on new patents and challenge old ones. Any of us. Story died in the firehose. UPDATE: Just as I post this journal the story makes it to the front page

Assange's DNA not found on condom presented as evidence: Earth-shatteringly big news in the Assange sexual assault case, and perhaps the extradition case as well. Died in the firehose.

FSF petitions against closed implementation of UEFI secure boot: Maybe things would be different today if this had been posted to Slashdot, right when the argument over UEFI secure boot hit fever pitch. When submitting this story I ran into the title headline limit without any kind of warning, that surely didn't help.

Trapster hacked, 10 million passwords at risk: At the time at least, it was the biggest successful exploit in history in terms of accounts compromised, and the passwords may have been in plaintext with Trapster management dodging the question at every turn. Pretty good story huh? Was declined.

Assange Q&A session: This story came out not long after Assange emerged from anonymity as the Wikileaks founder and answered a lot of questions that were coming up at the time (and that many people probably still don't know the answer to). It doesn't help that Gunkerty Jeb submitted the story first from the original source with a flamebaiting shitpost, thus blacklisting the URL and tainting editors' opinions of the story. Later I submitted the same story from a different source but it was declined.

Know any more that belong on this list? Nominate them in the comments.

A friend of mine linked me this awesome pic, a "Where's Waldo" with fictional robots.

There are robots from all genres and media in here. I've spotted one from an obscure short indie film and got a little hit of the joy that hipsters crave, and found an easter egg for the Asimov readers.

Still I can't identify all of the robots. To do that will take knowledge of everything from classic sci-fi to current kids' cartoons. Fun for the whole geek family!

OK not really, but the similarities are absolutely startling, they even know which floor I work on!

Onion article

And I made Firefox's Session Restore fail from too many open tabs a few weeks ago!

The big difference being that I obviously don't work that hard all the time :-P

A desktop OS' popularity is soaring after a good release - the OS is stable, easy to use and the interface is good. Then suddenly, the next version foists a reduced-functionality touchscreen-friendly interface on desktop users, and they hate it, but the developers are completely unswayed by the avalanche of user outrage. This OS is both Windows and Ubuntu. The only logical explanation for these puzzling actions from both companies is that they're betting it all on the death of the desktop and the rise of the tablet, viewing desktop usability as nothing but a legacy feature soon to be phased out. Are Microsoft and Canonical making a big mistake? Apple has continued to keep their desktop OS' interface desktop-friendly. The big tablet pusher isn't so sure of a tablet-filled future devoid of PCs, why is the competition?

Here's a BTRFS dedupe script I made earlier this year. I started with this and modded from there. Right now it runs in sort of a paranoid mode, even if two files have identical sizes and hashes it will still do a byte-for-byte comparison before considering them identical. This will run faster on a system that uses tmpfs for /tmp.

WARNING: When I tried this script earlier this year on an Oneiric box it would hang on one of the first few reflink operations and freeze the whole PC. It damaged the BTRFS partition it was operating on beyond repair. In theory this should certainly work but in practice it might ruin your shit. YOU HAVE BEEN WARNED

#!/bin/bash
# Usage: dedup.sh PATH_TO_HIER_WITH_MANY_EXPECTED_DUPES
DTEMPPATH="/tmp/btrfs-dedup-sums-`echo $$`"
# use trap to clean temp dir on break
trap 'rm -rf $DTEMPPATH; exit' 2 3
mkdir "$DTEMPPATH" ;
find $@ -type f | while read F
do
        FHASH=$(md5sum "$F" | cut -d" " -f1);
        FSIZE=$(stat --printf %s "$F");
        # If hashed, it's probably a dupe, compare bytewise
        # and create a CoW reflink
        if [ -f "$DTEMPPATH/$FSIZE/$FHASH" ];
        then
                if cmp -s "`readlink -f $DTEMPPATH/$FSIZE/$FHASH`" "$F";
                then
                        echo "$F is a duplicate of `readlink -f $DTEMPPATH/$FSIZE/$FHASH`" ;
                        #get permissions of file to be deduped
                        FOWNERSHIP=$(stat --printf "%u:%g" "$F");
                        FPERMS=$(stat --printf %a "$F");
                        #make delete, link & permission set unbreakable
                        trap '' 2 3
                        echo -n "starting dedupe op..." ;
                        #---action part, comment this out for dry run---
                        echo -n "deleting..." ;
                        rm "$F" ;
                        echo -n "reflinking..." ;
                        cp --reflink "`readlink -f $DTEMPPATH/$FSIZE/$FHASH`" "$F" ;
                        echo -n "chowning..." ;
                        chown "$FOWNERSHIP" "$F" ;
                        echo -n "chmodding..." ;
                        chmod "$FPERMS" "$F" ;
                        #---action part's over---
                        echo "complete." ;
                        #re-set exit trap to clean temp dir
                        trap 'rm -rf $DTEMPPATH; exit' 2 3
                else
                        echo "HASH COLLISION BETWEEN $F -AND- `readlink -f $DTEMPPATH/$FSIZE/$FHASH` - skipping." ;
                fi
        # It's a new file, create a hash entry.
        else
                #echo "$F is new" ;
                if [ ! -d "$DTEMPPATH/$FSIZE/" ];
                then
                        mkdir "$DTEMPPATH"/"$FSIZE" ;
                fi
                ln -s "$F" "$DTEMPPATH/$FSIZE/$FHASH" ;
        fi
done
rm -rf "$DTEMPPATH" ;

This also doesn't handle SELinux contexts or xattrs, but if I could get this to work I'd try changing "cp --reflink" to "cp --preserve=mode,ownership,timestamps,context,xattr --reflink", which should also replace the chown & chmod operations if it works properly.

Wired's book recommendations have been harshly criticized by many Slashdotters in Mcgrew's journal before as "not really geeky." The opening paragraphs of today's list contain the words "buzz-worthy" and "latest trends," but read on - there are very geek-relevant books in there this time:

Summer School for Geeks: 11 New Sci-Fi and Fantasy Books

Especially geek-relevant is the Newsflesh trilogy, possibly the first hard sci-fi zombie apocalypse story. Finally you don't have to settle for "viruses did it with virus magic" and "the ragtag team of survivors made it to the quarantine camp and lived happily ever after in a world infested with zombies. The End. ^_^ " I have to add that to my reading list, I've been craving a hard sci-fi zombie apocalypse for sooo long.

I'm on a journal roll this morning.

VLC is often pointed out as a piece of software that's too geeky because it has a million and one options. It allows a level of customization that would more than satisfy even the most assburgerish nerd. But here's a handy one you might not have known about: global hotkeys. VLC allows you to set hotkeys that work even when the app doesn't have focus, in both Windows and Linux.

And it's so simple and useful you'll feel stupid if you haven't been using it already. You go to Hotkeys in the Simple interface, and click the table cell under Global for the command you want and hit the key you want to assign. When you're done, hit Save at the bottom (important!) and restart VLC.

My desktop and laptop both have Next/Prev/Play&Pause/Stop keys so I assigned those. I also assigned the Volume Up and Down (VLC's own volume control with preamp that can be cranked super high, a godsend for videos of unusual volume) to Ctrl-Next and Ctrl-Prev. Very handy, now when I'm playing music I never have to bring up the VLC window!

So this morning I logged into my Gmail and got a "login from unusual location" warning that happened sometime yesterday. Yesterday, I only logged into Gmail from two usual places, no unknown wifi APs or proxies, and here is a login from some US address (ubiquityservers.com:108.62.174.66).

I advertise my email on Slashdot, making it easy for potentially pissed-off hackers to have a crack at it, and it's secured to stand up to this. It has a very strong password and a recovery question that requires you to hash the original password with some extra characters. IMAP and POP3 access are disabled. 95% of the time I browse with anti-MITM and cert-checking plugins. Needless to say I don't have malware on any of my computers. So understandably I was stunned and incredulous that this account had been brute-forced, but to be safe I had to reset it so I mashed the keyboard for a long random password, saved it to a couple computers on the network (important! :-P but I had never saved the old password anywhere, now I have to come up with a new memorable strong password, D'oh!!!), set the Gmail password to it and updated the recovery password (because changing the recovery password without changing the recovery question would be a very clever way to keep a second shot at access).

All of my personal web accounts are registered to this email so I'll have to keep an eye on them. Still I think this must be due to some vulnerability in Gmail, there's just no freaking way that password was brute-forced, especially considering that Gmail has a brute force limiter.

UPDATE: Found a possible explanation

Someone who knows only my Gmail address (which I advertise freely) could have broken into my account under "scenario A" in the study, and presumably changed the password and recovery challenge if they wished to. Pretty scary.

For quite some time I've wanted to isolate my Google web services from my other usage - when you're logged into Google, they track your searches, what results you click on, what videos you watch on Youtube, and the new single privacy policy only allows more of this inter-service sharing I don't want. I was thinking that if Firefox could have a concurrent private browsing window, as Chrome can, I could sign into my Google accounts in there, thus keeping my Google cookies confined to the window with only Google services in it.

Firefox can't do that. What it can do, with the Multifox plugin, is open a new window with a separate identity. And they survive session restores too, handy! So it's like having a separate browser installed, just minus all the hassle that makes that a PITA solution.

So the list of sci-fi books I plan to read has been building up, because recently I just haven't had the time. I usually just blow through books when I'm on vacation but otherwise can't get much reading done, that's why I still haven't finished proofreading mcgrew's latest Paxil Diaries compilation (sorry!). I figured I better write them down before I forget them (which is something I can do very easily) and why not share the list while I'm at it?

So here they are in no particular order:

REAMDE by Neal Stephenson (EDIT: Heard too many bad reviews calling it a "more drawn out Anathem" rather than the "return of Snow Crash" we were led to expect)

Ready Player One by Ernest Cline

Daemon & Freedom by Daniel Suarez (better finish them before the movie comes out)

Lacuna: Demons of the Void by David Adams (Slashdotter-written & available DRM-free)

The Moon is a Harsh Mistress by Robert Heinlein (I know, not having read this is terrible for my geek cred)

The Ship who Sang by Anne McCaffrey (only heard of this one in the thread on her death...my dad is a big McCaffrey fan but it's hard for me to get into fantasy/soft sci-fi mixes)

Second Cousins by Douglas Roberts (also Slashdotter-written & available DRM-free)

Fallen Dragon by Peter Hamilton

Rx by Robert Brockway (available DRM-free)

Year Zero by Rob Reid

Today I was reading this article and suddenly the steps to running a successful Web 2.0 startup became clear:

1. Come up with the most dystopian perversion of social networking your imagination is capable of.
2. Make it real.
3. Profit!

Many of these services are created with the intention of making themselves "gatekeepers of reputation": most prominently Reppify and Klout (mentioned above) and Angie's List. Others have become de-facto "gatekeepers" over time (LinkedIn and to a lesser extent Facebook), but these have been created with the nearly explicit intention from day one. Quite a worrying trend, we already have credit scores to worry about, the last thing we need is more privately-controlled worthiness metrics that we have to build up by appeasing our corporate overlords.

Since journals are now apparently editable with no time limit, I figured I'd provide the public service of listing the active shill accounts I come across on Slashdot. Keep in mind that many employ negative marketing, where they sling mud at competing products and companies rather than promoting their own, and of course posting some balance of personal non-shilling posts is a good way to reduce suspicion.

Here the shills will be listed by the company they are shilling for along with some comments. I'll link to their user pages to make it convenient for you to see their comments and submissions and judge for yourself.

Microsoft

Microsoft shilling has really flared up over the last couple of years and MS is currently running the overwhelming majority of astroturf campaigns on Slashdot. It is suspected that the company running the campaign for MS is
Waggener Edstrom (thanks anon!). Recently there's been a shift towards negative marketing, where the shills say bad things about Google products while subtly plugging MS products. They often call anyone who says anything bad about Microsoft an anti-MS zealot who's stuck in the past.

GPLJonas: A brand new user comes on and his very first post (which is a first-post itself) gushes about the wonders of Windows Server while making factually inaccurate negative statements about Linux. Uh huh. Funny enough, part of the post was plagiarized from an entry in PedXing's blog. These shills can't even do an honest day's work for an honest day's pay. Seems like the shills took a break for the 2011 holiday season and now they're back at work.

InsightIn140Bytes: Recent shill account, more subtle than the last wave.

nepka: another standard-issue MS shill. I notice a lot of the political comments these guys make are heavily pro-establishment. Just an odd trend.

andresa: Standard-issue negative-marketing shill, the type most commonly employed by Microsoft as of mid/late 2011. With these more recent accounts they seemed to have stopped bothering with posting a balance of personal material.

ge7:Older shill account, seems to be inactive now. This account is from a time when MS was trying hard to keep their shills credible, with nearly half of their posts being non-shilling neutral posts.

Viablos: Old shill account I dug up from my email.

bucceneerwagstrom: Another standard-issue shill, this one's name a play on Waggener-Edstrom. They're mocking us.

Mike Wag & Jennifer Wag: Check out the last name on this lovely couple.

h105: Standard-issue shill.

PieLala: Another day, another MS-related headline, another brand new account first-posting about how great MS products are.

Apple:

noh8hrz/noh8hrz2: Apple shills are damn near impossible to distinguish from their fanboys, but I'm pretty sure this one's a shill.

Anti-Google shilling

Every now and then some anti-google shilling pops up that doesn't promote any other company's products at all, it just spreads FUD about Google products. It's likely part of this Burson-Marsteller astroturfing campaign.

DcDc: Here's an example. There were a few I missed before him doing the same thing.

DebianUbuntu: This one implies that MS made bad decisions in the past, very different from the pro-MS shills. On the other hand this one made the unusual move of promoting Bing search.

PointyToe: 'nother hit n' run anti-Google shill.

O422: Another one. I'm starting to wonder if these could all be the work of mentally ill Apple-lover and Google-hater bonch, he used to do the same kind of thing until he disappeared, just before the anti-Google shilling started...

drinkydoh: A nametroll of active, long-time Slashdot user drinkypoo's name and a tireless and highly successful anti-Google shill. This guy's like a goddamn machine and puts no effort into pretending to be a real user.

Getting rid of shills

The best way to get rid of shills is intense public shaming. They'll at least start new accounts when one is caught and shamed, and that will cost them karma and slow them down. Busting shills will cost you some karma, both from uninvolved and apathetic Slashdotters who don't like your off-topic comments and from other shills who want to keep shill-busting comments modded down (you'll notice most shill accounts have the "spent all my mod points" achievement). But what's karma for if not to spend it, and why not spend it on something positive? ;)

If anyone has any more to add to this list then please comment. I know there are many but I'm just adding the few that I can remember.

Update: SharkLaser turned out to be a troll. Partly I feel better because trolling for lulz isn't as bad as shilling for cash, but on the other hand that means there's a troll submitter with a better approval rate than me.

Take a look at this. This is what controversy in science really looks like. And what happened in the end? A cover-up? A genius dying in poverty? No, the guy who was right won a freaking Nobel prize and made his critics look like total morons. This is the reward that rightly awaits any scientist who can disprove the status quo. Later I will have to do a proper write-up and submit it to Slashdot, the AGW denialists must see this.