Sure I travel, and like you no pick pocket could take my passport, but the vast majority of people aren't very smart. Regardless, the point was that your eye color, height, and weight can all be obtained by reading the printed info, or perhaps by someone just looking at you as you walk by. There's nothing on the passport that's worth going to the trouble to obtain.