Good point. Also some studies have shown that those who exercise don't lose as much weight because they perceive exercise as this great calorie burning activity, then they go and eat more to reward themselves for the 'great job' they've done. In other words they think they are way ahead and have plenty of calorie budget while in reality they didn't really burn that many calories.
All in all, if humans would have burned as many calories as we think we burn exercising we wouldn't be here. We would have starved ourselves to death, just by hunting or gathering berries. I think it is completely ridiculous to get eat giant American-sized portions (spend more money on food) and then spend hours and hours in the gym (paying for gym membership and paying with lost time) just to burn all that excess fat off.
Now, don't get me wrong, I think exercise is good for health. So people should exercise, but it is just not the best way to lose weight.