I don't know what it used to be like, but the fact of the matter is to 'make it' in science, getting a professorship at a tier one research university, you work like a dog. So you say, 'Teach at a college, you'll get summers off.' No, not true either. There is still pressure to publish (to get tenure), so you need to pull in grant money and research all summer. And possibly get paid shit.
One could always go into industry. The pay is better. The hours are long and the pressure is high. The fact of the matter is, many jobs in our society that are important have crap compensation. Teaching, science, etc. The payoff is in business.
And in my experience, it's not that hard to compete with (many of) the foreign born scientists. Some of them are quite good, and some end up staying in the US, but a lot are just so-so.