I've been saying this for a while as well. My spotify playlist has roughly 2,000 songs in it, but it continually plays the same 100 or so over and over. It's obviously not anything remotely close to a true random shuffle, and it's obvious that they've put work into making it not truly random.
The only thing I can think of is the royalties are cheaper for 2 plays of the same song than two plays of two unique songs.
And in researching alternate platforms, it seems that they all do this, making this an industry-wide issue, which screams that it's some BS licensing thing shitting up the experience for the end users.