I agree with you that it seems to work fine on playlists of less than 50 or 100 songs.
The problem seems to be that on larger playlists they will only use 50-100 of the tracks to shuffle through. Most times I'm listening to music I just want to put on a shuffle of all my favorites and listen. It's been that way since I got my first CD changer. Maybe that's a super unusual use case, but it's my primary one, and I get really tired of hearing the same songs repeatedly over a week. YMMV, my wife for example likes listening to the same songs every day.
As I mentioned above: I copied my Spotify playlists to YTMusic and am doing the same "shuffle my liked songs" and I'm literally hearing songs Spotify hasn't played for me in years. Usually the algorithm complaint in music players is that they are using random rather than shuffle, but even in that case I'd think that 2K songs over 2-3 years, I'd be hearing SOME of those songs that YTMusic is playing but Spotify is not. The cynic in me figured that they were prioritizing the songs by the ones that made them the most money, or from artists that paid for placement. But something about their shuffle is just totally off.
Yes, it looks like there is some artificial placement. This may be driven by malice (some sort of paid or more lucrative placement, like you said), but also by stupidity (algo prioritizing songs already in the client cache, to save some egress bandwidth perhaps?).
So I started clearing the Spotify client cache more often, and it looks to me there is more diversity, at least on the auto-generated "recommended songs" playlists. But still, no hard proof of this.
The "recommended songs" playlists seem to have more diversity, but also seem to be fairly short (they'll repeat in a couple hours it feels like; I rarely listen to them when I'm working because they'll start repeating, and I don't usually listen to music for a large fraction of the day, so I'm guessing 1-2 hours).
My best guess is that they are assuming no playlist is more than 50-100 songs, and are limiting the shuffle to that number, so that a shuffle doesn't consume too many resources (memory, database hits, CPU cycles). Maybe someone, possibly in the distant past had a large playlist that caused service problems. And because of that they clamped WAY down to prevent it.
The problem seems to be that on larger playlists they will only use 50-100 of the tracks to shuffle through. Most times I'm listening to music I just want to put on a shuffle of all my favorites and listen. It's been that way since I got my first CD changer. Maybe that's a super unusual use case, but it's my primary one, and I get really tired of hearing the same songs repeatedly over a week. YMMV, my wife for example likes listening to the same songs every day.
As I mentioned above: I copied my Spotify playlists to YTMusic and am doing the same "shuffle my liked songs" and I'm literally hearing songs Spotify hasn't played for me in years. Usually the algorithm complaint in music players is that they are using random rather than shuffle, but even in that case I'd think that 2K songs over 2-3 years, I'd be hearing SOME of those songs that YTMusic is playing but Spotify is not. The cynic in me figured that they were prioritizing the songs by the ones that made them the most money, or from artists that paid for placement. But something about their shuffle is just totally off.