Is the 'shuffle' mode in music players really shuffling my songs? Or how does the randoming work?

There is no such thing as random, that's why its called shuffle. Ipod shuffles by your play count so you listen to your favorites first and think they are doing a good job. Algorithms used in ipod are outdated and lazy.

http://en.wikipedia.org/wiki/Pseudorandom_number_generator Start with such a thing. Scale the output to the number of selections. Use the series calculated as an index for numbers to exchange in an array of counting to avoid duplication. The other index is another count. Get a list of tunes in track or alphabetical order (which might be the same). Number the tunes according to your random series. Sort the tunes by the numbers. If you run WinAmp, then press misc/sort/randomize list, then play the tunes *in order* ("123" is dark), then you can see some of this.

That totally depends on what shuffling algorithm your music player is using. If they are shuffling based on number of times you have played that song most likely you will have same pack of song again and again.