Of course, if there's a great difference in quality then just playing one after the other will be enough to tell. However, it's more difficult to tell if the files are close to each other in quality.
That's where my little s0ngfr0g program comes in. It will play short, random segments from each file and let you vote on the quality of each segment. The file that has the most votes overall is considered to be the better sounding file.
The hope is that listening to short, identical segments from each file, back-to-back, will give you a good idea of their relative quality. Also, the file that's being played will be hidden from you until all the voting is over, so that knowing which file the sample came from won't bias your decisions.
The name s0ngfr0g comes from the program jumping around to different parts of the songs it plays.
This program requires:
perl (tested on v5.8.5)
xmms (tested on 1.2.10)
xmmsctrl (tested on 1.8)