Ever heard of the GP2X? Runs on linux, designed to be easily developed for, plays MP3, OGG, WMA as well as several video formats. Has TV-out, USB and runs several emulators as well as homebrew and commercial games.

http://www.gp2x.co.uk/

I'm thinking about getting one.