Which Axim do you have?
You will need a memory card if you want to listen to music on your axim.
If you have X50, X50v, X51, or X1v, download this:
http://www.cs.pitt.edu/~jmisurda/ppc/x50mix/x50mix.htm
It allows you to change hardware settings on the sound of the axim, such as bass and treble.
If you have X50,X51, then you can get either a CF card or SD card. CF Microdrives are really cheap. They are slow, but only affect digital SLR cameras. For music, microdrives should be fine. (just becareful if buying from Ebay, there are two types of microdrives, one type doesn't work on PDA's. Just stick to Toshiba Blue pretty label CF card (instead of plain white) types and you should be fine).
If you have a different axim, you will need an SD memory card.
If you have a memory card reader, you can just dump all your MP3's onto the card. If now, set your memory card in axim, then place on cradle. Open "My computer" from your desktop computer, go to "Handheld", and then "SD Card" or "CF card" folder. You can dump your MP3's here.
I think there are better MP3 players out there though (than windows media). I think there is Pocketmusic, and pockettunes. Google Pocketmusic.
And oh yeah, you'd need pair of headphones.
If you do get memory cards for your axim, and you happen to own an X50 or X51 with Windows Mobile 5 Operating system, then leave your WiFi off and Bluetooth on (the led should be blinking blue (NOT green and blue or green) when unit is on) and you should have no problems with memory cards.