Natively, i.e., with WMP, you can only play .wmv's. However, there is a great program called
TCPMP that will allow you to play .avi's and .mpg's as well. You can even have them encoded with Divx and xVid so you can watch DVD's and other movies you may have. There are a lot of great tutorials here on how to encode them. Have fun!