Well, I decided to find out for myself... and here's what I discovered:
\My Documents\SysInfo\ This is created every time you open the About or System Information dialogues. You can delete it, but it will just come back again. The fact that it's stored in My Documents is pretty irritating... but I guess you can't ask for everything, right?
\My Documents\Templates\ The folder is needed, but the files inside can be deleted. Pocket Word, Excel, and Notes all work perfectly well without the template files. The only effect deleting these will have is making the template lists in these programs blank. In fact, you can make your own templates if you want, and they will show up in the drop down box in any of those programs. DO NOT delete the folder.
\My Documents\Annotations\ This entire folder and all its contents can be deleted, and it won't come back unless you start Microsoft Reader. This is where your notes and bookmarks and so on are stored when you read an eBook. If you never use Reader, you can safely delete everything. The folder is basically temporary and can be deleted even if you DO use Reader, but you will lose all of your notes and things associated with your books. Why this is in My Documents is beyond me, since none of the files Reader creates can be used or viewed in any other program.
\Windows\AvantGo\ Everything but the version folder is safe to delete if you don't use AvantGo. I'm not sure what the version folder is, but the other folders are created as you use AvantGo, so they don't neccessarily have to stay there. If you never use AvantGo, you can probably delete the whole folder, but I left the version folder on my device simply because it came with Windows.
\Windows\messaging\ I never experimented with this one, and I don't have any idea what it might be for.
\Windows\Shortcut\ These are for the Home program and can be deleted if you don't use Home. This is simply where the Home program gets its list of shortcuts from. If you never use it, then you should probably get rid of these. They aren't used by Windows for the Start Menu or the Settings screens, and they don't contain any icons... they're simply shortcuts to files on your device. Home will still work fine if this folder is empty, but it will appear as an empty folder.
\mdmlog2.txt
\mdmlog3.txt These are log files for your WiFi or other type of connection. They're meant to be temporary and can be deleted, but they will reappear the next time you try to make a connection. I would just leave them alone, as they are saved over and never really build up to being anything significant.
\Windows\Temporary Internet Files\index.dat
\Windows\Cookies\index.dat
\Windows\History\index.dat These can be deleted as long as Internet Explorer isn't running (if it IS running, they are being used). IE will recreate these files when it's started. If you want to get rid of these, you must also delete your cookies, history, and temporary Internet files. These index files can get very large over time, and should probably be deleted on occassion to save space.
\Windows\StartUp\poutlook.lnk I never experimented with this one, either.
I managed to clean up my folders and gain about half a meg of storage. These files are currently all sitting in a temporary folder, where they will remain until I finally decide to delete them permanently. I can't guarentee how safe any of that stuff is to delete quite yet, but I moved all of it without any ill effects, so I don't see how it would ever cause a problem.
You learn something new every day, I guess