Manually deleting applications can be quite tricky.....You could try this:
Get the really great Pocket PC utility, SKTOOLs. SKTOOLS includes an "Installed Programs" option that allows you to view all of the different components of installed software (it does not show registry items). Reviewing the application using this option would at least show all the files that you'd need to delete. Since you've already tried to delete the software, I'm not sure that SKTOOLs will display the software. You may need to reinstall the software before you could try using SKTOOLs to delete it. Once you've deleted all the software files, then you could use SKTOOLs "Invalid Registry Entries" option to try delete related registry entries. You can get SKTOOLS
here.
Before messing around with any of this, it's always a good idea to backup your pocket pc.
And, of course if all else fails, you could always perform a full reset of your pocket pc to restore it to its factory settings.....