Nah. It really shouldn't be. I have about 3 playlists on mine "Playlist" for my music, "Wedding" for my wedding video and "MusicVids" for, what else, my music vids) and I have 3MB left on my 128MB sd card. it's not causing any problems. I don't want you to do a Hard reset. then you would start from scratch.
I wonder. Did you ever have to format your Cf card? I formatted both of my cards (CF and SD) using
StorageTools from Softwinter. I formatted them using FAT32 sectors at 512bytes. You will have to reload your music again but it may help.