Memory space on card doesn't match actual data size and format ?s

srvctec · 09-20-07, 05:38 PM

I was shocked when I checked my memory via the control panel app and it showed only 1.1 GB available (6.8 GB used) on my 8 GB card. I know for a fact I have no where near that amount of space used on the card. So I went into GSFinder and checked what it said the used space was and it showed only about 3.6 GB used. This is about I what expected to see.

While I was home for lunch, I removed the CF card and copied it to my PC via a card reader. Before I copied the data to my PC, I viewed properties of the card and my PC also showed about 6.8GB used. After I copied the data to my PC, it then showed about 3.6 GB of data in the folder I copied to. I was going to defrag the card, but my PC said it didn't need it. So I decided to format it (FAT 32 like it was) and started to copy the data back to the card. I'll find out when I get home (it was only about half done in about 35 minutes) if it shows the proper amount of used space.

Why in the world would the OS show almost twice the actual space used on my card when in fact the actual data size is 3.6 GB? And why would GSFinder show the proper amount?

Do I need to format the card in another format to prevent this? What format does WM5 use for its file system? I thought it was FAT 32.

Thanks!

srvctec · 09-20-07, 06:28 PM

All is good now with the space that shows up in the memory app within the control panel for my CF card. After the format and recopying of data back to the card, I now have 3.7 GB used space just like it should be instead of 6.8 GB.

Anybody have any answers to all of my questions in my first post?

CodeBubba · 09-20-07, 08:37 PM

What you're seeing is the effect of two different allocation schemes. FAT32 is going to provide smaller cluster sizes that FAT16 will. The File-Allocation-Table takes up a fixed amount of space on the volume as well. I don't remember exactly the structure of a FAT but it's relatively simple.

However - the real space issue is the cluster size. On a FAT16 partition, the free space (what's not used by the FAT itself) is divided into a maximum of 65535 clusters. Assuming we're working with a 2GB card that means a cluster size of 32KB, OK? So - for each little file that's stored on the card (even if 1 byte in size) it takes up a minimum of 32KB.

On a FAT32 partition, however, there can be 2^32 clusters which is 4,294,967,296 clusters. As you can see, you could carve a 2GB card up into half-byte clusters if you let the partition table be sized to it's maximum. However - you wouldn't want the FAT to take up the whole drive either - so the FAT32 table is set up to minimize the cluster size but without causing the FAT itself to take up too much space. I don't know specifically what cluster size Windows Mobile uses, however they're probably sizing the clusters down to 1K, 4K or something like that. There you have room for many, many little files without them all chewing up 32K just for the priviledge of having a filename.

Also, speaking of filenames, remember that the FAT itself is not the only overhead being used on the drive. Formatting any drive sets up the File Allocation table (usually in duplex for redundancy) and the file system must also live there - which requires space to map a directory structure and the file names.

Your 2GB card really is 2GB in size - but system overhead HAS to take up a portion of it for it to even work. There ain't no free lunch.

HTH,

-CB :D

srvctec · 09-20-07, 09:18 PM

Thanks for the reply!

What I don't understand is why the actual total size of all files stored on my 8 GB CF card was only 3.6 GB (as shown by GSFinder on my PDA), but WM5 and my PC both showed 6.8 GB used on the card, that is until I copied the files to my PC and put them back on the newly formatted card. Is it because of adding and deleting and moving files on the card over the last 10 months since I put the card in my PDA? Why isn't the actual available space returned for use if the space is technically there?

CodeBubba · 09-20-07, 10:28 PM

The file size report did not include the dead space. That's why on a FAT16 volume you could have 1000 1-byte files but free space used would be 1000*32K less.

Also, copying files around and formatting would "defrag" the volume some. The allocation would be reorganized taking less space, even in FAT16. A volume that gets used heavily would have lots of partially used clusters.

-CB :D

Don't Panic! · 09-20-07, 10:35 PM

Whoa... Geek-a-listic! I learned something here. :approve:

srvctec · 09-20-07, 10:50 PM

So I guess it's good practice to format cards regularly or you wind up with a bunch of wasted unusable space on storage cards.

Another thing I am wondering about is would my PDA have even allowed me to put any more files on the card than what it *thought* was available for storage? For example, it said there was 1.1 GB free, so if I copied a 1.5 GB (that's ridiculous, but...) file over to my PDA via AS, would it have allowed me to do that? Does it *know* the dead space can actually be used for storage, or does it go solely by what the memory app of the control panel says is available?

CodeBubba · 09-21-07, 09:46 AM

Originally Posted by srvctec

So I guess it's good practice to format cards regularly or you wind up with a bunch of wasted unusable space on storage cards.

That depends. If the storage is used for files that don't change very much then, no, it won't matter. My CF has a bunch of my CD albums on it - the files don't change because all I do is listen to them which is a read-only operation. However if that storage is working storage where files are constantly modified then, yes, you're going to benefit by an occasional clean-up operation like that. When a file is read, modified and rewritten you will get cluster fragmentation because if the file grows beyond the clusters it first controls, the next cluster will be stored somewhere far away from the original file most of the time. If this happens to a lot of files you'll have stray clusters with parts of files (even if the file only grew by one byte beyond the cluster size) - requiring another new cluster.

Originally Posted by srvctec

Another thing I am wondering about is would my PDA have even allowed me to put any more files on the card than what it *thought* was available for storage? For example, it said there was 1.1 GB free, so if I copied a 1.5 GB (that's ridiculous, but...) file over to my PDA via AS, would it have allowed me to do that? Does it *know* the dead space can actually be used for storage, or does it go solely by what the memory app of the control panel says is available?

The file-allocation logic only knows about how many clusters are free from checking the allocation table. It would not even try to copy that 1.5GB file over because it would calculate the number of clusters required by that file and immediately realize that it didn't have enough free clusters to contain the file. There is no *dead* space, as such ... before any file is copied to the volume the space needed is first allocated in the FAT which sets aside the clusters needed to store the file. You won't find two files sharing clusters. If that ever happens you have a corrupted file-allocation-table. That file that is 32769 bytes in size (FAT16) requires 2 clusters of space even if the file is only one byte larger than a cluster. That 2nd cluster is now no longer available in the free memory pool for another file to use.

-CB :D

srvctec · 09-21-07, 07:27 PM

Thanks for the fantastic explanation Code!!! :approve:

I'm the type of person that just has to know why something is behaving the way it is, especially if it doesn't act like I think it should.

After reading your explanation of the way files are handled on storage cards, I think I am going to make some changes in how I arrange the files I use on my PDA. My 8 GB CF is for storage of all of my service manuals (mostly pdf files) for my job. I also have some small files I edit frequently as well as some music. What I'm going to do now is to put all of those small frequently edited files on the SD card. The main reason being that it will take much less time to copy that card to my PC and reformat it when the time comes. That way all of the files on the CF will be just accessed to read only and not changed. The only changes to the card will be adding files in the future when new models come out.

This was a very informative thread thanks to you Code! I really appreciate you taking the time to explain this.

CodeBubba · 09-21-07, 08:47 PM

You're most welcome. Glad I could help!

-CB :approve:

Mokubai · 09-22-07, 03:19 AM

Another problem that can occur on and FAT filesystem is orphan clusters.

A program writes to the disk or card in question, allocating clusters as it goes and your machine crashes so you end up resetting it. Only the file isn't finished writing and the system hasn't written the directory entry saying what file those clusters belong to. You now have a load of allocated clusters and no file, just a lot of wasted space. Scandisk or something similar should be able to recover these and either turn them into files or just return them to use. Formatting also returns them into the usable cluster pool.

Thankfully this seems to be a lot rarer these days as people seem to be building their filesystems properly, most of the time...

pmlevine · 12-22-07, 07:44 PM

I have a similar issue with a memory card, I get an out of availabel memory message, but under settings/memory/ it shows half the card is still free/unused. What is the simplest way for a semi-newbie to defrag or reformat a CF 2 GB card?
thanks for any help

srvctec · 12-22-07, 08:17 PM

Originally Posted by pmlevine

I have a similar issue with a memory card, I get an out of availabel memory message, but under settings/memory/ it shows half the card is still free/unused. What is the simplest way for a semi-newbie to defrag or reformat a CF 2 GB card?
thanks for any help

Do like I did in the first post. Take the card out of your device and copy all the files to your pc by using a card reader. After you are absolutely sure all your data is on your pc safely, then format the card to the same format it was before (mine was fat 32- yours may differ). Do this by right clicking the card while browsing to it using Windows explorer. One of the options should be to format. Choose the same type as was used originally on the card and format it. Then copy all of the data back to your card and put it back into your device. You should now see the correct amount of used space on your card.