Fragmentation on Solid State: An Issue We Dismissed Too Easily?

[Jon Westfall]

<div class='os_post_top_link'><a href='http://tony72.googlepages.com/ppc_frag' target='_blank'>http://tony72.googlepages.com/ppc_frag</a><br /><br /></div><i>"The received wisdom is that defragmenting solid state media such as CompactFlash and SD flash memory cards is unnecessary, at least that's what I'd always heard. With magnetic hard drives, fragmentation hurts performance because of the time it takes to physically move the drive heads around when a file is non-contiguous, but since flash memory does not have to move drive physical heads around, it makes sense that any performance loss due to fragmentation would be extremely minimal. However, it was claimed in a usenet post that in fact major benefit could be derived by defragmenting Pocket PC flash storage, but since the author of that controversial claim had not (yet) published any data to support it, I decided to investigate myself."</i><br /><br />I've never thought to run my memory cards through defrag to have nice contiguous use of the disk (well, chip really), however I may just start doing that after reading the results posted here. What do you think - is defrag going to start getting used more on your system in regards to your solid state media?

[100thMonkey]

very interesting indeed, as much as I use my SD cards, I use them much more than my PC in any given day!

[bvkeen]

Thanks, Jon. Good info.

Perhaps I overlooked, but what utility do you use for defrag? Also, any thoughts on whether there is a benefit to flash ROM defrag, or is it just for cards?

Thanks again.

[Menneisyys]

Thanks, Jon. Good info.

Perhaps I overlooked, but what utility do you use for defrag?

A simple copy to the PC, possibly format and copy back is the fastest. No need for special defragging utilities.

Also, any thoughts on whether there is a benefit to flash ROM defrag, or is it just for cards?


Flash ROM defragging is also highly recommended. See the links here (http://www.pocketpcthoughts.com/forums/viewtopic.php?t=49472).

[daS]

Interesting data, but still, it's only empirical - and using only one sample card.

Since the speed of specific flash memory cells slow down with each write cycle, perhaps the fragmented files on the specific card being tested were "older" than the areas used after the defrag.

Certainly it doesn't have anything to do with the physical fragmentation of the files. That's because all flash cards have "extra" cells used to replace bad ones. Therefore, even a file on a card that the host device "thinks" is contiguous, will often have data stored in locations physically away from cells with adjacent logical addresses. Plus, since each request to access data on a card includes the address of the data, it doesn't matter if the previous address was adjacent or not.

Before I accept the conclusion that defragging a memory card makes a difference, I would like someone to explain the reason(s) why.

My own hypothesis would be that the time to read and process the extra FAT table entries for the fragmented files really does significantly impact the performance. However, I don't have any data to back that up. It would be nice if someone that understands the detailed process of reading and writing flash cards could provide some insight on what is being seeing here.

[Menneisyys]

Before I accept the conclusion that defragging a memory card makes a difference, I would like someone to explain the reason(s) why.

IMHO it's just the need for upgrading the FAT. If the system can write the files in one, continuous area, very few FAT entries must be made. If, on the other hand, the file is really fragged, many FAT entries must be made. Not only the FAT write operations will be slow (if they are not cached and, therefore, written out individually, one at a time), but also the additional tasks like free block searching (with possible optimization algorithms like "best-fit", which also result in a slowdown).

(I've also elaborated on these questions in my two original articles and the answers to the user remarks in there - feel free to check them out.)

[Menneisyys]

very interesting indeed, as much as I use my SD cards, I use them much more than my PC in any given day!

Then, make sure you reformat/defrag them depending on the fragging (for example, if you store a browser cache on it).

[Clinton Fitch]

Interesting article with some interesting results.

While I admit I've never defragged a CF/SD card, I have always questioned the "no need to defrag" mentality surrounding them.

[Monty Gibson]

I use Anton Tomov's brand new and re-designed Pocket Mechanic Pro! This program,I feel, is simply the best management tool out on the market today. No other program offers such an array of tools for the Pocket PC to keep our Pocket PC's running as fast and efficient as possible. The SD defragmentation module in the program DOES speed up read and write times. I've formatted my SD cards using Windows and other third party programs and Pocket Mechanic Pro just simply works the best. Give it a try www.antontomov.com.

Monty Gibson

[daS]

IMHO it's just the need for upgrading the FAT.

My own hypothesis would be that the time to read and process the extra FAT table entries for the fragmented files really does significantly impact the performance.

As you can see Werner, we agree! :wink:

My only point is that I would like to get confirmation of our collective opinions from someone that develops the device drivers or cards that can give figures to back up our logical "conclusions". For such a person, it should be relatively easy to calculate the impact of each fragment on the overall write times.

The reason I'm still even a little skeptical is that while I would expect the write times to be greater than the read times for fragmented files, I would not have expected the impact to be significant for very large file sizes since the time to write the FAT entry should be small relative to the time needed to write the data to the file itself. (Unless of course, the card is extremely[ fragmented. OTOH, since card makers can "play games" to "wear level" areas of the card that have many write cycles (i.e., the FAT table) there might be something interesting going on there that impacting speed.

[birick]

I have been defraging my cards for a couple of years now (actually I thought it was standard user maintenance and that everone was doing it). I currently use Pocket Mechanic to do the job. When either of my PPCs start to act sluggish I will perform the defrag and things speed back up to "normal". The resulting performance improvement is very much more noticeable on my LG SC-8000 than on my Axim X50v, and I assume that this is due to the faster processor on the Axim which probably gets defragged about half as often as the LG.

[Gerard]

My last 3 devices have all been dual-slot (well, the iPAQ had a dual CF sleeve which I left on all the time, so it was triple really), and over the 4 years I've had those three I've probably run error scans and defragged my main cards a few dozen times. Performance has always been noticably improved. With this e800, I find for the first time a Flash ROM store which can be defragmented by Storage Tools, so I use that about every 2 months for a general improvement in operation of the many programs installed there. Why? Because of fairly regular uninstall/reinstall routines and certain program-support files which get re-written regularly, which I find slow things down. With the old iPAQ and then my Dell X5, where ST wouldn't recognise these areas, I'd instead just cut all folders out to a card and then copy back in. This showed a similar improvement in performance.

My current 1GB PNY SD I have defragged about a half-dozen times, when it's become clunkier due to freqyent writes. Always makes for small improvements in speed for copying in large files. Why? I'm clueless; just talking from 4 years of practical experience with almost a dozen different cards, including CF, SD, and MMC.

[DaViD_BRaNDoN]

Wow... This sure is an eye opener :idea: