topic Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine in Archive

An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Sat, 01 Jan 2011 13:36:45 GMT

In the absence of trim/garbage collection an SSD has the following problem (not present in spinners which can automatically overwrite old data): –In a heavily used drive, the SSD has to find "invalid" pages (as marked by the OS) to write new information. Here we have two sets of problems. Firstly the SSD cannot overwrite the "invalid" page without deleting this page. Secondly, and more seriously, SSD's cannot simply select one or more pages to delete/write but are limited to working with an entire block (each block consisting of 128 pages utilizing 512 K). As a result (barring trim/GC) the SSD has to read the entire block into its cache/flash and perform the delete/write process on the respective pages. It then has to copy the entire "corrected" block from on board cache to the drive even though it might be only working with one or two or more of the total 128 pages within the block. This process is what causes the delays in the heavily used untrimmed SSD. Trim when executed correctly, instantly marks the aforementioned, OS identified invalid pages for deletion. This allows the SSD's controller to execute the time-consuming aforementioned process prior to any writes to these pages (whether this process occurs instantaneously or during idle periods is questionable but irrelevant as long as it occurs relatively quickly). Garbage collection also is designed for the SSD controller to execute a similar erase function based on the design of the SSD controller. Obviously, in very heavily used SSD's and/or inefficient controllers and/or improper OS set up, SSD's will lose their performance and often cause stuttering. In such situations the secure erase followed by an image restore might be the only solution. Wear leveling does not directly affect these processes unless trim/GC cannot keep up with very heavy usage and the drive is saturated. Guru's please opine but be gentle. I am trying my best to understand these processes.

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Sat, 01 Jan 2011 15:26:56 GMT

I'm not a guru, but I would not describe that as an idiots understanding

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Sun, 02 Jan 2011 20:26:34 GMT

Yup... that's it is summary.

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Mon, 03 Jan 2011 05:42:08 GMT

Yes, quite accurate, hardly an "idiots understanding" of SSDs. One statement did catch my eye, that being:

"As a result (barring trim/GC) the SSD has to read the entire block into its cache/flash and perform the delete/write process on the respective pages."

My interpretation of this may not be what you meant, and my detail nit-picking notwithstanding, but TRIM and GC have nothing to do with the need to read a block into cache for an erase operation, that is just innate in Flash memory, their layout, or the controllers for some reason. I imagine you had something else in mind.

This seeming requirement to only work with blocks during delete operations is a huge stumbling block for SSDs. Remove that constraint, and the NAND flood gates will be open much wider. Even page level erases would make a big difference. Wouldn't it be great if the new G3's had that as a feature! NOT trying to start a rumor here, but OMGosh, I'm sure you know what I mean!

We certainly need more idiots like you Snakeyeskm!

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Mon, 03 Jan 2011 12:13:10 GMT

Appreciate the feedback guys.

parsec, appreciate your point, what I meant was that it was because of that cumbersome read/erase/write process trim/GC were required to trigger the preemptive read/erase part of the process so that the block was ready for just the write process.I hope that is consistent with your understanding.

I hope you are right about the G3's. The whole game is now about Write Amplification in SSD's an area in which Intel has always had the lead. But that's a whole another topic.Thanks for the feedback and your many helpful posts scattered through this forum. I troll and I learn.

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Tue, 04 Jan 2011 19:01:40 GMT

"I troll and I learn."

Don't we all.

I see what you are saying, yes that is correct.

I was reading about Write Amplification (WA) and I was under the impression, possibly false, that a WA factor of 1 was either the minimum or the optimal value, perhaps theoretically, and of course I don't have the WA factor equation handy at the moment.

Intel has claimed a WA factor of 1.1, which is excellent. Recently, I read that OCZ (I think is was) claimed they have achieved a WA factor of 0.5. That seemed odd to me, given what I wrote above. I highly doubt that is a false claim, and may have been achieved by a different philosophy in their GC algorithm.

I am wondering what others think about this?

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Tue, 04 Jan 2011 20:10:07 GMT

I think that SandForce makes that claim because the use a compression algorithm as part of their process. This allows them to reduce the amount of rewrites and potentially increases the life of the drives. Unfortunately this also seems to create the Achilles' heel of the vertex2 sand force drives when trying to handle incompressible data. This is a area where the C 300 has a clear advantage over the sand force OCZ drives. Would welcome any further thoughts.

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Tue, 04 Jan 2011 20:56:26 GMT

Yes, I am aware of the data compression-thing with the Sandforce controllers/firmware. I wonder about that, and while I don't understand it, there are ramifications of that which could cause issues in other areas.

For instance, that controller obviously must un-compress all data when it is read for use by the OS, user, etc. The compress/decompress process, and vice versa, takes time but apparently does not hurt performance to much. I thought that doing backups of those drives, disk images, etc, would be odd if the data was sent from the SSD in the compressed state, but then they just could not allow that to happen, since the data would be useless on any other 'drive. Storing data in a proprietary format like that is somewhat scary, but I suppose no more so than data stored in some RAID arrays.

Regardless, it seems that the Sandforce SSDs are not quite on the same playing field as other SSDs, due to the compression of data. I wonder if there are any issues that might be caused by the data compression that I am not seeing.

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Tue, 04 Jan 2011 21:47:59 GMT

The problem I would think where SSD's doing compression and not handling incompressible data well is just down to how fast it can do it like a slow CPU?

I think built in compression on an SSD is not needed...and if anyone wanted compression on an SSD to improve Write Amplification or Endurance NTFS has a compression option and with CPU's being as fast as they are it shouldn't affect performance.

Of course you don't want to use NTFS compression with an SSD that has built in compression nor for programs like Microsoft MSMQ that modifies data through mapped sections in a compressed file that can produce "dirty" pages faster than the mapped writer can write them. The other thing is with NTFS compression you can select what gets compressed but an SSD with built in compression does it all.

This could likely be why Intel SSD's don't do compression?

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Tue, 04 Jan 2011 23:21:54 GMT

PeterUK wrote:

The problem I would think where SSD's doing compression and not handling incompressible data well is just down to how fast it can do it like a slow CPU?

This could likely be why Intel SSD's don't do compression?

The compression processing is done by the SandForce controller. It does not rely on the host system CPU at all.

The compression is not just for improving write amplification. It helps wih performance as there is less data to read/write. You can see this performance difference on compressable vs uncompressable benchmarks. I won't comment on how the compression algorithm actually works as I doubt anyone outside of SandForce knows and it is hard to run tests.

I believe the concept of on-the-compression is a relatively novel idea.

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Tue, 04 Jan 2011 23:51:53 GMT

The compression processing is done by the SandForce controller. It does not rely on the host system CPU at all.

Yes I know that I mean like a CPU limit for the SandForce controller itself yes....Which is why I said "like a slow CPU" .

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Wed, 05 Jan 2011 00:51:41 GMT

If I understand it right, trim on the SF drives occurs in a very controlled manner and is designed to minimize write amplification. As a result, unlike the more traditional (indilinx) SSD's where there can be a rapid implementation of trim/garbage collection resulting in rapid recovery of speed (at the cost of increased write amplification), sand force drives on the other hand, will sacrifice "as new" speeds in favor of a more measured and selective implementation of trim with selective blocks being pushed through for erase and rewrite. There is also reasonable speculation that SF drives monitor the amounts of writes during a given period and use that data to adjust write speed to secure nand longevity consistent with warranty periods. This combination of maximizing nand longevity/minimizing write amplification does result in an overall decline in performance to a "settled state" that can be 15 to 20% below "as new". Furthermore extensive use of incompressible data (video, music, zip and rar etc.) will slow these drives in both performance and ability to minimize write amplification.

Curiouser and curiouser.

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Wed, 05 Jan 2011 07:04:46 GMT

PeterUK wrote:

The compression processing is done by the SandForce controller. It does not rely on the host system CPU at all.

Yes I know that I mean like a CPU limit for the SandForce controller itself yes....Which is why I said "like a slow CPU" .

Even if there was a processor-bottleneck, the gains are still noticable. BTW, the SF-2xxx will be released next year and are suppose to hit 500MB/s sequential...

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Wed, 05 Jan 2011 10:29:37 GMT

Even if there was a processor-bottleneck, the gains are still noticable. BTW, the SF-2xxx will be released next year and are suppose to hit 500MB/s sequential...

Very noticeable if you like writing ones and zeros.

^ and by that I mean a file that consists when written just ones or zeros so it can compress it well.

Of course there is a disadvantaged (advantages too just posting the disadvantage) by doing compression on a SSD which is you are limited to the port speed since it decompresses at the SSD and so can't push data out faster then the port but with NTFS compression you pull data off the SSD compressed and decompress it after.

Message was edited by: PeterUK

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Thu, 06 Jan 2011 04:23:17 GMT

PeterUK wrote:

Even if there was a processor-bottleneck, the gains are still noticable. BTW, the SF-2xxx will be released next year and are suppose to hit 500MB/s sequential...

Very noticeable if you like writing ones and zeros.

^ and by that I mean a file that consists when written just ones or zeros so it can compress it well.

Message was edited by: PeterUK

.44x write amplification with Vista + Office 2007 install... that means compression is probably in the 60-70% range: http://images.anandtech.com/reviews/storage/SandForce/SF-2000/durawrite.jpg http://images.anandtech.com/reviews/storage/SandForce/SF-2000/durawrite.jpg

Don't forget that port bandwidth is really only an issue on sequential read/writes with relatively deep queue depths. The SF-2xxx is going to be SATA 6Gb/s so port bandwidth won't be a problem.

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Thu, 06 Jan 2011 09:09:43 GMT

The SF-2xxx is going to be SATA 6Gb/s so port bandwidth won't be a problem.

I was thinking more on the connecting port then what the SSD has.

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Thu, 06 Jan 2011 15:40:09 GMT

I've just had a brain wave!

So SSD's in RAID can't TRIM and lets say can't GC when the SSD's in the array are filled with data and garbage the SSD does the read-modify-write thing to write new data.

Now here's the thing what if the RAID controller tells the OS its going to write some data...the OS looks at the array to see where its free (from it point of view) which the SSD does the read-modify-write thing but the RAID controller that was going to write data in the free space doesn't so that the read-modify-write operation just writes back data that is valid and frees the free space.

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Thu, 06 Jan 2011 17:37:37 GMT

Anand goes some preliminary SF-2582 benchmarks with compressible and uncompressible data: http://www.anandtech.com/show/4100/ocz-vertex-pro-3-demo-worlds-first-sandforce-sf2000 http://www.anandtech.com/show/4100/ocz-vertex-pro-3-demo-worlds-first-sandforce-sf2000

I'm a bit confused on your proposal. Is it that you want the RAID controller to hold the write in cache while SSD is going through the long read-modify-write process? This only works if the controller has enough cache to hold the requests while the SSD is trying to catch up on completing them all.

Also, the OS doesn't look at where the array is free though. It just tells the controller to do something with its member disks.

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Thu, 06 Jan 2011 18:58:48 GMT

You need coffee...no this has nothing to do with cache this is to do with what the read-modify-write process writes back by faking write so that valid data is written back and garbage is not.

The OS shows 80GB free on a array of a total 160GB but that free space has garbage with the only option to read-modify-write but if by some way you can fake a 70GB write the SSD's in the array will read-modify-write but what you don't do is write 70GB of data you read-modify-write the valid data back should it be a block with some valid data on and with other blocks with garbage on read-modify-write nothing back to the SSD's so as to free space up.

Re: An idiots understanding of SSD limitations/trim/GC – guru's please opine

idata — Thu, 06 Jan 2011 22:56:35 GMT

I see... but would increase write amplification drastically though?