Why does "fio write" change the performance of "fio read"

[YLin34]

I have a P4600 2TB NVMe SSD. Right after I used Intel SSD toolbox to secure erase it, fio reports 4kB random read at 809K IOPS.

sudo fio --name=dummy --size=10G --filename=/dev/nvme0n1 --ioengine=libaio --direct=1 --rw=randread -bs=4k** --iodepth=64 --numjobs=4 --group_reporting

However, after I tried to pre-conditioning the SSD with the following write command

sudo fio --name=dummy --size=2000G --filename=/dev/nvme0n1 --ioengine=libaio --direct=1 --rw=write -bs=4k** --iodepth=64 --numjobs=1 --group_reporting (I also tried --size=1700G, the result is the same)

the randread command above is now reporting 4kB at 520k IOPS.

So why such a severe drop?

[idata]

Hi yaolin,

Thank you for contacting our support community. We understand your situation regarding the Intel® SSD DC P4600 Series. We recommend you to check the following https://itpeernetwork.intel.com/how-to-benchmark-ssds-with-fio-visualizer/ article related to how to benchmark SSDs with FIO*. You will be able to find the settings for the test. Regarding the number, we recommend you to check the https://ark.intel.com/products/96999/Intel-SSD-DC-P4600-Series-2_0TB-12-Height-PCIe-3_1-x4-3D1-TLC performance information related to the SSD. As per the website, the SSD has the following stats: Performance :Sequential Read (up to) 3200 MB/sSequential Write (up to) 1575 MB/sRandom Read (100% Span) 610000 IOPSRandom Write (100% Span) 196650 IOPSIf you get the 75% of the numbers or better, it means that the SSD is working fine. As an example: Random Read: 610000 IOPS75% of 610000 = 457500 IOPS and you are getting 809K which is above of the advertised speed. Keep in mind that SSD was empty before the test. For the Random Write: 196650 IOPS75% of 196650 = 147487.5 IOPS.Based on the information, after pre-conditioning the SSD you got 520k IOPS which represents the 85.6% of the random reads based on the drive specs. Keep in mind that at this point there is information written in the SSD so the numbers may vary. Regards, Junior M.

[YLin34]

Hi,

Thanks for the quick response and the great links.

No, I was not complaining about the performance itself. I was just trying to understand why the IOPS drops.

I was thinking that, right after secure erase the SSD, FTL table is also erased. This table will need to be build and thus the first randread could be slower. Yet I got 809k IOPS.

Then I sequentially wrote through most of the drive once (1.7TB out of 2.0TB). Since I wrote only once, there shall be no erase involved and thus there shall be no garbage collection.

But then, why did the IOPS drop to 520k when I did randread again? You mentioned this is a normal IOPS. Then maybe I shall ask why IOPS is so high in the first randread. I don't know why SSD is empty vs has data would impact the read throughput.

Thanks,

Yao

[idata]

Hi yaolin,

Thanks for the reply. In order to answer your question regarding why the SSD performance slows down as it becomes full, we recommend you to check the following articles: https://www.howtogeek.com/165542/why-solid-state-drives-slow-down-as-you-fill-them-up/ Why Solid-State Drives Slow Down As You Fill Them Up https://pureinfotech.com/why-solid-state-drive-ssd-performance-slows-down/ Why solid-state drive (SSD) performance slows down as it becomes full If you have further questions then feel free to contact us again. NOTE: Any links provided for third party tools or sites are offered for your convenience and should not be viewed as an endorsement by Intel® of the content, products, or services offered there. We do not offer support for any third party tool mentioned here.Regards, Junior M.

[YLin34]

I understand that the write perform will drop as the SSD fills up. But I am doing random read only in this case.