I/O request timeouts on Linux with Intel P3520/P4600 NVMe PCIe SSDs
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-06-2018 08:04 AM
Hi,
We are experiencing persistent I/O request timeouts on Linux with P3520/P4600 SSDs. We have tried multiple different kernels (3.10, 4.4, 4.9) and see the timeouts on all of them. The P4600 seems to be more prone to these than the P3520 though we see them on the latter as well. We have the latest firmware installed on both drives which are housed in the same machine (Supermicro 5018R-WR with X10SRW-F motherboard and E5-1650 V4 CPU). We can reproduce the timeouts by simply running mkfs -t xfs on the drive.
Here is the output from isdct (version isdct-3.0.9.400-17.x86_64):
- Intel SSD DC P3520 Series CVPF717100L01P2JGN -
Bootloader : MB1B0105
DevicePath : /dev/nvme0n1
DeviceStatus : Healthy
Firmware : MDV10271
FirmwareUpdateAvailable : The selected Intel SSD contains current firmware as of this tool release.
Index : 0
ModelNumber : INTEL SSDPEDMX012T7
ProductFamily : Intel SSD DC P3520 Series
SerialNumber : CVPF717100L01P2JGN
- Intel SSD DC P4600 Series BTLE736007F54P0KGN -
Bootloader : 0110
DevicePath : /dev/nvme1n1
DeviceStatus : Healthy
Firmware : QDV10150
FirmwareUpdateAvailable : The selected Intel SSD contains current firmware as of this tool release.
Index : 1
ModelNumber : INTEL SSDPEDKE040T7
ProductFamily : Intel SSD DC P4600 Series
SerialNumber : BTLE736007F54P0KGN
Here are the messages the 4.9 kernel prints when using the P4600
[ 151.297903] nvme nvme1: I/O 568 QID 1 timeout, aborting
[ 151.303130] nvme nvme1: I/O 569 QID 1 timeout, aborting
[ 151.308347] nvme nvme1: I/O 570 QID 1 timeout, aborting
[ 151.313562] nvme nvme1: I/O 571 QID 1 timeout, aborting
[ 151.355465] nvme nvme1: completing aborted command with status: 0000
[ 151.411273] nvme nvme1: completing aborted command with status: 0000
[ 151.466903] nvme nvme1: completing aborted command with status: 0000
[ 151.522609] nvme nvme1: completing aborted command with status: 0000
[ 151.578226] nvme nvme1: completing aborted command with status: 0000
...
[ 165.395295] nvme nvme1: Abort status: 0x0
[ 165.399296] nvme nvme1: Abort status: 0x0
[ 165.403299] nvme nvme1: Abort status: 0x0
[ 165.407304] nvme nvme1: Abort status: 0x0
We would appreciate your help in resolving this issue.
Regards,
Shantanu Goel
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-06-2018 03:44 PM
Hello Shantanu Goel,
Thank you for your interest in the Intel® SSD P3520 Series and the Intel® SSD P4600 Series.I understand that your system is experiencing persistent I/O request timeouts.Could you please tell me which is the specific Linux* OS distribution that you are using, and also provide a brief description of the intended use of the SSDs?Additionally, in order to provide the adequate assistance, please share the report generated by the Intel® System Support Utility for the Linux* Operating System ( https://downloadcenter.intel.com/download/26735/).I'll be waiting for your response.Regards,Andres V.- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-07-2018 07:39 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-07-2018 09:17 AM
Hello Shantanu,
Thank you for providing the requested file. We will analyze the provided data and get back to you via this community thread as soon as we have relevant information. Thank you for your patience. Regards,Andres V.- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-07-2018 02:09 PM
Hello Shantanu,
In order to further understand the issue, your system, and the troubleshooting that you have performed, could you please answer the following questions?- How many SSDs of each model are experiencing the timeout issue? How many SSDs do you have of each series?
- Have you tried connecting the SSDs to different slots? Have you been able to test the drives in another motherboard or system?
- Are you using any king of RAID controller?
Regarding the Intel® SSD DC P4600 Series, a new firmware version will tentatively be available within the next couple of weeks as part of the latest Intel® Solid State Drive Data Center Tool version, so please keep checking the download link https://downloadcenter.intel.com/download/27248?v=t https://downloadcenter.intel.com/download/27248?v=t, update your firmware and test again.
I'll be waiting for your response. Regards,Andres V.