NVMe is a specification for connecting an SSD directly to a machine’s PCIe interface. This greatly increases storage performance. Learn everything you need to know about NVMe SSDs and their benefits for big data.
SSDs have revolutionized data storage with significantly higher transfer speeds. However, The performance of these devices can be bridged by SAS or SATA connections. inherited from traditional hard drives. In fact, for intensive use cases such as big data analysis, it is preferable to use SSDs based on the open NVMe standard.
NVMe: What is it and how does it work?
The NVM Express or NVMe is a software interface for PCI Express SSDs using non-volatile memory (NVM). This open standard allows flash storage devices to operate at the maximum write and read speeds that their flash memory allows.
Simply put, NVMe allows you to connect the SSD directly to the PCIe interface. Aisni, the SSD is no longer limited by intermediate communication protocols. NVMe takes advantage of the internal parallelism and low latency associated with flash storage devices.
NVMe vs SATA and SAS
Until 2014, most of the SSDs produced were based on the historical storage buses such as SATA or SAS to interface with computers. These two buses, along with the SCSI control protocol, were designed and engineered for use with HDDs.
The HDDs are mechanical devicesThe data is stored on the internal platter, each internal platter of which is divided into tracks and sectors on which the data is stored. The playback head can only play back one sector at a time. In contrast, an SSD has no mechanical components. It consists of an assembly of Flash modules, and it is possible to access several memory blocks simultaneously. Read and write operations can also be performed on different blocks.
The difference is therefore major, and standard buses such as SATA and SAS do not allow the full potential of SSDs to be exploited. Even if the performance gain is significant compared to an HDD, it is even more significant when the SSD is directly connected to a fast bus and controlled via an optimised control set.
This is why the Non-Volatile Memory Host Controller Interface (NVMHCI) working group, which brings together nearly a hundred companies, has developed the NVMe specification whose version 1.0 was launched in March 2011 with reference drivers for Windows, Linux and FreeBSD.
The NVMe offers a reduction in latency, an increase in the IOPS level, and a reduction in power consumption. In addition, NVMe enables multipathing: multiple PCI-e controllers in a single machine can deliver paths to a single device. This increases resiliency. A final benefit is advanced namespace management, for example, to enable sharing of an NVMe SSD device between multiple servers via a virtualization mechanism such as Multi-Root-IOV.
By way of comparison, the SATA III on which modern motherboards are based has a maximum throughput of 6000 MB per second. This connection enables the SSD to achieve a read speed of 530 MB/s, and a write speed of 500 MB/s. An NVMe SSD, on the other hand, will be able to reach a write speed of 3500 MB/s. The writing speed is thus multiplied by 7 compared to a SATA SSD.
Why is NVMe ideal for Big Data?
Real-time Big Data analysis or high-performance database environments are workloards that require keep latency to a minimum. That’s why it’s best to choose NVMe SSDs for Big Data.
According to IDC analysts, between 60 and 70 percent of Fortune 2000 companies will have at least one critical workload using real-time Big Data analysis by 2020. Indeed, we can expect a real growth of NVMe in the enterprise.