What is remote direct memory access (RDMA)?

RMDA is a way of moving data between CPUs, GPUs and storage devices – and it's crucial for cloud migrations. Read on for a deep dive into the topic.

Remote Direct Memory Access, which is abbreviated to RDMA, is a technology that allows data to be transferred rapidly and without difficulty from the primary cloud storage of one computer to another.

In the context of working with enormous data sets or sophisticated procedures, such as those required for machine learning, this is a must-have.

In this article, we're taking a deep dive into the topic of RDMA and seeing how it operates.

What is RDMA?

RDMA refers to the procedure in which one computer in a network directly accesses the memory of another computer in the network without going via the operating system, processor, or cache of any of those computers. This helps boost system throughput and performance by freeing up resources.

A distant machine's CPU doesn't need to be used for read/write activities. Thanks to this innovation, both the data transfer rate and the networking latency are streamlined. Through the activation of network adapters, it uses zero-copy networking to transport data straight into the buffers of the systems.

Features of RDMA

1. The zero-copy networking protocol

Without having to copy the data across network layers, it's feasible to transmit and receive information directly between the buffers of different programs.

2. Reduced involvement of the CPU

Applications can access data stored on faraway servers without taxing those servers' processing power. Accessed data won't be stored in the distant server's CPU cache memory.

3. Transactional efficiency

When exchanging information, it's feasible to do so in the form of individual messages rather than continuous streams, thereby eliminating the requirement for message separation.

Working with RDMA

For the RDMA data exchange method to function properly, the network cards on your computer must include the appropriate protocol set.

For the most part, TCP/IP serves as the foundation. A good transport protocol is required for the technology to provide zero-copy networking, for instance, and won't tax the CPU.

Data transfers between systems with RDMA capabilities are quicker than those between systems without RDMA support.

RDMA support in vSphere

RDMA is a protocol that's supported by vSphere.

vSphere's many storage features, such as iSCSI extensions via RDMA, which were introduced in version 6.7, and NVMeoF, which used RDMA in version 7, are good examples of RDMA's use with the platform.

RDMA is another option that contributes significantly to the success of vSphere Bitfusion. vSphere 7 Update 1 adds RDMA functionality for native endpoints (non-PVRDMA endpoints) such as with storage arrays.

Within a vSphere context, the use of RDMA for virtual workloads is supported. Applications for high-performance computing (HPC), database backends and big data platforms are just a few examples of the large range of applications that may benefit from RDMA's ability to achieve faster I/O speeds while maintaining reduced latency.

RMDA support for operating on Virtual Machines

There are many different ways to make RDMA accessible to virtual machines (VMs).

The first possibility is to pass through an HCA or an RDMA-capable NIC to the VM by using a (Dynamic) DirectPath I/O. When you use passthrough, a one-to-one link is established between the VM and the RDMA network device. The fact that vSphere capabilities such as vMotion aren't available with passthrough is a drawback in this situation.

Utilising paravirtual RDMA (PVRDMA), which is also known as vRDMA, is a second alternative that allows vMotion to maintain task mobility. This method is also known as vRDMA. PVRDMA is supported for an x64 Linux guest operating system as of VM hardware version 13, which was released with vSphere 6.5.

PVRDMA provides support for the RDMA over converged Ethernet (RoCE) protocol. The fact that an Ethernet fabric may be used is one of RoCE's many attractive features. In contrast to the case with InfiniBand, there is no need for a separate fabric. RDMA must be supported by the ethernet fabric, and the primary way this may be accomplished is by implementing Priority Flow Control (PFC).

PVRDMA is compatible with both RoCE v1 and v2 protocols. The main difference between RoCE v1 and RoCE v2 is that the former only supports switched networks, while the latter supports routed networks.

The pros and cons of using RDMA

When compared to other cloud data transfer technologies and protocols, such as iSCSI (TCP-based SCSI), fibre channel (FC), or fibre channel over Ethernet (FCoE), RDMA's exceptional speed is one of its primary benefits.

The RDMA version also plays a role in the ultimate data transfer speed. Ethernet and InfiniBand are widely used because they can provide transmission rates of 10–100 Gbps. Applications like distributed databases, large data analyses and those run in data centres may all benefit greatly from this.

But there are certain drawbacks to RDMA that make it less attractive than fibre channels, which are nevertheless widely used today. Because RDMA necessitates the purchase of new hardware and protocol components, its adoption by businesses necessitates substantial investment.

As a result, the fees associated with RDMA are much more than those of FC or FCoE. In addition, RDMA's rapid data transmission capabilities are contingent on its widespread adoption.

Are you looking for a managed cloud migration solution? Please don't hesitate to get in touch to speak with one of our VMware experts.