Internet Engineering Task Force (IETF)                  A. Kanevsky, Ed.
Request for Comments: 6581                                     Dell Inc.
Updates: 5043, 5044                                      C. Bestler, Ed.
Category: Standards Track                                Nexenta Systems
ISSN: 2070-1721                                                 R. Sharp
                                                                   Intel
                                                                 S. Wise
                                                     Open Grid Computing
                                                              April 2012
        
Internet Engineering Task Force (IETF)                  A. Kanevsky, Ed.
Request for Comments: 6581                                     Dell Inc.
Updates: 5043, 5044                                      C. Bestler, Ed.
Category: Standards Track                                Nexenta Systems
ISSN: 2070-1721                                                 R. Sharp
                                                                   Intel
                                                                 S. Wise
                                                     Open Grid Computing
                                                              April 2012
        

Enhanced Remote Direct Memory Access (RDMA) Connection Establishment

增强的远程直接内存访问(RDMA)连接建立

Abstract

摘要

This document updates RFC 5043 and RFC 5044 by extending Marker Protocol Data Unit (PDU) Aligned Framing (MPA) negotiation for Remote Direct Memory Access (RDMA) connection establishment. The first enhancement extends RFC 5044, enabling peer-to-peer connection establishment over MPA / Transmission Control Protocol (TCP). The second enhancement extends both RFC 5043 and RFC 5044, by providing an option for standardized exchange of RDMA-layer connection configuration.

本文档通过扩展用于远程直接内存访问(RDMA)连接建立的标记协议数据单元(PDU)对齐帧(MPA)协商来更新RFC 5043和RFC 5044。第一个增强扩展了RFC 5044,支持通过MPA/传输控制协议(TCP)建立对等连接。第二个增强扩展了RFC 5043和RFC 5044,为RDMA层连接配置的标准化交换提供了一个选项。

Status of This Memo

关于下段备忘

This is an Internet Standards Track document.

这是一份互联网标准跟踪文件。

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 5741.

本文件是互联网工程任务组(IETF)的产品。它代表了IETF社区的共识。它已经接受了公众审查,并已被互联网工程指导小组(IESG)批准出版。有关互联网标准的更多信息,请参见RFC 5741第2节。

Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc6581.

有关本文件当前状态、任何勘误表以及如何提供反馈的信息,请访问http://www.rfc-editor.org/info/rfc6581.

Copyright Notice

版权公告

Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved.

版权所有(c)2012 IETF信托基金和确定为文件作者的人员。版权所有。

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

本文件受BCP 78和IETF信托有关IETF文件的法律规定的约束(http://trustee.ietf.org/license-info)自本文件出版之日起生效。请仔细阅读这些文件,因为它们描述了您对本文件的权利和限制。从本文件中提取的代码组件必须包括信托法律条款第4.e节中所述的简化BSD许可证文本,并提供简化BSD许可证中所述的无担保。

Table of Contents

目录

   1. Introduction ....................................................3
      1.1. Summary of Changes Affecting RFC 5044 ......................4
      1.2. Summary of Changes Affecting RFC 5043 ......................4
   2. Requirements Language ...........................................4
   3. Definitions .....................................................4
   4. Motivations .....................................................7
      4.1. Standardization of RDMA Read Parameter Configuration .......7
      4.2. Enabling MPA Mode ..........................................9
      4.3. Lack of Explicit RTR in MPA Request/Reply Exchange ........10
      4.4. Limitations on ULP Workaround .............................11
           4.4.1. Transport Neutral APIs .............................11
           4.4.2. Work/Completion Queue Accounting ...................11
           4.4.3. Host-based Implementation of MPA Fencing ...........12
   5. Enhanced MPA Connection Establishment ..........................13
   6. Enhanced MPA Request/Reply Frames ..............................14
   7. Enhanced SCTP Session Control Chunks ...........................15
   8. MPA Error Reporting ............................................16
   9. Enhanced RDMA Connection Establishment Data ....................17
      9.1. IRD and ORD Negotiation ...................................18
      9.2. Peer-to-Peer Connection Negotiation .......................20
      9.3. Enhanced Connection Negotiation Flow ......................21
   10. Interoperability ..............................................21
   11. IANA Considerations ...........................................22
   12. Security Considerations .......................................23
   13. Acknowledgements ..............................................23
   14. References ....................................................23
      14.1. Normative References .....................................23
      14.2. Informative References ...................................24
        
   1. Introduction ....................................................3
      1.1. Summary of Changes Affecting RFC 5044 ......................4
      1.2. Summary of Changes Affecting RFC 5043 ......................4
   2. Requirements Language ...........................................4
   3. Definitions .....................................................4
   4. Motivations .....................................................7
      4.1. Standardization of RDMA Read Parameter Configuration .......7
      4.2. Enabling MPA Mode ..........................................9
      4.3. Lack of Explicit RTR in MPA Request/Reply Exchange ........10
      4.4. Limitations on ULP Workaround .............................11
           4.4.1. Transport Neutral APIs .............................11
           4.4.2. Work/Completion Queue Accounting ...................11
           4.4.3. Host-based Implementation of MPA Fencing ...........12
   5. Enhanced MPA Connection Establishment ..........................13
   6. Enhanced MPA Request/Reply Frames ..............................14
   7. Enhanced SCTP Session Control Chunks ...........................15
   8. MPA Error Reporting ............................................16
   9. Enhanced RDMA Connection Establishment Data ....................17
      9.1. IRD and ORD Negotiation ...................................18
      9.2. Peer-to-Peer Connection Negotiation .......................20
      9.3. Enhanced Connection Negotiation Flow ......................21
   10. Interoperability ..............................................21
   11. IANA Considerations ...........................................22
   12. Security Considerations .......................................23
   13. Acknowledgements ..............................................23
   14. References ....................................................23
      14.1. Normative References .....................................23
      14.2. Informative References ...................................24
        
1. Introduction
1. 介绍

When used over the Transmission Control Protocol (TCP), the current Remote Direct Data Placement (RDDP) [RFC5041] suite of protocols relies on the MPA [RFC5044] protocol for both connection establishment and for markers for TCP layering.

当通过传输控制协议(TCP)使用时,当前的远程直接数据放置(RDDP)[RFC5041]协议套件依赖MPA[RFC5044]协议来建立连接和标记TCP分层。

A typical model for establishing an RDMA connection has the following steps:

建立RDMA连接的典型模型具有以下步骤:

o The passive side (responder) Upper Layer Protocol (ULP) listens for connection requests.

o 被动端(响应者)上层协议(ULP)侦听连接请求。

o The active side (initiator) ULP submits a connection request using an RDMA endpoint, the desired destination, and the parameters to be used for the connection. Those parameters include both RDMA-layer characteristics, such as the number of simultaneous RDMA Read Requests to be allowed, and application-specific data.

o 主动端(启动器)ULP使用RDMA端点、所需目的地和用于连接的参数提交连接请求。这些参数包括RDMA层特征(例如允许的同时RDMA读取请求的数量)和特定于应用程序的数据。

o The passive side ULP receives a connection request that includes the identity of the active side and the requested connection characteristics. The passive side ULP uses this information to decide whether to accept the connection, and if it is to be accepted, how to create and/or configure the local RDMA endpoint.

o 被动侧ULP接收包括主动侧的标识和请求的连接特征的连接请求。被动端ULP使用此信息决定是否接受连接,以及如果要接受连接,如何创建和/或配置本地RDMA端点。

o If accepting, the responder submits its acceptance of the connection request, which in turn generates the accept message to the initiator. This responder accept operation includes the RDMA endpoint to be used and the connection characteristics (both the RDMA configuration and any application-specific Private Data to be transferred to the initiator).

o 如果接受,响应者将提交其对连接请求的接受,从而向发起方生成接受消息。此响应程序接受操作包括要使用的RDMA端点和连接特征(RDMA配置和要传输到启动器的任何特定于应用程序的私有数据)。

o The active side receives confirmation that the connection has been accepted, what the configured connection characteristics are, and any application-supplied Private Data.

o 主动端收到连接已被接受的确认、配置的连接特征以及任何应用程序提供的私有数据。

Currently, MPA only supports a client-server model for connection establishment, forcing peer-to-peer applications to interact as though they had a client-server relationship. In addition, negotiation of some parameters specific to the Remote Direct Memory Access Protocol (RDMAP) [RFC5040] are left to ULP negotiation. Providing an optional ULP-independent format for exchanging these parameters would be of benefit to transport neutral RDMA applications.

目前,MPA仅支持用于建立连接的客户机-服务器模型,强制对等应用程序进行交互,就像它们具有客户机-服务器关系一样。此外,特定于远程直接存储器访问协议(RDMAP)[RFC5040]的一些参数的协商留给ULP协商。为交换这些参数提供可选的ULP独立格式将有利于传输中性RDMA应用程序。

1.1. Summary of Changes Affecting RFC 5044
1.1. 影响RFC 5044的变更汇总

This document enhances the MPA connection setup protocol [RFC5044]. First, it adds exchange and negotiation of the parameters necessary to support RDMA Read Requests. Second, it adds a message that serves as a Ready to Receive (RTR) indication from the initiator to the responder as the last message of connection establishment and adds negotiation of which type of message to use for carrying the RTR indication into MPA Request/Reply Frames.

本文档增强了MPA连接设置协议[RFC5044]。首先,它添加了支持RDMA读取请求所需的参数交换和协商。第二,它添加一条消息,作为连接建立的最后一条消息,作为从发起方到响应方的准备接收(RTR)指示,并添加用于将RTR指示携带到MPA请求/应答帧中的哪种类型的消息的协商。

RTR indications are optional and are carried by existing RDMA message types, specifically a zero-length FULPDU Send message, a zero-length RDMA Read message, or a zero-length RDMA write message. The presence vs. absence of the RTR indication and the type of RDMA message to use are negotiated by control flags in Enhanced RDMA connection establishment data specified by this document (see Section 9). RDMA implementations are often tightly integrated with application libraries and hardware, hence the flexibility to use more than one type of RDMA message enables implementations to choose message types that are less disruptive to the implementation structure. When an RTR indication is used, and MPA connection setup negotiation indicates support for multiple RDMA message types as RTR indications by both the initiator and responder, the initiator selects one of the supported RDMA message types as the RTR indication at the initiator's sole discretion.

RTR指示是可选的,由现有RDMA消息类型携带,具体来说是零长度FULPDU发送消息、零长度RDMA读取消息或零长度RDMA写入消息。RTR指示的存在与否以及要使用的RDMA消息的类型由本文件规定的增强RDMA连接建立数据中的控制标志协商(见第9节)。RDMA实现通常与应用程序库和硬件紧密集成,因此使用多种类型的RDMA消息的灵活性使实现能够选择对实现结构破坏性较小的消息类型。当使用RTR指示,并且MPA连接设置协商表明发起方和响应方都支持多个RDMA消息类型作为RTR指示时,发起方可自行选择一种受支持的RDMA消息类型作为RTR指示。

1.2. Summary of Changes Affecting RFC 5043
1.2. 影响RFC 5043的变更汇总

This document enhances [RFC5043] by adding new Enhanced Session Control Chunks that extend the currently defined Chunks with the addition of Inbound RDMA Read Queue Depth (IRD) and Outbound RDMA Read Queue Depth (ORD) negotiation.

本文档通过添加新的增强会话控制区块来增强[RFC5043],该区块通过添加入站RDMA读取队列深度(IRD)和出站RDMA读取队列深度(ORD)协商来扩展当前定义的区块。

2. Requirements Language
2. 需求语言

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].

本文件中的关键词“必须”、“不得”、“必需”、“应”、“不应”、“应”、“不应”、“建议”、“可”和“可选”应按照[RFC2119]中所述进行解释。

3. Definitions
3. 定义

Active Side: See Initiator.

主动端:请参阅启动器。

Consumer: The ULPs or applications that lie above MPA and Direct Data Placement (DDP). The Consumer is responsible for making TCP or Stream Control Transmission Protocol (SCTP) connections, starting MPA and DDP connections, and generally controlling operations. See [RFC5044] and [RFC5043].

消费者:ULP或高于MPA和直接数据放置(DDP)的应用程序。使用者负责建立TCP或流控制传输协议(SCTP)连接,启动MPA和DDP连接,并通常控制操作。参见[RFC5044]和[RFC5043]。

CRC: Cyclic Redundancy Check

循环冗余校验

Completion Queue (CQ): A Consumer-accessible queue where the RDMA device reports completions of Work Requests. A Consumer is able to reap completions from a CQ without requiring per-transaction support from the kernel or other privileged entity. See [RDMAC].

完成队列(CQ):RDMA设备报告工作请求完成情况的消费者可访问队列。消费者能够从CQ中获取完成,而无需内核或其他特权实体提供每事务支持。见[RDMAC]。

Completion Queue Entry (CQE): Transport- and device-specific representation of a Work Completion. A CQ holds CQEs. See [RDMAC].

完成队列条目(CQE):工作完成的特定于传输和设备的表示。CQ持有CQ。见[RDMAC]。

FULPDU: Framed Upper Layer Protocol PDU. See FPDU of [RFC5044].

FULPDU:框架上层协议PDU。参见[RFC5044]的FPDU。

Inbound RDMA Read Request Queue (IRRQ): A queue that is associated with an RDMA connection that tracks active incoming simultaneous RDMA Read Request Messages. See [RDMAC].

入站RDMA读取请求队列(IRRQ):与RDMA连接关联的队列,该连接跟踪活动的同时传入的RDMA读取请求消息。见[RDMAC]。

Inbound RDMA Read Queue Depth (IRD): The maximum number of incoming simultaneous RDMA Read Request Messages an RDMA connection can handle. See [RDMAC].

Inbound RDMA Read Queue Depth(IRD):RDMA连接可以处理的同时传入RDMA读取请求消息的最大数量。见[RDMAC]。

Initiator: The endpoint of a connection that sends the MPA Request Frame. The initiator is the active side of the connection establishment. See [RFC5044].

启动器:发送MPA请求帧的连接的端点。启动器是连接建立的活动端。参见[RFC5044]。

IRD: See Inbound RDMA Read Queue Depth.

IRD:请参阅入站RDMA读取队列深度。

MPA Fencing: MPA responder connection establishment logic that ensures that no ULP messages will be transferred until the initiator's first message has been received.

MPA围栏:MPA响应程序连接建立逻辑,确保在收到启动器的第一条消息之前不会传输ULP消息。

MPA Request Frame: Data sent from the MPA initiator to the MPA responder during the Startup Phase. See [RFC5044].

MPA请求帧:启动阶段从MPA启动器发送到MPA响应程序的数据。参见[RFC5044]。

MPA Reply Frame: Data sent from the MPA responder to the MPA initiator during the Startup Phase. See [RFC5044].

MPA应答帧:启动阶段从MPA应答器发送到MPA启动器的数据。参见[RFC5044]。

ORD: See Outbound RDMA Read Queue Depth.

ORD:请参阅出站RDMA读取队列深度。

Outbound RDMA Read Queue Depth (ORD): The maximum number of simultaneous RDMA Read Requests that can be issued for the RDMA connection. This should be less than or equal to the peer's IRD. See [RDMAC].

Outbound RDMA Read Queue Depth(ORD):可以为RDMA连接同时发出的最大RDMA读取请求数。这应小于或等于对等方的IRD。见[RDMAC]。

Passive Side: See Responder.

被动方:见响应方。

Private Data: A block of data exchanged between MPA endpoints during initial connection setup. See [RFC5044].

私有数据:初始连接设置期间MPA端点之间交换的数据块。参见[RFC5044]。

Queue Pair (QP): A Queue Pair is the set of Work Queues associated exclusively with a single Endpoint (first defined in [VIA]). The Send Queue (SQ), Receive Queue (RQ), and Inbound RDMA Read Queue (IRQ) are considered to be part of the Queue Pair. The potentially shared Completion Queue (CQ) and Shared Receive Queue (SRQ) are not. See [RDMAC].

队列对(QP):队列对是专门与单个端点(首先在[VIA]中定义)关联的一组工作队列。发送队列(SQ)、接收队列(RQ)和入站RDMA读取队列(IRQ)被视为队列对的一部分。潜在共享完成队列(CQ)和共享接收队列(SRQ)不可用。见[RDMAC]。

Remote Peer: The MPA protocol implementation on the opposite end of the connection. Used to refer to the remote entity when describing protocol exchanges or other interactions between two nodes. See [RFC5044].

远程对等:连接另一端的MPA协议实现。在描述两个节点之间的协议交换或其他交互时,用于指代远程实体。参见[RFC5044]。

Responder: The connection endpoint that responds to an incoming MPA connection request (the MPA Request Frame). The responder is the passive side of the connection establishment. See [RFC5044].

Responder:响应传入MPA连接请求(MPA请求帧)的连接端点。响应者是连接建立的被动端。参见[RFC5044]。

Ready to Receive (RTR): RTR is an indication provided by the last connection establishment message sent from the initiator to the responder. An RTR indicates that the initiator is ready to receive messages and that connection establishment is completed.

准备接收(RTR):RTR是由发起方发送给响应方的最后一条连接建立消息提供的指示。RTR表示启动器已准备好接收消息,并且连接建立已完成。

Startup Phase: The initial exchanges of an MPA connection that serves to more fully identify MPA endpoints to each other and pass connection-specific setup information to each other. See [RFC5044].

启动阶段:MPA连接的初始交换,用于更全面地相互识别MPA端点,并相互传递特定于连接的设置信息。参见[RFC5044]。

Shared Receive Queue (SRQ): A shared pool of Receive Work Requests posted by the Consumer that can be allocated by multiple RDMA endpoints (QP). See [RDMAC].

共享接收队列(SRQ):消费者发布的接收工作请求的共享池,可由多个RDMA端点(QP)分配。见[RDMAC]。

Tagged (DDP) Message: A DDP Message that targets a Tagged Buffer that is explicitly advertised to the Remote Peer through exchange of an STag (memory handle), offset in the memory region identified by STag, and length [RFC5040].

标记的(DDP)消息:一种DDP消息,其目标是标记的缓冲区,该缓冲区通过STag(内存句柄)、STag标识的内存区域中的偏移量和长度(RFC5040)的交换显式通告给远程对等方。

Untagged (DDP) Message: A DDP Message that targets an Untagged Buffer associated with a queue specified the by Queue Number (QN). [RFC5040].

未标记(DDP)消息:一种DDP消息,目标是与队列号(QN)指定的队列相关联的未标记缓冲区。[RFC5040]。

Work Queue: An element of a QP that allows user-space applications to submit Work Requests directly to network hardware (first defined in [VIA]). Specific Work Queues include the Send Queue (SQ) for transmit requests, Receive Queue (RQ) for receive requests specific to a single endpoint, and Shared Receive Queues (SRQs) for receive requests that can be allocated by one or more endpoints. See [RDMAC].

工作队列:QP的一个元素,允许用户空间应用程序直接向网络硬件提交工作请求(首先在[VIA]中定义)。特定的工作队列包括用于发送请求的发送队列(SQ)、用于特定于单个端点的接收请求的接收队列(RQ)以及用于可由一个或多个端点分配的接收请求的共享接收队列(SRQ)。见[RDMAC]。

Work Queue Element (WQE): Transport- and device-specific representation of a Work Request. See [RDMAC].

工作队列元素(WQE):工作请求的特定于传输和设备的表示。见[RDMAC]。

Work Request: An elementary object used by Consumers to enqueue a requested operation (WQEs) onto a Work Queue. See [RDMAC].

工作请求:使用者用于将请求的操作(WQEs)排入工作队列的基本对象。见[RDMAC]。

4. Motivations
4. 动机

The goal of this document is two-fold. The first is to extend support from the current client-server model for RDMA connection setup to a peer-to-peer model. The second is to add negotiation of the RDMA Read Queue size for both sides of an RDMA connection.

本文件的目标有两个方面。第一个是将对RDMA连接设置的支持从当前的客户机-服务器模型扩展到对等模型。第二个是为RDMA连接的两侧添加RDMA读取队列大小的协商。

4.1. Standardization of RDMA Read Parameter Configuration
4.1. RDMA读取参数配置的标准化

Most RDMA applications are developed using a transport-neutral Application Programming Interface (API) to access RDMA services based on a "Queue Pair" paradigm as originally defined by the Virtual Interface Architecture [VIA], refined by the Direct Access Programming Library [DAPL], and most commonly deployed with the OpenFabrics API [OFA].

大多数RDMA应用程序是使用与传输无关的应用程序编程接口(API)开发的,以访问基于“队列对”范式的RDMA服务,该范式最初由虚拟接口体系结构[VIA]定义,由直接访问编程库[DAPL]细化,最常与OpenFabrics API[OFA]一起部署。

These transport-neutral APIs seek to provide a common set of RDMA services whether the underlying transport is, for example, RDDP over MPA, RDDP over SCTP, or InfiniBand.

这些与传输无关的API寻求提供一组通用的RDMA服务,无论底层传输是RDDP over MPA、RDDP over SCTP还是InfiniBand。

The common model for establishing an RDMA connection has the following steps:

用于建立RDMA连接的通用模型具有以下步骤:

o The passive side ULP listens for connection requests.

o 被动端ULP侦听连接请求。

o The active side ULP submits a connection request using an RDMA endpoint ("Queue Pair"), the desired destination, and the parameters to be used for the connection. Those parameters include both RDMA-layer characteristics, such as the number of simultaneous RDMA Read Requests to be allowed, and application-specific data (typically referred to as "Private Data").

o 主动端ULP使用RDMA端点(“队列对”)、所需目的地和用于连接的参数提交连接请求。这些参数包括RDMA层特征(例如允许的同时RDMA读取请求的数量)和特定于应用程序的数据(通常称为“私有数据”)。

o The passive side ULP receives a connection request, which includes the identity of the active side and the requested connection characteristics. The passive side ULP uses this information to decide whether to accept the connection, and if it is to be accepted, how to create and/or configure the RDMA endpoint.

o 被动侧ULP接收连接请求,该连接请求包括主动侧的标识和请求的连接特征。被动端ULP使用此信息决定是否接受连接,以及如果要接受连接,如何创建和/或配置RDMA端点。

o If accepting, the passive side ULP submits its acceptance of the connection request. This local accept operation includes the RDMA endpoint to be used and the connection characteristics (both the RDMA configuration and any application-specific Private Data to be returned).

o 如果接受,被动侧ULP提交其对连接请求的接受。此本地接受操作包括要使用的RDMA端点和连接特征(RDMA配置和要返回的任何特定于应用程序的私有数据)。

o The active side receives confirmation that the connection has been accepted, what the configured connection characteristics are, and any application-supplied Private Data.

o 主动端收到连接已被接受的确认、配置的连接特征以及任何应用程序提供的私有数据。

As currently defined, DDP connection establishment requires the ULP to encode the RDMA configuration in the application-specific Private Data. This results in undesirable duplication of logic to cover RDMA characteristics of both InfiniBand and RDDP for each ULP, and to specify for InfiniBand and RDDP the extraction of the RDMA characteristics for each ULP.

按照目前的定义,DDP连接建立需要ULP在特定于应用程序的专用数据中对RDMA配置进行编码。这会导致不希望的逻辑复制,以覆盖每个ULP的InfiniBand和RDDP的RDMA特性,并为InfiniBand和RDDP指定每个ULP的RDMA特性提取。

Both RDDP and InfiniBand support an initial Private Data exchange; therefore, a standard definition of the RDMA characteristics within the Private Data section would enable common connection establishment APIs to format the RDMA characteristics based on the same API information used when establishing either protocol to form the connection. The application would then only have to indicate that it was using this standard format to enable common connection establishment procedures to apply common code to properly parse these fields and configure the RDMA endpoints accordingly. Exchange of parameters necessary to perform RDMA Read operations is a common usage of the initial Private Data exchange.

RDDP和InfiniBand都支持初始私有数据交换;因此,专用数据部分中RDMA特性的标准定义将使公共连接建立API能够基于建立任一协议以形成连接时使用的相同API信息格式化RDMA特性。然后,应用程序只需指示它正在使用此标准格式来启用公共连接建立过程,以应用公共代码来正确解析这些字段并相应地配置RDMA端点。交换执行RDMA读取操作所需的参数是初始私有数据交换的常见用法。

One of the RDMA operations that is defined in [RDMAC] is an RDMA Read. RDMA Read operations are performed using an untagged message sent from a Queue Pair (QP) on the local endpoint to a QP on the remote endpoint targeting the Inbound RDMA Read Request Queue (QN=1 or Inbound RDMA Read Request Queue (IRRQ)) associated with the connection. RDMA Read responses transfer data associated with each RDMA Read Request from the remote endpoint to the local endpoint using tagged messages. An inbound RDMA Read Request remains on the IRRQ from the time that it is received until the time that the last tagged message associated with the RDMA request is acknowledged. The IRRQ is associated with a QP but is not a Work Queue. Instead, the IRRQ is a stand-alone queue that is used to manage RDMA Read Requests associated with a QP. See [RDMAC], Section 6 for more information regarding QPs and IRRQ. One of the characteristics that must be configured for a QP is the size of the IRRQ. This parameter is called the Inbound RDMA Read Queue Depth (IRD). Another characteristic of a QP that must be configured is a local limit on the number of simultaneous outbound RDMA Read Requests based on the size of the remote endpoint QP's IRRQ. This parameter is call the

[RDMAC]中定义的RDMA操作之一是RDMA读取。RDMA读取操作是使用从本地端点上的队列对(QP)发送到远程端点上的QP的未标记消息来执行的,目标是与连接相关联的入站RDMA读取请求队列(QN=1或入站RDMA读取请求队列(IRRQ))。RDMA读取响应使用标记消息将与每个RDMA读取请求相关联的数据从远程端点传输到本地端点。从接收到入站RDMA读取请求到确认与RDMA请求关联的最后一条标记消息为止,该请求一直保留在IRRQ上。IRRQ与QP关联,但不是工作队列。相反,IRRQ是一个独立队列,用于管理与QP关联的RDMA读取请求。有关QPs和IRRQ的更多信息,请参见[RDMAC],第6节。必须为QP配置的特征之一是IRRQ的大小。此参数称为入站RDMA读取队列深度(IRD)。必须配置的QP的另一个特征是基于远程端点QP的IRRQ的大小对同时出站RDMA读取请求的数量的本地限制。此参数是调用

Outbound RDMA Read Queue Depth (ORD). ORD is used to limit the number of simultaneous RDMA Read Requests such that the local endpoint does not overrun the remote endpoint's IRRQ depth or IRD. Note that outbound RDMA Reads are submitted to a QP's Send Queue at the local peer, not to a separate outbound RDMA Read Request queue on the local peer. The local endpoint uses ORD to strictly limit simultaneous Read Requests so that IRRQ overruns do not occur at the remote endpoint.

出站RDMA读取队列深度(ORD)。ORD用于限制同步RDMA读取请求的数量,以便本地端点不会超出远程端点的IRRQ深度或IRD。请注意,出站RDMA读取被提交到本地对等方的QP发送队列,而不是本地对等方上的单独出站RDMA读取请求队列。本地端点使用ORD严格限制同时读取请求,以便IRRQ溢出不会在远程端点发生。

Determination of the values of the ORD and IRD are left to the ULP by the current RDDP suite of protocols and also by [RDMAC]. Since this negotiation of ORD and IRD is typical, it is desirable to provide a common mechanism as described in this document.

ORD和IRD值的确定由当前RDDP协议套件和[RDMAC]留给ULP。由于ORD和IRD的协商是典型的,因此希望提供本文件所述的通用机制。

4.2. Enabling MPA Mode
4.2. 启用MPA模式

MPA defines encoding of DDP Segments in Framed Upper Layer Protocol PDUs (FULPDUs). Generation of FULPDUs requires the ability to periodically insert MPA Markers and to generate the MPA CRC-32c for each frame. Reception may require parsing/removing the markers after using them to identify MPA Frame boundaries and validation of the MPA-CRC32c.

MPA定义了帧上层协议PDU(FULPDU)中DDP段的编码。FULPDU的生成需要能够定期插入MPA标记,并为每个帧生成MPA CRC-32c。接收可能需要在使用标记识别MPA帧边界和验证MPA-CRC32c后解析/移除标记。

A major design objective for MPA was to ensure that the resulting TCP stream would be fully compliant for any and all TCP-aware middleboxes. The challenge is that while only some TCP payload streams are a valid stream of MPA FULPDUs, any sequence of bytes is a valid TCP payload stream. The determination that a given stream is in a specific MPA mode cannot be made at the MPA or TCP layer. Therefore, enabling of MPA mode is handled by the ULP.

MPA的一个主要设计目标是确保生成的TCP流完全符合任何和所有支持TCP的中间盒。挑战在于,虽然只有一些TCP有效负载流是MPA FULPDU的有效流,但任何字节序列都是有效的TCP有效负载流。无法在MPA或TCP层确定给定流处于特定MPA模式。因此,MPA模式的启用由ULP处理。

The MPA protocol can be viewed as having two parts:

MPA协议可分为两部分:

o a specification of generation and reception of MPA FULPDUs. This is unchanged by enhanced RDMA connection establishment.

o MPA FULPDUs的生成和接收规范。增强的RDMA连接建立不会改变这一点。

o a pre-MPA exchange of messages to enable a specific MPA mode for the TCP connection. Enhanced RDMA connection establishment extends this protocol with two new features.

o 为TCP连接启用特定MPA模式的消息预MPA交换。增强的RDMA连接建立通过两个新特性扩展了该协议。

In typical implementations, generation and reception of MPA FULPDUs is handled by hardware. The exchange of the MPA Request and Reply Frames is then handled by host software. As will be explained, this implementation split impedes applications that are not compatible with the client-server assumptions in the current MPA Request/Reply exchange.

在典型实现中,MPA FULPDU的生成和接收由硬件处理。MPA请求和应答帧的交换随后由主机软件处理。正如将要解释的那样,这种实现分离阻碍了与当前MPA请求/应答交换中的客户机-服务器假设不兼容的应用程序。

4.3. Lack of Explicit RTR in MPA Request/Reply Exchange
4.3. MPA请求/应答交换中缺少显式RTR

The exchange of MPA Request and Reply messages to place a TCP connection in MPA mode is specified in [RFC5044]. This protocol provides many benefits to the design of MPA FULPDU hardware:

[RFC5044]中规定了在MPA模式下交换MPA请求和回复消息以建立TCP连接。该协议为MPA FULPDU硬件的设计提供了许多好处:

o The ULP is responsible for specifying the exact MPA Mode (Markers enabled or disabled, CRC-32c enabled or suppressed) and the point in the TCP streams (inbound and outbound) where MPA Frames will begin.

o ULP负责指定确切的MPA模式(标记启用或禁用、CRC-32c启用或抑制)以及TCP流中MPA帧开始的点(入站和出站)。

o Before the first MPA Frame is transmitted, all pre-MPA mode TCP payloads will have been acknowledged by the peer. Therefore, it is never necessary to generate a retransmission that mixes pre-MPA and MPA payload.

o 在发送第一个MPA帧之前,对等方将确认所有MPA前模式TCP有效载荷。因此,不必生成混合了pre-MPA和MPA有效载荷的重传。

o Before MPA reception is enabled, all incoming pre-MPA mode TCP payloads will have been acknowledged. Therefore, the host will never receive a TCP segment that mixes pre-MPA and MPA payload.

o 在启用MPA接收之前,将确认所有传入的MPA前模式TCP有效载荷。因此,主机将永远不会收到混合了MPA前和MPA有效负载的TCP段。

The limitation of the current MPA Request/Reply exchange is that it does not define a Ready to Receive (RTR) indication that the active side would send, so that the passive side can know that the last non-MPA payload (the MPA Reply) had been received.

当前MPA请求/应答交换的限制在于,它没有定义主动方将发送的准备接收(RTR)指示,以便被动方可以知道最后一个非MPA有效负载(MPA应答)已被接收。

Instead, the role of an RTR indication is piggybacked on the first MPA FULPDU sent by the active side. This is actually a valuable optimization for all applications that fit the classic client-server model. The client only initiates the connection when it has a request to send to the server, and the server has nothing to send until it has received and processed the client request.

相反,RTR指示的作用由活动侧发送的第一个MPA FULPDU承载。对于所有适合经典客户机-服务器模型的应用程序来说,这实际上是一个有价值的优化。客户机只有在有请求发送到服务器时才启动连接,而服务器在接收并处理客户机请求之前无需发送任何内容。

Even applications where the server sends some configuration data immediately can easily send the same information as application Private Data in the MPA Reply. So the currently defined exchange works for almost all applications.

即使是服务器立即发送一些配置数据的应用程序也可以轻松地在MPA应答中发送与应用程序私有数据相同的信息。因此,当前定义的exchange适用于几乎所有的应用程序。

Many peer-to-peer applications, especially those involving cluster calculations (frequently using Message Passing Interface (MPI) [UsingMPI] or [RDS]), have no natural client or server roles ([PPMPI] [OpenMP]). Typically, one member of the cluster is arbitrarily selected to initiate the connection when the distributed task is launched, while the other accepts it. At startup time, however, there is no way to predict which node will have the first message to actually send. Immediately establishing the connections is valuable because it reduces latency once results are ready to transmit and it validates connectivity throughout the cluster.

许多对等应用程序,特别是涉及群集计算的应用程序(经常使用消息传递接口(MPI)[UsingMPI]或[RDS]),没有自然的客户端或服务器角色([PPMPI][OpenMP])。通常,当分布式任务启动时,集群的一个成员被任意选择以启动连接,而另一个成员则接受连接。但是,在启动时,无法预测哪个节点将实际发送第一条消息。立即建立连接很有价值,因为一旦结果准备好传输,它可以减少延迟,并验证整个集群的连接。

The lack of an explicit RTR indication in the MPA Request/Reply exchange forces all applications to have a first message from the connection initiator, whether or not this matches the application communication model.

MPA请求/应答交换中缺少明确的RTR指示,这迫使所有应用程序都具有来自连接启动器的第一条消息,无论该消息是否与应用程序通信模型匹配。

4.4. Limitations on ULP Workaround
4.4. ULP解决方案的限制

The requirement that the RDMA connection initiator sends the first message does not appear to be onerous on first examination. The natural question is why the application layer would not simply generate a dummy message when there is no other message to submit.

RDMA连接启动器发送第一条消息的要求在第一次检查时似乎并不繁重。自然的问题是,当没有其他消息要提交时,为什么应用程序层不会简单地生成一条伪消息。

There are three factors that make this workaround unsuitable for many peer-to-peer applications:

有三个因素使得此解决方案不适用于许多对等应用程序:

o Transport-Neutral APIs.

o 传输中立的API。

o Work/Completion Queue Accounting.

o 工作/完成队列记帐。

o Host-based implementation of MPA Fencing.

o MPA围栏的基于主机的实现。

4.4.1. Transport-Neutral APIs
4.4.1. 与传输无关的API

Many of these applications access RDMA services using a transport-neutral API such as [DAPL] or [OFA]. Only RDDP over TCP [RFC5044] has a first message requirement. Other RDMA transports, including RDDP over SCTP (see [RFC5043]) and InfiniBand (see [IBTA]), do not.

其中许多应用程序使用传输无关的API(如[DAPL]或[OFA])访问RDMA服务。只有TCP上的RDDP[RFC5044]有第一条消息要求。其他RDMA传输,包括SCTP上的RDDP(参见[RFC5043])和InfiniBand(参见[IBTA]),则没有。

Application or middleware communications can be expressed as transport-neutral RDMA operations, allowing lower software layers to translate to transport and device specifics. Having a distinct extra message that is required only for one transport undermines the application's goal of being transport neutral.

应用程序或中间件通信可以表示为与传输无关的RDMA操作,从而允许较低的软件层转换为传输和设备细节。只有一个传输需要一条独特的额外消息,这会破坏应用程序实现传输中立的目标。

4.4.2. Work/Completion Queue Accounting
4.4.2. 工作/完成队列记帐

RDMA local APIs conventionally use Work Queues to submit requests (Work Queue elements or WQEs) and to asynchronously receive completions (in Completion Queues or CQs).

RDMA本地API通常使用工作队列来提交请求(工作队列元素或WQE)和异步接收完成(在完成队列或CQ中)。

Each Work Request can generate a Completion Queue Entry (CQE). Completions for successful transmit Work Requests are frequently suppressed, but the CQ capacity must account for the possibility that each will complete in error. A CQ can receive completions from multiple Work Queues.

每个工作请求都可以生成完成队列条目(CQE)。成功传输工作请求的完成通常被抑制,但CQ容量必须考虑到每个请求都将错误完成的可能性。CQ可以从多个工作队列接收完成。

CQs are defined to allow hardware RDMA implementations to generate CQEs directly to a user-space-mapped buffer. This enables a user-space RDMA Consumer to reap completions without requiring kernel intervention.

CQ被定义为允许硬件RDMA实现直接向用户空间映射的缓冲区生成CQ。这使得用户空间RDMA使用者能够在不需要内核干预的情况下获得完成。

A hardware RDMA implementation cannot reasonably wait for an available slot in the CQ. The queue must be sized such that an overflow will not occur. When an overflow does occur, it is considered a catastrophic error and will typically require tearing down all RDMA connections using that CQ.

硬件RDMA实现不能合理地等待CQ中的可用插槽。队列的大小必须确保不会发生溢出。当溢出确实发生时,它被视为灾难性错误,通常需要使用该CQ断开所有RDMA连接。

This style of interface is very efficient, but places a burden on the application to properly size each CQ to match the Work Queues that feed it.

这种类型的接口非常有效,但会给应用程序带来负担,使其无法正确调整每个CQ的大小,以匹配为其提供服务的工作队列。

While the format of both WQEs and CQEs is transport and device dependent, a transport-neutral API can deal with WQEs and CQEs as abstract transport- and device-neutral objects. Therefore, the number of WQEs and CQEs required for an application can be transport and device neutral.

虽然WQE和CQE的格式都依赖于传输和设备,但传输无关API可以将WQE和CQE作为抽象的传输和设备无关对象处理。因此,应用所需的wqe和cqe的数量可以是传输和设备中性的。

The capacity of the Work Queues and CQs can be calculated in an abstract transport- and device-neutral fashion. If a dummy operation approach is used, it would require lower layers to know the usage model, and would disrupt the calculations by inserting a dummy "operation" Work Request and filtering out the matching completion. The lower layer does not know the usage model on which the queue sizes are built, nor does it know how frequently an insertion will be required.

工作队列和CQ的容量可以以抽象的传输和设备中立的方式计算。如果使用虚拟操作方法,则需要较低层了解使用模型,并通过插入虚拟“操作”工作请求并过滤出匹配的完成来中断计算。下层不知道队列大小是基于什么使用模型构建的,也不知道插入的频率。

4.4.3. Host-based Implementation of MPA Fencing
4.4.3. 基于主机的MPA围栏实现

Many hardware implementations of RDDP using MPA/TCP do not handle the MPA Request/Reply exchange in hardware, rather they are handled by the host processor in software. With such designs, it is common for the MPA Fencing to be implemented in the user-space, device-specific library (commonly referred to as a 'User Verbs' library or module).

许多使用MPA/TCP的RDDP硬件实现并没有在硬件中处理MPA请求/应答交换,而是由主机处理器在软件中处理。对于此类设计,MPA围栏通常在用户空间、设备特定库(通常称为“用户动词”库或模块)中实现。

When the generation and reception of MPA FULPDUs are already dedicated to hardware, a Work Completion can only be generated by an untagged message, since arrival of a message for a tagged buffer does not necessarily generate a completion and is done without any interaction with ULP [RFC5040].

当MPA FULPDU的生成和接收已经专用于硬件时,工作完成只能由未标记的消息生成,因为标记缓冲区的消息的到达不一定生成完成,并且在不与ULP进行任何交互的情况下完成[RFC5040]。

5. Enhanced MPA Connection Establishment
5. 增强MPA连接建立

Below we provide an overview of Enhanced Connection Setup. The goal is to allow standard negotiation of the ORD/IRD setting on both sides of the RDMA connection and/or to negotiate the initial data transfer operation by the initiator when the existing 'client sends first' rule does not match application requirements.

下面我们提供增强连接设置的概述。目标是允许在RDMA连接两侧对ORD/IRD设置进行标准协商,和/或在现有“客户机先发送”规则不符合应用程序要求时协商发起方的初始数据传输操作。

The RDMA connection initiator sends an MPA Request, as specified in [RFC5044]; the new format defined here allows for:

RDMA连接启动器发送MPA请求,如[RFC5044]中所述;此处定义的新格式允许:

o Standardized negotiation of ORD and IRD.

o ORD和IRD的标准化谈判。

o Negotiation of RTR functionality and the RDMA message type to use as the RTR indication.

o 协商RTR功能和用作RTR指示的RDMA消息类型。

The RDMA connection responder processes the MPA Request and generates an MPA Reply, as specified in [RFC5044]; the new format completes the negotiation.

RDMA连接响应程序处理MPA请求并生成MPA回复,如[RFC5044]中所述;新格式完成了协商。

The local interface needs to provide a way for a ULP to request the use of explicit RTR indication on a per-application or per-connection basis when an explicit RTR indication will be required. Piggybacking the RTR on a Client's first message is a valuable optimization for most connections.

当需要显式RTR指示时,本地接口需要为ULP提供一种方式,以请求在每个应用程序或每个连接的基础上使用显式RTR指示。对大多数连接来说,在客户端的第一条消息上搭载RTR是一种有价值的优化。

The RDMA connection initiator MUST NOT allow any later FULPDUs to be transmitted before the RTR indication. One method to achieve this is to delay notifying the ULP that the RDMA connection has been established until after any required RTR indication has been transmitted.

RDMA连接启动器不得允许在RTR指示之前传输任何后续FULPDU。实现这一点的一种方法是延迟通知ULP RDMA连接已经建立,直到任何所需的RTR指示已经发送。

All MPA exchanges are performed via TCP prior to RDMA establishment, and are therefore signaled via TCP and not via RDMA completion.

所有MPA交换在RDMA建立之前通过TCP执行,因此通过TCP而不是RDMA完成发出信号。

6. Enhanced MPA Request/Reply Frames
6. 增强的MPA请求/应答帧

Enhanced RDMA connection establishment uses an alternate format for MPA Requests and Replies as follows:

增强型RDMA连接建立使用MPA请求和回复的替代格式,如下所示:

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    0  |                                                               |
       +         Key (16 bytes containing "MPA ID Req Frame")          +
    4  |      (4D 50 41 20 49 44 20 52 65 71 20 46 72 61 6D 65)        |
       +         Or  (16 bytes containing "MPA ID Rep Frame")          +
    8  |      (4D 50 41 20 49 44 20 52 65 70 20 46 72 61 6D 65)        |
       +                                                               +
    12 |                                                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    16 |M|C|R|S| Res   |     Rev       |          PD_Length            |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                                                               |
       ~                                                               ~
       ~                   Private Data                                ~
       |                                                               |
       |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        
        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    0  |                                                               |
       +         Key (16 bytes containing "MPA ID Req Frame")          +
    4  |      (4D 50 41 20 49 44 20 52 65 71 20 46 72 61 6D 65)        |
       +         Or  (16 bytes containing "MPA ID Rep Frame")          +
    8  |      (4D 50 41 20 49 44 20 52 65 70 20 46 72 61 6D 65)        |
       +                                                               +
    12 |                                                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    16 |M|C|R|S| Res   |     Rev       |          PD_Length            |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                                                               |
       ~                                                               ~
       ~                   Private Data                                ~
       |                                                               |
       |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        

Key: Unchanged from [RFC5044].

键:与[RFC5044]相同。

M: Unchanged from [RFC5044].

M:与[RFC5044]相同。

C: Unchanged from [RFC5044].

C:与[RFC5044]相同。

R: Unchanged from [RFC5044].

R:与[RFC5044]相同。

S: One, if the Private Data begins with the enhanced RDMA connection establishment data; 0 otherwise.

S:一,如果私有数据以增强的RDMA连接建立数据开始;否则为0。

Res: One bit smaller than in [RFC5044]; otherwise unchanged. In [RFC5044], the 'Res' field, in which the newly defined 'S' bit resides, is reserved for future use. [RFC5044] specifies that 'Res' MUST be set to zero when sending and MUST NOT be checked on reception, making use of 'S' bit backwards compatibility with the original MPA Frame format. When the 'S' bit is set to zero, no additional Private Data is used for enhanced RDMA connection establishment; therefore, the resulting MPA Request and Reply Frames are identical to the unenhanced protocol.

Res:比[RFC5044]中小一位;其他方面不变。在[RFC5044]中,新定义的“S”位所在的“Res”字段保留供将来使用。[RFC5044]指定发送时必须将“Res”设置为零,并且在接收时不得检查,利用“S”位与原始MPA帧格式的向后兼容性。当“S”位设置为零时,没有额外的专用数据用于增强的RDMA连接建立;因此,得到的MPA请求和应答帧与未增强协议相同。

Rev: This field contains the revision of MPA. To use any enhanced connection establishment feature, this MUST be set to two or higher. If no enhanced connection establishment features are desired, it MAY be set to one. A host accepting MPA connections MUST continue to accept MPA Requests with version one, even if it supports version two.

版次:此字段包含MPA的版次。要使用任何增强的连接建立功能,必须将其设置为2或更高。如果不需要增强的连接建立功能,可以将其设置为一个。接受MPA连接的主机必须继续接受版本1的MPA请求,即使它支持版本2。

PD_Length: Unchanged from [RFC5044]. This is the total length of the Private Data field, including the enhanced RDMA connection establishment data, if present.

PD_长度:从[RFC5044]起保持不变。这是专用数据字段的总长度,包括增强的RDMA连接建立数据(如果存在)。

Private Data: Unchanged from [RFC5044]. However, if the 'S' flag is set, Private Data MUST begin with enhanced RDMA connection establishment data (see Section 9).

私有数据:从[RFC5044]起保持不变。但是,如果设置了“S”标志,则私有数据必须以增强的RDMA连接建立数据开始(参见第9节)。

7. Enhanced SCTP Session Control Chunks
7. 增强的SCTP会话控制块

Enhanced RDMA connection establishment uses the first 32 bits of the Private Data field for IRD and ORD negotiation in the "DDP Stream Session Initiate" and "DDP Stream Session Accept" SCTP Session Control Chunks.

增强型RDMA连接建立在“DDP流会话启动”和“DDP流会话接受”SCTP会话控制块中使用专用数据字段的前32位进行IRD和ORD协商。

The type of the SCTP Session Control Chunk is defined by a Function Code (see [RFC4960]). [RFC5043] already defines codes for 'DDP Stream Session Initiate' and 'DDP Stream Session Accept', which are equivalent to an MPA Request Frame and an accepting MPA Reply Frame.

SCTP会话控制块的类型由功能代码定义(请参见[RFC4960])。[RFC5043]已经定义了“DDP流会话启动”和“DDP流会话接受”的代码,它们相当于MPA请求帧和接受MPA应答帧。

Enhanced RDMA connection establishment requires three additional function codes listed below:

增强的RDMA连接建立需要以下列出的三个附加功能代码:

Enhanced DDP Stream Session Initiate: 0x005

增强的DDP流会话启动:0x005

Enhanced DDP Stream Session Accept: 0x006

增强的DDP流会话接受:0x006

Enhanced DDP Stream Session Reject: 0x007

增强的DDP流会话拒绝:0x007

The Enhanced Reject function code MUST be used to indicate rejection of enhanced DDP stream session for a configuration that would have been accepted for unenhanced DDP stream session negotiation.

“增强拒绝”功能代码必须用于指示拒绝接受非增强DDP流会话协商的配置的增强DDP流会话。

The enhanced DDP stream session establishment follows the same rules as the standard DDP stream session establishment as defined in [RFC5043]. ULP-supplied Private Data MUST be included for Enhanced DDP Stream Session Initiate, Enhanced DDP Stream Session Accept, and Enhanced DDP Stream Session Reject messages, and MUST follow the enhanced RDMA connection establishment data in the DDP Stream Session Initiate and the Enhanced DDP Stream Session Accept messages.

增强的DDP流会话建立遵循与[RFC5043]中定义的标准DDP流会话建立相同的规则。对于增强DDP流会话启动、增强DDP流会话接受和增强DDP流会话拒绝消息,必须包含ULP提供的专用数据,并且必须遵循DDP流会话启动和增强DDP流会话接受消息中的增强RDMA连接建立数据。

Private Data length MUST NOT exceed 512 bytes in any message, including enhanced RDMA connection establishment data.

任何消息中的私有数据长度不得超过512字节,包括增强的RDMA连接建立数据。

Private Data MUST NOT be included in the DDP Stream Session TERM message.

专用数据不得包含在DDP流会话术语消息中。

Received Extended DDP Stream Session Control messages SHOULD be reported to the ULP. If reported, any supplied Private Data MUST be available for the ULP to examine. For example, a received Extended DDP Stream Session Control message is not reported to ULP if none of the requested RTR indication types are supported by the receiver. In this case, the Provider MAY generate a reject reply message indicating which RTR indication types it supports.

收到的扩展DDP流会话控制消息应报告给ULP。如果报告,任何提供的私有数据必须可供ULP检查。例如,如果接收机不支持任何请求的RTR指示类型,则接收到的扩展DDP流会话控制消息不会报告给ULP。在这种情况下,提供者可以生成拒绝回复消息,指示其支持的RTR指示类型。

The enhanced DDP stream management MUST use the DDP stream session termination function code to terminate a stream established using enhanced DDP stream session function codes.

增强型DDP流管理必须使用DDP流会话终止功能代码来终止使用增强型DDP流会话功能代码建立的流。

[RFC5043] already supports either side sending the first DDP Message since the Payload Protocol Identifier (PPID) already distinguishes between Session Establishment and DDP Segments. The enhanced RDMA connection establishment provides the ULP a transport-independent way to support the peer-to-peer model.

[RFC5043]已经支持任何一方发送第一条DDP消息,因为有效负载协议标识符(PPID)已经区分了会话建立和DDP段。增强的RDMA连接建立为ULP提供了一种独立于传输的方式来支持对等模型。

The following additional Legal Sequences of DDP Stream Session messages are defined:

定义了以下DDP流会话消息的附加合法序列:

o Enhanced Active/Passive Session Accepted: as with Section 6.2 of [RFC5043], but with the extended opcodes as defined in this document.

o 接受增强型主动/被动会话:与[RFC5043]第6.2节相同,但使用本文件中定义的扩展操作码。

o Enhanced Active/Passive Session Rejected: as with Section 6.3 of [RFC5043], but with the extended opcodes as defined in this document.

o 增强型主动/被动会话被拒绝:与[RFC5043]第6.3节相同,但使用本文件中定义的扩展操作码。

o Enhanced Active/Passive Session Non-ULP Rejected: as with Section 6.4 of [RFC5043], but with the extended opcodes as defined in this document.

o 增强型主动/被动会话非ULP被拒绝:与[RFC5043]第6.4节相同,但使用本文件中定义的扩展操作码。

8. MPA Error Reporting
8. MPA错误报告

The RDMA connection establishment protocol is layered upon the protocols defined in [RFC5040] and [RFC5041]. Any enhanced RDMA connection establishment error generates an MPA termination message to a peer. [RFC5040] defines a triplet of protocol layers, error types, and error codes for error specification. MPA negotiation for RDMA connection establishment uses the following layer and error type for MPA error reporting:

RDMA连接建立协议基于[RFC5040]和[RFC5041]中定义的协议分层。任何增强型RDMA连接建立错误都会向对等方生成MPA终止消息。[RFC5040]为错误规范定义了三重协议层、错误类型和错误代码。RDMA连接建立的MPA协商使用以下层和错误类型进行MPA错误报告:

   Layer:      0x2 - LLP Error Type: 0x0 - MPA
        
   Layer:      0x2 - LLP Error Type: 0x0 - MPA
        

While [RFC5044] defines four error codes, [RFC5043] does not define any. Enhanced RDMA connection establishment extends the error codes defined in [RFC5044] by adding three new error codes. Thus, enhanced RDMA connection establishment is backward compatible with both [RFC5043] and [RFC5044].

[RFC5044]定义了四个错误代码,而[RFC5043]没有定义任何错误代码。增强的RDMA连接建立通过添加三个新的错误代码扩展了[RFC5044]中定义的错误代码。因此,增强的RDMA连接建立与[RFC5043]和[RFC5044]向后兼容。

The following error codes are defined for enhanced RDMA connection establishment negotiation:

为增强RDMA连接建立协商定义了以下错误代码:

      Error Code         Description
      --------------------------------------------------------
      0x05               Local catastrophic
      0x06               Insufficient IRD resources
      0x07               No matching RTR option
        
      Error Code         Description
      --------------------------------------------------------
      0x05               Local catastrophic
      0x06               Insufficient IRD resources
      0x07               No matching RTR option
        
9. Enhanced RDMA Connection Establishment Data
9. 增强的RDMA连接建立数据

Enhanced RDMA connection establishment places the following 32 bits at the beginning of the Private Data field of the MPA Request and Reply Frames or the "DDP Stream Session Initiate" and "DDP Stream Session Accept" SCTP Session Control Chunks. ULP-specified Private Data follows this field. The maximum amount of ULP-specified Private Data is therefore reduced by 4 bytes. Note that this field MUST be sent in network byte order, with the IRD and ORD encoded as 14-bit unsigned integers.

增强的RDMA连接建立将以下32位放在MPA请求和应答帧或“DDP流会话启动”和“DDP流会话接受”SCTP会话控制块的私有数据字段的开头。此字段后面是ULP指定的专用数据。因此,ULP指定的私有数据的最大数量减少了4个字节。请注意,此字段必须以网络字节顺序发送,IRD和ORD编码为14位无符号整数。

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    0  |A|B|        IRD                |C|D|        ORD                |
    4  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        
        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    0  |A|B|        IRD                |C|D|        ORD                |
    4  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        

IRD: Inbound RDMA Read Queue Depth.

IRD:入站RDMA读取队列深度。

ORD: Outbound RDMA Read Queue Depth.

ORD:出站RDMA读取队列深度。

A: Control Flag for connection model.

A:连接模型的控制标志。

B: Control Flag for use of a zero-length FULPDU (Send) RTR indication.

B:用于零长度FULPDU(发送)RTR指示的控制标志。

C: Control Flag for use of a zero-length RDMA Write RTR indication.

C:使用零长度RDMA写入RTR指示的控制标志。

D: Control Flag for use of a zero-length RDMA Read RTR indication.

D:使用零长度RDMA读取RTR指示的控制标志。

9.1. IRD and ORD Negotiation
9.1. 税务局与税务局谈判

The IRD and ORD are used for negotiation of Inbound RDMA Read Request Queue depths for both endpoints of the RDMA connection. The IRD is used to configure the depth of the Inbound RDMA Read Request Queue (IRRQ) on each endpoint. ORD is used to limit the number of simultaneous outbound RDMA Read Requests allowed at any given point in time in order to avoid IRRQ overruns at the remote endpoint. In order to describe the negotiation of both local endpoint and remote endpoint ORD and IRD values, four terms are defined:

IRD和ORD用于协商RDMA连接的两个端点的入站RDMA读取请求队列深度。IRD用于配置每个端点上入站RDMA读取请求队列(IRRQ)的深度。ORD用于限制在任何给定时间点允许的同时出站RDMA读取请求的数量,以避免远程端点的IRRQ溢出。为了描述本地端点和远程端点ORD和IRD值的协商,定义了四个术语:

Initiator IRD: The IRD value sent in the MPA Request or "DDP Stream Session Initiate" SCTP Session Control Chunk. This is the value of the initiator's IRD at the time of the MPA Request generation. The responder sets its local ORD value to this value or less. The initiator IRD is the maximum number of simultaneous inbound RDMA Read Requests that the initiator can support for the requested connection.

启动器IRD:在MPA请求或“DDP Stream Session Initiate”SCTP会话控制块中发送的IRD值。这是生成MPA请求时启动器的IRD值。响应程序将其本地ORD值设置为该值或更小。启动器IRD是启动器可为请求的连接支持的最大同时入站RDMA读取请求数。

Initiator ORD: The ORD value in the MPA Request or "DDP Stream Session Initiate" SCTP Session Control Chunk. This is the initial value of the initiator's ORD at the time of the MPA Request generation and also a request to the responder to support a responder IRD of at least this value. The initiator ORD is the maximum number of simultaneous outbound RDMA Read operations that the initiator desires the responder to support for the requested connection.

启动器ORD:MPA请求或“DDP流会话启动”SCTP会话控制块中的ORD值。这是MPA请求生成时发起方ORD的初始值,也是对响应方的请求,以支持至少该值的响应方IRD。启动器ORD是启动器希望响应程序支持请求连接的最大并发出站RDMA读取操作数。

Responder IRD: The IRD value returned in the MPA Reply or "DDP Stream Session Accept" SCTP Session Control Chunk. This is the actual value that the responder sets for its local IRD. This value is greater than or equal to the initiator ORD for successful negotiations. The responder IRD is the maximum number of simultaneous inbound RDMA Read Requests that the responder actually can support for the requested connection.

响应者IRD:MPA应答或“DDP流会话接受”SCTP会话控制块中返回的IRD值。这是响应程序为其本地IRD设置的实际值。此值大于或等于成功协商的发起人ORD。响应程序IRD是响应程序实际可支持的请求连接的最大同时入站RDMA读取请求数。

Responder ORD: The ORD value returned in the MPA Reply or "DDP Stream Session Accept" SCTP Session Control Chunk. This is the actual value that the responder used for ORD and is less than or equal to the initiator IRD for successful negotiations. The responder ORD is the maximum number of simultaneous outbound RDMA Read operations that the responder will allow for the requested connection.

响应者ORD:MPA应答或“DDP流会话接受”SCTP会话控制块中返回的ORD值。这是响应方用于ORD的实际值,小于或等于成功协商的发起方IRD。响应程序ORD是响应程序将允许请求的连接同时进行出站RDMA读取操作的最大数量。

The relationships between these parameters after a successful negotiation is complete are the following:

成功协商完成后,这些参数之间的关系如下:

initiator ORD <= responder IRD

发起方ORD<=响应方IRD

responder ORD <= initiator IRD

响应者ORD<=发起者IRD

The responder and initiator MUST pass the peer's provided IRD and ORD values to the ULP, in addition to using the values as calculated by the preceding rules.

响应方和发起方必须将对等方提供的IRD和ORD值传递给ULP,此外还必须使用根据上述规则计算的值。

The responder ORD SHOULD be set to a value less than or equal to the initiator IRD. If the initiator ORD is insufficient to support the selected connection model, the responder IRD MAY be increased; for example, if the initiator ORD is 0 (RDMA Reads will not be used by the ULP) and the responder supports use of a zero-length RDMA Read RTR indication, then the responder IRD can be set to 1. The responder MUST set its ORD at most to the initiator IRD. The responder MAY reject the connection request if the initiator IRD is not sufficient for the ULP-required ORD and specify the required ORD in the MPA Reject Frame responder ORD. Thus, the TERM message MUST contain Layer 2, Error Type 0, Error Code 6.

响应程序ORD应设置为小于或等于启动器IRD的值。如果发起方ORD不足以支持所选连接模型,则可增加响应方IRD;例如,如果启动器ORD为0(ULP将不使用RDMA读取),并且响应程序支持使用零长度RDMA读取RTR指示,则响应程序IRD可以设置为1。响应者必须将其ORD最多设置为启动器IRD。如果发起方IRD不足以满足ULP要求的ORD,则响应方可拒绝连接请求,并在MPA拒绝帧响应方ORD中指定要求的ORD。因此,术语消息必须包含第2层,错误类型0,错误代码6。

Upon receiving the MPA Accept Frame from the responder, the initiator MUST set its IRD at least to the responder ORD and its ORD at most to the responder IRD. If the initiator does not have sufficient resources for the required IRD, it MUST send a TERM message to the responder indicating insufficient resources and terminate the connection due to insufficient resources. Thus, the TERM message MUST contain Layer 2, Error Type 0, Error Code 6.

从响应程序接收MPA接受帧后,发起程序必须将其IRD至少设置为响应程序ORD,将其ORD最多设置为响应程序IRD。如果发起方没有足够的资源用于所需的IRD,则必须向响应方发送一条表示资源不足的TERM消息,并由于资源不足而终止连接。因此,术语消息必须包含第2层,错误类型0,错误代码6。

The initiator MUST pass the responder provided IRD and ORD to the ULP for both MPA Accept and Reject messages. The initiator ULP can decide its course of action. For example, the initiator ULP may terminate the established connection and renegotiate the responder ORD.

发起者必须将响应者提供的IRD和ORD传递给ULP,以获得MPA接受和拒绝消息。启动器ULP可以决定其操作过程。例如,发起方ULP可以终止已建立的连接并重新协商响应方ORD。

An all ones value (0x3FFF) indicates that automatic negotiation of the IRD or ORD is not desired, and that the ULP will be responsible for it. The responder MUST respond to an initiator ORD value of 0x3FFF by leaving its local endpoint IRD value unchanged and setting the IRD to 0x3FFF in its reply message. The initiator MUST leave its local endpoint ORD value unchanged upon receiving a responder IRD value of 0x3FFF. The responder MUST respond to an initiator IRD value of 0x3FFF by leaving its local endpoint ORD value unchanged, and setting ORD to 0x3FFF in its reply message. The initiator MUST leave its local endpoint IRD value unchanged upon receiving a responder ORD value of 0x3FFF.

all ones值(0x3FFF)表示不需要IRD或ORD的自动协商,ULP将对此负责。响应程序必须通过保持其本地端点IRD值不变并在其回复消息中将IRD设置为0x3FFF来响应启动器ORD值0x3FFF。在接收到响应程序IRD值0x3FFF时,启动器必须保持其本地端点ORD值不变。响应程序必须通过保持其本地端点ORD值不变并在其回复消息中将ORD设置为0x3FFF来响应启动器IRD值0x3FFF。在接收到响应程序ORD值0x3FFF时,启动器必须保持其本地端点IRD值不变。

9.2. Peer-to-Peer Connection Negotiation
9.2. 对等连接协商

Control Flag A value 1 indicates that a peer-to-peer connection model is being performed, and value 0 indicates a client-server model. Control Flag B value 1 indicates that a zero-length FULPDU (Send) RTR indication is requested for the initiator and supported by the responder, respectively, 0 otherwise. Control Flag C value 1 indicates that a zero-length RDMA Write RTR indication is requested for the initiator and supported by the responder, respectively, 0 otherwise. Control Flag D value 1 indicates that a zero-length RDMA Read RTR indication is requested for the initiator and supported by the responder, respectively, 0 otherwise. The initiator MUST set Control Flag A to 1 for the peer-to-peer model. The initiator MUST set each Control Flag B, C, and D to 1 for each of the options it supports, if Control Flag A is set to 1.

控制标志A值1表示正在执行对等连接模型,值0表示客户端-服务器模型。控制标志B值1表示为启动器请求零长度FULPDU(发送)RTR指示,并分别由响应程序支持,否则为0。控制标志C值1表示发起者请求零长度RDMA写入RTR指示,响应者分别支持该指示,否则为0。控制标志D值1表示发起者请求了零长度RDMA Read RTR指示,响应者分别支持该指示,否则为0。发起方必须将对等模型的控制标志A设置为1。如果控制标志A设置为1,则启动器必须为其支持的每个选项将每个控制标志B、C和D设置为1。

The responder MUST support at least one RTR indication option if it supports Enhanced RDMA connection establishment. If Control Flag A is 1 in the MPA Request message, then the responder MUST set Control Flag A to 1 in the MPA reply message. For each initiator-supported RTR indication option, the responder SHOULD set the corresponding Control Flag if the responder can support that option in an MPA reply. The responder is not required to specify all RTR indication options it supports. The responder MUST set at least one RTR indication option if it supports more than one initiator-specified RTR indication option. The responder MAY include additional RTR indication options it supports, even if not requested by any initiator specified RTR indication options. If the responder does not support any of the initiator-specified RTR indication options, then the responder MUST set at least one RTR indication type option it supports.

如果响应程序支持增强的RDMA连接建立,则它必须至少支持一个RTR指示选项。如果MPA请求消息中的控制标志A为1,则响应者必须在MPA回复消息中将控制标志A设置为1。对于每个启动器支持的RTR指示选项,如果响应程序可以在MPA回复中支持该选项,则响应程序应设置相应的控制标志。响应者无需指定其支持的所有RTR指示选项。如果响应程序支持多个启动器指定的RTR指示选项,则响应程序必须至少设置一个RTR指示选项。响应者可包括其支持的附加RTR指示选项,即使未经任何启动器指定的RTR指示选项请求。如果响应程序不支持任何启动器指定的RTR指示选项,则响应程序必须至少设置一个其支持的RTR指示类型选项。

Upon receiving the MPA Accept Frame with Control Flag A set to 1, the initiator MUST generate one of the negotiated RTR indications. If the initiator is not able to generate any of the responder-supported RTR indications, then it MUST send a TERM message to the responder indicating failure to negotiate a mutually compatible connection model or RTR option, and terminate the connection. Thus, the TERM message MUST contain Layer 2, Error Type 0, Error Code 7. The ULP can negotiate a ULP-level RTR indication when a Provider-level RTR indication cannot be negotiated.

收到控制标志A设置为1的MPA接受帧后,启动器必须生成一个协商RTR指示。如果发起方无法生成响应方支持的任何RTR指示,则必须向响应方发送一条TERM消息,指示未能协商相互兼容的连接模型或RTR选项,并终止连接。因此,术语消息必须包含第2层,错误类型0,错误代码7。当提供商级RTR指示无法协商时,ULP可以协商ULP级RTR指示。

The initiator MUST set Control Flag A to 0 for the client-server model. The responder MUST set Control Flag A to 0 if Control Flag A is 0 in the request. If Control Flag A is set to 0, then Control Flags B, C, and D MUST also be set to 0. On reception, if Control Flag A is set to 0, then Control Flags B, C, and D MUST be ignored.

启动器必须将客户端-服务器模型的控制标志A设置为0。如果请求中的控制标志A为0,则响应程序必须将控制标志A设置为0。如果控制标志A设置为0,则控制标志B、C和D也必须设置为0。接收时,如果控制标志A设置为0,则必须忽略控制标志B、C和D。

9.3. Enhanced Connection Negotiation Flow
9.3. 增强的连接协商流

The RTR indication type and ORD/IRD negotiation follows the following order:

RTR指示类型和ORD/IRD协商遵循以下顺序:

initiator (MPA Request) --> The initiator sets Control Flag A to 1 to indicate the peer-to-peer connection model and sets its initial IRD/ORD on the local endpoint of the connection. The initiator also sets Control Flags B, C, and D to 1 for each initiator-supported option of RTR indication.

启动器(MPA请求)-->启动器将控制标志A设置为1,以指示对等连接模型,并在连接的本地端点上设置其初始IRD/ORD。对于每个受启动器支持的RTR指示选项,启动器还将控制标志B、C和D设置为1。

responder (MPA Reply) <-- The responder matches the initiator's Control Flag A value and sets ORD/IRD to its local endpoint values based upon the initiator's initial ORD/IRD values and the number of simultaneous RDMA Read Requests required by the ULP. The responder sets Control Flags B, C, and D to 1 for each responder-supported option of RTR indication options for the peer-to-peer connection model. The responder also sets its IRD/ORD to actual values.

响应程序(MPA应答)<--响应程序匹配启动器的控制标志A值,并根据启动器的初始ORD/IRD值和ULP要求的同步RDMA读取请求数,将ORD/IRD设置为其本地端点值。响应程序为对等连接模型的每个响应程序支持的RTR指示选项将控制标志B、C和D设置为1。响应者还将其IRD/ORD设置为实际值。

initiator (First RDMA Message) --> After the initiator modifies its ORD/IRD to match the responder's values as stated above, the initiator sends the first message of the negotiated RTR indication option. If no matching RTR indication option exists, then the initiator sends a TERM message.

发起者(第一条RDMA消息)-->发起者修改其ORD/IRD以匹配上述响应者的值后,发起者发送协商RTR指示选项的第一条消息。如果不存在匹配的RTR指示选项,则启动器将发送一条TERM消息。

The initiator or responder MUST generate the TERM message that contains Layer 2, Error Type 0, Error Code 5 when it encounters any error locally for which the special Error Code is not defined in Section 8 before resetting the connection.

在重置连接之前,当启动器或响应程序在本地遇到第8节中未定义特殊错误代码的任何错误时,必须生成包含第2层、错误类型0、错误代码5的术语消息。

10. Interoperability
10. 互操作性

The initiator requests enhanced RDMA connection establishment by sending an enhanced RDMA establishment request; an enhanced responder is REQUIRED to respond with an enhanced RDMA connection establishment response, whereas an unenhanced responder treats the enhanced request as incorrectly formatted and closes the TCP connection. All responders are REQUIRED to issue unenhanced RDMA connection establishment responses in response to unenhanced RDMA connection establishment requests.

发起方通过发送增强的RDMA建立请求来请求增强的RDMA连接建立;增强的响应程序需要使用增强的RDMA连接建立响应进行响应,而未增强的响应程序将增强的请求视为格式错误并关闭TCP连接。所有响应者都需要发出未增强的RDMA连接建立响应,以响应未增强的RDMA连接建立请求。

The initiator MUST NOT use the enhanced RDMA connection establishment formats or function codes when no enhanced functionality is desired.

当不需要增强功能时,启动器不得使用增强的RDMA连接建立格式或功能代码。

The responder MUST continue to accept unenhanced connection requests.

响应者必须继续接受未增强的连接请求。

There are three initiator/responder cases that involve enhanced MPA: both the initiator and responder, only the responder, and only the initiator. The enhanced MPA Frame is defined by field 'S' set to 1.

有三种启动器/响应程序案例涉及增强MPA:启动器和响应程序、仅响应程序和仅启动器。增强MPA帧由设置为1的字段“S”定义。

Enhanced MPA initiator and responder: If the responder receives an enhanced MPA message, it MUST respond with an enhanced MPA message.

增强型MPA启动器和响应程序:如果响应程序收到增强型MPA消息,则必须使用增强型MPA消息进行响应。

Enhanced MPA responder only: If the responder receives an unenhanced MPA message ('S' is set to 0), it MUST respond with an unenhanced MPA message.

仅限增强型MPA响应程序:如果响应程序收到未增强型MPA消息(“S”设置为0),则必须使用未增强型MPA消息进行响应。

Enhanced MPA initiator only: If the responder receives an enhanced MPA message and it does not support enhanced RDMA connection establishment, it MUST close the TCP connection and exit MPA. From a standard RDMA connection establishment point of view, the enhanced MPA Frame is improperly formatted as stated in [RFC5044]. Thus, both the initiator and responder report TCP connection termination to an application locally. In this case, the initiator MAY attempt to establish an RDMA connection using the unenhanced MPA protocol as defined in [RFC5044] if this protocol is compatible with the application, and let the ULP deal with ORD and IRD and peer-to-peer negotiations.

仅限增强型MPA启动器:如果响应程序收到增强型MPA消息且不支持增强型RDMA连接建立,则必须关闭TCP连接并退出MPA。从标准RDMA连接建立的角度来看,增强型MPA帧的格式不正确,如[RFC5044]所述。因此,启动器和响应程序都会在本地向应用程序报告TCP连接终止。在这种情况下,发起者可以尝试使用[RFC5044]中定义的未增强MPA协议建立RDMA连接,前提是该协议与应用程序兼容,并让ULP处理ORD和IRD以及对等协商。

A note for potential future enhancements for connection establishment negotiation: It is possible to further extend formatting of Private Data of the MPA Request and Reply Frames and to use other bits from the "Res" field to indicate additional Private Data formatting.

连接建立协商的潜在未来增强注意事项:可以进一步扩展MPA请求和应答帧的私有数据格式,并使用“Res”字段中的其他位来指示其他私有数据格式。

11. IANA Considerations
11. IANA考虑

IANA has added the following entries to the "SCTP Function Codes for DDP Session Control" registry created by Section 3.5 of [RFC6580]:

IANA已将以下条目添加到[RFC6580]第3.5节创建的“DDP会话控制的SCTP功能代码”注册表中:

0x0005, Enhanced DDP Stream Session Initiate, [RFC6581]

0x0005,增强DDP流会话启动,[RFC6581]

0x0006, Enhanced DDP Stream Session Accept, [RFC6581]

0x0006,增强DDP流会话接受,[RFC6581]

0x0007, Enhanced DDP Stream Session Reject, [RFC6581]

0x0007,增强DDP流会话拒绝,[RFC6581]

IANA has added the following entries to the "MPA Errors" registry created by Section 3.3 of [RFC6580]:

IANA已将以下条目添加到[RFC6580]第3.3节创建的“MPA错误”注册表中:

   0x2/0x0/0x05,  - MPA Error / Local catastrophic error, [RFC6581]
        
   0x2/0x0/0x05,  - MPA Error / Local catastrophic error, [RFC6581]
        
   0x2/0x0/0x06  - MPA Error / Insufficient IRD resources, [RFC6581]
        
   0x2/0x0/0x06  - MPA Error / Insufficient IRD resources, [RFC6581]
        
   0x2/0x0/0x07  - MPA Error / No matching RTR option, [RFC6581]
        
   0x2/0x0/0x07  - MPA Error / No matching RTR option, [RFC6581]
        
12. Security Considerations
12. 安全考虑

The security considerations from RFC 5044 and RFC 5043 apply and the changes in this document do not introduce new security considerations. However, it is recommended that implementations do sanity checking for the input parameters, including ORD, IRD, and the control flags used for RTR indication option negotiation.

RFC 5044和RFC 5043中的安全注意事项适用,本文档中的更改不会引入新的安全注意事项。但是,建议实现对输入参数进行健全性检查,包括ORD、IRD和用于RTR指示选项协商的控制标志。

13. Acknowledgements
13. 致谢

The authors wish to thank Sean Hefty, Dave Minturn, Tom Talpey, David Black, and David Harrington for their valuable contributions and reviews of this document.

作者感谢Sean Hefty、Dave Minturn、Tom Talpey、David Black和David Harrington对本文件的宝贵贡献和评论。

14. References
14. 工具书类
14.1. Normative References
14.1. 规范性引用文件

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.

[RFC2119]Bradner,S.,“RFC中用于表示需求水平的关键词”,BCP 14,RFC 2119,1997年3月。

[RFC4960] Stewart, R., "Stream Control Transmission Protocol", RFC 4960, September 2007.

[RFC4960]Stewart,R.,“流控制传输协议”,RFC 49602007年9月。

[RFC5040] Recio, R., Metzler, B., Culley, P., Hilland, J., and D. Garcia, "A Remote Direct Memory Access Protocol Specification", RFC 5040, October 2007.

[RFC5040]Recio,R.,Metzler,B.,Culley,P.,Hilland,J.,和D.Garcia,“远程直接内存访问协议规范”,RFC 50402007年10月。

[RFC5041] Shah, H., Pinkerton, J., Recio, R., and P. Culley, "Direct Data Placement over Reliable Transports", RFC 5041, October 2007.

[RFC5041]Shah,H.,Pinkerton,J.,Recio,R.,和P.Culley,“可靠传输上的直接数据放置”,RFC 50412007年10月。

[RFC5043] Bestler, C. and R. Stewart, "Stream Control Transmission Protocol (SCTP) Direct Data Placement (DDP) Adaptation", RFC 5043, October 2007.

[RFC5043]Bestler,C.和R.Stewart,“流控制传输协议(SCTP)直接数据放置(DDP)自适应”,RFC 50432007年10月。

[RFC5044] Culley, P., Elzur, U., Recio, R., Bailey, S., and J. Carrier, "Marker PDU Aligned Framing for TCP Specification", RFC 5044, October 2007.

[RFC5044]Culley,P.,Elzur,U.,Recio,R.,Bailey,S.,和J.Carrier,“TCP规范的标记PDU对齐帧”,RFC 5044,2007年10月。

[RFC6580] Ko, M. and D. Black, "IANA Registries for the Remote Direct Data Placement (RDDP) Protocols", RFC 6580, April 2012.

[RFC6580]Ko,M.和D.Black,“远程直接数据放置(RDDP)协议的IANA注册”,RFC 65802012年4月。

14.2. Informative References
14.2. 资料性引用

[DAPL] "Direct Access Programming Library", <http://www.datcollaborative.org/uDAPL_doc_062102.pdf>.

[DAPL]“直接访问编程库”<http://www.datcollaborative.org/uDAPL_doc_062102.pdf>.

[IBTA] "InfiniBand Architecture Specification Release 1.2.1", <http://www.infinibandta.org>.

[IBTA]“InfiniBand体系结构规范1.2.1版”<http://www.infinibandta.org>.

[OFA] "OFA verbs & APIs", <http://www.openfabrics.org/>.

[OFA]“OFA动词和API”<http://www.openfabrics.org/>.

[OpenMP] McGraw-Hill, "Parallel Programming in C with MPI and OpenMP", 2003.

[OpenMP]McGraw-Hill,“使用MPI和OpenMP的C并行编程”,2003年。

[PPMPI] Morgan Kaufmann Publishers Inc., "Parallel Programming with MPI", 2008.

[PPMPI]摩根·考夫曼出版公司,“使用MPI的并行编程”,2008年。

[RDMAC] "RDMA Protocol Verbs Specification (Version 1.0)", <http://www.rdmaconsortium.org/home/ draft-hilland-iwarp-verbs-v1.0-RDMAC.pdf>.

[RDMAC]“RDMA协议动词规范(1.0版)”<http://www.rdmaconsortium.org/home/ 草稿-hilland-iwarp-verbs-v1.0-RDMAC.pdf>。

[RDS] Open Fabrics Association, "Reliable Datagram Socket", 2008, <http://www.openfabrics.org/archives/spring2008sonoma>.

[RDS]开放结构协会,“可靠数据报套接字”,2008年<http://www.openfabrics.org/archives/spring2008sonoma>.

[UsingMPI] MIT Press, "Using MPI-2: Advanced Features of the Message Passing Interface", 1999.

[使用MPI]麻省理工学院出版社,“使用MPI-2:消息传递接口的高级功能”,1999年。

[VIA] Cameron, Don and Greg Regnier, "Virtual Interface Architecture", Intel, April 2002.

[通过]Cameron,Don和Greg Regnier,“虚拟接口体系结构”,英特尔,2002年4月。

Authors' Addresses

作者地址

Arkady Kanevsky (editor) Dell Inc. One Dell Way, MS PS2-47 Round Rock, TX 78682 USA

Arkady Kanevsky(编辑)戴尔公司,美国德克萨斯州圆岩市,邮编:78682

   Phone: +1-512-728-0000
   EMail: arkady.kanevsky@gmail.com
        
   Phone: +1-512-728-0000
   EMail: arkady.kanevsky@gmail.com
        

Caitlin Bestler (editor) Nexenta Systems 555 E El Camino Real #104 Sunnyvale, CA 94087 USA

Caitlin Bestler(编辑)Nexenta Systems 555 E El Camino Real#104 Sunnyvale,CA 94087 USA

   Phone: +1-949-528-3085
   EMail: Caitlin.Bestler@nexenta.com
        
   Phone: +1-949-528-3085
   EMail: Caitlin.Bestler@nexenta.com
        

Robert Sharp Intel LAD High Performance Message Passing, Mailstop: AN1-WTR1 1501 South Mopac, Suite 400 Austin, TX 78746 USA

Robert Sharp Intel LAD高性能消息传递,邮箱:AN1-WTR1 1501 South Mopac,美国德克萨斯州奥斯汀400号套房,邮编78746

   Phone: +1-512-493-3242
   EMail: robert.o.sharp@intel.com
        
   Phone: +1-512-493-3242
   EMail: robert.o.sharp@intel.com
        

Steve Wise Open Grid Computing 4030 Braker Lane STE 130 Austin, TX 78759 USA

Steve Wise开放网格计算美国德克萨斯州奥斯汀Braker Lane街4030号,邮编:78759

   Phone: +1-512-343-9196 x101
   EMail: swise@opengridcomputing.com
        
   Phone: +1-512-343-9196 x101
   EMail: swise@opengridcomputing.com