Featured Post

YouTube and link library for S2D.dk


Monitor RoCE - Cisco

Monitor RoCE and configuration examples for Cisco Switch

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.

Performance Monitor
Monitor Priority and Pause Frames for RDMA (RoCE) on Cisco Switch

Microsoft references:

Cisco Nexus 9000 Series Configuring Priority Flow Control

S2D# show interface ethernet 1/x priority-flow-control
    If you do not specify an interface (ethernet x/x) the show interface priority-flow-control command will display the priority flow control information for all interfaces (layer 2 and layer 3).

    S2D# show interface priority-flow-control
      Example, Port 3 and 4 has PFC On/On (no Auto/On), and we can see it receive and send (Rx/Tx) Pause Frames

      S2D# show interface priority-flow-control
      Port Mode Oper(VL bmap) RxPPP TxPPP

      Ethernet1/1 Auto Off 0 0
      Ethernet1/2 Auto Off 0 0
      Ethernet1/3 On   On (0) 13734 2546
      Ethernet1/4 On   On (0) 13865 2634
      Ethernet1/5 Auto Off 0 0
      Ethernet1/6 Auto Off 0 0
      Ethernet1/7 Auto Off 0 0


      • Port specific:
        • switchport mode trunk
        • switchport trunk native vlan <Your native vlan, Default is 1>
        • switchport trunk allowed vlan <Your vlan list>
        • spanning-tree port type edge
        • flowcontrol receive off
        • flowcontrol send off
        • priority-flow-control mode on
        • no shutdown
      • Global:
        • class-map type qos match-all RDMA
        • match cos 3
        • class-map type queuing RDMA
        • match qos-group 3
        • policy-map type qos QOS_MARKING
        • class RDMA
        • set qos-group 3
        • class class-default
        • policy-map type queuing QOS_QUEUEING
        • class type queuing RDMA
        • bandwidth percent 50
        • class type queuing class-default
        • bandwidth percent 50
        • class-map type network-qos RDMA
        • match qos-group 3
        • policy-map type network-qos QOS_NETWORK
        • class type network-qos RDMA
        • mtu xxxx (Please check with pNIC settings*)
        • pause no-drop
        • class type network-qos class-default
        • mtu 9216 (Please check with pNIC settings, pNIC for Windows Jumbo Frames is normally 9014, the Physical Switch end-to-end connection is recommended to be the max size 9216)
        • system qos
        • service-policy type qos input QOS_MARKING
        • service-policy type queuing output QOS_QUEUEING
        • service-policy type network-qos QOS_NETWORK