Featured Post

YouTube and link library for S2D.dk

2019/10/09

Monitor RoCE - HPE

Monitor RoCE and configuration examples for HPE Switch

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

Performance Monitor
Monitor Priority and Pause Frames for RDMA (RoCE) on HPE Switch

Microsoft references:
HPE references:

HPE FlexFabric 5950 Command Reference

S2D# display priority-flow-control interface HundredGigE1/1/x
(List the PFC Settings and the Priority and Pause Frames (Recv/Send))

Auto: Specifies PFC in auto mode. In this mode, the Ethernet interface automatically negotiates the PFC status with its peer over LLDP.
Enable: Forcibly enables PFC.

Interface           AdminMode  OperMode  Dot1pList   Prio  Recv       Send
--------------------------------------------------------------------------------
HGE1/1/x            Enabled    Enabled   3           3     176597     34268

S2D# reset counters interface HundredGigE1/1/x
(To reset the Recv/Send counters)

******************************************************************************
HPE Switch Port Example for PFC, Priority 3 SMB traffic:
******************************************************************************
interface HundredGigE1/1/x
port link-mode bridge
description S2D Storage Port
port link-type trunk
port trunk permit vlan <S2D Storage vlan list>
stp edged-port
priority-flow-control enable
priority-flow-control no-drop dot1p 3
qos trust dot1p

***
ETS
***

Draft - need to be validated

Microsoft use ETS 3 for SMB (50% bandwidth in the below example) and 7 for Cluster traffic
Note: Priority 7 dont need PFC only the ETS


qos map-table dotlp-lp
import 0 export 0
import 2 export 0
import 3 export 1
import 4 export 0
import 5 export 0
import 6 export 0
import 7 export 2


interface interface HundredGigE1/1/x 
qos wrr be group 1 byte-count 15
qos wrr af1 group 1 byte-count 15
qos wrr af2 group sp
qos wrr af3 group sp
qos wrr af4 group sp
qos wrr ef group sp
qos wrr ca6 group sp
qos wrr ca7 group sp

******************************************************************************
Switch Configuration for only Storage Traffic
ETS 95% for SMB Traffic on the Host
ETS 1% for Cluster Traffic on the Host
SP = Strict Priority

List the HPE TCs:
qos wrr be group
qos wrr af1 group
qos wrr af2 group
qos wrr af3 group
qos wrr af4 group
qos wrr ef group
qos wrr ca6 group
qos wrr ca7 group
******************************************************************************

Draft - need to be validated

qos map-table dotlp-lp
import 0 export 0
import 2 export 0
import 3 export 1
import 4 export 0
import 5 export 0
import 6 export 0
import 7 export 2


interface interface HundredGigE1/1/x 
qos wrr af1 group sp
qos wrr af2 group sp