Monitor RoCE and configuration examples for Lenovo Switch
*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***
Performance Monitor
Monitor Priority and Pause Frames for RDMA (RoCE) on Lenovo Switch.
Microsoft references:
Lenovo:
Lenovo System X
******************************************************************************
Lenovo Display Commands
******************************************************************************
Editing is done below, removed lines with repeated information replaced by x (port numbers)
Note:
G8332 running CNOS 10.4.3 – Syntax changed with CNOS 10.7 and higher
The Example use priorities 3 and 5 (The Cluster traffic just need to have the highest priority 5 or 7)
******************************************************************************
S2D# display cee ets information
ETS Global Admin Configuration:
PGID BW% COSq Priorities Description
==============================================================================
0 49 0 0 1 2 Default
1 0 NA
2 0 2 4 6 7
3 50 3 3 RoCEv2_Traffic (SMB)
4 0 4
5 1 5 5 Cluster
6 0 6
7 0 7
15 NA 1
ETS Global Operational Configuration:
PGID BW% COSq Priorities Description
==============================================================================
0 49 0 0 1 2 Default
1 0 NA
2 0 2 4 6 7
3 50 3 3 RoCEv2_Traffic (SMB)
4 0 4
5 1 5 5 Cluster
6 0 6
7 0 7
15 NA 1
ETS Interface Admin Configuration
--------------------
ETS Willing mode is not supported on the Switch.
Interface Advertise
================================================================================
Ethernet1/x On
Ethernet1/x On
******************************************************************************
S2D# display cee pfc
Global Admin PFC State: On
Priority State Description
============================
0 Dis
1 Dis
2 Dis
3 Ena RoCEv2_Priority (SMB)
4 Dis
5 Dis Cluster
6 Dis
7 Dis
Admin Configuration:
--------------------
PFC Willing mode is not supported on the Switch.
Interface State Advertise Cap Priorities
================================================================================
Ethernet1/x On On 2 3 5
Ethernet1/x On On 2 3 5
--------------------------------------------------------------------------------
Operational Configuration
-------------------------
Global Operational PFC State: On
Priority State Description
============================
0 Dis
1 Dis
2 Dis
3 Ena RoCEv2_Priority (SMB)
4 Dis
5 Dis Cluster
6 Dis
7 Dis
Interface State Cap Priorities
================================================================================
Ethernet1/x On 2 3 5
Ethernet1/x On 2 3 5
******************************************************************************
Ports used for S2D (PFC) need to show traffic/Packets on Rx/Tx
S2D# display cee pfc counters
Priority Flow Control Packets:
Interface Received(Rx) Sent(Tx)
=======================================================
Ethernet1/x 0 0
Ethernet1/x 0 0
Ethernet1/7 4988911 17067089
Ethernet1/8 1852123 12482442
Ethernet1/9 9800282 12982588
Ethernet1/10 5229729 20209402
Ethernet1/11 7244549 16976660
Ethernet1/x 0 0
Ethernet1/23 12245734 14795755
Ethernet1/24 12245683 8309129
Ethernet1/x 0 0
******************************************************************************
S2D# display cee interface Ethernet 1/7
ETS information for the interface Ethernet1/7:
ETS Global Admin Configuration:
PGID BW% COSq Priorities Description
==============================================================================
0 49 0 0 1 2 Default
1 0 NA
2 0 2 4 6 7
3 50 3 3 RoCEv2_Traffic (SMB)
4 0 4
5 1 5 5 Cluster
6 0 6
7 0 7
15 NA 1
ETS Global Operational Configuration:
PGID BW% COSq Priorities Description
==============================================================================
0 49 0 0 1 2 Default
1 0 NA
2 0 2 4 6 7
3 50 3 3 RoCEv2_Traffic (SMB)
4 0 4
5 1 5 5 Cluster
6 0 6
7 0 7
15 NA 1
Admin Configuration
--------------------
Advertise Willing
================================================================================
On Off
Operational Configuration
--------------------
Advertise Willing
================================================================================
Off Off
PFC information for the interface Ethernet1/7:
Admin Configuration
--------------------
Interface State Advertise Cap Priorities
================================================================================
Ethernet1/7 On On 2 3 5
--------------------------------------------------------------------------------
Operational Configuration
-------------------------
State Cap Priorities
================================================================================
On 2 3 5
******************************************************************************
