Featured Post

YouTube and link library for S2D.dk

2019/10/09

RoCE Troubleshooting

RDMA/RoCE Troubleshooting

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

RoCE Troubleshooting

The RDMA Activity Performance Counter can give you some useful information

RDMA Accepted Connections will increase for each new connection. The Picture show a lab system with a RoCE Error. RDMA Accepted Connections will normally only change if you move VMs (SOFS), disable/enable pNIC/vNIC or reboot a Node.

The picture show that I have thousands of Accepted Connections and after just a few seconds I have 4 Accepted Connections. With no change in the Cluster.

Active Connections

Show the Active Connections on each vNIC/pNIC to see the connect use the "netstat -xan"













Monitor RoCE - Windows Server

Performance Monitor and examples of Physical Switch configuration for Mellanox, Cisco, HPE, Fujitsu and Dell

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

Performance Monitor

Monitor Priority and Pause Frames for RDMA (RoCE) with Mellanox QoS

Examples of Physical Switch configuration for Mellanox, Cisco, HPE, Fujitsu and Dell

For all the physical switch examples, always get your hardware vendor to verify your settings before use.
  1. Mellanox
  2. Cisco
  3. HPE
  4. Fujitsu
  5. Dell
  6. Lenovo
Lab setup:
Storages Spaces Direct (S2D) clusters with 2,4 or 6 nodes, Windows Server 2016 and Windows Server 2019
    The Host and Switch need to be configured for DCB/PFC/ETS
    Note: Microsoft does not currently support DCBx/LLDP if you use SET. You need to be sure that the Switch is not set to "PFC Auto" on the used ports. But use "PFC Enable" or "PFC On"

    Use the Microsoft Deployment Guide and Validate DCB Scripts for Host validation
    In the Examples we use Priority 3 for Storage and Priority 7 for Cluster Traffic.
    • Storage Traffic = SMB 445 Traffic (Storage, CSV and Live Migration)
      • Storage Traffic need PFC/ETS
    • Cluster Traffic = UDP 3343 Traffic
      • Cluster Traffic only use the ETS
    In the lab I use Jumbo Frames for Ethernet and auto size for RDMA.
    (RDMA/RoCE Max Frame Size is in Auto Mode).

    Sent Pause Frames
    The total number of pause frames sent from this priority to the far-end port. The untagged instance indicates the number of global pause frames that were sent.
    Sent Pause Duration
    The total duration of packets transmission being paused on this priority in microseconds.
    Received Pause Frames
    The number of pause frames that were received to this priority from the far-end port. The untagged instance indicates the number of global pause frames that were received.
    Received Pause Duration
    The total duration that far-end port was requested to pause for the transmission of packets in microseconds.
    Sent Discard Frames 
    The number of packets discarded by the transmitter.  Note: this counter is per Traffic Class (TC) and not per priority.


    Performance Monitor:
    The pictures below shows the Priority 3 (SMB) and also Priority 7 (Cluster)
    (There need to be high load on the Physical Switch Port and/or Network Adapter before you will see that values change for Pause Frames).

    New-NetQosPolicy "Cluster" -Cluster -PriorityValue8021Action 7
    New-NetQosPolicy "SMB" -NetDirectPortMatchCondition 445 -PriorityValue8021Action 3
    New-NetQosPolicy "DEFAULT" -Default -PriorityValue8021Action 0

    Set-VMNetworkAdapter -ManagementOS -Name "SMB1" -IeeePriorityTag on
    Set-VMNetworkAdapter -ManagementOS -Name "SMB2" -IeeePriorityTag on




    Performance Monitor:
    The pictures below shows the Priority 3 (SMB) and Pause Frames (Rcv/Sent) on the pNICs with 10 minuttes between each other. We can see that the Frames increased, so the system sent and recived it correct. (There need to be high load on the Physical Switch Port and/or Network Adapter before you will see that values change).














    Monitor RoCE - Dell

    Monitor RoCE and configuration examples for Dell Switch

    *** Disclaimer ***
    s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
    For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
    The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
    ***

    ******************************************************************************
    Dell Show Commands:
    ******************************************************************************


    Microsoft references:
    Show DCB and ETS for Dell Switch:

    S2D# show dcb
    S2D# show qos priority-groups
    S2D# show qos dcb-map RDMA
    S2D# show qos dot1p-queue-mapping

    S2D ports

    S2D# show interfaces tengigabitethernet 1/x pfc summary
    S2D# show interfaces tengigabitethernet 1/x pfc statistics
    S2D# show interfaces tengigabitethernet 1/x ets summary
    S2D# show interfaces tengigabitethernet 1/x ets detail

    S2D# show stack-unit all stack-ports all pfc details
    S2D# show stack-unit all stack-ports all ets details

    ******************************************************************************
    Dell switch configuration example (S4810):

    ******************************************************************************

    S2D# dcb enable

    *** Example #1 ***

    Example for S2D/SOFS ports for Storage only:
    PFC control on QoS Class 3 (SMB)

    Note: Host have more pNICs

    S2D# configure
    S2D (Config)# dcb-map dcb-RDMA
    S2D# priority-group 0 bandwidth 5 pfc off
    S2D# priority-group 3 bandwidth 95 pfc on
    S2D# priority-pgid 0 0 0 3 0 0 0 0
    S2D# exit


    *** Example #2 ***

    Example for ports for Storage and Virtual Guest (VM) traffic with one Physical Network Adapter (2 Ports):
    PFC control on QoS Class 3 (SMB) and Class 5 (Cluster)

    S2D# configure
    S2D (Config)# dcb-map dcb-RDMA
    S2D# priority-group 0 bandwidth 39 pfc off
    S2D# priority-group 3 bandwidth 60 pfc on
    S2D# priority-group 5 bandwidth 1 pfc off
    S2D# priority-pgid 0 0 0 3 0 5 0 0
    S2D# exit


    *** Example #3 ***

    Example for Host ports for Storage and Virtual Guest (VM) traffic with one Physical Network Adapter (2 Ports):
    PFC control on QoS Class 3 (SMB) and Class 7 (Cluster)


    S2D# configure
    S2D (Config)# dcb-map dcb-RDMA
    S2D# priority-group 0 bandwidth 39 pfc off
    S2D# priority-group 3 bandwidth 60 pfc on
    S2D# priority-group 7 bandwidth 1 pfc off
    S2D# priority-pgid 0 0 0 3 0 0 0 7
    S2D# exit


    Apply map to ports 0-xx

    S2D# configure
    S2D# interface range tengigabitethernet 0/0-xx
    S2D# dcb-map dcb-RDMA
    S2D# exit

    Monitor RoCE - Cisco

    Monitor RoCE and configuration examples for Cisco Switch

    *** Disclaimer ***
    s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
    For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
    The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
    ***

    Performance Monitor
    Monitor Priority and Pause Frames for RDMA (RoCE) on Cisco Switch


    Microsoft references:


    ******************************************************************************
    Cisco:
    ******************************************************************************
    Cisco Nexus 9000 Series Configuring Priority Flow Control

    S2D# show interface ethernet 1/x priority-flow-control
      If you do not specify an interface (ethernet x/x) the show interface priority-flow-control command will display the priority flow control information for all interfaces (layer 2 and layer 3).

      S2D# show interface priority-flow-control
        Example, Port 3 and 4 has PFC On/On (no Auto/On), and we can see it receive and send (Rx/Tx) Pause Frames

        S2D# show interface priority-flow-control
        ============================================================
        Port Mode Oper(VL bmap) RxPPP TxPPP
        ============================================================

        Ethernet1/1 Auto Off 0 0
        Ethernet1/2 Auto Off 0 0
        Ethernet1/3 On   On (0) 13734 2546
        Ethernet1/4 On   On (0) 13865 2634
        Ethernet1/5 Auto Off 0 0
        Ethernet1/6 Auto Off 0 0
        Ethernet1/7 Auto Off 0 0

        (Draft)

        • Port specific:
          • switchport mode trunk
          • switchport trunk native vlan <Your native vlan, Default is 1>
          • switchport trunk allowed vlan <Your vlan list>
          • spanning-tree port type edge
          • flowcontrol receive off
          • flowcontrol send off
          • priority-flow-control mode on
          • no shutdown
        • Global:
          • class-map type qos match-all RDMA
          • match cos 3
          • class-map type queuing RDMA
          • match qos-group 3
          • policy-map type qos QOS_MARKING
          • class RDMA
          • set qos-group 3
          • class class-default
          • policy-map type queuing QOS_QUEUEING
          • class type queuing RDMA
          • bandwidth percent 50
          • class type queuing class-default
          • bandwidth percent 50
          • class-map type network-qos RDMA
          • match qos-group 3
          • policy-map type network-qos QOS_NETWORK
          • class type network-qos RDMA
          • mtu xxxx (Please check with pNIC settings*)
          • pause no-drop
          • class type network-qos class-default
          • mtu 9216 (Please check with pNIC settings, pNIC for Windows Jumbo Frames is normally 9014, the Physical Switch end-to-end connection is recommended to be the max size 9216)
          • system qos
          • service-policy type qos input QOS_MARKING
          • service-policy type queuing output QOS_QUEUEING
          • service-policy type network-qos QOS_NETWORK

        Monitor RoCE - HPE

        Monitor RoCE and configuration examples for HPE Switch

        *** Disclaimer ***
        s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
        For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
        The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
        ***

        Performance Monitor
        Monitor Priority and Pause Frames for RDMA (RoCE) on HPE Switch

        Microsoft references:
        HPE references:

        HPE FlexFabric 5950 Command Reference

        S2D# display priority-flow-control interface HundredGigE1/1/x
        (List the PFC Settings and the Priority and Pause Frames (Recv/Send))

        Auto: Specifies PFC in auto mode. In this mode, the Ethernet interface automatically negotiates the PFC status with its peer over LLDP.
        Enable: Forcibly enables PFC.

        Interface           AdminMode  OperMode  Dot1pList   Prio  Recv       Send
        --------------------------------------------------------------------------------
        HGE1/1/x            Enabled    Enabled   3           3     176597     34268

        S2D# reset counters interface HundredGigE1/1/x
        (To reset the Recv/Send counters)

        ******************************************************************************
        HPE Switch Port Example for PFC, Priority 3 SMB traffic:
        ******************************************************************************
        interface HundredGigE1/1/x
        port link-mode bridge
        description S2D Storage Port
        port link-type trunk
        port trunk permit vlan <S2D Storage vlan list>
        stp edged-port
        priority-flow-control enable
        priority-flow-control no-drop dot1p 3
        qos trust dot1p

        ***
        ETS
        ***

        Draft - need to be validated

        Microsoft use ETS 3 for SMB (50% bandwidth in the below example) and 7 for Cluster traffic
        Note: Priority 7 dont need PFC only the ETS


        qos map-table dotlp-lp
        import 0 export 0
        import 2 export 0
        import 3 export 1
        import 4 export 0
        import 5 export 0
        import 6 export 0
        import 7 export 2


        interface interface HundredGigE1/1/x 
        qos wrr be group 1 byte-count 15
        qos wrr af1 group 1 byte-count 15
        qos wrr af2 group sp
        qos wrr af3 group sp
        qos wrr af4 group sp
        qos wrr ef group sp
        qos wrr ca6 group sp
        qos wrr ca7 group sp

        ******************************************************************************
        Switch Configuration for only Storage Traffic
        ETS 95% for SMB Traffic on the Host
        ETS 1% for Cluster Traffic on the Host
        SP = Strict Priority

        List the HPE TCs:
        qos wrr be group
        qos wrr af1 group
        qos wrr af2 group
        qos wrr af3 group
        qos wrr af4 group
        qos wrr ef group
        qos wrr ca6 group
        qos wrr ca7 group
        ******************************************************************************

        Draft - need to be validated

        qos map-table dotlp-lp
        import 0 export 0
        import 2 export 0
        import 3 export 1
        import 4 export 0
        import 5 export 0
        import 6 export 0
        import 7 export 2


        interface interface HundredGigE1/1/x 
        qos wrr af1 group sp
        qos wrr af2 group sp

        Monitor RoCE - Fujitsu

        Monitor RoCE and configuration examples for Fujitsu Switch

        *** Disclaimer ***
        s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
        For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
        The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
        ***

        Performance Monitor
        Monitor Priority and Pause Frames for RDMA (RoCE) on Fujitsu Switch

        Microsoft references:

        ******************************************************************************
        Show Commands for PFC
        ******************************************************************************
        Microsoft dont support DCBx if the system use vNIC/SET Switch "NDKm2". So PFC need to be "on" don't use DCBX in this case


        S2D# show qos flow interface TenGigabitEthernet 1/0/2

        S2D# Interface TenGigabitEthernet 1/0/2
          Mode PFC
          DCBX enabled for PFC negotiation
          TX 865261786 frames
          RX 9002 frames
                 TX   TX    RX   RX Output Paused
          CoS Admin Oper Admin Oper  512 BitTimes
          ---------------------------------------
            0   Off  Off   Off  Off             0
            1   Off  Off   Off  Off             0
            2   Off  Off   Off  Off             0
            3    On   On    On   On      11879696
            
            4   Off  Off   Off  Off             0
            5   Off  Off   Off  Off             0
            6   Off  Off   Off  Off             0
            7   Off  Off   Off  Off             0

        ******************************************************************************
        Fujitsu configuration Example 
        ******************************************************************************

        S2D# Interface TenGigabitEthernet 1/0/2
        S2D# qos flowcontrol pfc 3 tx on rx on
        S2D# S2D# exit


        S2D# show running-config interface TenGigabitEthernet 1/0/2
        interface TenGigabitEthernet 1/0/2
        cee default
        mtu 9216
        switchport
        switchport mode trunk
        switchport trunk allowed vlan add 222
        switchport trunk tag native-vlan
        switchport trunk native-vlan 222
        spanning-tree shutdown
        lldp disable
        fabric isl enable
        fabric trunk enable
        no shutdown

        Monitor RoCE - Lenovo

        Monitor RoCE and configuration examples for Lenovo Switch

        *** Disclaimer ***
        s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
        For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
        The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
        ***

        Performance Monitor
        Monitor Priority and Pause Frames for RDMA (RoCE) on Lenovo Switch.

        Microsoft references:

        Lenovo:
        Lenovo System X

        ******************************************************************************
        Lenovo Display Commands
        ******************************************************************************

        Editing is done below, removed lines with repeated information replaced by x (port numbers)

        Note: G8332 running CNOS 10.4.3 – Syntax changed with CNOS 10.7 and higher

        The Example use priorities 3 and 5 (The Cluster traffic just need to have the highest priority 5 or 7)

        ****************************************************************************** 

        S2D# display cee ets information
        ETS Global Admin Configuration:
        PGID    BW%   COSq  Priorities          Description
        ==============================================================================
        0       49    0      0 1 2              Default
        1       0     NA
        2       0     2      4 6 7
        3       50    3      3                  RoCEv2_Traffic (SMB)
        4       0     4
        5       1     5      5                  Cluster
        6       0     6
        7       0     7
        15      NA    1

        ETS Global Operational Configuration:
        PGID    BW%   COSq  Priorities          Description
        ==============================================================================
        0       49    0      0 1 2              Default
        1       0     NA
        2       0     2      4 6 7
        3       50    3      3                  RoCEv2_Traffic (SMB)
        4       0     4
        5       1     5      5                  Cluster
        6       0     6
        7       0     7
        15      NA    1

        ETS Interface Admin Configuration
        --------------------
        ETS Willing mode is not supported on the Switch.

        Interface           Advertise
        ================================================================================
        Ethernet1/x         On
        Ethernet1/x         On

        ******************************************************************************
        S2D# display cee pfc
        Global Admin PFC State: On
        Priority  State  Description
        ============================
           0      Dis
           1      Dis
           2      Dis
           3      Ena    RoCEv2_Priority (SMB)
           4      Dis
           5      Dis    Cluster
           6      Dis
           7      Dis
        Admin Configuration:
        --------------------
        PFC Willing mode is not supported on the Switch.

        Interface      State  Advertise Cap   Priorities
        ================================================================================
        Ethernet1/x    On     On        2     3 5
        Ethernet1/x    On     On        2     3 5

        --------------------------------------------------------------------------------

        Operational Configuration
        -------------------------
        Global Operational PFC State: On
        Priority  State  Description
        ============================
           0      Dis
           1      Dis
           2      Dis
           3      Ena    RoCEv2_Priority (SMB)
           4      Dis
           5      Dis    Cluster
           6      Dis
           7      Dis
        Interface      State Cap   Priorities
        ================================================================================
        Ethernet1/x    On    2     3 5
        Ethernet1/x    On    2     3 5

        ******************************************************************************
        Ports used for S2D (PFC) need to show traffic/Packets on Rx/Tx

        S2D# display cee pfc counters
        Priority Flow Control Packets:
        Interface              Received(Rx)            Sent(Tx)
        =======================================================
        Ethernet1/x                       0                   0
        Ethernet1/x                       0                   0
        Ethernet1/7                 4988911            17067089
        Ethernet1/8                 1852123            12482442
        Ethernet1/9                 9800282            12982588
        Ethernet1/10                5229729            20209402
        Ethernet1/11                7244549            16976660
        Ethernet1/x                       0                   0
        Ethernet1/23               12245734            14795755
        Ethernet1/24               12245683             8309129
        Ethernet1/x                       0                   0

        ******************************************************************************

        S2D# display cee interface Ethernet 1/7

        ETS information for the interface Ethernet1/7:
        ETS Global Admin Configuration:
        PGID    BW%   COSq  Priorities          Description
        ==============================================================================
        0       49    0      0 1 2              Default
        1       0     NA
        2       0     2      4 6 7
        3       50    3      3                  RoCEv2_Traffic (SMB)
        4       0     4
        5       1     5      5                  Cluster
        6       0     6
        7       0     7
        15      NA    1

        ETS Global Operational Configuration:
        PGID    BW%   COSq  Priorities          Description
        ==============================================================================
        0       49    0      0 1 2              Default
        1       0     NA
        2       0     2      4 6 7
        3       50    3      3                  RoCEv2_Traffic (SMB)
        4       0     4
        5       1     5      5                  Cluster
        6       0     6
        7       0     7
        15      NA    1

        Admin Configuration
        --------------------
        Advertise Willing
        ================================================================================
        On        Off

        Operational Configuration
        --------------------
        Advertise Willing
        ================================================================================
        Off       Off

        PFC information for the interface Ethernet1/7:
        Admin Configuration
        --------------------
        Interface      State  Advertise Cap   Priorities
        ================================================================================
        Ethernet1/7    On     On        2     3 5

        --------------------------------------------------------------------------------

        Operational Configuration
        -------------------------
        State Cap   Priorities
        ================================================================================
        On    2     3 5

        ******************************************************************************







        Mellanox SR-IOV

        Enable Virtual Switch with SR-IOV for Mellanox ConnectX-3 Pro and Mellanox ConnectX-5


        *** Disclaimer ***
        s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
        For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
        The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
        ***

        Enable Virtual Switch with SR-IOV for Mellanox ConnectX-3 Pro and Mellanox ConnectX-5
        1. List the Physical Network Adapter in the Host. NIC1, NIC2 and NIC3 is used in this blogpost. (NIC3 and NIC4 is used for the Chelsio blogpost #link will be add#)
          Get-NetAdapter | sort Name | ft Name, InterfaceDescription, LinkSpeed -AutoSize
        2. List the default number of VF in the OS
          (Note: This is not the same as the configured)
          Get-NetAdapterSriov | sort name | ft Name, InterfaceDescription, SriovSupport, NumVFs -AutoSize
        3. List the Mellanox ConnectX-3 Pro numbers of VF enabled for each pNIC Port.
          Mellanox ConnectX-3 Pro support 32 VF in total. For more information about the configuration of the SR-IOV in the pNIC BIOS and Speed, ses the blogpost #link will be add#)
          Mellanox 56 Gigabit
          Get-MlnxPCIDeviceSriovSetting
        4. The Mellanox ConnectX-5 support up to 1000 VF, the Physical Server don´t support that, therefor we have to change the number to a supported value. We used the Mellanox Tools to get the PCI Device Name. Link to Mellanox Tool
        5. PCI Device Name for the Mellanox ConnectX-5
          mlxfwmanager.exe --query
        6. Change the numbers of VF to 64 (1-127 is supported)
          mlxconfig.exe -d mt4119_pciconf0 set SRIOV_EN=1 NUM_OF_VFS=64More information in Mellanox Firmware Tools (MFT) User Manual
        7. List the current configuration and the new values
        8. Reboot
        9. Create the Virtual Switch "iovSwitch1" with the two Mellanox ConnetX-3 Pro ports
          New-VMSwitch -Name iovSwitch1 -NetAdapterName NIC1, NIC2 -AllowManagementOS $false -EnableEmbeddedTeaming $true -EnableIov $true
        10. Create the Virtual Switch "iovSwitch3" with the two Mellanox ConnetX-5
          New-VMSwitch -Name iovSwitch3 -NetAdapterName NIC5 -AllowManagementOS $true -EnableEmbeddedTeaming $true -EnableIov $true
        11. List the two Virtual Switch with SR-IOV enabled
        12. Only the iovSwitch3 have "-AllowManagement $true"
        13. The vNIC created for Management Access (Default name is the vSwitch name)
          Get-VMNetworkAdapter -ManagementOS
        14. Rename the vNIC iovSwitch3 to MGMT
          Rename-VMNetworkAdapter -ManagementOS -VMNetworkAdapterName iovSwitch3 -NewName MGMT
          Rename-NetAdapter "*MGMT*" MGMT
        15. List the pNIC/vNIC now with the MGMT name for the vNIC
          Get-NetAdapter | sort Name | ft Name, InterfaceDescription, LinkSpeed -AutoSize
        16. Show the new Virtual Switch with the correct numbers of VF
          Get-NetAdapterSriov | sort name | ft Name, InterfaceDescription, SriovSupport, NumVFs -AutoSize