Convert-LBFO2SET

Convert LBFO to SET

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

Azure Stack HCI: Convert LBFO to SET (including conversion in SCVMM) Convert-LBFO2SET This tool was written by Microsoft PG and CSS Support to assist customers in their migration of load balancing and failover (LBFO) teams to Switch Embedded Teams (SET). https://aka.ms/Convert-LBFO2SET The video shows the steps from SCVMM and the physical host to convert the LBFO team and virtual switch to the switch embedded (SET). Convert the standard virtual switch in SCVMM back to a logical switch

(00:55) Create the LBFO Team/Switch from SCVMM for the Demo (01:24) Show the LBFO on the Host (01:50) Show the Virtual Switch and run the convert command (02:49) Show the SET Switch (03:34) Convert the standard virtual switch in SCVMM back to a logical switch PowerShell: Convert-LBFO2SET -LBFOTeam LBFOSwitch -SETTeam SETSwitch Demo with VMs running by: Dan Cuomo https://youtu.be/Aedjp2_y4bU

*** Build a lab setup in minutes on your laptop https://aka.ms/wslab

S2D and Extents

S2D and Extents Part 1 and 2 - How small a vDisk can you create

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

The following Demo is performed in a S2D test environment with 6 Cluster Nodes.

How small a vDisk can you create ?

Part 1

Part 2

The Windows Version is important in this Demo, the Extent size in this Demo is 256MB or 1GB

Each Node have 5 SSDs used for the Storage Pool.

If you don´t have access to physical lab servers you can use WSLab:

• https://aka.ms/wslab

Task:
Create the smallest possible vDisk

Columns and Extents:
Know you Columns count and the Extents Size ?

The vDisk is created with the default maximum columns count of 8 and 3 way Mirror

6x5=30 (Host x Disk) the total of physical disks in the Storage pool
3x8=24 Extents is used (3 way Mirror x default column count of 8)

Try to create a 1GB vDisk, is that posibel ?
No, Why ?
You have Columns of  8 and each Extents is 1GB so the smallest vDisk will be 8GB (See Video for Part 1)
You have Columns of  8 and each Extents is 256MB so the smallest vDisk will be 2GB (See Video for Part 2)

Windows Server 2016 and Windows Server 2019 (Build: 17763) use a Extents size from 256MB to 1GB in a 3 way Mirror.

Part 1:
The Test "vDisk" was create with the PowerShell command:

New-Volume -StoragePoolFriendlyName S2D* -FriendlyName vDisk1 -FileSystem CSVFS_ReFS -ProvisioningType Fixed -Size 1GB -Verbose

Part 2:
The Test "vDisk" was create with the PowerShell command:

New-Volume -StoragePoolFriendlyName S2D* -FriendlyName vDisk1 -FileSystem CSVFS_ReFS -ProvisioningType Fixed -StorageTierFriendlyNames "Performance" -StorageTierSizes 1GB -Verbose

Split Mellanox Port and connect to Cisco Switch

Split Mellanox Port from 1x100 Gigabit to 4x10 Gigabit ports and connect to Cisco Switch

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

Split Mellanox Port from 1x100 Gigabit to 4x10 Gigabit ports and connect to Cisco Switch

The Example use a Cisco 2360G (4x10 Gigabit Ports) 
Mellanox Split Cable with Prolabs Transceiver, Cisco compatible and has been programmed to match the Cisco Switch 2360G

***

Split Mellanox Port 15

***

enable

configure terminal 

interface ethernet 1/15

shutdown

module-type qsfp-split-4

Yes

For port configuration or reset port, please go to:
Split Mellanox Port from 1x100 Gigabit to 4x10 Gigabit

Mellanox
HowTo Use Mellanox Interconnect Products with Cisco Switches

Cisco Switch show command for idprom and transceiver:
show interfaces tenGigabitEthernet 0/1 status

show idprom interface tenGigabitEthernet 0/1
show idprom interface tenGigabitEthernet 0/1 detail
show interfaces tenGigabitEthernet 0/1 transceiver
show interfaces tenGigabitEthernet 0/1 transceiver properties

Split Mellanox Port from 1x100 Gigabit to 4x10 Gigabit

Split Mellanox Port from 1x100 Gigabit to 4x10 Gigabit ports and connect to Windows Server Host

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

Split Mellanox Port from 1x100 Gigabit to 4x10 Gigabit ports and connect to Windows Server Host

***

Split Mellanox Port 16

***

enable

configure terminal 

interface ethernet 1/16

shutdown

module-type qsfp-split-4

Yes

***

To used Port 16 (the 4 split ports) for Host running S2D  (RDMA/RoCE)

***

dcb priority-flow-control enable force

dcb priority-flow-control priority 3 enable

***

WRR or Strict - Example with Strict

***

interface ethernet 1/16/1-1/16/4 traffic-class 3 dcb ets strict
interface ethernet 1/16/1-1/16/4 traffic-class 7 dcb ets strict

interface ethernet 1/16/1-1/16/4 flowcontrol send off force

interface ethernet 1/16/1-1/16/4 flowcontrol receive off force

interface ethernet 1/16/1-1/16/4 speed 10000 force

interface ethernet 1/16/1-1/16/4 mtu 9216 force

interface ethernet 1/16/1-1/16/4 dcb priority-flow-control mode on force

interface ethernet 1/16/1 switchport mode hybrid

interface ethernet 1/16/2 switchport mode hybrid

interface ethernet 1/16/3 switchport mode hybrid

interface ethernet 1/16/4 switchport mode hybrid

interface ethernet 1/16/1 switchport hybrid allowed-vlan (your VLAN list for that port)

interface ethernet 1/16/2 switchport hybrid allowed-vlan (your VLAN list for that port)

interface ethernet 1/16/3 switchport hybrid allowed-vlan (your VLAN list for that port)

interface ethernet 1/16/4 switchport hybrid allowed-vlan (your VLAN list for that port)

***

To reset the port 16 back to one port
***
interface ethernet 1/16/1  shutdown
interface ethernet 1/16/2  shutdown
interface ethernet 1/16/3  shutdown
interface ethernet 1/16/4  shutdown
interface ethernet 1/16/1  module-type qsfp force

***

For more information about how to configure and monitor the Mellanox Switch with RoCE

• Mellanox SN2100
• Mellanox SX1012

1 or 2 vlan/subnets for S2D Cluster - Troubleshooting series

1 or 2 vlan/subnets for S2D Cluster - Troubleshooting series

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

Configuration of VLAN/Subnets for Storage Spaces Direct (S2D) Cluster
Configuration of VLAN/Subnets for the Troubleshooting Series, Part 1-3
Shows the steps for the Part 1, how to change from 1x VLAN to 2x VLANs.
Configuration of System Center Virtual Machine Manager (SCVMM)

Demo Part #1 - Demo with two vlan/subnets.

The demo shows a Windows Server 2019 cluster that has been configured with two vlan/subnets. The subnet only exist in one physical switch, if the vNIC (SMB vNIC) has to move to the second physical NIC it will not have connectivity. The Failover Cluster will "know" that the network connectivity is down...

SET Switch, created from SCVMM

• vNIC "MGMT"
• vNIC "SMB11" - vlan 211 > Team Mapping to pNIC1
• vNIC "SMB12" - vlan 212 > Team Mapping to pNIC2

Demo Part #2 - Demo with two vlan/subnets but the vlans is present in both ToR Switches


The demo shows a Windows Server 2019 Storage Spaces Direct (S2D) Cluster that has been configured with two VLAN/Subnets. The subnet exist in both physical Switches, if the vNIC (SMB vNIC) has to move to the second physical NIC it will have Ethernet connectivity.


SET Switch, created from SCVMM

• vNIC "MGMT"
• vNIC "SMB01" - vlan 201 > Team Mapping to pNIC1
• vNIC "SMB02" - vlan 202 > Team Mapping to pNIC2

Demo Part #3 - Demo with one vlan/subnets


The demo shows a Windows Server 2019 cluster that has been configured with one vlan/subnet. The subnet exist in all physical switchs, if the vNIC (SMB vNIC) has to move to the second physical NIC it will have connectivity.


SET Switch, created from SCVMM

• vNIC "MGMT"
• vNIC "SMB01" - vlan 18 > Team Mapping to pNIC1
• vNIC "SMB02" - vlan 18 > Team Mapping to pNIC2

Replace Node - Troubleshooting series

Replace Node - Troubleshooting series

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

Replace Storage Spaces Direct Cluster Node - Troubleshooting series for Storage Spaces Direct (S2D)

4 Node Storage Spaces Direct (S2D) Cluster with failed physical disks

• S048001
• S048002 (Failed Node)
• S048003
• S048004

Part 1: Removed the failed disks from the Storage Pool and removed the Cluster Node

Part 2: Add the Node back to the Cluster

Windows Admin Center list the failed and retired disks

Remove the failed disk from the Storage Pool
Get-StorageNode -Name S048002.contoso.com | Get-PhysicalDisk -PhysicallyConnected | sort PhysicalLocation | ft SerialNumber, OperationalStatus, PhysicalLocation

The Cluster used in the Demo is a SOFS Cluster so we don´t have VMs on the Host.
However if the Solution is a HCI then perform a live migration to move running VMs off the server you're about to remove.
Pause the cluster server using the following PowerShell command on the Host that you plan to remove: Suspend-ClusterNode -Drain
The Disks are Retired by the system, so I only need to remove the from the Storage Pool
$Disk = Get-PhysicalDisk |? Usage -like Retired
Get-StoragePool *S2D* | Remove-PhysicalDisk -PhysicalDisks $Disk

More information about how to manually Retired or Reset Disks please see the Blog:
Retire, Remove, Reset or Add Disk to Storage Pool


Remove "Evict" the server "S048002" from the cluster by running the following PowerShell command Remove-ClusterNode S048002

Test RDMA Connectivity

Test RDMA Connectivity

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

Test RDMA Connectivity
Note: The Test-RDMA Script is a connectivity tools and not for performance tests...

Use the Microsoft Deployment Guide and Validate DCB Scripts for Host validation

• Windows Server 2016 and 2019 RDMA Deployment Guide
• Validate DCB

Download the DiskSpd.exe utility and extract into C:\Temp

• DiskSpd.exe utility

Download the Test-RDMA powershell script to C:\Temp

• Test-RDMA script

Test with a two Node cluster

• Node 1 = S048033
• Node 2 = S048034

S048033

• SMB01 = Interface Index 7 = 172.18.148.33
• SMB02 = Interface Index 9 = 172.18.248.33

S048034

• SMB01 = Interface Index 9 = 172.18.148.34
• SMB02 = Interface Index 8 = 172.18.248.34

Test from S048033 to S048034

Get-SmbClientNetworkInterface

.\Test-RDMA.PS1 -IfIndex 9 -IsRoCE $true -RemoteIpAddress 172.18.248.34 -PathToDiskspd C:\Temp\

.\Test-RDMA.PS1 -IfIndex 7 -IsRoCE $true -RemoteIpAddress 172.18.148.34 -PathToDiskspd C:\Temp\