Featured Post

YouTube and link library for S2D.dk

2019/11/24

YouTube and link library for S2D.dk

YouTube and link library for S2D.dk

 Please subscribe to S2D.dk YouTube… to support the channel

S2D.dk YouTube Channel

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

YouTube
Update
News
Microsoft Failover Cluster
Storage Spaces Direct (S2D)
Micron Tools
Microsoft Tools
Mellanox
Marvell
Chelsio
Cisco

 Intel
Dell
HPE
DataON
SuperMicro

2019/11/23

DiskSpd remote disk Performance and Host impact

DiskSpd (diskspd.exe) - Disk Read/Write impact on the Host CPU/LPs (Remote Disk with RDMA/iWARP)
Examples with 1, 10 and 32 threads per target. 1 or 8 Number of outstanding I/O requests per-target per-threads.

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

diskspd.exe - Disk Read/Write impact on the Host CPU/LPs 
The performance counters shows the Remote Client Host CPU/LPs impact, while the diskspd.exe creating workload

*** Note: Before you run the test with DiskSpd on a Cluster, set the CSV cache to 0. DiskSpd create synthetic workloads on Windows Server 2019 Clusters or later (including Storage Spaces Direct S2D). The Read only cache causes performance slowness with synthetic workloads, however with real workloads it increases performance.
***
Both the File Server and the Client Host use Chelsio 40G Network Adapteres with RDMA/iWARP
The File Server use a "SET Switch" with two vNICs and the Client use pNIC
  • S046001E - File Server
    • 172.18.0.119
    • 172.18.0.120
  • S046002A - Client (Running DiskSpd)
    • 172.18.0.121
    • 172.18.0.122
Part 1 - Performance test with DiskSpd (Remote Disk Performance - Part 1)


Part 2 - Performance test with DiskSpd (Remote Disk Performance - Part 2)


The first Picture shows the SMB RDMA Connections
Note: The direction is opposite of the job, Read Test will show SMB Direct Write traffic

SMB Direct Connections from Client Server to File Server
***

Client Server Read Performance from the remote File Share (Remote Disk)
Note:
The Client Server use a pNIC from Chelsio with RDMA/iWARP enabled
***

Client Server Read Performance from the remote File Share (Remote Disk)
***

Client Server Write Performance from the remote File Share (Remote Disk)
***


Client Server Write Performance from the remote File Share (Remote Disk)
***

File Server Host Read Performance from the remote Client (Local Disk impact on the File Server)
Note:
The File Server use a Virtual Switch (SET Switch) with vNICs and RDMA enabled (NDKm2)
The pNICs are also Chelsio with RDMA/iWARP
***

File Server Host Write Performance from the remote Client (Local Disk impact on the File Server)
***

2019/11/22

DiskSpd local disk Performance and Host impact

DiskSpd (diskspd.exe) - Disk Read/Write impact on the Host CPU/LPs (Local Disk)
Examples with 1, 10 and 32 threads per target. 1 or 8 Number of outstanding I/O requests per-target per-threads.

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

 diskspd.exe - Disk Read/Write impact on the Host CPU/LPs 
The performance counters shows the Host CPU/LPs impact, while the diskspd.exe creating workloads

*** Note: Before you run the test with DiskSpd on a Cluster, set the CSV cache to 0. DiskSpd create synthetic workloads on Windows Server 2019 Clusters or later (including Storage Spaces Direct S2D). The Read only cache causes performance slowness with synthetic workloads, however with real workloads it increases performance.



Links for download DiskSpd and Command help on GitHub

Example:
Create the Test file "io.dat" with the size of 100GB add the command "-c100G" the first time
Diskspd.exe -b8K -d60 -Su -L -o1 -t1 -r -w0 -W20 -c100G D:\Temp\io.dat
Read Test:
Diskspd.exe -b8K -d60 -Su -L -o1 -t1 -r -w0 -W30 D:\Temp\io.dat
Diskspd.exe -b8K -d60 -Su -L -o8 -t1 -r -w0 -W30 D:\Temp\io.dat

 Diskspd.exe -b8K -d60 -Su -L -o1 -t10 -r -w0 -W20 D:\Temp\io.dat
Diskspd.exe -b8K -d60 -Su -L -o8 -t10 -r -w0 -W20 D:\Temp\io.dat

 Diskspd.exe -b8K -d60 -Su -L -o1 -t32 -r -w0 -W20  D:\Temp\io.dat
Diskspd.exe -b8K -d60 -Su -L -o8 -t32 -r -w0 -W20  D:\Temp\io.dat
Write Tests:
Diskspd.exe -b8K -d60 -Su -L -o1 -t1 -r -w100 -W30 D:\Temp\io.dat
Diskspd.exe -b8K -d60 -Su -L -o8 -t1 -r -w100 -W30 D:\Temp\io.dat

 Diskspd.exe -b8K -d60 -Su -L -o1 -t10 -r -w100 -W20 D:\Temp\io.dat
Diskspd.exe -b8K -d60 -Su -L -o8 -t10 -r -w100 -W20 D:\Temp\io.dat

 Diskspd.exe -b8K -d60 -Su -L -o1 -t32 -r -w100 -W20  D:\Temp\io.dat
Diskspd.exe -b8K -d60 -Su -L -o8 -t32 -r -w100 -W20  D:\Temp\io.dat

Examples with 1, 10 and 32 threads per target. 1 or 8 Number of outstanding I/O requests per-target per-thread
Read, Number of threads 1 and 1 I/O requests per shared thread
***

Read, Number of threads 1 and 1 I/O requests per shared thread
***

Write, Number of threads 1 and 1 I/O requests per shared thread
***

Write, Number of threads 1 and 1 I/O requests per shared thread
***


Read, Number of threads 10 and 1 I/O requests per shared thread
***

Read, Number of threads 10 and 1 I/O requests per shared thread
***

Read, Number of threads 10 and 8 I/O requests per shared thread
***

Read, Number of threads 10 and 8 I/O requests per shared thread
***

Write, Number of threads 10 and 1 I/O requests per shared thread
***

Write, Number of threads 10 and 1 I/O requests per shared thread
***

Write, Number of threads 10 and 8 I/O requests per shared thread
***

Write, Number of threads 10 and 8 I/O requests per shared thread
***


Read, Number of threads 32 and 1 I/O requests per shared thread
***

Read, Number of threads 32 and 1 I/O requests per shared thread
***

Read, Number of threads 32 and 8 I/O requests per shared thread
***

Read, Number of threads 32 and 8 I/O requests per shared thread
***
Write, Number of threads 32 and 1 I/O requests per shared thread
***

Write, Number of threads 32 and 1 I/O requests per shared thread
***

Write, Number of threads 32 and 8 I/O requests per shared thread
***

Write, Number of threads 32 and 8 I/O requests per shared thread
***









2019/11/20

Perfmon /sys

Perfmon (perfmon /sys) - Save and Load Windows Performance Counters Settings

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

 Save and Load Windows Performance Counters


perfmon /sys


2019/11/19

Validator for RDMA Configuration and Best Practices

Validator for RDMA Configuration and Best Practices. How to do the Installation and first run in 5 minutes with "Validate-DCB".

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

Validator for RDMA Configuration and Best Practices

The Microsoft Docs version is now released

How to do the Installation and first run in 5 minutes with Validate-DCB

More information on GitHub
Video part 1 to 4

 Part 1 - Installation

Part 2

Part 3

Part 4

Installation
  • Install-Module Validate-DCB
Update
  • Remove the old version from the Module (C:\Program Files\WindowsPowerShell\Modules if pester can not be removed use the command, to have both versions)
  • Install-Module validate-dcb -SkipPublisherCheck


After installation of the Validate-DCB Module you can run the wizard:
  1. Run the Validate-DCB from PowerShell and the wizard will start <Next>
  2. Add the Cluster name
  3. Resolve
  4. The Cluster Nodes are listed
  5. Next
  6. Add vSwitch Name
  7. Add pNIC, vNIC and vlan information
  8. Select RDMA type iWARP or RoCE
  9. Select the Jumbo Frames (Ethernet MTU size)
    Note: the MTU for RDMA is not the same as Ethernet MTU
    For Mellanox list the Ethernet and RoCE/RDMA MTU with "mlx5cmd.exe -Stat"
    Both the pNIC and the vNIC for SMB need to have the same MTU Size
  10. Next
  11. Policy Name
  12. Bandwidth Reservation
  13. Next
  14. Save the configuration file
  15. Export (the job will start)








Validate-DCB Disconnected installation

Validator for RDMA Configuration and Best Practices. Validate-DCB Disconnected installation

*** Disclaimer ***
s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
***

Validator for RDMA Configuration and Best Practices

Validate-DCB Disconnected installation



Validate-DCB Fix 58 errors




 More information about installation and review the result for Validate-DCB
More information on GitHub

Validate-DCB Disconnected installation (Step-by-Step)
  1. The installation failed do to no internet access (Disconnected)
  2. From a Host with Internet access run "Save-Module Validate-DCB -Path C:\Temp"
  3. The packages are downloaded
  4. The Download complete
  5. The six folders are created with all the modules
  6. Select and copy the six folders
  7. Change back to the disconnected Host, change to the folder "C:\Program Files\WindowsPowerShell\Modules"
  8. Paste the files to the folder
  9. Files are copied
  10. Check that the files and folders are copied
    (Restart PowerShell CLI, to load with the new modules)
  11. Now you can run the "Validate-DCB"
  12. The prerequisite part "Global Unit" is check
  13. If the "Global Unit" pass, the wizard will start and you are ready to add the information about your system
  14. For more information about the next step go to the online installation guide [Link]
    (The next part is the same for online and disconnected systems)








                          How to review the result from Validate-DCB

                          Validator for RDMA Configuration and Best Practices. How to review the result from Validate-DCB

                          *** Disclaimer ***
                          s2d.dk is not responsible for any errors, or for the results obtained from the use of this information on s2d.dk. All information in this site is provided as "draft notes" and "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information. Always test in a lab setup, before use any of the information in production environment.
                          For any reference links to other websites we encourages you to read the privacy statements of the third-party websites.
                          The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
                          ***

                          Validator for RDMA Configuration and Best Practices

                          How to review the result from Validate-DCB


                          More information on GitHub
                          The results output consists of two unit
                          1. [Global Unit] Prerequisites and requirements to run the modal tests. 
                          2. [Modal Unit] Each Cluster Host configuration and best practices feedback

                          How to review the Error(s)

                          Example 1: Jumbo Packet Missing from vNIC SMB02
                          1. The Validate-DCB list one FailedCount
                          2. Scroll back to we find the red lines. Expect 9014 but the value was 1514 looks like the Jumbo Packet was not Enabled on the vNIC
                          3. Review the Properties for the vNIC SMB02 on Host S046036, The finding is correct, the Jumbo Packet was Disabled. To fix the Error just enable Jumbo Packet
                          4. Change the Value to 9014 Bytes
                          5. Scan the Host S046036 again with Validate-DCB
                          6. Now the Failed Count is back on 0


                          Review with PowerBI

                          Run the Validate-DCB and add the Report Path, copy the XML to the machine with PowerBI
                          Download the Validate-DCB.pbix fle from GitHub (Link)
                          1. Run "validate-dcb -ConfigFilePath C:\Temp\S046036.ps1 -ReportPath C:\Temp"
                          2. Copy the XML to the machine with PowerBI
                          3. Click on "Validate-DCB.pbix" to open PowerBI
                          4. Import the XML
                            A: Edit Queries
                            B: Data source settings
                            C: Change Source... Select the XML
                          5. Review the "Failure" and make the necessary changes


                          Tilføj billedtekst





                          Subscribe to: Posts (Atom)