Step by Step Server-to-server storage replication with Windows Server 2019 Storage Replica #WindowsAdminCenter #StorageReplica #WindowsServer2019 #ReFS #SR #Azure

In the old days all File servers where on one place, and if you want to replicate data you needed a extra tool to do this. Now days its already build in into Windows server. Storage replica can be used in several ways, replicate data from one Cluster to another or to Azure. but in this case I do a server to server replication as not everyone has a cluster.

For moving data to the Cloud there are currently several other applications like Azure file sync or Azure Migrate https://docs.microsoft.com/en-us/azure/migrate/migrate-overview Blog about Azure File Sync https://robertsmit.wordpress.com/2017/09/28/step-by-step-azure-file-sync-on-premises-file-servers-to-azure-files-storage-sync-service-afs-cloud-msignite/

Storage Replica is Windows Server technology that enables replication of volumes between servers or clusters for disaster recovery. It also enables you to create stretch failover clusters that span two sites, with all nodes staying in sync.

Storage Replica supports synchronous and asynchronous replication:

• • Synchronous replication mirrors data within a low-latency network site with crash-consistent volumes to ensure zero data loss at the file-system level during a failure.
  • Asynchronous replication mirrors data across sites beyond metropolitan ranges over network links with higher latencies, but without a guarantee that both sites have identical copies of the data at the time of a failure.

Storage Replica allows more efficient use of multiple datacenters. By stretching clusters or replicating clusters, workloads can be run in multiple datacenters for quicker data access by local proximity users and applications, as well as better load distribution and use of compute resources. If a disaster takes one datacenter offline, you can move its typical workloads to the other site temporarily.

Storage Replica may allow you to decommission existing file replication systems such as DFS Replication that were pressed into duty as low-end disaster recovery solutions. While DFS Replication works well over extremely low bandwidth networks, its latency is very high – often measured in hours or days. This is caused by its requirement for files to close and its artificial throttles meant to prevent network congestion. With those design characteristics, the newest and hottest files in a DFS Replication replica are the least likely to replicate. Storage Replica operates below the file level and has none of these restrictions.

Storage Replica also supports asynchronous replication for longer ranges and higher latency networks. Because it is not checkpoint-based, and instead continuously replicates, the delta of changes will tend to be far lower than snapshot-based products. Furthermore, Storage Replica operates at the partition layer and therefore replicates all VSS snapshots created by Windows Server or backup software; this allows use of application-consistent data snapshots for point in time recovery, especially unstructured user data replicated asynchronously.

The Setup I used two servers both domain joined, And there are different ways to configure the Storage Replica, the easy way and the 10 second way.

First we are installing the Storage replica feature and the File server Role.  The Storage replica feature needs a reboot.

Or use Powershell

install-WindowsFeature “Storage-Replica” –IncludeAllSubFeature

If you don’t know the module name you can find it easily

A reboot is needed.

Doing this server by server is not handy, So placing this together saves us some time.

$Servers = “Building-5”,”Building-9”

$Servers | ForEach { Install-WindowsFeature -ComputerName $_ -Name Storage-Replica,FS-FileServer -IncludeManagementTools -restart }

The –restart does an automatic restart if this is needed.

Storage Replica prerequisites

• • Active Directory Domain Services forest.
  • Storage Spaces with SAS JBODs, Storage Spaces Direct, fibre channel SAN, shared VHDX, iSCSI Target, or local SAS/SCSI/SATA storage. SSD or faster recommended for replication log drives. Microsoft recommends that the log storage be faster than the data storage. Log volumes must never be used for other workloads.
  • At least one Ethernet/TCP connection on each server for synchronous replication, but preferably RDMA.
  • At least 2GB of RAM and two cores per server. (with less memory the replication won’t start)
  • A network between servers with enough bandwidth to contain your IO write workload and an average of 5ms round trip latency or lower, for synchronous replication. Asynchronous replication does not have a latency recommendation.

As there is no Gui on the replica part we need to configure this by PowerShell or with the new Windows Admin Center

• • Download and install Windows Admin Center.

Both our servers had Two extra disks. One log and Data Disk.

• • You must create two volumes on each enclosure: one for data and one for logs.
  • Log and data disks must be initialized as GPT, not MBR.
  • The two data volumes must be of identical size.
  • The two log volumes should be of identical size.
  • All replicated data disks must have the same sector sizes.
  • All log disks must have the same sector sizes.
  • The log volumes should use flash-based storage, such as SSD. Microsoft recommends that the log storage be faster than the data storage. Log volumes must never be used for other workloads.
  • The data disks can use HDD, SSD, or a tiered combination and can use either mirrored or parity spaces or RAID 1 or 10, or RAID 5 or RAID 50.
  • The log volume must be at least 9GB by default and may be larger or smaller based on log requirements.
  • The File Server role is only necessary for Test-SRTopology to operate, as it opens the necessary firewall ports for testing.

As you can see there are some needs for the Replication As I show you below with the performance test why you need this.

First we are configuring the Disks on both servers. with some PowerShell commands but this can also be done with Disk manager.

Get-Disk | Where FriendlyName -eq ‘Msft Virtual Disk’

Get-Disk | Where FriendlyName -eq ‘Msft Virtual Disk’|Initialize-Disk -PartitionStyle GPT –PassThru

1..2 | % { Get-Disk $_ }| Where FriendlyName -eq ‘Msft Virtual Disk’|New-Partition -AssignDriveLetter -UseMaximumSize | Format-Volume -FileSystem ReFS -NewFileSystemLabel “SR01-disk” -Confirm:$false

I formatted the disk with ReFS and not with NTFS.

Now that the disks are in place we can start but before we start building the replica I want to make sure the connection and the network is fast and the server can deliver the performance we need.

Therefor I download a test tool Diskspd. https://aka.ms/diskspd

Important is that the network speed between the server is good as this is the life line for the storage replica. We can test the replication before the build things for real.

With this test tool we bring up a small load to test the server.

Using the Diskspd with the line below.

Diskspd.exe -c1g -d600 -W5 -C5 -b8k -t2 -o2 -r -w5 –i100 –j2 E:\test

Storage replica has a great test tool report. So with this we configure the test. Using Powershell

MD c:\temp 

Test-SRTopology -SourceComputerName “Building-5” -SourceVolumeName “e:” -SourceLogVolumeName “f:” -DestinationComputerName “Building-9” -DestinationVolumeName “e:” -DestinationLogVolumeName “f:” -DurationInMinutes 30 -ResultPath c:\Temp

#set output file
$outputfile=”$Env:TEMP”

Test-SRTopology -SourceComputerName “Building-5” -SourceVolumeName “e:” -SourceLogVolumeName “f:” -DestinationComputerName “Building-9” -DestinationVolumeName “e:” -DestinationLogVolumeName “f:” -IntervalInSeconds 5 -DurationInMinutes 30 -ResultPath $outputfile

#open output file
If (Test-Path $outputFile) { Invoke-Item $outputFile\TestSrTopologyReport.html } Else { Write-Host “FAILED: Output file not found: $url” -fore red }
Write-Host “Done” -ForegroundColor Cyan

while running the Test-SRTopology  with the -DurationInMinutes 30  option we also run Diskspd.

Diskspd.exe -c1g -d600 -W5 -C5 -b8k -t2 -o2 -r -w5 –i100 –j2 E:\test

It is a 1 Gb file placed on our E drive that is our Data disk for replication.

As you can see I have just one network adapter and no RDMA and in this config I hit the limit of the CPU and the network card max 4.4 Gbps not bad for a test config. (if you use a better machine in Azure Pick a Azure H-series those have RDMA

One CPU with 99% usage.

When the test is done the is a log file created  in    -ResultPath c:\Temp
Open the log file and detailed information is there about the test. this is why I choose 30 min duration.

Nice graph about the Data throughput, in this case not bad.

the Latency is always a issue this could change you from sync to async or more network adapters or better disks.  But for now it is good.

Log Volume Free Disk Space Test: The log volume F: in Building-5 has enough free space to hold the recommended log volume size of 8GB

Log Volume Free Disk Space Test: The log volume F: in Building-9 has enough free space to hold the recommended log volume size of 8GB

Storage replica has not that much PowerShell commands

#list all the commands
get-command *sr*

Setting up the actual replica is done with a long PowerShell command

The default log size is 8GB. Depending on the results of the Test-SRTopology cmdlet, you may decide to use -LogSizeInBytes with a higher or lower value.

New-SRPartnership -SourceComputerName “Building-5” –SourceRGName rg01 -SourceVolumeName “e:” -SourceLogVolumeName “f:” -DestinationComputerName “Building-9” –DestinationRGName rg02 -DestinationVolumeName “e:” -DestinationLogVolumeName “f:”

The default log size is 8GB. Depending on the results of the Test-SRTopology cmdlet, you may decide to use -LogSizeInBytes with a higher or lower value.

New-SRPartnership -SourceComputerName “Building-5” –SourceRGName rg01 -SourceVolumeName “e:” -SourceLogVolumeName “f:” -DestinationComputerName “Building-9” –DestinationRGName rg02 -DestinationVolumeName “e:” -DestinationLogVolumeName “f:” -LogSizeInBytes 1gb

here you can see the disk setup between both servers, the active side you can access the data disk, on the passive side the disk is not accessible.

Don’t place files on the Log disk.

To get replication source and destination state, use Get-SRGroup and Get-SRPartnership

Get-SRGroup

Get-SRGroup |fl *

Get-SRPartnership

(Get-SRGroup).replicas

This is just after the creation so no data yet for the last time in sync.

New-SRPartnership -SourceComputerName “Building-5” –SourceRGName rg01 -SourceVolumeName “e:” -SourceLogVolumeName “f:” -DestinationComputerName “Building-9” –DestinationRGName rg02 -DestinationVolumeName “e:” -DestinationLogVolumeName “f:”

For troubleshooting there are some events that you can check, go to the event viewer and check for the Storage replica events.

Or check the events with PowerShell

Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica -max 20

On the destination server, we can do the same or look for the events in the eventlog.

Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica | Where-Object {$_.ID -eq “1215”} | fl

(Get-SRGroup).Replicas | Select-Object numofbytesremaining

There are also a lot of performance counters that can be viewed with PowerShell

Get-Counter -Counter “\Storage Replica Statistics(*)\Total Bytes Received”
Get-Counter -Counter “\Storage Replica Statistics(*)\Total Bytes Sent”
Get-Counter -Counter “\Storage Replica Statistics(*)\Avg. Network Send Latency”
Get-Counter -Counter “\Storage Replica Statistics(*)\Replication State”
Get-Counter -Counter “\Storage Replica Statistics(*)\Last Recovery Elapsed Time”
Get-Counter -Counter “\Storage Replica Partition I/O Statistics(*)\Number of times flush paused”
Get-Counter -Counter “\Storage Replica Statistics(*)\Number of Flushed Recovery Transactions”
Get-Counter -Counter “\Storage Replica Statistics(*)\Number of Recovery Transactions”
Get-Counter -Counter “\Storage Replica Statistics(*)\Number of Flushed Replication Transactions”
Get-Counter -Counter “\Storage Replica Statistics(*)\Number of Replication Transactions”
Get-Counter -Counter “\Storage Replica Statistics(*)\Number of Messages Received”
Get-Counter -Counter “\Storage Replica Statistics(*)\Number of Messages Sent”
Get-Counter -Counter “\Storage Replica Partition I/O Statistics(*)\Avg. App Write Latency”
Get-Counter -Counter “\Storage Replica Partition I/O Statistics(*)\Avg. App Read Latency”
Get-Counter -Counter “\Storage Replica Statistics(*)\Target RPO”
Get-Counter -Counter “\Storage Replica Statistics(*)\Current RPO”
Get-Counter -Counter “\Storage Replica Statistics(*)\Avg. Log Queue Length”
Get-Counter -Counter “\Storage Replica Statistics(*)\Current Log Queue Length”
Get-Counter -Counter “\Storage Replica Statistics(*)\Total Bytes Received”
Get-Counter -Counter “\Storage Replica Statistics(*)\Total Bytes Sent”
Get-Counter -Counter “\Storage Replica Statistics(*)\Avg. Network Send Latency”
Get-Counter -Counter “\Storage Replica Statistics(*)\Replication State”
Get-Counter -Counter “\Storage Replica Statistics(*)\Avg. Message Round Trip Latency”
Get-Counter -Counter “\Storage Replica Statistics(*)\Last Recovery Elapsed Time”
Get-Counter -Counter “\Storage Replica Statistics(*)\Number of Flushed Recovery Transactions”
Get-Counter -Counter “\Storage Replica Statistics(*)\Number of Recovery Transactions”
Get-Counter -Counter “\Storage Replica Statistics(*)\Number of Flushed Replication Transactions”
Get-Counter -Counter “\Storage Replica Statistics(*)\Number of Replication Transactions”
Get-Counter -Counter “\Storage Replica Statistics(*)\Max Log Sequence Number”
Get-Counter -Counter “\Storage Replica Statistics(*)\Number of Messages Received”
Get-Counter -Counter “\Storage Replica Statistics(*)\Number of Messages Sent”

these counters look like this

To remove the Replication we run the following command :

Get-SRPartnership Get-SRPartnership | Remove-SRPartnership Get-SRGroup | Remove-SRGroup

Or change the direction of the replication just run the PowerShell command

#move the replication direction from one site, use the

Set-SRPartnership -NewSourceComputerName “Building-9” -SourceRGName rg02 -DestinationComputerName “Building-5” -DestinationRGName rg01

Why not use Windows Admin Center ?

But all this PowerShell my fear you on using this. Good news than when using Windows Admin Center

Windows Admin Center is a locally deployed, browser-based app for managing servers, clusters, hyper-converged infrastructure, and Windows 10 PCs. It comes at no additional cost beyond Windows and is ready to use in production.

Get it here

When opening the Source Storage Replica server you will see a quick over view of you configuration

Easy switch replication direction.

Notifications on the preformed actions

With an overview of the current configuration.

But the best part of Windows Admin Center is creating a new Replica. I removed the old replica and create a new one with the WAC.

Fill in the source and destination and your done.

With the Admin center you got a GUI wrapper for creating the Storage replica, No PowerShell needed

So removing the replication or in case one server is dead.

Normaly you would do

Get-SRPartnership | Remove-SRPartnership –confirm:$false

this removes the replication and both locations will show the Data.

But if source server is no longer there this will not work

Remove-SRPartnership –Name RG02 -IgnoreRemovalFailure so that it breaks the partnership completely

Remove-SRPartnership [[-SourceComputerName] <String>] [-SourceRGName] <String> [-DestinationComputerName] <String> [-DestinationRGName] <String> [-IgnoreRemovalFailure] [-Force] [-CimSession <CimSession[]>] [-ThrottleLimit <Int32>] [-AsJob] [-WhatIf] [-Confirm] [<CommonParameters>]

here is the source link

https://docs.microsoft.com/en-us/powershell/module/storagereplica/remove-srpartnership?view=win10-ps

Clear-SRMetadata Removes unreferenced Storage Replica metadata.

There are more options in Windows Admin Center that could be useful to you just try it.

And if you want to use file replication to Azure take a look at the Azure File Sync https://docs.microsoft.com/en-us/azure/storage/files/storage-files-introduction

Step by Step Azure File Sync – on-premises file servers to #Azure Files Storage Sync Service

https://robertsmit.wordpress.com/2017/09/28/step-by-step-azure-file-sync-on-premises-file-servers-to-azure-files-storage-sync-service-afs-cloud-msignite/

Follow Me on Twitter @ClusterMVP

Follow My blog https://robertsmit.wordpress.com

Linkedin Profile Robert Smit MVP Linkedin profile

Google  : Robert Smit MVP profile