Step by Step Server-to-server storage replication with Windows Server 2019 Storage Replica #WindowsAdminCenter #StorageReplica #WindowsServer2019 #ReFS #SR #Azure   2 comments

In the old days all File servers where on one place, and if you want to replicate data you needed a extra tool to do this. Now days its already build in into Windows server. Storage replica can be used in several ways, replicate data from one Cluster to another or to Azure. but in this case I do a server to server replication as not everyone has a cluster.

For moving data to the Cloud there are currently several other applications like Azure file sync or Azure Migrate https://docs.microsoft.com/en-us/azure/migrate/migrate-overview Blog about Azure File Sync https://robertsmit.wordpress.com/2017/09/28/step-by-step-azure-file-sync-on-premises-file-servers-to-azure-files-storage-sync-service-afs-cloud-msignite/

Storage Replica is Windows Server technology that enables replication of volumes between servers or clusters for disaster recovery. It also enables you to create stretch failover clusters that span two sites, with all nodes staying in sync.

Storage Replica supports synchronous and asynchronous replication:

  • Synchronous replication mirrors data within a low-latency network site with crash-consistent volumes to ensure zero data loss at the file-system level during a failure.
  • Asynchronous replication mirrors data across sites beyond metropolitan ranges over network links with higher latencies, but without a guarantee that both sites have identical copies of the data at the time of a failure.

Storage Replica allows more efficient use of multiple datacenters. By stretching clusters or replicating clusters, workloads can be run in multiple datacenters for quicker data access by local proximity users and applications, as well as better load distribution and use of compute resources. If a disaster takes one datacenter offline, you can move its typical workloads to the other site temporarily.

Storage Replica may allow you to decommission existing file replication systems such as DFS Replication that were pressed into duty as low-end disaster recovery solutions. While DFS Replication works well over extremely low bandwidth networks, its latency is very high – often measured in hours or days. This is caused by its requirement for files to close and its artificial throttles meant to prevent network congestion. With those design characteristics, the newest and hottest files in a DFS Replication replica are the least likely to replicate. Storage Replica operates below the file level and has none of these restrictions.

Storage Replica also supports asynchronous replication for longer ranges and higher latency networks. Because it is not checkpoint-based, and instead continuously replicates, the delta of changes will tend to be far lower than snapshot-based products. Furthermore, Storage Replica operates at the partition layer and therefore replicates all VSS snapshots created by Windows Server or backup software; this allows use of application-consistent data snapshots for point in time recovery, especially unstructured user data replicated asynchronously.

The Setup I used two servers both domain joined, And there are different ways to configure the Storage Replica, the easy way and the 10 second way.

Diagram showing a server in Building 5 replicating with a server in Building 9

First we are installing the Storage replica feature and the File server Role.  The Storage replica feature needs a reboot.

image

Or use Powershell

install-WindowsFeature "Storage-Replica" –IncludeAllSubFeature

If you don’t know the module name you can find it easily

install-WindowsFeature "Storage-Replica" -IncludeAllSubFeature

A reboot is needed.

install-WindowsFeature "Storage-Replica" -IncludeAllSubFeature

Doing this server by server is not handy, So placing this together saves us some time.

$Servers = “Building-5”,”Building-9”

$Servers | ForEach { Install-WindowsFeature -ComputerName $_ -Name Storage-Replica,FS-FileServer -IncludeManagementTools -restart }

The –restart does an automatic restart if this is needed.

image

Storage Replica prerequisites

  • Active Directory Domain Services forest.
  • Storage Spaces with SAS JBODs, Storage Spaces Direct, fibre channel SAN, shared VHDX, iSCSI Target, or local SAS/SCSI/SATA storage. SSD or faster recommended for replication log drives. Microsoft recommends that the log storage be faster than the data storage. Log volumes must never be used for other workloads.
  • At least one Ethernet/TCP connection on each server for synchronous replication, but preferably RDMA.
  • At least 2GB of RAM and two cores per server. (with less memory the replication won’t start)
  • A network between servers with enough bandwidth to contain your IO write workload and an average of 5ms round trip latency or lower, for synchronous replication. Asynchronous replication does not have a latency recommendation.

As there is no Gui on the replica part we need to configure this by PowerShell or with the new Windows Admin Center

Both our servers had Two extra disks. One log and Data Disk.

image

image

  • You must create two volumes on each enclosure: one for data and one for logs.
  • Log and data disks must be initialized as GPT, not MBR.
  • The two data volumes must be of identical size.
  • The two log volumes should be of identical size.
  • All replicated data disks must have the same sector sizes.
  • All log disks must have the same sector sizes.
  • The log volumes should use flash-based storage, such as SSD. Microsoft recommends that the log storage be faster than the data storage. Log volumes must never be used for other workloads.
  • The data disks can use HDD, SSD, or a tiered combination and can use either mirrored or parity spaces or RAID 1 or 10, or RAID 5 or RAID 50.
  • The log volume must be at least 9GB by default and may be larger or smaller based on log requirements.
  • The File Server role is only necessary for Test-SRTopology to operate, as it opens the necessary firewall ports for testing.

As you can see there are some needs for the Replication As I show you below with the performance test why you need this.

First we are configuring the Disks on both servers. with some PowerShell commands but this can also be done with Disk manager.

Get-Disk | Where FriendlyName -eq ‘Msft Virtual Disk’

image

Get-Disk | Where FriendlyName -eq ‘Msft Virtual Disk’|Initialize-Disk -PartitionStyle GPT –PassThru

image

1..2 | % { Get-Disk $_ }| Where FriendlyName -eq ‘Msft Virtual Disk’|New-Partition -AssignDriveLetter -UseMaximumSize | Format-Volume -FileSystem ReFS -NewFileSystemLabel "SR01-disk" -Confirm:$false

image

I formatted the disk with ReFS and not with NTFS.

Now that the disks are in place we can start but before we start building the replica I want to make sure the connection and the network is fast and the server can deliver the performance we need.

Therefor I download a test tool Diskspd. https://aka.ms/diskspd

Important is that the network speed between the server is good as this is the life line for the storage replica. We can test the replication before the build things for real.

With this test tool we bring up a small load to test the server.

image

Using the Diskspd with the line below.

Diskspd.exe -c1g -d600 -W5 -C5 -b8k -t2 -o2 -r -w5 –i100 –j2 E:\test

Storage replica has a great test tool report. So with this we configure the test. Using Powershell

MD c:\temp 

Test-SRTopology -SourceComputerName "Building-5" -SourceVolumeName "e:" -SourceLogVolumeName "f:" -DestinationComputerName "Building-9" -DestinationVolumeName "e:" -DestinationLogVolumeName "f:" -DurationInMinutes 30 -ResultPath c:\Temp

#set output file
$outputfile="$Env:TEMP"

Test-SRTopology -SourceComputerName "Building-5" -SourceVolumeName "e:" -SourceLogVolumeName "f:" -DestinationComputerName "Building-9" -DestinationVolumeName "e:" -DestinationLogVolumeName "f:" -IntervalInSeconds 5 -DurationInMinutes 30 -ResultPath $outputfile

#open output file
If (Test-Path $outputFile) { Invoke-Item $outputFile\TestSrTopologyReport.html } Else { Write-Host "FAILED: Output file not found: $url" -fore red }
Write-Host "Done" -ForegroundColor Cyan

imageimage

while running the Test-SRTopology  with the -DurationInMinutes 30  option we also run Diskspd.

Diskspd.exe -c1g -d600 -W5 -C5 -b8k -t2 -o2 -r -w5 –i100 –j2 E:\test

It is a 1 Gb file placed on our E drive that is our Data disk for replication.

imageimage

As you can see I have just one network adapter and no RDMA and in this config I hit the limit of the CPU and the network card max 4.4 Gbps not bad for a test config. (if you use a better machine in Azure Pick a Azure H-series those have RDMA

image

One CPU with 99% usage.

When the test is done the is a log file created  in    -ResultPath c:\Temp
Open the log file and detailed information is there about the test. this is why I choose 30 min duration.

image

Nice graph about the Data throughput, in this case not bad.

image

the Latency is always a issue this could change you from sync to async or more network adapters or better disks.  But for now it is good.

image

Log Volume Free Disk Space Test: The log volume F: in Building-5 has enough free space to hold the recommended log volume size of 8GB

Log Volume Free Disk Space Test: The log volume F: in Building-9 has enough free space to hold the recommended log volume size of 8GB

Storage replica has not that much PowerShell commands

#list all the commands
get-command *sr*

Setting up the actual replica is done with a long PowerShell command

The default log size is 8GB. Depending on the results of the Test-SRTopology cmdlet, you may decide to use -LogSizeInBytes with a higher or lower value.

New-SRPartnership -SourceComputerName "Building-5" –SourceRGName rg01 -SourceVolumeName "e:" -SourceLogVolumeName "f:" -DestinationComputerName "Building-9" –DestinationRGName rg02 -DestinationVolumeName "e:" -DestinationLogVolumeName "f:"

image

The default log size is 8GB. Depending on the results of the Test-SRTopology cmdlet, you may decide to use -LogSizeInBytes with a higher or lower value.

New-SRPartnership -SourceComputerName "Building-5" –SourceRGName rg01 -SourceVolumeName "e:" -SourceLogVolumeName "f:" -DestinationComputerName "Building-9" –DestinationRGName rg02 -DestinationVolumeName "e:" -DestinationLogVolumeName "f:" -LogSizeInBytes 1gb

image

here you can see the disk setup between both servers, the active side you can access the data disk, on the passive side the disk is not accessible.

Don’t place files on the Log disk.

To get replication source and destination state, use Get-SRGroup and Get-SRPartnership

Get-SRGroup

Get-SRGroup |fl *

image

Get-SRPartnership

image

(Get-SRGroup).replicas

image

This is just after the creation so no data yet for the last time in sync.

New-SRPartnership -SourceComputerName "Building-5" –SourceRGName rg01 -SourceVolumeName "e:" -SourceLogVolumeName "f:" -DestinationComputerName "Building-9" –DestinationRGName rg02 -DestinationVolumeName "e:" -DestinationLogVolumeName "f:"

For troubleshooting there are some events that you can check, go to the event viewer and check for the Storage replica events.

image

Or check the events with PowerShell

Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica -max 20

image

On the destination server, we can do the same or look for the events in the eventlog.

Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica | Where-Object {$_.ID -eq "1215"} | fl

image

(Get-SRGroup).Replicas | Select-Object numofbytesremaining

There are also a lot of performance counters that can be viewed with PowerShell

Get-Counter -Counter "\Storage Replica Statistics(*)\Total Bytes Received"
Get-Counter -Counter "\Storage Replica Statistics(*)\Total Bytes Sent"
Get-Counter -Counter "\Storage Replica Statistics(*)\Avg. Network Send Latency"
Get-Counter -Counter "\Storage Replica Statistics(*)\Replication State"
Get-Counter -Counter "\Storage Replica Statistics(*)\Last Recovery Elapsed Time"
Get-Counter -Counter "\Storage Replica Partition I/O Statistics(*)\Number of times flush paused"
Get-Counter -Counter "\Storage Replica Statistics(*)\Number of Flushed Recovery Transactions"
Get-Counter -Counter "\Storage Replica Statistics(*)\Number of Recovery Transactions"
Get-Counter -Counter "\Storage Replica Statistics(*)\Number of Flushed Replication Transactions"
Get-Counter -Counter "\Storage Replica Statistics(*)\Number of Replication Transactions"
Get-Counter -Counter "\Storage Replica Statistics(*)\Number of Messages Received"
Get-Counter -Counter "\Storage Replica Statistics(*)\Number of Messages Sent"
Get-Counter -Counter "\Storage Replica Partition I/O Statistics(*)\Avg. App Write Latency"
Get-Counter -Counter "\Storage Replica Partition I/O Statistics(*)\Avg. App Read Latency"
Get-Counter -Counter "\Storage Replica Statistics(*)\Target RPO"
Get-Counter -Counter "\Storage Replica Statistics(*)\Current RPO"
Get-Counter -Counter "\Storage Replica Statistics(*)\Avg. Log Queue Length"
Get-Counter -Counter "\Storage Replica Statistics(*)\Current Log Queue Length"
Get-Counter -Counter "\Storage Replica Statistics(*)\Total Bytes Received"
Get-Counter -Counter "\Storage Replica Statistics(*)\Total Bytes Sent"
Get-Counter -Counter "\Storage Replica Statistics(*)\Avg. Network Send Latency"
Get-Counter -Counter "\Storage Replica Statistics(*)\Replication State"
Get-Counter -Counter "\Storage Replica Statistics(*)\Avg. Message Round Trip Latency"
Get-Counter -Counter "\Storage Replica Statistics(*)\Last Recovery Elapsed Time"
Get-Counter -Counter "\Storage Replica Statistics(*)\Number of Flushed Recovery Transactions"
Get-Counter -Counter "\Storage Replica Statistics(*)\Number of Recovery Transactions"
Get-Counter -Counter "\Storage Replica Statistics(*)\Number of Flushed Replication Transactions"
Get-Counter -Counter "\Storage Replica Statistics(*)\Number of Replication Transactions"
Get-Counter -Counter "\Storage Replica Statistics(*)\Max Log Sequence Number"
Get-Counter -Counter "\Storage Replica Statistics(*)\Number of Messages Received"
Get-Counter -Counter "\Storage Replica Statistics(*)\Number of Messages Sent"

these counters look like this

image

To remove the Replication we run the following command :

Get-SRPartnership Get-SRPartnership | Remove-SRPartnership Get-SRGroup | Remove-SRGroup

Or change the direction of the replication just run the PowerShell command

#move the replication direction from one site, use the

Set-SRPartnership -NewSourceComputerName "Building-9" -SourceRGName rg02 -DestinationComputerName "Building-5" -DestinationRGName rg01

Why not use Windows Admin Center ?

But all this PowerShell my fear you on using this. Good news than when using Windows Admin Center

Windows Admin Center is a locally deployed, browser-based app for managing servers, clusters, hyper-converged infrastructure, and Windows 10 PCs. It comes at no additional cost beyond Windows and is ready to use in production.

Get it here

When opening the Source Storage Replica server you will see a quick over view of you configuration

image

Easy switch replication direction.

imageimage

Notifications on the preformed actions

image

With an overview of the current configuration.

But the best part of Windows Admin Center is creating a new Replica. I removed the old replica and create a new one with the WAC.

Fill in the source and destination and your done.

imageimageimage

With the Admin center you got a GUI wrapper for creating the Storage replica, No PowerShell needed

image

There are more options in Windows Admin Center that could be useful to you just try it.

 

And if you want to use file replication to Azure take a look at the Azure File Sync https://docs.microsoft.com/en-us/azure/storage/files/storage-files-introduction

Step by Step Azure File Sync – on-premises file servers to #Azure Files Storage Sync Service

https://robertsmit.wordpress.com/2017/09/28/step-by-step-azure-file-sync-on-premises-file-servers-to-azure-files-storage-sync-service-afs-cloud-msignite/

 

Follow Me on Twitter @ClusterMVP

Follow My blog https://robertsmit.wordpress.com

Linkedin Profile Robert Smit MVP Linkedin profile

Google  : Robert Smit MVP profile

Posted October 30, 2018 by Robert Smit [MVP] in Windows Server 2019

Tagged with

2 responses to “Step by Step Server-to-server storage replication with Windows Server 2019 Storage Replica #WindowsAdminCenter #StorageReplica #WindowsServer2019 #ReFS #SR #Azure

Subscribe to comments with RSS.

  1. How does this compare to DFS since it does not allow you to have 2 writable file systems in 2 different Geo regions?

    • Hi,
      Yes SR is replication blocks from one location to the other. it is fast and Synchronous replication mirrors data on low-latency high-bandwidth networks to ensure zero data loss at the file-system level in the case of a failure,asynchronous replication which is designed for use across high-latency networks.
      DFS is provision namespaces to access files without needing to know their physical location.DFS Replication was designed for use over low-bandwidth networks. DFS-R also requires that files be closed before they can be replicated.
      Storage Replica works at block-level, it can replicate open files.It replicates all VSS snapshots, enabling administrators to run application-aware, point-in-time restore operations using backups created by Windows Backup or third-party backup solutions that use Microsoft’s APIs.

      But if you want to have multiple locations and access to the files maybe Azure file sync it the right solution.
      https://robertsmit.wordpress.com/2017/09/28/step-by-step-azure-file-sync-on-premises-file-servers-to-azure-files-storage-sync-service-afs-cloud-msignite/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  • Twitter

  • %d bloggers like this: