Finally Azure File Sync is there in public preview, for the last months I had the pleasure to work with the Azure File Sync team and tested the product and thought about some great ideas where Azure File Sync (AFS) could be useful. And I guess you all have Ideas where you could use AFS. Placing your File server somewhere and get your files to the cloud. Our use a Azure Data Box ADB https://azure.microsoft.com/nl-nl/updates/azure-data-box-preview/
With Azure File Sync (preview), shares can be replicated on-premises or in Azure and accessed through SMB or NFS shares on Windows Server. Azure File Sync is useful for scenarios in which data needs to be accessed and modified far away from an Azure datacenter, such as in a branch office scenario. Data may be replicated between multiple Windows Server endpoints, such as between multiple branch offices.
Azure File Sync is a multi-master sync solution, it makes it easy to solve global access problems introduced by having a single point of access on-premises, or in Azure by replicating data between Azure File shares and servers anywhere in the world. With Azure File Sync, we’ve introduced a very simple concept, the Sync Group, to help you manage the locations that should be kept in sync with each other. Every Sync Group has one cloud endpoint, which represents an Azure File share, and one or more server endpoints, which represents a path on a Windows Server. That’s it! Everything within a Sync Group will be automatically kept in sync!
- Azure File Sync enables organizations to:
- Centralize file services in Azure storage
- Cache data in multiple locations for fast, local performance
- Eliminate local backup and DR
The Azure File Sync agent is supported on Windows Server 2016 and Windows Server 2012 R2 and consists of three main components:
- FileSyncSvc.exe: The background Windows service responsible for monitoring changes on Server Endpoints and initiating sync sessions to Azure.
- StorageSync.sys: The Azure File Sync file system filter, responsible for tiering cold files to Azure Files (when cloud tiering is enabled).
- PowerShell management cmdlets: PowerShell cmdlets for interacting with the Microsoft.StorageSync Azure Resource Provider. The cmdlets can be found at the following locations (by default):
- %ProgramFiles%\Azure\StorageSyncAgent\StorageSync.Management.PowerShell.Cmdlets.dll
- %ProgramFiles%\Azure\StorageSyncAgent\StorageSync.Management.ServerCmdlets.dll
The Azure File Sync agent also includes a preview version of the Work Folders server feature which has been updated to support Azure File Sync. This preview version of Work Folders does not have a UI and must be managed via PowerShell: https://docs.microsoft.com/en-us/powershell/module/syncshare/?view=win10-ps
- To learn more about Work Folders, please refer to the overview: https://docs.microsoft.com/en-us/windows-server/storage/work-folders/work-folders-overview
To learn more about Azure File Sync, please refer to the planning guide: https://docs.microsoft.com/azure/storage/files/storage-sync-files-planning
But In the Preview I’m a bit Confused, what is the name of the product this Azure File Sync Or Storage Sync Service So looking it up in the Azure Store and in the quick list the name is not the Same.
So when created the Azure File Sync <> you need to look under Storage Sync Services
Now that said how to built a Replica to Azure and back to my other Data Center ?
So what do we need for this scenario, We need two File servers and a storage account in Azure.
I created on a file server mvpafs01 with an extra disk that is hosted my onprem files. on the other server MVPAFS02 the share is in a different location.
Azure File Sync extends on premises files servers into Azure providing cloud benefits while maintaining performance and compatibility.
Azure File Sync provides:
- Multi-site access – provide write access to the same data across Windows Servers and Azure Files
- Cloud Tiering – store only recently accessed data on local servers
- Integrates with Azure backup – no need to back up your data on premises
- Rapid DR – restore file metadata immediately and recall data as needed
Open your Azure subscription and look into the store for Azure File Sync.
Create the Azure File Sync components
First we make a New Storage Account, this storage account will hold the on premise files
When the Storage account is created we create a file share on this storage account.
Currently the share has a maximum of 5TB !
Max size of a file share 5 TB
Max size of a file in a file share 1 TB
Max number of files in a file share Only limit is the 5 TB total capacity of the file share
Max IOPS per share 1000
Max number of files in a file share Only limit is the 5 TB total capacity of the file share
In this a limit of 4TB is more than enough to hold my files.
Now that the Azure File Sync is created we can configure the Azure File Sync.
First we create a sync group in this group we can sync the files from one to many.
If you didn’t create the Storage account and the File share you will need to create this first.
Create a sync Group
A Sync Group contains a list of endpoints that define where a set of files sync to. Servers and Azure File Shares can participate in syncing the same set of files when they are listed in the same Sync Group.
At the moment only one Azure File Share can participate in a Sync Group and it must be in the same region as this Storage Sync Service. Below you can create the Sync Group and its first and only Cloud Endpoint in one step. In the future you will be able to add more Cloud Endpoints. You can add Server Endpoints after this step completes.
After creating this Sync Group and its first Cloud Endpoint, the next step is adding one or more Server Endpoints to the Sync Group.
Next step is preparing the on premise file server and install the Agent and add the Azure PowerShell modules.
To register a server:
- Download the Azure Storage Sync agent and install it on all servers you want to sync.
- After finishing the agent install, use the server registration utility that opens to register the server to this Storage Sync Service.
When finishing the download of the right files we start the installation of the Agent.
- Download and run the StorageSyncAgent.msi.
- Follow the instructions to complete the installation.
- At the conclusion of the Azure File Sync agent installation, the Server Registration UI will auto-start.
- Follow the instructions to register the server with your Storage Sync Service.
Before we start the Agent we need to disable the enhanced security ( for admins only)
The installation of the Agent is simple and Quick unless the Azure Modules are not on the Server.
Now that the Agent is installed we can register this server in Azure File Sync (AFS)
I did not have the Azure PowerShell modules on this server So I need to install the modules first
https://go.microsoft.com/fwlink/?linkid=856959
You can check the version with the Powershell command lets
Get-Module PowerShellGet -list | Select-Object Name,Version,Path
# Install the Azure Resource Manager modules from the PowerShell Gallery
Install-Module AzureRM
This can take sometime but you don’t need a reboot for this.
just login to your azure subscription where the Azure File Sync (AFS) is installed
Pick the right subscription and Resource Group with the Storage Sync Service.
The next step after the registration of the server is creating an endpoint this End point is linking the File share to the Sync service
Creating an Endpoint is the final step but remember as soon as this is in place the Sync services on the on premise server starts the initial sync!
Creating the Azure File Sync (AFS) Endpoint
A Server Endpoint integrates a subfolder of a volume from a Registered Server as a location to sync. The following considerations apply:
- Servers must be registered to the Storage Sync Service that contains this Sync Group before you can add a location on them here.
- A specific location on the server can only sync with one Sync Group. Syncing the same location or even a part of it – with a different Sync Group doesn’t work.
- Make sure that the path you specify for this server is correct and not the root of a volume before hitting Create.
- Cloud Tiering: A switch to enable or disable cloud tiering, which enables infrequently used or accessed files to be tiered to Azure Files.
- Volume Free Space: the amount of free space to reserve on the volume on which the Server Endpoint resides. For example, if the Volume Free Space is set to 50% on a volume with a single Server Endpoint, roughly half the amount of data will be tiered to Azure Files. Note that regardless of whether cloud tiering is enabled, your Azure File share always has a complete copy of the data in the Sync Group.
Data traffic on the File server in this case it is just with one CPU. The upload speed is around the 300Mbps with almost 100% CPU
After checking the same upload with 4 Cores and the upload is more than doubled so keep this in mind when uploading the files. Unless your line is the throttle neck
Perfect the files are synced and ready for cloud usage.
But I also want these files in my other datacenter, I could just copy those files and in a few days I run robocopy with the delta’s but I can also use a second endpoint in Azure File Sync (AFS) and keep all files in sync.
The first step is the same as any server to register install the Azure File Sync (AFS) Agent with the Powershell Modules
Connect with the same Azure subscription
As you can see the server is online and registered.
As this server doesn’t have a second disk I place all the files on a different share
But after filling in the share name and applied it the server gets very busy but there are no files in the folder.
Check this : all the files are cached in the System volume information folder under HFS. After the caching it placed all the files in the right folder.
Just keep in mind that this is the process and your Monitoring agents could alarm you for this.
After the initial sync I have two file servers and a Azure Storage account with the same files. I can Edit files on 3 point and still it got synced.
The synced files on the Second server and as you can see that the System files are gone and placed in the share.
Hope this blog gives you the start on using the Azure File Sync (AFS) it is very useful as you could sync file between subscriptions or regions or just between your data centers.
Follow Me on Twitter @ClusterMVP
Follow My blog https://robertsmit.wordpress.com
Linkedin Profile Http://nl.linkedin.com/in/robertsmit
Google Me : https://www.google.nl
Bing Me : http://tinyurl.com/j6ny39w
LMGTFY : http://lmgtfy.com/?q=robert+smit+mvp+blog
Share this:
Robert Smit MVP Blog 
Hi
Do you know when Azure File Sync will be out of Preview and production ready?
Well They announced it on Ignite and Microsoft announced always new products on events. so it could be on Ignite. Else I have no idea.
Currently the preview is working great. check Microsoft Doc’s for more info
https://docs.microsoft.com/en-us/azure/storage/files/storage-sync-files-planning
Azure File Sync is now GA
https://azure.microsoft.com/en-us/blog/score-one-for-the-it-pro-azure-file-sync-is-now-generally-available/
Looking at the diagram, it suggests that there are 2 cloud regions involved. How is that configured? I’ve looked for ages and all I can see is enabling Geo Redundant storage, but that does not give you control of which regions to replicate to.
Hi Robert,
Thank you for your detailed article.
Sorry if my question is not so related to the topic:
“Currently we are finding a solution for File Sharing over SMB for External Users. We want the Internet users can map network-drive and access files over the internet via SMB 445 port.”
We know Azure Files could help but it is not yet support for managing permissions, so do you have any suggestion?
Hi John,
everything is in the same region the Azure Storage Sync services is basicly an agent who is syncing your data to a storage account. and from there you can sync the to any other server but it need a conection to the sync services.
So you are free to replicate the data. the Geo redundant storage is a way different option.
https://docs.microsoft.com/en-us/azure/storage/files/storage-sync-files-deployment-guide?tabs=portal
Hi,
Well a combination of the two could be your solution using the onprem file server sync this to Azure with azure File sync. then give the internet user a specific share. the permissions can only be chanhed on the on prem server.
Hi Robert,
This seems to be quite dangerous if we publish SMB port over the Internet on On-Prem File Server. So now I am contacting Azure Files Team for information about the support of Authentication over Azure AD. Hope that with this feature, it could be helpful.
The Data is all encrypted. But I understand your point.
Looking to see if anyone has used the Azure Data Box to pre-seed an Azure File Sync. We have ~140 TB in our current file shares and not looking forward to syncing that over a 1Gbps link.
Hi Databox is your best option. Microsoft will send you a device and you can copy this to this devices and ship the device back.
you need to make the folders first in the blob storage but all is explained well https://azure.microsoft.com/en-us/services/storage/databox/
Hi, I used your instructions, very clear and helpful however I am having one issue with a file. Its a VHD file I am looking to use for a VM on azure. I have tried uploading this via the browser and it just fails (its 460GB) but I thought that this method would work however it has synced other files but not the .vhd file. Is this expected, have I missed something here?
the file size limit is 100GB but it is not a hard limit but depends on the line speed etc. the real hard limit is <1TB file size but syncing large files is always a critical thing.
Hi there a simple way to configure an email alert for the health of the registered servers and sync groups? I disconnected the internet from my lab vm to see what happens. There is no way to tell that its broken other than checking the azure portal or looking in the event logs of the vm.
HI, Thanks for reading my blog. In this case you could set alerts on the events that happened in the eventlog, or run a powershell check every 5 min and then do a send email. this is all free and already in the Server 2019 product. else you will need some tooling to alert you. or use Azure monitoring an Alerting as soon as the connection drops it triggers an alert. but it all depends on the needs and setup that you want alerts. or check this blog post from a fellow MVP its about a service but the same is for an event https://clusteringformeremortals.com/2018/10/29/step-by-step-how-to-trigger-an-email-alert-when-a-windows-service-starts-or-stops-on-windows-server-2016/
Thanks, the alerting could also a good follow up blog post.
This doesn’t sound simple sorry Robert. The alerting would make an excellent follow up to this blog post. How hard would this be for MS to build this into Azure File Sync setup. I feel like its such an important feature to leave out of the product.
What if after succesfully setting up the sync service and registering the file server you cannot add a server endpoint because it is greyed out? I run into this and evertyhing looks/reports itself healthy, but I just can’t get the sync to start.
Then it looks like that there is no connection, make sure the ports are open and no FW is blocking the ports of virusscanner is blocking files
I am thinking of syncing two Azure file shares to the same local server. The first one is to the share where users are working in normally. The second should be to a new file share for storing archived projects.
Is this possible or do I need a second server for the second share? Or what other things I need to think of?
It depends on the load of these file shares but it they have running on the same server then just sync them to azure, using the same agent just configure a new share and sync that. I did not test this in the blog but the method is the same. Let me know how it goes.