Archive for the ‘Azure File service’ Tag

Step by Step Azure File Sync – on-premises file servers to #Azure Files Storage Sync Service #AFS #Cloud #MSIgnite   2 comments

Finally Azure File Sync is there in public preview, for the last months I had the pleasure to work with the Azure File Sync team and tested the product and thought about some great ideas where Azure File Sync (AFS) could be useful. And I guess you all have Ideas where you could use AFS. Placing your File server somewhere and get your files to the cloud.  Our use a Azure Data Box ADB https://azure.microsoft.com/nl-nl/updates/azure-data-box-preview/

With Azure File Sync (preview), shares can be replicated on-premises or in Azure and accessed through SMB or NFS shares on Windows Server. Azure File Sync is useful for scenarios in which data needs to be accessed and modified far away from an Azure datacenter, such as in a branch office scenario. Data may be replicated between multiple Windows Server endpoints, such as between multiple branch offices.

Azure File Sync (AFS)

Azure File Sync is a multi-master sync solution, it makes it easy to solve global access problems introduced by having a single point of access on-premises, or in Azure by replicating data between Azure File shares and servers anywhere in the world. With Azure File Sync, we’ve introduced a very simple concept, the Sync Group, to help you manage the locations that should be kept in sync with each other. Every Sync Group has one cloud endpoint, which represents an Azure File share, and one or more server endpoints, which represents a path on a Windows Server. That’s it! Everything within a Sync Group will be automatically kept in sync!

    Azure File Sync enables organizations to:

    • Centralize file services in Azure storage
    • Cache data in multiple locations for fast, local performance
    • Eliminate local backup and DR

    The Azure File Sync agent is supported on Windows Server 2016 and Windows Server 2012 R2 and consists of three main components:

    • FileSyncSvc.exe: The background Windows service responsible for monitoring changes on Server Endpoints and initiating sync sessions to Azure.
    • StorageSync.sys: The Azure File Sync file system filter, responsible for tiering cold files to Azure Files (when cloud tiering is enabled).
    • PowerShell management cmdlets: PowerShell cmdlets for interacting with the Microsoft.StorageSync Azure Resource Provider. The cmdlets can be found at the following locations (by default):
  • %ProgramFiles%\Azure\StorageSyncAgent\StorageSync.Management.PowerShell.Cmdlets.dll
  • %ProgramFiles%\Azure\StorageSyncAgent\StorageSync.Management.ServerCmdlets.dll

The Azure File Sync agent also includes a preview version of the Work Folders server feature which has been updated to support Azure File Sync. This preview version of Work Folders does not have a UI and must be managed via PowerShell: https://docs.microsoft.com/en-us/powershell/module/syncshare/?view=win10-ps

But In the Preview I’m a bit Confused, what is the name of the product this Azure File Sync Or Storage Sync Service So looking it up in the Azure Store and in the quick list the name is not the Same.

imageimage

So when created the Azure File Sync <> you need to look under Storage Sync Services

image

Now that said how to built a Replica to Azure and back to my other Data Center ?

 

 Azure File Sync (AFS)

So what do we need for this scenario, We need two File servers and a storage account in Azure.

imageimage

I created on a file server mvpafs01 with an extra disk that is hosted my onprem files. on the other server MVPAFS02 the share is in a different location.

Azure File Sync extends on premises files servers into Azure providing cloud benefits while maintaining performance and compatibility.

Azure File Sync provides:

  • Multi-site access – provide write access to the same data across Windows Servers and Azure Files
  • Cloud Tiering – store only recently accessed data on local servers
  • Integrates with Azure backup – no need to back up your data on premises
  • Rapid DR – restore file metadata immediately and recall data as needed

Open your Azure subscription and look into the store for Azure File Sync.

image

 

image

Create the Azure File Sync components

imageAzure File Sync (AFS)

First we make a New Storage Account, this storage account will hold the on premise files

image

image

When the Storage account is created we create a file share on this storage account.

image

Currently the share has a maximum of 5TB !

image

Max size of a file share  5 TB

Max size of a file in a file share 1 TB

Max number of files in a file share Only limit is the 5 TB total capacity of the file share

Max IOPS per share 1000

Max number of files in a file share Only limit is the 5 TB total capacity of the file share

image

In this a limit of 4TB is more than enough to hold my files.

image

Now that the Azure File Sync is created we can configure the Azure File Sync.

First we create a sync group in this group we can sync the files from one to many.

image

If you didn’t create the Storage account and the File share you will need to create this first.

Create a sync Group

A Sync Group contains a list of endpoints that define where a set of files sync to. Servers and Azure File Shares can participate in syncing the same set of files when they are listed in the same Sync Group.

At the moment only one Azure File Share can participate in a Sync Group and it must be in the same region as this Storage Sync Service. Below you can create the Sync Group and its first and only Cloud Endpoint in one step. In the future you will be able to add more Cloud Endpoints. You can add Server Endpoints after this step completes.

After creating this Sync Group and its first Cloud Endpoint, the next step is adding one or more Server Endpoints to the Sync Group.

 

Azure File Sync (AFS)

Next step is preparing the on premise file server and install the Agent and add the Azure PowerShell modules.

To register a server:

  • Download the Azure Storage Sync agent and install it on all servers you want to sync.
  • After finishing the agent install, use the server registration utility that opens to register the server to this Storage Sync Service.

 

image image

When finishing the download of the right files we start the installation of the Agent.

  1. Download and run the StorageSyncAgent.msi.
  2. Follow the instructions to complete the installation.
  3. At the conclusion of the Azure File Sync agent installation, the Server Registration UI will auto-start.
  4. Follow the instructions to register the server with your Storage Sync Service.

Before we start the Agent we need to disable the enhanced security ( for admins only)

 

image

The installation of the Agent is simple and Quick unless the Azure Modules are not on the Server.

Azure File Sync (AFS)Azure File Sync (AFS)Azure File Sync (AFS)Azure File Sync (AFS)imageAzure File Sync (AFS)Azure File Sync (AFS) image

Now that the Agent is installed we can register this server in Azure File Sync (AFS)

Azure File Sync (AFS)

I did not have the Azure PowerShell modules on this server So I need to install the modules first

https://go.microsoft.com/fwlink/?linkid=856959

image

You can check the version with the Powershell command lets

Get-Module PowerShellGet -list | Select-Object Name,Version,Path

# Install the Azure Resource Manager modules from the PowerShell Gallery

Install-Module AzureRM

imageimage

This can take sometime but you don’t need a reboot for this.

image

just login to your azure subscription where the Azure File Sync (AFS) is installed

imageimage

Pick the right subscription and Resource Group with the Storage Sync Service.

image

The next step after the registration of the server is creating an endpoint this End point is linking the File share to the Sync service

image

 

Creating an Endpoint is the final step but remember as soon as this is in place the Sync services on the on premise server starts the initial sync!

image

Creating the Azure File Sync (AFS) Endpoint

image

A Server Endpoint integrates a subfolder of a volume from a Registered Server as a location to sync. The following considerations apply:

  • Servers must be registered to the Storage Sync Service that contains this Sync Group before you can add a location on them here.
  • A specific location on the server can only sync with one Sync Group. Syncing the same location or even a part of it – with a different Sync Group doesn’t work.
  • Make sure that the path you specify for this server is correct and not the root of a volume before hitting Create.

image

  • Cloud Tiering: A switch to enable or disable cloud tiering, which enables infrequently used or accessed files to be tiered to Azure Files.
  • Volume Free Space: the amount of free space to reserve on the volume on which the Server Endpoint resides. For example, if the Volume Free Space is set to 50% on a volume with a single Server Endpoint, roughly half the amount of data will be tiered to Azure Files. Note that regardless of whether cloud tiering is enabled, your Azure File share always has a complete copy of the data in the Sync Group.

image

Data traffic on the File server in this case it is just with one CPU. The upload speed is around the 300Mbps with almost 100% CPU

imageimage

After checking the same upload with 4 Cores and the upload is more than doubled so keep this in mind when uploading the files. Unless your line is the throttle neck

imageimage

Perfect the files are synced and ready for cloud usage.

But I also want these files in my other datacenter, I could just copy those files and in a few days I run robocopy with the delta’s but I can also use a second endpoint in Azure File Sync (AFS) and keep all files in sync.

The first step is the same as any server to register install the Azure File Sync (AFS)  Agent with the Powershell Modules

 

image

Connect with the same Azure subscription

image

As you can see the server is online and registered.

 

image

As this server doesn’t have a second disk I place all the files on a different share

image

But after filling in the share name and applied it the server gets very busy but there are no files in the folder.

Check this : all the files are cached in the System volume information folder under HFS. After the caching it placed all the files in the right folder.

Just keep in mind that this is the process and your Monitoring agents could alarm you for this. 

image

After the initial sync I have two file servers and a Azure Storage account with the same files. I can Edit files on 3 point and still it got synced.

image

The synced files on the Second server and as you can see that the System files are gone and placed in the share.

image

Hope this blog gives you the start on using the Azure File Sync (AFS) it is very useful as you could sync file between subscriptions or regions or just between your data centers.

 

Follow Me on Twitter @ClusterMVP

Follow My blog https://robertsmit.wordpress.com

Linkedin Profile Http://nl.linkedin.com/in/robertsmit

Google Me : https://www.google.nl

Bing Me : http://tinyurl.com/j6ny39w

LMGTFY : http://lmgtfy.com/?q=robert+smit+mvp+blog

Advertisements

Posted September 28, 2017 by Robert Smit [MVP] in Azure

Tagged with , , ,

Configuring cloud witness in Windows Server 2016 Cluster Azure Files #cloud #azure #winserv   7 comments

Cloud Witness is a new type of Failover Cluster quorum witness being introduced in Windows Server 2016. But you will need an Azure Subscription to use this unless you are using your own private cloud to use a file share witness in a third  DataCenter. As this post is an edited post but still actual

Earlier I create a blog post about creating a file share in Azure.

https://robertsmit.wordpress.com/2014/06/16/microsoft-azure-file-server-system-error64-or-new-azurestorageshare-cannot-bind-parameter-context-azure-cloud-mvpbuzz/

image

But now this file share can also be used for your Private Cluster Or Azure Cluster but remember your cluster needs internet access to connect to Microsoft Azure.

 

image

Edited 5-11-2014 <>>>>>>

So if you want to have a file witness then you will need the fileshare option but the cloud witness is using the BLOB storage and will cost you almost noting with Azure Credits

However the File share is still in preview under Windows Azure Files  http://msdn.microsoft.com/en-us/library/azure/dn167006.aspx

Sign up for a preview http://azure.microsoft.com/en-us/solutions/storage-backup-recovery/

Now that we have created a new storage account We can use this for the cluster.

If you select the storage account you will see something like this.

image

We are using the blob storage and skip the above but you can use this for other things

<>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

image

At the bottom you can select the Manage Access Keys. This is the information we needed for the cluster.

image

Keep this save ! You will need this in your cluster.

image

When opening my windows Cluster Failover manager under more actions you can configure the quorum settings.

image

As always we do the Advance configuration.

imageimage

Now we check the Cloud Witness option.

image

We use the Account name and the Key from the Azure portal that we created earlier.

The screen may differ as you could have a different version!

imageimage

And we have a Cloud Witness here configured in my Public Cloud.

image

We don’t need a site to site VPN just connect the Cluster nodes to Internet ( atleast a Azure connection )

image

If we check the Azure Configuration and there you can see the creation of MSFT-Cloud-Witness. With the files in the blob storage.

 

 

You can also configure this in azure.

With the Get-ClusterQuorum we find quickly what witness we have.

image

and Configuring this is also quickly done put your account name and key in the set-clusterquorum and it will create a Cloud Witness.

Set-ClusterQuorum -AccessKey V7CR1/DijezGyA== -AccountName clusterw10 –CloudWitness

image

Happy clustering

Flickr Tags: ,,,,,

Follow Me on Twitter @ClusterMVP

Follow My blog https://robertsmit.wordpress.com

Linkedin Profile Http://nl.linkedin.com/in/robertsmit

Google Me : https://www.google.nl

Bing Me : http://tinyurl.com/j6ny39w

LMGTFY : http://lmgtfy.com/?q=robert+smit+mvp+blog

  • Twitter

  • %d bloggers like this: