Author Archive

#Azure Storage Spaces direct #S2D Standard Storage vs Premium Storage   Leave a comment

I see this often in the Forums Should I use Standard Storage or should I use Premium storage. Well it Depends Premium cost more that Standard but even that depends in the basic. Can a $ 4000 Azure Storage space configuration  out perform a $ 1700 Premium configuration. this blog post is not on how to configure Storage spaces but more an overview on concepts, did I pick the right machine or did I build the right configuration well it all depends.

I love the HPC vm sizes https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-hpc but also expensive.

So in these setups I did create a storage space direct configuration all almost basic. but Key is here pick the Right VM for the job.

Standard 6 node cluster 4 core 8GB memory total disks 96 Type S30 (1TB) RAW disk space 96TB  and 32TB for the vDisk

Premium 3 node Cluster 2 core 16GB memory Total disks 9 Type P30 (1TB) RAW disk space 9TB  and 3TB for the vDisk

Standard A8 (RDMA) 5 node cluster 8 core 56GB memory total disks 80 Type p20 (500GB) RAW disk space 40TB

So basically comparing both configs makes no sense Couse  both configs are different. bigger machines vs little VM

and a lot less storage.

Standard Storage storage vs Premium

The performance of standard disks varies with the VM size to which the disk is attached, not to the size of the disk.

image

So the nodes have 16 disk each 16 * 500 IOPS  and with a max bandwidth of 480 Mbps. that could be a issue as would I use the full GB network than I need atleast  125 MB/s

image

In the Premium it is all great building the same config as in the standard the cost would be $3300 vs $12000. If you have a solution and you need the specifications then this is the way to go.

Can I out perform the configuration with standard disks ? In an old blog post I did the performance test on a 5 node A8 machine and 16 premium storage P20- 500GB 40TB RAW and got a network throughput of 4.2Gbps 

image

https://robertsmit.wordpress.com/2016/01/05/using-windows-storage-spaces-direct-with-hyper-converged-in-microsoft-azure-with-windows-server-2016/

Measurements are different on different machines and basically there is no one size fits all it all depends on the workload or config or needs.

using the script from (by Mikael Nystrom, Microsoft MVP) on the basic disk not very impressive list  high latency but that’s the Standard storage.

imageimage

The premium Storage is way faster and constant. So when using Azure and you need an amount of load or VM’s there is so much choice if you pick a different machine the results can be better. when hitting the IOPS ceiling of the VM. Prepare some calculations when building your new solution.  Test some configurations first before you go in production.

Azure is changing everyday today this may be the best solution but outdated tomorrow.

Below are some useful links on the Machine type and storage type.

https://docs.microsoft.com/en-us/azure/virtual-machines/windows/acu

 https://docs.microsoft.com/en-us/azure/virtual-machines/windows/standard-storage

https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-memory#ds-series

 

Thanks for reading my blog. Did you check my other blog post about Azure File Sync : https://robertsmit.wordpress.com/2017/09/28/step-by-step-azure-file-sync-on-premises-file-servers-to-azure-files-storage-sync-service-afs-cloud-msignite/

 

 

Follow Me on Twitter @ClusterMVP

Follow My blog https://robertsmit.wordpress.com

Linkedin Profile Http://nl.linkedin.com/in/robertsmit

Google Me : https://www.google.nl

Bing Me : http://tinyurl.com/j6ny39w

LMGTFY : http://lmgtfy.com/?q=robert+smit+mvp+blog

Advertisements

Posted November 9, 2017 by Robert Smit [MVP] in Windows Cluster, Windows Server 2016

Tagged with

Getting Started with #Azure Data Science Virtual Machine on Windows 2016 #DSVM #winserv #VSTS #DevOps   Leave a comment

 

The Data Science Virtual Machine (DSVM) is a ‘Windows Server 2016 with Containers’ VM & includes popular tools for data exploration, analysis, modeling & development.

Highlights:

  • Microsoft R Server – Dev. Ed. (Scalable R)
  • Anaconda Python
  • SQL Server 2017 Dev. Ed. – With In-Database R and Python analytics
  • Microsoft Office 365 ProPlus BYOL – Shared Computer Activation
  • Julia Pro + Juno Editor
  • Jupyter notebooks
  • Visual Studio Community Ed. + Python, R & node.js tools
  • Power BI Desktop
  • Deep learning tools e.g. Microsoft Cognitive Toolkit (CNTK 2.1), TensorFlow & mxnet
  • ML algorithm libraries e.g. xgboost, Vowpal Wabbit
  • Azure SDKs + libraries for various Azure Cloud offerings. Integration tools are included for: 
    1. Azure Machine Learning
    2. Azure Data Factory
    3. Stream Analytics
    4. SQL Data Warehouse
    5. Hadoop + Apache Spark (HDICluster)
    6. Data Lake
    7. Blob storage
    8. ML & Data Science tutorials as Jupyter notebooks

    Tools for ML model operationalization as web services in the cloud, using Azure ML or Microsoft R Server.

    Pre-configured and tested with Nvidia drivers, CUDA Toolkit, & NVIDIA cuDNN library for GPU workloads available if using NC class VM SKUs.

  •  

    Starting in the Azure Portal

    GO to New or +

    image

    Search for Data Science Virtual Machine (DSVM)

    image

    Select the {csp} Data Science Virtual Machine  – Windows 2016 option. 

    image

    Next fill in the username and password with resource group.

    image 

    Pick a machine type. When you pick a higher machine type when deploying every thing is way faster than just picking a Standard_A1 size.

    image

     

    As you can see there is a orange image mark in the text that the cost will be billed separately.

    Offer details

    Data Science Virtual Machine – Windows 2016

    0.0000 EUR/hr

    Good to know there are no cost and this is free. you need to pay for the Azure VM! in my case a E32s v3

    The highlighted Marketplace purchase(s) are not covered by your Azure credits, and will be billed separately.
    You cannot use your Azure monetary commitment funds or subscription credits for these purchases. You will be billed separately for marketplace purchases.

    image

    not bad 9 minute install with a long list of tools Office, visual studio , Visual studio Code,etc

    There is not a free license for the office and studio product but you can sign in with your credentials.

    image

    Thanks to the Big compute everything is running awesome.

    image

    As you can see all the tools are there, some needs a configuration so no default things that needs to be removed first just ready to start with out the long installation of all the tools.

    image

    What was missing on the Data Science Virtual Machine (DSVM) as it is a DevOps VM I installed the RSAT tools and project Honolulu single box for Azure management and development.

    https://robertsmit.wordpress.com/2017/09/25/projecthonolulu-the-new-future-of-windows-server-gui-management-servermgmt-smt-winserv/

     

    Follow Me on Twitter @ClusterMVP

    Follow My blog https://robertsmit.wordpress.com

    Linkedin Profile Http://nl.linkedin.com/in/robertsmit

    Google Me : https://www.google.nl

    Bing Me : http://tinyurl.com/j6ny39w

    LMGTFY : http://lmgtfy.com/?q=robert+smit+mvp+blog

    Posted October 30, 2017 by Robert Smit [MVP] in Azure

    Tagged with ,

    Step by Step Azure File Sync – on-premises file servers to #Azure Files Storage Sync Service #AFS #Cloud #MSIgnite   2 comments

    Finally Azure File Sync is there in public preview, for the last months I had the pleasure to work with the Azure File Sync team and tested the product and thought about some great ideas where Azure File Sync (AFS) could be useful. And I guess you all have Ideas where you could use AFS. Placing your File server somewhere and get your files to the cloud.  Our use a Azure Data Box ADB https://azure.microsoft.com/nl-nl/updates/azure-data-box-preview/

    With Azure File Sync (preview), shares can be replicated on-premises or in Azure and accessed through SMB or NFS shares on Windows Server. Azure File Sync is useful for scenarios in which data needs to be accessed and modified far away from an Azure datacenter, such as in a branch office scenario. Data may be replicated between multiple Windows Server endpoints, such as between multiple branch offices.

    Azure File Sync (AFS)

    Azure File Sync is a multi-master sync solution, it makes it easy to solve global access problems introduced by having a single point of access on-premises, or in Azure by replicating data between Azure File shares and servers anywhere in the world. With Azure File Sync, we’ve introduced a very simple concept, the Sync Group, to help you manage the locations that should be kept in sync with each other. Every Sync Group has one cloud endpoint, which represents an Azure File share, and one or more server endpoints, which represents a path on a Windows Server. That’s it! Everything within a Sync Group will be automatically kept in sync!

      Azure File Sync enables organizations to:

      • Centralize file services in Azure storage
      • Cache data in multiple locations for fast, local performance
      • Eliminate local backup and DR

      The Azure File Sync agent is supported on Windows Server 2016 and Windows Server 2012 R2 and consists of three main components:

      • FileSyncSvc.exe: The background Windows service responsible for monitoring changes on Server Endpoints and initiating sync sessions to Azure.
      • StorageSync.sys: The Azure File Sync file system filter, responsible for tiering cold files to Azure Files (when cloud tiering is enabled).
      • PowerShell management cmdlets: PowerShell cmdlets for interacting with the Microsoft.StorageSync Azure Resource Provider. The cmdlets can be found at the following locations (by default):
    • %ProgramFiles%\Azure\StorageSyncAgent\StorageSync.Management.PowerShell.Cmdlets.dll
    • %ProgramFiles%\Azure\StorageSyncAgent\StorageSync.Management.ServerCmdlets.dll

    The Azure File Sync agent also includes a preview version of the Work Folders server feature which has been updated to support Azure File Sync. This preview version of Work Folders does not have a UI and must be managed via PowerShell: https://docs.microsoft.com/en-us/powershell/module/syncshare/?view=win10-ps

    But In the Preview I’m a bit Confused, what is the name of the product this Azure File Sync Or Storage Sync Service So looking it up in the Azure Store and in the quick list the name is not the Same.

    imageimage

    So when created the Azure File Sync <> you need to look under Storage Sync Services

    image

    Now that said how to built a Replica to Azure and back to my other Data Center ?

     

     Azure File Sync (AFS)

    So what do we need for this scenario, We need two File servers and a storage account in Azure.

    imageimage

    I created on a file server mvpafs01 with an extra disk that is hosted my onprem files. on the other server MVPAFS02 the share is in a different location.

    Azure File Sync extends on premises files servers into Azure providing cloud benefits while maintaining performance and compatibility.

    Azure File Sync provides:

    • Multi-site access – provide write access to the same data across Windows Servers and Azure Files
    • Cloud Tiering – store only recently accessed data on local servers
    • Integrates with Azure backup – no need to back up your data on premises
    • Rapid DR – restore file metadata immediately and recall data as needed

    Open your Azure subscription and look into the store for Azure File Sync.

    image

     

    image

    Create the Azure File Sync components

    imageAzure File Sync (AFS)

    First we make a New Storage Account, this storage account will hold the on premise files

    image

    image

    When the Storage account is created we create a file share on this storage account.

    image

    Currently the share has a maximum of 5TB !

    image

    Max size of a file share  5 TB

    Max size of a file in a file share 1 TB

    Max number of files in a file share Only limit is the 5 TB total capacity of the file share

    Max IOPS per share 1000

    Max number of files in a file share Only limit is the 5 TB total capacity of the file share

    image

    In this a limit of 4TB is more than enough to hold my files.

    image

    Now that the Azure File Sync is created we can configure the Azure File Sync.

    First we create a sync group in this group we can sync the files from one to many.

    image

    If you didn’t create the Storage account and the File share you will need to create this first.

    Create a sync Group

    A Sync Group contains a list of endpoints that define where a set of files sync to. Servers and Azure File Shares can participate in syncing the same set of files when they are listed in the same Sync Group.

    At the moment only one Azure File Share can participate in a Sync Group and it must be in the same region as this Storage Sync Service. Below you can create the Sync Group and its first and only Cloud Endpoint in one step. In the future you will be able to add more Cloud Endpoints. You can add Server Endpoints after this step completes.

    After creating this Sync Group and its first Cloud Endpoint, the next step is adding one or more Server Endpoints to the Sync Group.

     

    Azure File Sync (AFS)

    Next step is preparing the on premise file server and install the Agent and add the Azure PowerShell modules.

    To register a server:

    • Download the Azure Storage Sync agent and install it on all servers you want to sync.
    • After finishing the agent install, use the server registration utility that opens to register the server to this Storage Sync Service.

     

    image image

    When finishing the download of the right files we start the installation of the Agent.

    1. Download and run the StorageSyncAgent.msi.
    2. Follow the instructions to complete the installation.
    3. At the conclusion of the Azure File Sync agent installation, the Server Registration UI will auto-start.
    4. Follow the instructions to register the server with your Storage Sync Service.

    Before we start the Agent we need to disable the enhanced security ( for admins only)

     

    image

    The installation of the Agent is simple and Quick unless the Azure Modules are not on the Server.

    Azure File Sync (AFS)Azure File Sync (AFS)Azure File Sync (AFS)Azure File Sync (AFS)imageAzure File Sync (AFS)Azure File Sync (AFS) image

    Now that the Agent is installed we can register this server in Azure File Sync (AFS)

    Azure File Sync (AFS)

    I did not have the Azure PowerShell modules on this server So I need to install the modules first

    https://go.microsoft.com/fwlink/?linkid=856959

    image

    You can check the version with the Powershell command lets

    Get-Module PowerShellGet -list | Select-Object Name,Version,Path

    # Install the Azure Resource Manager modules from the PowerShell Gallery

    Install-Module AzureRM

    imageimage

    This can take sometime but you don’t need a reboot for this.

    image

    just login to your azure subscription where the Azure File Sync (AFS) is installed

    imageimage

    Pick the right subscription and Resource Group with the Storage Sync Service.

    image

    The next step after the registration of the server is creating an endpoint this End point is linking the File share to the Sync service

    image

     

    Creating an Endpoint is the final step but remember as soon as this is in place the Sync services on the on premise server starts the initial sync!

    image

    Creating the Azure File Sync (AFS) Endpoint

    image

    A Server Endpoint integrates a subfolder of a volume from a Registered Server as a location to sync. The following considerations apply:

    • Servers must be registered to the Storage Sync Service that contains this Sync Group before you can add a location on them here.
    • A specific location on the server can only sync with one Sync Group. Syncing the same location or even a part of it – with a different Sync Group doesn’t work.
    • Make sure that the path you specify for this server is correct and not the root of a volume before hitting Create.

    image

    • Cloud Tiering: A switch to enable or disable cloud tiering, which enables infrequently used or accessed files to be tiered to Azure Files.
    • Volume Free Space: the amount of free space to reserve on the volume on which the Server Endpoint resides. For example, if the Volume Free Space is set to 50% on a volume with a single Server Endpoint, roughly half the amount of data will be tiered to Azure Files. Note that regardless of whether cloud tiering is enabled, your Azure File share always has a complete copy of the data in the Sync Group.

    image

    Data traffic on the File server in this case it is just with one CPU. The upload speed is around the 300Mbps with almost 100% CPU

    imageimage

    After checking the same upload with 4 Cores and the upload is more than doubled so keep this in mind when uploading the files. Unless your line is the throttle neck

    imageimage

    Perfect the files are synced and ready for cloud usage.

    But I also want these files in my other datacenter, I could just copy those files and in a few days I run robocopy with the delta’s but I can also use a second endpoint in Azure File Sync (AFS) and keep all files in sync.

    The first step is the same as any server to register install the Azure File Sync (AFS)  Agent with the Powershell Modules

     

    image

    Connect with the same Azure subscription

    image

    As you can see the server is online and registered.

     

    image

    As this server doesn’t have a second disk I place all the files on a different share

    image

    But after filling in the share name and applied it the server gets very busy but there are no files in the folder.

    Check this : all the files are cached in the System volume information folder under HFS. After the caching it placed all the files in the right folder.

    Just keep in mind that this is the process and your Monitoring agents could alarm you for this. 

    image

    After the initial sync I have two file servers and a Azure Storage account with the same files. I can Edit files on 3 point and still it got synced.

    image

    The synced files on the Second server and as you can see that the System files are gone and placed in the share.

    image

    Hope this blog gives you the start on using the Azure File Sync (AFS) it is very useful as you could sync file between subscriptions or regions or just between your data centers.

     

    Follow Me on Twitter @ClusterMVP

    Follow My blog https://robertsmit.wordpress.com

    Linkedin Profile Http://nl.linkedin.com/in/robertsmit

    Google Me : https://www.google.nl

    Bing Me : http://tinyurl.com/j6ny39w

    LMGTFY : http://lmgtfy.com/?q=robert+smit+mvp+blog

    Posted September 28, 2017 by Robert Smit [MVP] in Azure

    Tagged with , , ,

    #ProjectHonolulu the new future of Windows Server GUI management #servermgmt #SMT #winserv   1 comment

    As Azure Server management tools discontinued the SMT preview service in Azure on June 30, 2017 and we where stuck to Windows Server management such as Remote Desktop, Server Manager, Remote Server Administration Tools (RSAT), and other MMC-based management tools. See my old blog post about this : https://robertsmit.wordpress.com/2016/02/12/azure-server-management-tools-offers-a-set-of-web-gui-tools-to-manage-azurestack-servers-rsmt-asmt/

    But Microsoft created a fresh new tool to manage all our servers, Project “Honolulu” is the next step in our journey to deliver on our vision for Windows Server graphical management experiences.

    Looking at the interface it is great, real-time graphs, single point of management. Loading of some components can take some time(Seconds). But it runs not in the IE 11 version. So if you run this on a management server you will need  Google chrome . I had the chance to work with Microsoft during the last couple of months in the Alpha versions. there is a lot of improvement done. There are some options disappeared in the Project ‘Honolulu’ (Technical Preview) and there is a huge whish list and probably when you test the tool you think he this would be nice also.  Then go to the Uservoice page and create or vote for your item.  There are a lot of items in UserVoice with some of the more popular requests from Private Preview so vote for you item and make Project “Honolulu” a piece of your self   https://aka.ms/HonoluluFeedback

    Below is a overview of the standard tool set that Project “Honolulu” is offering.

    image

    And there is also a light foot print on memory

    image

    So what does it take to run this a huge server ? no just a quick install and you are ready to go. it runs with a self signed certificate if you don’t have a public one.

    https://robertsmit.wordpress.com/2016/02/12/azure-server-management-tools-offers-a-set-of-web-gui-tools-to-manage-azurestack-servers-rsmt-asmt/

    imageimage

    imageimage 

    As you can see the Installation is quick and simple easy to setup. Just pick a port number for the website and a Certificate. If you doesn’t have one there will be a self signed cert created.  It does say 60 Days and you can look this up in the local computer Certificate store

    image

    image

    After the Installation you can open the icon or open a Chrome session to server name and the port number. eh wait what was the port number again ?

    The port number is stored in the register 

    image

    [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ServerManagementGateway]
    "SmePort"="51358"
    "UseHttps"="1"

    Opening the Wrong Browser :

    image

    After starting Honolulu in the right browser there is a quick tour. But as always who does this. just skip the tour brave IT person.

    image

    In case the tool hangs or is not responding just restart the service.

    image

    imageimage

    So after opening we all want to see the nice dashboards and overview. Well you need to add the machines first and that is a lot of work.

    No AD select all all typing and fill in the Credentials Luckily there is also a Import.

     

    image

     

      image

    And the best part is it is just a Text file TXT fill in the names comma or line separated and you are good to go.

    Wait for the credentials as you are doing this with the last server and check the box us this for all servers.

    imageConfigure-SMremoting.exe -enable

    Running this on Server 2012R2 you will need WMF 5 or Windows Management Framework 5.1 Preview

    Windows Management Framework 5.1 includes updates to Windows PowerShell, Windows PowerShell Desired State Configuration (DSC), Windows Remote Management (WinRM), Windows Management Instrumentation (WMI). Release notes: https://msdn.microsoft.com/en-us/powershell/wmf/5.1/release-notes

    https://www.microsoft.com/en-us/download/details.aspx?id=53347

    But running a Quick Cluster in Azure does not bring me the nice dashboard yet. 

    image

    Well In a few days I have this in an environment where the dashboards are showing but for now I used the screenshot from Msignite

    There are two sessions on ignite about Honolulu

    image

     

    Don’t forget your Feedback on Uservoice  https://aka.ms/HonoluluFeedback

    More info : https://blogs.technet.microsoft.com/servermanagement/2017/09/21/video-series-an-inside-look-at-project-honolulu/

     

    Follow Me on Twitter @ClusterMVP

    Follow My blog https://robertsmit.wordpress.com

    Linkedin Profile Http://nl.linkedin.com/in/robertsmit

    Google Me : https://www.google.nl

    Bing Me : http://tinyurl.com/j6ny39w

    LMGTFY : http://lmgtfy.com/?q=robert+smit+mvp+blog

    Posted September 25, 2017 by Robert Smit [MVP] in Windows Server 2016

    Tagged with

    Step by Step Azure network security groups NSG – Security Center #Azure #NSG #Network   Leave a comment

    Now Days I see that people not fully understand  the security needs in Azure. There are a lot of options in Azure to improve the security.

    A great option is the Security Center. This is a great dashboard to get a quick over view an the security status of your subscription.

    image

     

    image

     

    But the other Option is setting up a network security group (NSG)

    image

    A network security group (NSG) contains a list of security rules that allow or deny network traffic to resources connected to Azure Virtual Networks (VNet). NSGs can be associated to subnets, individual VMs (classic), or individual network interfaces (NIC) attached to VMs (Resource Manager).

    When an NSG is associated to a subnet, the rules apply to all resources connected to the subnet. Traffic can further be restricted by also associating an NSG to a VM or NIC.

    Associating NSGs

    You can associate an NSG to VMs, NICs, and subnets, depending on the deployment model you are using, as follows:

    • VM (classic only): Security rules are applied to all traffic to/from the VM.
    • NIC (Resource Manager only): Security rules are applied to all traffic to/from the NIC the NSG is associated to. In a multi-NIC VM, you can apply different (or the same) NSG to each NIC individually.
    • Subnet (Resource Manager and classic): Security rules are applied to any traffic to/from any resources connected to the VNet.

    You can associate different NSGs to a VM (or NIC, depending on the deployment model) and the subnet that a NIC or VM is connected to. Security rules are applied to the traffic, by priority, in each NSG, in the following order:

    • Inbound traffic

      1. NSG applied to subnet: If a subnet NSG has a matching rule to deny traffic, the packet is dropped.

      2. NSG applied to NIC (Resource Manager) or VM (classic): If VM\NIC NSG has a matching rule that denies traffic, packets are dropped at the VM\NIC, even if a subnet NSG has a matching rule that allows traffic.

    • Outbound traffic

      1. NSG applied to NIC (Resource Manager) or VM (classic): If a VM\NIC NSG has a matching rule that denies traffic, packets are dropped.

      2. NSG applied to subnet: If a subnet NSG has a matching rule that denies traffic, packets are dropped, even if a VM\NIC NSG has a matching rule that allows traffic.

    image

    As most items in Azure there are Limits to the number of NSGs you can have in a subscription and number of rules per NSG. To learn more about the limits, read the Azure limits article.

    image

    Creating a network security group (NSG) is easy you can do this in the portal or with Powershell

    imageimage

    As I mentioned above you can set the network security group (NSG) on a subnet or VM. Add multiple items in a network security group (NSG)

    image

    By default all is set to basic just pick a service and open or close the port.

    imageimage

    But when checking the Advanced option the Rule pane will change into a rich and flexible option menu.

     

    image   Instead of selecting just a service You can also add a IP range to exclude others for accessing this machine.

    image

    Setting this in the GUI is nice but when you need to change or add a lot of these you will need Powershell or ARM templates.

    Below are just some examples on how to use them

    Login-AzureRmAccount
     
    # Select a subscription
    $subscriptionId = (Get-AzureRmSubscription | Out-GridView -Title ‘Select your Azure Subscription:’ -PassThru)
    Select-AzureRmSubscription -SubscriptionId $subscriptionId.Id
     
    # Select a Resource Group
    $rgName = (Get-AzureRmResourceGroup | Out-GridView -Title ‘Select your Azure Resource Group:’ -PassThru).ResourceGroupName
     
    # Set the NSG name and Azure region
    $nsgName = "Trusted-Nsg01"
    $location = "West Europe"
    $source1 = "8.8.8.8/32"
    $source2 = "8.8.4.4/32"
    $source3 = "*"
    $dest1="3389"
    $dest2="443"
    $dest3="80"
    $tag="blog"

    #Below are Sample Rules
    $rule1 = New-AzureRmNetworkSecurityRuleConfig -Name rdp-rule -Description "Allow RDP" `
    -Access Allow -Protocol Tcp -Direction Inbound -Priority 100 `
    -SourceAddressPrefix $source1 -SourcePortRange * `
    -DestinationAddressPrefix * -DestinationPortRange $dest1

    $rule2 = New-AzureRmNetworkSecurityRuleConfig -Name web-rule2 -Description "Allow Port" `
    -Access Allow -Protocol Tcp -Direction Inbound -Priority 101 `
    -SourceAddressPrefix $source2 -SourcePortRange * -DestinationAddressPrefix * `
    -DestinationPortRange $dest2

    $rule3 = New-AzureRmNetworkSecurityRuleConfig -Name web-rule3 -Description "Allow Port" `
    -Access Allow -Protocol Tcp -Direction Inbound -Priority 103 `
    -SourceAddressPrefix $source3 -SourcePortRange * -DestinationAddressPrefix * `
    -DestinationPortRange $dest3

    $rule4 = New-AzureRmNetworkSecurityRuleConfig -Name web-rule4 -Description "Allow Port" `
    -Access Allow -Protocol Tcp -Direction Inbound -Priority 104 `
    -SourceAddressPrefix Internet -SourcePortRange * -DestinationAddressPrefix * `
    -DestinationPortRange 88

     

    Now that the port Rules are created we need to put them in a security group

    #applying the Rules
    $nsg = New-AzureRmNetworkSecurityGroup -ResourceGroupName $rgName -Location $location -Name $nsgName -SecurityRules $rule1,$rule2,$rule3,$rule4

    image

    # Display default and security rules for NSG
     
    (Get-AzureRmNetworkSecurityGroup -Name $nsgName -ResourceGroupName $rgName).SecurityRules | Select-Object * | Out-GridView
    (Get-AzureRmNetworkSecurityGroup -Name $nsgName -ResourceGroupName $rgName).DefaultSecurityRules | Select-Object * | Out-GridView

    #Remove NSG

    Remove-AzureRmNetworkSecurityGroup -Name $nsgName -ResourceGroupName $rgName

     

    Now that we created a network security group (NSG) we can add it to a VM this can also be done with PowerShell but there is a BUT.

    let me show you, Go to the VM and select the network card.

    image

    The Nic can be named nic245768323 something, I always use named NIC’s so that is easy but if not the NSG could be applied on an other VM and maybe it will fail.

    imageimage

    When selecting this manual you can see the nic and if you are sure on the other machines you can do this with PowerShell also.

     

    Follow Me on Twitter @ClusterMVP

    Follow My blog https://robertsmit.wordpress.com

    Linkedin Profile Http://nl.linkedin.com/in/robertsmit

    Google Me : https://www.google.nl

    Bing Me : http://tinyurl.com/j6ny39w

    LMGTFY : http://lmgtfy.com/?q=robert+smit+mvp+blog

    Posted September 11, 2017 by Robert Smit [MVP] in Azure

    Tagged with ,

    Step by Step Azure Network watcher #Azure #ANW #Network #Cloud #diagnose #troubleshooting   Leave a comment

     

    Network Watcher is a regional service that enables you to monitor and diagnose conditions at a network scenario level in, to, and from Azure. Network diagnostic and visualization tools available with Network Watcher help you understand, diagnose, and gain insights to your network in Azure. Use Network Watcher, a service that enables you to monitor and diagnose conditions at a network scenario level.

    Network Watcher currently has the following capabilities:

    • Topology – Provides a network level view showing the various interconnections and associations between network resources in a resource group.
    • Variable Packet capture – Captures packet data in and out of a virtual machine. Advanced filtering options and fine-tuned controls such as being able to set time and size limitations provide versatility. The packet data can be stored in a blob store or on the local disk in .cap format.
    • IP flow verify – Checks if a packet is allowed or denied based on flow information 5-tuple packet parameters (Destination IP, Source IP, Destination Port, Source Port, and Protocol). If the packet is denied by a security group, the rule and group that denied the packet is returned.
    • Next hop – Determines the next hop for packets being routed in the Azure Network Fabric, enabling you to diagnose any misconfigured user-defined routes.
    • Security group view – Gets the effective and applied security rules that are applied on a VM.
    • NSG Flow logging – Flow logs for Network Security Groups enable you to capture logs related to traffic that are allowed or denied by the security rules in the group. The flow is defined by a 5-tuple information – Source IP, Destination IP, Source Port, Destination Port and Protocol.
    • Virtual Network Gateway and Connection troubleshooting – Provides the ability to troubleshoot Virtual Network Gateways and Connections.
    • Network subscription limits – Enables you to view network resource usage against limits.
    • Configuring Diagnostics Log – Provides a single pane to enable or disable Diagnostics logs for network resources in a resource group.
    • Connectivity (Preview) – Verifies the possibility of establishing a direct TCP connection from a virtual machine to a given endpoint.

     

    Lets start with creating the Network Watcher.

    Open Powershell  :

    Login-AzureRmAccount

    Register-AzureRmProviderFeature -FeatureName AllowNetworkWatcher -ProviderNamespace Microsoft.Network

    Get-AzureRmProviderFeature -FeatureName AllowNetworkWatcher -ProviderNamespace  Microsoft.Network

     

    Go to the https://portal.azure.com

    https://robertsmit.wordpress.com/

    As you can see I have several with status disabled and one with partially enabled

    image

    Enabling the Network Watcher is easy just do a right click on the 3 dots and enable this for all regions or just one, or set this as a default.

     

    image

    image

    Now that we enabled the Network Watcher We create a separate Storage Account for this, as all the logging goes to this storage account. We don’t want to place log files all over the subscription.

    image

    Our just run a PowerShell command to do this. I use a own resource group for this

    New-AzureRmResourceGroup -Name "rsg-netwatcher01" -Location "westeurope"
    New-AzureRmStorageAccount -ResourceGroupName "rsg-netwatcher01" -Location "westeurope" -Name "stnetwatcher01" -SkuName Standard_LRS

     

    Topology – Provides a network level view showing the various interconnections and associations between network resources in a resource group.

    TO view the topology from your network could be very handy- Remember this is only ARM so no ASM

    image

    and yes the pictures getting large

    image

    This is all the basic stuff IP flow verify is the more interesting part.

     

    IP flow verify

    IP flow verify checks if a packet is allowed or denied to or from a virtual machine based on 5-tuple information. This information consists of direction, protocol, local IP, remote IP, local port, and remote port. If the packet is denied by a security group, the name of the rule that denied the packet is returned. While any source or destination IP can be chosen, this feature helps administrators quickly diagnose connectivity issues from or to the internet and from or to the on-premises environment.

    IP flow verify targets a network interface of a virtual machine. Traffic flow is then verified based on the configured settings to or from that network interface. This capability is useful in confirming if a rule in a Network Security Group is blocking ingress or egress traffic to or from a virtual machine.

    Remember If you have multiple regions you must enable Network Watcher in all regions.

    image

    The flow is easy the Source Machine and Port number and the destination Machine and Port number. as this is all in the same subnet but If you are running this in more complex networks then this could be very useful.

     

    Checking the Security Groups all settings in just one overview that is also very handy when troubleshooting.

    image

    So all thing in the Network Watcher is nice but one this that is always a pain is troubleshoot the VPN connections and get the log files etc.

    In the Network Watcher there is an option on troubleshoot the VPN connection

    Network Watcher – VPN Diagnostics

    This is also the place where the storage container is needed. Just select the Virtual network gateway and add the Storageaccount with the Start Troubleshooting. This could take a few Minutes to complete!

    imageimage image

    When the trace is done there is a Zipfile GatewayTenantWorker_IN_0.zip placed in the folder with a date folder structure so no overwrite of the file.

    In the Zip file are 2 files unless you have issues.

    image

    Connectivity State : Connected
    Remote Tunnel Endpoint :
    Ingress Bytes (since last connected) : 202242292718 B
    Egress Bytes (Since last connected) : 2435917732003 B
    Connected Since : 8/15/2017 9:41:08 AM

    In the connection stats you can see the traffic between the VPN connection.

    When you have issues with the VPN connection then there will be more files in the zip file. Beside the ConnectionStats.txt and the CPUStat.txt, we got IKEErrors.txt, Scrubbed-wfpdiag.txt, wfpdiag.txt.sum and wfpdiag.xml.

    The IKEErrors.txt and Scrubbed-wpfdiag.txt will get you the most detail about the error of the VPN connection

     

    Pricing details

    There are no charges to use Network Watcher today. On October 1, 2017, the pricing model below goes into effect.

    Feature Monthly allotment Overage charge
    Network Logs Ingested 5 GB €0.422 per GB
    Network Diagnostic Tools 1,000 checks €0.844 per 1,000 checks
     
    • Network logs are stored within a storage account and have a retention policy that can be set from one day to 365 days. If a retention policy is not set, the logs are maintained forever. Corresponding charges will apply for storage, Log Analytics, and event hubs respectively.
    • Network Watcher Diagnostic Tools and Topology features are billed for the number of Network Diagnostic checks initiated via Azure Portal, PowerShell, CLI, or Rest.

    As the Cost are minimal and easy to use so enable this today.

     

    Follow Me on Twitter @ClusterMVP

    Follow My blog https://robertsmit.wordpress.com

    Linkedin Profile Http://nl.linkedin.com/in/robertsmit

    Google Me : https://www.google.nl

    Bing Me : http://tinyurl.com/j6ny39w

    LMGTFY : http://lmgtfy.com/?q=robert+smit+mvp+blog

    Posted August 16, 2017 by Robert Smit [MVP] in Azure

    Tagged with

    How to: Resize virtual machines in #Azure With #Powershell Multiple or Single virtual machines   Leave a comment

    With the new VM sizes in Azure you may want to change the Size as you get more VM for less money. but remember the VM will restart! so better fi

    RDSTWEAKERS.COM

    But changing the VM by hand is a time consuming  job So Powershell could be very handy in this case. you can change the Vm size easily with a one-liner

    So first we need to login into the azure Subscription.

    Login-AzureRmAccount

    If you have multiple Subscriptions you need to select the right subscription.

    $subscrip=Get-AzureRmSubscription | Out-GridView -OutputMode Single -Title ‘Please select a Azure Subscription.’
    Select-AzureRmSubscription -TenantId  $subscrip.TenantId

    Get-AzureRmVM

    image

     

    $vm = Get-AzureRmVM -VMName MVPCB10 -ResourceGroupName RSG-VNET
    $vm.HardwareProfile.VmSize = "Standard_D2_v3"
    Update-AzureRmVM -VM $vm -ResourceGroupName RSG-VNET

    Ok this seems nice but I have 50 VM’s that I like to change

    #set new Size to VM
    1..5 | % {
    $vm = Get-AzureRmVM -ResourceGroupName RSG-VNET -VMName MVPCB1$_
    $vm.HardwareProfile.VmSize = "Standard_D13_v2_Promo"
    Update-AzureRmVM -VM $vm -ResourceGroupName RSG-VNET

    }

    Better But if you used random names then the above will not really help you in quick size changing. The next step would be selecting all the VM that needs to be changed and selecting a Size for changing. That sounds great but how to start ?

    With the Out-GridView you can do great things. to bad that the price is not available in this.

    image

     

    The script would be like this :

     

    $VMList = Get-AzureRmVm | Out-GridView -OutputMode Multiple -Title ‘Please select an Azure Virtual Machine to resize.’;
    $TargetSize = Get-AzureRmVmSize -Location westeurope | Out-GridView -OutputMode Single -Title ‘Please select a target Azure Virtual Machine size.’;
    foreach ($VM in $VMList) {
      Write-output "Resizing Microsoft Azure Virtual Machine" $VM.Name "in Resource Group" $VM.ResourceGroupName "to size" $TargetSize
     
      Update-AzureRmVm -VM $VM -ResourceGroupName $VM.ResourceGroupName -Verbose
    }
    Get-AzureRmVm

    After this the VM’s are all changed in a other Size.

    Follow Me on Twitter @ClusterMVP

    Follow My blog https://robertsmit.wordpress.com

    Linkedin Profile Http://nl.linkedin.com/in/robertsmit

    Google Me : https://www.google.nl

    Bing Me : http://tinyurl.com/j6ny39w

    LMGTFY : http://lmgtfy.com/?q=robert+smit+mvp+blog

    https://rdstweakers.com

    Posted July 18, 2017 by Robert Smit [MVP] in Azure

    Tagged with

  • Twitter

  • %d bloggers like this: