Azure Notebooks for Azure Sentinel

Azure Sentinel is Microsoft’s cloud-native SIEM that provides intelligent security analytics for your entire enterprise at cloud scale.
This SIEM as a Service (SIEMaaS) solution is designed as a cloud-based security-monitoring platform that leverages the power of the cloud for analytics and detections.

https://azure.microsoft.com/en-us/services/azure-sentinel/

there is a good video  https://www.youtube.com/watch?v=XXZp6LQZSJU&feature=youtu.be 

Limitless cloud speed and scale
Azure Sentinel is the first SIEM built into a public cloud platform to help your security analysts focus on what really matters.

Easily connect your data sources
Azure Sentinel provides simple and easy integration with signals and intelligence from security solutions whether they are on premises, in Azure, or in other clouds.
Azure Sentinel provides seamless integration with Microsoft 365, Azure, and other Microsoft products, including Microsoft’s security products.

Detect suspicious activities in your organization
Azure Sentinel fuses together unique machine learning algorithms, world-class security research, and the breadth and depth of the critical security data available to Microsoft as a major enterprise vendor. Azure Sentinel helps you detect both known and unknown attack vectors, detecting threats across all stages of the kill chain.

Investigate and remediate breaches
Azure Sentinel gives you visibility into all the entities involved in an alert and provides a simple and instinctive UI to investigate the detection, helping you easily understand the scope of the breach.
To cut down on the volume of alerts you get, Azure Sentinel automatically investigates alerts to help you determine what action to take, enabling you to move from alert to remediation in minutes, at scale.
Leveraging the power of Logic Apps, Azure Sentinel helps you respond to incidents instantly, using built-in orchestration and automation playbooks.

Joining the Preview program give you the enable option and you will need some configuration in the Azure portal. Overall a great overview in the new dash boarding. one thing is I need more screens to show all this.

 

You will need a workspace I you have already one you can use this or just create a new one

I’ll pick my current one as all my VM’s are reporting into this.

 

Now we can install the add-on for data collection, there is already a big list.

As I already had a workspace there is already some content to use, at this point I don’t have any incidents, so no cases and alerts

I think this is a grate feature the “hunting” predefined query’s ready to run and adjustable to your need.

Reuse the custom query, for better adjustment in your site.

You can find more samples on github https://github.com/Azure/Azure-Sentinel

Also the Azure Notebooks for Azure Sentinel is a new option, create your Project in Jupyter

 

What is Azure Notebooks?

Azure Notebooks is a free hosted service to develop and run Jupyter notebooks in the cloud with no installation. Jupyter is an open source project that lets you easily combine markdown text, executable code (Python, R, and F#), persistent data, graphics, and visualizations onto a single, sharable canvas called a notebook.

How do Azure Notebooks work?

Interactive Azure Notebooks provides security insights and actions to investigate anomalies and hunt for malicious behaviors. Each Azure Notebook is purpose-built with a self-contained workflow for a specific use case. Visualizations are included in each Azure Notebook for faster data exploration and threat hunting. Click on the button below to clone our prebuilt investigation and hunting Azure Notebooks into projects that belong to you. Modify and tailor your projects to your environment. Either run the Azure Notebooks for free or, for better performance, run them on a dedicated virtual host. Click here to learn more.

Using the Notebooks locally or in other environments

Azure Sentinel will provision notebooks and supporting modules for you in Azure Notebooks. You can also download the notebooks and modules and use them locally in a supported Python environment (Anaconda is recommended) or another notebook hosting environment such as Azure Databricks or a JupyterHub environment that supports Python 3.6 or later.

 

 

With the import a copy will be made from the Github to your own repository to get you started.

 

this take some time after this the project page is opening for you.

You can check the samples and adjust them for your needs

 

Checking the Logs in the Azure Sentinel will give you a nice dashboard with all the content. I have limited amount of data in this so no big lines or exceptions.

A Sample dashboard with the infrastructure query in Azure Sentinel

A Sample dashboard with the infrastructure query in Azure Sentinel

A Sample dashboard with the general overview query in Azure Sentinel

Some are based on multiple pages, big screens are needed or smaller fonts but overall this is a nice addition to the Azure Family.

Azure Sentinel will take some time to get this running and configuring but once there is data you will see a very nice new tool that can help you to solve your problems in Azure better an quickly.

See and stop threats before they cause harm, with SIEM reinvented for a modern world. Azure Sentinel is your birds-eye view across the enterprise. Put the cloud and large-scale intelligence from decades of Microsoft security experience to work. Make your threat detection and response smarter and faster with artificial intelligence (AI). Eliminate security infrastructure setup and maintenance, and elastically scale to meet your security needs—while reducing IT costs.

 

Follow Me on Twitter @ClusterMVP

Follow My blog https://robertsmit.wordpress.com

Linkedin Profile Robert Smit MVP Linkedin profile

Google  : Robert Smit MVP profile

The Secure DevOps Kit for Azure is a collection of scripts, tools, extensions, automations.

The kit is based on Powershell and can be extended to Azure log analytics with some nice dashboarding. But if you have a large subscription the Powershell query can take some time. With this toolkit Devops teams using extensive automation and smoothly integrating security into native Devops workflows helping accomplish secure Devops with these 6 focus areas:

• Secure the subscription: A secure cloud subscription provides a core foundation upon which subsequent development and deployment activities can be conducted. An engineering team should have the capabilities to deploy and configure security in the subscription including elements such as alerts, ARM policies, RBAC, Security Center policies, JEA, Resource Locks, etc. Likewise, it should be possible to check that all settings are in conformance to a secure baseline
• Enable secure development: During the coding and early development stages, developers should have the ability to write secure code and to test the secure configuration of their cloud applications. Just like build verification tests (BVTs), we introduce the concept of security verification tests (SVTs) which can check for security of various resource types in Azure
• Integrate security into CICD: Test automation is a core tenet of devops. We emphasize this by providing the ability to run SVTs as part of the VSTS CICD pipeline. These SVTs can be used to ensure that the target subscription used to deploy a cloud application and the Azure resources the application is built upon are all setup in a secure manner
• Continuous Assurance: In the constantly changing dev ops environment, it is important to move away from the mindset of security being a milestone. We have to treat security as a continuously varying state of a system. This is made possible through capabilities that enable continuous assurance using a combination of automation runbooks, schedules, etc.
• Alerting & Monitoring: Visibility of security status is important for individual application teams and also for central enterprise teams. We provide solutions that cater to the needs of both. Moreover, the solution spans across all stages of dev ops in effect bridging the gap between the dev team and the ops team from a security standpoint through the single, integrated views it generates
• Cloud Risk Governance: Lastly, underlying all activities in the kit is a telemetry framework that generates events capturing usage, adoption, evaluation results, etc. This allows us to make measured improvements to security targeting areas of high risk and maximum usage before others.

Keep in mind that The OMS portal will is retired on January 15, 2019. You can continue to use your existing services and licensing in the Azure portal.

Even in the Azure portal you can still connect to OMS

Complete feature set of Secure DevOps Kit for Azure

Feature Area	Secure DevOps Kit Feature
Secure the subscription	Subscription Health Check Subscription Provisioning Alerts Configuration ARM Policy Configuration Azure Security Center Configuration Access control (IAM) Hygiene
Enable secure development	Security Verification Tests (SVT) Security IntelliSense- VS Extension
Integrate security into CICD	AzSK VS Extension-executes SVTs in a CICD pipeline
Continuous Assurance	Security scanning via Azure Automation Runbooks
Alerting & Monitoring	OMS Solution for AzSK containing: Security dashboard views covering security state/actions Alerts with pertinent search queries
Cloud Risk Governance	Control/usage telemetry through Insights

Setting up Secure DevOps Kit for Azure (AzSK)

First make sure you have the right Azure modules installed, I noticed the automation module failed So I added this manualy.

Import-Module AzureRM.Automation

Get-AzSKAzureServicesSecurityStatus -SubscriptionId

Installing the Secure DevOps Kit for Azure (AzSK)

Install-Module AzSK -Scope CurrentUser

Now that the Powershell modules are installed we can start the (AzSK) Scan

Get-AzSKAzureServicesSecurityStatus –SubscriptionId  ID

In this subscription there are 44 items that are been checked

Items are been checked on the security issues

Nice detailed overview is shown. Also a log folder is been created with all the issues. per resource Item.

As you can see I have some failed items and with a High, so I need to take a good look at this and fix this.

This maybe one of the best Items here an excel sheet with al the issues listed with the solution mentioned and if this can be automated.

If needed there is an URL that points you to the right solution.

As Azure log analytics is great and it can be integrated with some OMS (Azure monitoring Dashboards)

The OMS portal will be retired on January 15, 2019. You can continue to use your existing services and licensing in the Azure portal. So the current documentation need some updating.

Pressing the OMS button in the Azure portal brings you to the OMS portal but then nothing. As it is now all Azure portal.

Setting up the dashboards failed on me during the first installation but when I did run this a second time the dashboard was there.  (Timing) 

Creating the OMS default dashboard we need to run some powershell scripts.

$omsSubId =”id”   #subscription hosting the OMS workspace

$omsWSId =’OMS ID’

$omsRGName =’omsrsg’     #RG where the OMS workspace is hosted

$azSkViewName = ‘MVP_AzSK_view’ #This will identify the tile for AzSK view in OMS.

    #This command will deploy the AzSK view in the OMS workspace.  
    Install-AzSKOMSSolution -OMSSubscriptionId $omsSubId `

                    -OMSResourceGroup $omsRGName `

                    -OMSWorkspaceId $omsWSId `

                    -ViewName $azSkViewName

Note:

1) The blades of the OMS view created by this command will start populating only after AzSK scan events become available in the corresponding OMS workspace.

To understand how to send AzSK events to an OMS workspace see https://aka.ms/devopskit/oms.

2) The OMS view installed contains a basic set of queries over DevOps Kit scan events. Please feel free to customize them once you get familiar with the que

ries.

We also periodically publish updated/richer queries at: https://aka.ms/devopskit/omsqueries.

Checking the OMS – log analytics workspace it has not much issues as this is a test subscription and if it was all perfect then there is no fun.

and with longer logging and more Items in azure you will get a different overview.

There are lots of options you can set and there is a detailed description on how to use this on Github

Setting up ARM policys is also one of the options

Set-AzSKARMPolicies –SubscriptionId

So get started with the DevOpsKit https://github.com/azsk/DevOpsKit-docs 

https://github.com/azsk/DevOpsKit-docs/tree/master/05-Alerting-and-Monitoring

AzSK Security Controls Portal @https://aka.ms/azskosstcp

With this it’s a nice tool and yes a bit time consuming but learned a lot and make me see things different in the Azure Subscription 

And If you combine this directly and not afterwards then this could be your time saver to fix all the security items

 

Follow Me on Twitter @ClusterMVP

Follow My blog https://robertsmit.wordpress.com

Linkedin Profile Robert Smit MVP Linkedin profile

Google  : Robert Smit MVP profile

What’s new in SQL Server 2019

This blog is about creating a new SQL cluster with a Failover instance based on a storage space direct configuration. There are a lot off opinions on do’s and don’t for on prem and in Azure with S2D. This is not the blog post on this discussion.  This is just a post on SQL 2019 on Windows server 2019 storage space direct because I can!

And in a later blog post we are extending this to Azure. Still it always depends on running a Full SQL server or use a managed instance in Azure. But for now the new SQL server 2019 is the basic of our cluster running on a Windows Server 2019 cluster. As this time the installation will be GUI based as I got often the question on the screens. normally I run some PowerShell or a command line script that will do the job in several minutes

What are the new features in SQL Server 2019, the list below is just a short list for a full updated list go to the link below.

OR if you want to run a managed SQL server in Azure : https://azure.microsoft.com/en-us/pricing/details/sql-database/managed/

 

The Database Engine is the core service for storing, processing, and securing data. The Database Engine provides controlled access and rapid transaction processing to meet the requirements of the most demanding data consuming applications within your enterprise. The Database Engine also provides rich support for sustaining high availability.

 

 

https://docs.microsoft.com/en-us/sql/sql-server/sql-server-technical-documentation?view=sql-server-ver15

• Installation of SQL Server is supported on x64 processors only. It is no longer supported on x86 processors.
• SysPrep is supported for all installations of SQL Server. SysPrep now supports failover cluster installations
• Always On Availability Groups – secondary replica connection redirection
• SQL Server Machine Learning Services failover clusters
• the operating system requirements for the principal editions of SQL Server
• SQL Server Management Studio (SSMS) 18.0 (preview)
• Azure Data Studio
• Azure Data Studio

Always On Availability Groups – more synchronous replicas (CTP 2.0)

• Up to five synchronous replicas: SQL Server 2019 preview increases the maximum number of synchronous replicas to 5, up from 3 in SQL Server 2017 (14.x) . You can configure this group of 5 replicas to have automatic failover within the group. There is 1 primary replica, plus 4 synchronous secondary replicas.

• Secondary-to-primary replica connection redirection: Allows client application connections to be directed to the primary replica regardless of the target server specified in the connection string. This capability allows connection redirection without a listener. Use secondary-to-primary replica connection redirection in the following cases:

  • The cluster technology does not offer a listener capability.
  • A multi subnet configuration where redirection becomes complex.
  • Read scale-out or disaster recovery scenarios where cluster type is NONE.

SQL Server Enterprise /SQL Server Standard :

1. Windows Server 2019
2. Windows Server 2016
3. Windows Server 2012 R2
4. Windows Server 2012

The supported storage types for data files are:

• Local Disk

• Shared Storage

• Storage Spaces Direct (S2D)

• SMB File Share

As the SQL server 2019 is still in preview I download the evaluation version.  Get your SQL server here

 

Pick the version you want, in our case I select the download media.

Pick the ISO format do download this.

The SQL installation is done on my cluster. described in the blog post below

step by step Windows Server 2019 File Server clustering With powershell or GUI

https://robertsmit.wordpress.com/2018/11/29/step-by-step-windows-server-2019-file-server-clustering-with-powershell-or-gui-cluster-ha-azure-windowsadmincenter-windowsserver2019/

In this Windows server 2019 cluster I created a S2D formatted with ReFS. There is plenty of info to find on my blog how to create the S2D in the cluster.

https://robertsmit.wordpress.com/2016/01/05/using-windows-storage-spaces-direct-with-hyper-converged-in-microsoft-azure-with-windows-server-2016/

https://robertsmit.wordpress.com/2017/11/09/azure-storage-spaces-direct-s2d-standard-storage-vs-premium-storage/

We have a Windows server 2019 Cluster in place with the Storagespace direct configuration.

The specific cluster configuration is debatable, in a typical SQL configuration you would have multiple disks.

Our Cluster with storage spaces. In storage spaces all disks come from the same disk pool if you create things default. then multiple disk has a different purpose. As you want to manage the SQL different than others.

Our SQL Installation is done this time by the GUI, show you the screens and options. you can always use the one liner to install the SQL server unattended.

 

And no I do not use the normal cluster installation but I use the advanced Cluster prep and completion. these two steps gives me a bit more freedom and flexibility it the installation fails.

The choice is developer but can also be the one with a product key.

Just do the updates direct as it is a new version and on a new OS, just to make sure that I run the latest bits.

I got a warning on the Firewall as I turned this off during the installation. This is configured by a GPO when I place the Cluster in the Right AD OU

Selecting the features and the installation folders.

Do I go for a default instance or a named instance. I always go for a named instance. Its easy and better management. In the old days some applications just want to run on a default instance.

My service account that has JEA Just enough Access

C:\Program Files\Microsoft SQL Server\150\Setup Bootstrap\Log\20181217_034408\ConfigurationFile.ini

If you want to use a unattended install you can use this ini file to get the same or adjusted values during the install

Ini file content.

Now that the first step is completed we can do the completion step.

Now the second step is needed to finish the SQL 2019 installation on the first node

But running the wizard if failed on me, as the error said the Cluster is not verified!

I just need to run the Cluster validation.

Now with the Passed Cluster validation we try again.

Now this looks good lets stat the installation of the SQL 2019

With the Named instance and usage of an Alias, I can easy change this or move the SQL to another cluster.

The installation will show me what cluster groups I can use and which are used at the moment.

Selecting the right disks for the installation remember These are the Storage space direct disks that are created with the cluster installation. See my other blog post about creating S2D on Windows Server 2019

A fixed IP is needed unless you use the DHCP checkbox.

Some applications needs special Collation, when changing this it is an Instance setting and can’t be changed again.

Normally you will place the SQL admins or some groups. this is a demo cluster so admin access only

Select the right disks and change this in the location fields.

Selecting multiple files for the Temp DB and the DB location

Also in this step we get a Ini file as output this can be reused if needed.

 

Now that the installation on one node is ready you can see the SQL resources in the cluster. As I did only the install on one node we need to do the Add node installation to get the installation done for a full FCI.

The Step to add an extra cluster node with the SQL installation is an easy step.

Follow the Wizard and use the Same account that you used before.

 

As the SQL studio is no longer a default installation you need to download the bits from Microsoft.

SSMS 18.0 Public Preview 6 is now available, and is the latest generation of SQL Server Management Studio that provides support for SQL Server 2019 preview!

SSMS 17.9.1 is the current General Availability (GA) version of SSMS

Download SQL Server Management Studio 17.9.1 
Download SQL Server Management Studio 17.9.1 Upgrade Package (upgrades 17.x to 17.9.1)

Version Information

• Release number: 17.9.1
• Build number: 14.0.17289.0
• Release date: November 21, 2018

As we are using the 2019 We need to have the latest version

Now we have the studio ready and can access the SQL instance.

In the next blogs I’ll show you how to extend the SQL to Azure.

Below is an overview of the SQL disks based on storage spaces direct.

With All these steps I hope you can build your own cluster and play with this. As for production never use a next next Finish installation there is always some custom tweaks needed to get the best performance.

 

Follow Me on Twitter @ClusterMVP

Follow My blog https://robertsmit.wordpress.com

Linkedin Profile Robert Smit MVP Linkedin profile

Google  : Robert Smit MVP profile

Share this:

 

Olympia V2 is the next step for enabling Windows Insiders to try new and pre-release Windows 10 Enterprise features. Windows Insider Lab for Enterprise v2 provides a complete Microsoft 365 deployment and management testing environment that can be run directly on your own machines. The lab features both client and administrative functionality, including System Center Configuration Manager Preview plus connectivity to Office 365 and Enterprise Mobility Security evaluation trials. Customers can also add the latest Windows 10 Insider Preview Enterprise build to the lab. 

This is a great lab toolkit, to start with the new features. It is easy to setup with a great learning curve.

First we download the entire Lab. it around 14GB

The table below lists the virtual machines, which will be imported and created in Hyper-V:

Server Name	Roles & Products
HYD -DC1	Active Directory Domain Controller, DNS, DHCP, Certificate Services Windows Server 2016
HYD-CM1	System Center Configuration Manager Technical Preview Branch – Version 1808 (Note: After installing a baseline version, you can then use in-console updates to bring your installation up-to-date with the most recent preview version. See Section 4.) Windows Deployment Services Microsoft Deployment Toolkit Windows 10 ADK Windows Software Update Services Microsoft SQL Server 2014 Windows Server 2016
HYD-APP1	Microsoft BitLocker Administration and Monitoring Microsoft SQL Server 2014 Windows Server 2016
HYD-GW1	Remote Access for Internet Connectivity Windows Server 2016
HYD-CLIENT1 (Optional)	If the Windows 10 Insider Preview ISO image is imported, this machine will be created with windows installed and will be Domain Joined
HYD-CLIENT2 (Optional)	If the Windows 10 Insider Preview ISO image is imported, this machine will be created with windows installed and will be Domain Joined
HYD-CLIENT3 (Optional)	If the Windows 10 Insider Preview ISO image is imported, this machine will be created with windows installed and will be on Workgroup
HYD-CLIENT4 (Optional)	If the Windows 10 Insider Preview ISO image is imported, this machine will be created with windows installed and will be on Workgroup

The VM list in Hyper-v

The table below lists the credentials and access type available in the default implementation:

After that just extract the files, keep in mind the setup extract the files at the current location of the setup files. You can move the VM’s afterwards

Starting the setup and extracting the VM’s

Select your Vswitch on the Hyper-v server

Select a insiders ISO or download one,

Plenty of room in the Windows Server 2019 Hyper-v server with Storage Spaces direct.

The extracting can take up some time depends on the disks and CPU speed for extraction

After the Extraction Several VM’s are added to the Hyper-v Server

The Gateway will route all the data to internet.

The setup is done the full lab is installed, there are several laps that you can do and setup

The domain structure that is created is the basic for all the labs

A SCCM site is created and ready for use. As this is the Technical preview I already got the 1812 Build

 

In the Azure Active directory we set some custom pictures.

 

Customize these screens is easy done in the Azure portal

Next step is use SCCM and Intune to manage your systems. This lab is perfect for showing all the options.

 

The Setup is Complete and ready to use, this lab is a great way to self explore the new features.

     Lab Objectives

This guide is designed to provide step-by-step guidance in demonstrating the basic functionality of the feature.

·         Lab Setup

o   On-Premises Environment

o   Cloud Environment

o   On-Premises Environment Post Setup Manual Steps

·         Servicing

o   Windows Analytics Update Compliance

·         Deployment & Management

o   Modern Device Deployment

o   Modern Device Management with AutoPilot

o   Co-Management

o   Modern Application Management with Intune

o   Enterprise State Roaming

·         Security

o   Windows Information Protection

o   Windows Defender Advanced Threat Protection

o   Windows Defender Application Guard

o   Windows Defender Exploit Guard

o   Windows Hello

o   Credential Guard

o   Device Encryption (MBAM)

o   Device Guard – User Mode Code Integrity

·         Compatibility

o   Windows Analytics Upgrade Readiness

o   Browser Compatibility

o   Desktop Bridges

·         Additional Labs

o   MDM WINS over GP

o   MAM FAQ

The Windows Insider Lab for Enterprise was designed for Windows Insiders who want to try new experimental and pre-release Enterprise Privacy and Security features. There are two versions of the lab:

· Windows Insider Lab for Enterprise v1 – provides a client-side view of the latest Microsoft 365 enterprise features through access to Olympia Corp – a virtual corporation has been set up to reflect the IT infrastructure of real world business. 

· Windows Insider Lab for Enterprise v2 – provides a complete Microsoft 365 deployment and management testing environment that can be run directly on your own machines. The lab features both client and administrative functionality, including System Center Configuration Manager Preview plus connectivity to Office 365 and Enterprise Mobility + Security evaluation trials.

 

Follow Me on Twitter @ClusterMVP

Follow My blog https://robertsmit.wordpress.com

Linkedin Profile Robert Smit MVP Linkedin profile

Google  : Robert Smit MVP profile

Share this:

In the former blog post :https://robertsmit.wordpress.com/2018/11/29/step-by-step-windows-server-2019-file-server-clustering-with-powershell-or-gui-cluster-ha-azure-windowsadmincenter-windowsserver2019/

I created a File share on a Cluster to make the share HA. This is more the traditional way to make the share HA. But what if you have multiple locations and you want to use this share in Azure. Big internal lines between the Datacenter and copy the files to Azure (DFS) method. but that’s old. Better use the Azure File Sync option the files are synced to all the Server and available in Azure. Better and faster.

#bettertogether  

 With Azure File Sync , shares can be replicated on-premises or in Azure and accessed through SMB or NFS shares on Windows Server. Azure File Sync is useful for scenarios in which data needs to be accessed and modified far away from an Azure datacenter, such as in a branch office scenario. Data may be replicated between multiple Windows Server endpoints, such as between multiple branch offices. Azure File Sync transforms Windows Server into a quick cache of your Azure file share. You can use any protocol that’s available on Windows Server to access your data locally, including SMB, NFS, and FTPS. You can have as many caches as you need across the world.

To get started with the Azure File Sync we need a Storage account in Azure.

We create a storage account in Azure.

Remember this works only on Windows Servers ! System Requirements:

• A server running Windows Server 2012 R2, Windows Server 2016 or Windows Server 2019:

  Version
  Supported SKUs
  Supported deployment options

  Windows Server 2019
  Datacenter and Standard
  Full (server with a UI)

  Windows Server 2016
  Datacenter and Standard
  Full (server with a UI)

  Windows Server 2012 R2
  Datacenter and Standard
  Full (server with a UI)

 

Now that the storage account is created we are starting with the Azure File Sync creation in Azure.

 

Name the Storage Sync Service , and create a resource group.

The next step is register the Onpremise server to Azure with the Azure File Sync Agent

Azure File Sync  Agent download https://go.microsoft.com/fwlink/?linkid=858257

The installation is in two steps.

1. Installing the agent
2. Configuring the Agent

After the download install the Agent on the File server, As I use a Cluster install the Agent on every node of the Cluster.

Now that the agent is installed the Second wizard pops up for the configuration and if needed a update.

So far so good. As the Agent is connecting to Azure there are some additional components needed.

As this Cluster was a fresh installation and I did not used the PowerShell command for Azure here I need to install the AzureRM modules (or AZ module)

https://go.microsoft.com/fwlink/?linkid=856959

Installing and updating the modules.

Install-Module -Name AzureRM –AllowClobber

With this command you can see the current Powershell version

Get-Module -Name AzureRM -List | select Name,Version

 

Now that the PowerShell commands are installed we can refresh the page and the installation continues

If you are using a CSP subscription in Azure then you need to set this check box. and use your tenant ID

In all other subscriptions keep this default

Pick the right Resource group the one with the created Storage Sync services in it. else the field will be empty.

Select a resource group that contains a Storage Sync Service, or use the Azure portal to create one in this resource group.

When this process is done we can configure the rest in the Azure portal.

As you can see the Cluster CNO object is named here

In the pane that opens, enter the following information to create a sync group with a cloud endpoint:

• Sync group name: The name of the sync group to be created. This name must be unique within the Storage Sync Service, but can be any name that is logical for you.
• Subscription: The subscription where you deployed the Storage Sync Service.
• Storage account: If you select Select storage account, another pane appears in which you can select the storage account that has the Azure file share that you want to sync with.
• Azure file share: The name of the Azure file share with which you want to sync.

Next is creating the Sync group.

 

Pick a name for the Sync group name. and the proper Storage account that we created earlier. In this storage account we did not create a File share this is needed to hold the Files. so the azure file share check box is not showing you anything.

Go the the storage account and create a File share

With this created the creation of the Sync group can be completed.

Next step is creating some endpoints. this means bind the local share to the services and sync this to the Azure storage account share.

Adding the endpoint and pick the registered server and the file share that will be synced.

If you want to enable cloud Tiering and fill in the values. In this demo I don’t use this.

Note:

Only NTFS volumes are supported. ReFS, FAT, FAT32, and other file systems are not supported.

Failover Clustering

Windows Server Failover Clustering is supported by Azure File Sync for the "File Server for general use" deployment option. Failover Clustering is not supported on "Scale-Out File Server for application data" (SOFS) or on Clustered Shared Volumes (CSVs).

The Azure File Sync agent must be installed on every node in a Failover Cluster for sync to work correctly.

In my demo the Share is not listed, I already know why, As I used ReFS for the cluster disk.

This can be painful as you need to format that disk and move all the data to a temp location.

       

After changing the disk format and a refresh you can see that the deployment is pending and working.

 

After this you have a full Hybrid file share Fully redundant on premise and a off load to Azure.

As last the best option to get the data into this HA file share is using the Windows Admin Center 

In Windows Admin Center there is a great options Storage Migration Services

Opening Windows admin Center and select the source this will be scanned and when done the files can be migrated. (the scanning can take some time)

When the scanning is done the files and shares are listed. more info can be found here https://youtu.be/WCWxAp27ERk

 

Follow Me on Twitter @ClusterMVP

Follow My blog https://robertsmit.wordpress.com

Linkedin Profile Robert Smit MVP Linkedin profile

Google  : Robert Smit MVP profile

Share this:

Installing the Cluster is easy now days. But just this I post a little blog on how to do this, In my blog stats it shows that the 2012 post is still very active , so time for an update to Windows Server 2019. in the creation there isn’t much changed, it gets only easier. but If you still not in PowerShell you got more clicks to do an less Coffee. And Windows Admin Center is also a great addition to manage a cluster. This blog post is also usable in Azure Only you need to add Storagespacesdirect and a CSV file share. 

Just install a bare metal (VM) windows Server 2019 and do a domain join and the fun can start.

Installing the Cluster Feature in powershell

Install-WindowsFeature –Name Failover-Clustering –IncludeManagementTools

#Create cluster validation report
Test-Cluster -Node MVP19-01,MVP19-02

#Create new Cluster
New-Cluster -Name MVP1911-27 -Node MVP19-01,MVP19-02 -NoStorage -StaticAddress "10.255.255.45"

#place witness file on USB device from my router

Set-ClusterQuorum -FileShareWitness \\SERVER\SHARE -Credential $(Get-Credential)

Now that the basic cluster is ready we start with the HA share

File share witness enhancements We enabled the use of a file share witness in the following scenarios:

• Absent or extremely poor Internet access because of a remote location, preventing the use of a cloud witness.
• Lack of shared drives for a disk witness. This could be a Storage Spaces Direct hyperconverged configuration, a SQL Server Always On Availability Groups (AG), or an * Exchange Database Availability Group (DAG), none of which use shared disks.
• Lack of a domain controller connection due to the cluster being behind a DMZ.

• A workgroup or cross-domain cluster for which there is no Active Directory cluster name object (CNO). Find out more about these enhancements in the following post in Server & Management Blogs: Failover Cluster File Share Witness and DFS.

  We now also explicitly block the use of a DFS Namespaces share as a location. Adding a file share witness to a DFS share can cause stability issues for your cluster, and this configuration has never been supported. We added logic to detect if a share uses DFS Namespaces, and if DFS Namespaces is detected, Failover Cluster Manager blocks creation of the witness and displays an error message about not being supported.

that’s it the cluster is created, we can start with the File server

Next is installation of the file server role

A restart is needed! After the restart we can build the cluster with the HA file share

$servers = ("MVP19-01", "MVP19-02") 
foreach ($server in $servers) {Install-WindowsFeature -Name file-services -ComputerName $server}

Now that the File Server Role is added we can add the Disk. Or use a disk that you already added before.

First we need to add a disk this can be done in the Failover Cluster manager or with PowerShell

 

Get-ClusterAvailableDisk | Add-ClusterDisk

The Roles are there and the Disk is added

Next step is adding the File server Role to the Cluster and add the HA File Share.

In this case I have a fail over disk and I use the File Server for general use.

 

So when adding the Disk it is not showing the disk. This is The disk is added to the cluster but the disk isn’t formatted!

Keep in mind that formating the cluster disk while it is online is not possible. You need to set the disk in maintenance mode else the format will fail.

 

So after the disk format we will see the Disk appear and can be added to the File server

 

After this the File server is up and running. As you can see the setup is screen intense, building this with PowerShell is a lot faster.

Powershell

add-ClusterFileServerRole -Storage "Cluster Disk 1" -Name MyFiles

New-SmbShare -Name "Data" -Path "J:\Data" -EncryptData $True

Quick steps with powershell and even the share is created and encrypted

Next step is adding the file share.

go for the Quick setup

Pick the disk and select the folder with the data on the disk, if there is no data then create a folder that will hold the data later.

as you can see the UNC path from the File Server.

 

As you can see the settings can be adjusted for you needs and also set the right access, and keep in mind this needs to be don on the Cluster Level!

All Done

So creating a File Server and 2 file shares is Click intensive if you don’t use PowerShell.

But What about Windows Admin Center ? yes that would be an option also except here you can’t create a cluster role.

You can create a new role but no file server /share etc.

But when the share is created and running like now you can use Windows Admin Center for migration the data to the file share.

But more and more options are coming in Windows Admin Center below are some links that you can use to add your request to the UserVoice

https://docs.microsoft.com/en-us/windows-server/manage/windows-admin-center/use/manage-failover-clusters

More Coming

Failover cluster management in Windows Admin Center is actively under development and new features will be added in the near future. You can view the status and vote for features in UserVoice:

Feature Request

Show more clustered disk info

Support additional cluster actions

Support converged clusters running Hyper-V and Scale-Out File Server on different clusters

View CSV block cache

See all or propose new feature

+++++++++++++++

 

Follow Me on Twitter @ClusterMVP

Follow My blog https://robertsmit.wordpress.com

Linkedin Profile Robert Smit MVP Linkedin profile

Google  : Robert Smit MVP profile

Share this:

 

Distributed denial of service (DDoS) attacks are some of the largest availability and security concerns facing customers that are moving their applications to the cloud. A DDoS attack attempts to exhaust an application’s resources, making the application unavailable to legitimate users. DDoS attacks can be targeted at any endpoint that is publicly reachable through the internet.

What is DDoS Protection? Protecting applications from DDoS attacks has been one of the top security concerns for Azure customers. Azure DDoS protection service is an Azure Networking offering aimed at protecting publicly accessible endpoints from DDoS attacks. The offering gives customers access to the same protection that is used to protect Microsoft’s online assets, such as Xbox Live and Office 365. Azure DDoS protection service provides constant network flow monitoring of the protected endpoints, and when detecting a DDoS attack, automatically applies traffic scrubbing to make sure only legitimate requests are forwarded to the application.

Azure DDoS protection, combined with application design best practices, provide defense against DDoS attacks. Azure DDoS protection provides the following service tiers:

• Basic: Automatically enabled as part of the Azure platform. Always-on traffic monitoring, and real-time mitigation of common network-level attacks, provide the same defenses utilized by Microsoft’s online services. The entire scale of Azure’s global network can be used to distribute and mitigate attack traffic across regions. Protection is provided for IPv4 and IPv6 Azure public IP addresses.
• Standard: Provides additional mitigation capabilities over the Basic service tier that are tuned specifically to Azure Virtual Network resources. DDoS Protection Standard is simple to enable, and requires no application changes. Protection policies are tuned through dedicated traffic monitoring and machine learning algorithms. Policies are applied to public IP addresses associated to resources deployed in virtual networks, such as Azure Load Balancer, Azure Application Gateway, and Azure Service Fabric instances, but this protection does not apply to App Service Environments. Real-time telemetry is available through Azure Monitor views during an attack, and for history. Rich attack mitigation analytics are available via diagnostic settings. Application layer protection can be added through the Azure Application Gateway Web Application Firewall or by installing a 3rd party firewall from Azure Marketplace. Protection is provided for IPv4 Azure public IP addresses.

So how to start with DDoS in Azure.

First go to the Virtual Networks.

Next selecting the Network and in the left pane there is a section DDoS Protection.

Selecting the DDoS Protection there is the Basic and the Standard Setting

Pricing Details

There the Basic is the default and comes with free pricing.

The Standard is a different option and Cost you some real money! and these are monthly costs. For a demo I turned it on and forget to turned it of and spend 10K in 4 months so keep a track on your Azure costs.

The DDoS Protection service will have a fixed monthly charge, as well as a charge for data processed. The fixed monthly charge includes protection for 100 resources. Protection for additional resources will be charged on a monthly per-resource basis.

Monthly price for DDoS Protection (includes protection for 100 resources): €2,483/month

Overage charges (more than 100 resources): €25 per resource per month

 

When Enabling the DDoS Standard we need to create a DDoS protection plan first, if you have already one you can add the ID.

Check the create DDoS protection Plan

Now that we created a plan witch is more a resource place holder, we can add this to the DDoS protection plan

Now that the DDoS and the plan is in place we can create an alert rule in case we have a DDoS attack.

In the Azure Monitor we can create the alert rule and we can see the logging.

To see telemetry for a DDoS attack, log into the Azure Portal and navigate to the “Monitor” blade.

Within the monitor blade, click on “Metrics”, select the appropriate subscription, resource group, resource type of “Public IP” and the Public IP that was the target of the attack. After selecting the resource, a series of Available Metrics will appear on the left side. These metrics are selected and then will be graphed.

The metric names are relatively self-explanatory and the basic construct is that there are tag names on each metric as follows: • Dropped tag name (e.g. Inbound Packets Dropped DDoS): The number of packets dropped/scrubbed by the DDoS system

• Forwarded tag name (e.g: Inbound Packets Forwarded DDoS): The number of packets forwarded by the DDoS system to the destination VIP – traffic that was not filtered • No tag name (e.g: Inbound Packets DDoS): The total number of packets that came into the scrubbing system – representing the sum of the packets dropped and forwarded

The traffic shown in the Monitor dashboard.

To create a dashboard there are some options with counters. It all depends on your need.

 

now we create an alert rule.

Email Alerting To configure an email alert for a metric, click on the “Click to add an alert” text. An email alert can be created on any metric, but the most obvious metric to create an alert on is “Under DDoS attack or not”. This is a boolean value 1 or 0. “1” means you are under attack. “0” means you are not under attack. To be emailed when under attack, set the Metric for “Under DDoS attack or not” and “Condition” to “Greater than” zero (0) over the last 5 minutes. Similar alerts can be set up for other metrics. An example screenshot is provided below.

 

 

To divine the Severity I keep this as this is also be used in SCOM

Azure Monitor Alert Severity Levels

Sev 0 = Critical
Sev 1 = Error
Sev 2 = Warning
Sev 3 = Informational
Sev 4 = Verbose

Last part in selecting the email for this alert.

With this setup you got a good protection against DDoS attacks. below is the workflow how DDoS protection works.

 

Follow Me on Twitter @ClusterMVP

Follow My blog https://robertsmit.wordpress.com

Linkedin Profile Robert Smit MVP Linkedin profile

Google  : Robert Smit MVP profile