Building in Azure is easy and the wizard takes you to all the steps and you have a working VM. choosing the right size is different often it has a link to the on premise world 4 core CPU and 8 GB memory. and the disk I need 1 TB disk space. All simple but then things get complicated the performance needs to be better CPU is fine Memory well 60 % plenty of disk space. Bigger VM perfect.
Still slow Whole VM runs at 20-60% users are complaining must be this Azure thing someone else his computer runs slow.
I often hear this. But is it really slow or is your measurement wrong ?
When you pick a machine on premise what do you take performance or Cost ? <> performance and then cost right and at the end you settle with the cost vs. performance.
But in Azure what do you take performance or Cost ?<> 100% Costs, VM’s are expensive. This is not always wrong but sometimes is paying a bit more the best approach
In my sample here I show you the performance in a Disk with different machine types, and not picking the right components doesn’t give you the right performance. but it may well function on your workload, but then you may pay to much for you over sized configuration.
In my sample I need a VM with 300 IOPS and one with 4000 IOPS and I need one with 27000 IOPS CPU and Memory are in this case not important as it is more i/o intensive.
I pick a default Azure VM an D machine, put some disks to the machine one HDD-S30 ,SSD-E30 ,SSD-P30,SSD-P60
| VM Type | Disk Type | MiB/s | I/O per s |
| Standard D2s v3 (2 vcpus, 8 GiB memory) | HDD-S30 | 2.01 | 514.23 |
| SSD-E30 | 2.21 | 566.27 | |
| SSD-P30 | 13.29 | 3403.51 | |
| SSD-P60 | 12.33 | 3157.46 |
First goal met 500 IOPS and an cheap machine but this could also an Azure B type VM much cheaper. then I wonder why use SSD over HDD for the IOPS it’s the same speed and latency there is a point SDD are performance steady, but for normal workload. Costs If you have a lot of transactions then SDD may be cheaper. A fact is nobody knows how expensive the HDD disk are, have you ever calculated the Storage transactions ?
below is a overview of the disk latency.
25th | 100.325 | N/A | 100.325 HDD-S30
25th | 100.012 | N/A | 100.012 SSD-E30
25th | 4.545 | N/A | 4.545 SSD-P30
Comparing all the SSD disks and pick the right performance is not hard Microsoft did a great job on explaining this. on Microsoft docs
Disk size
| Premium SSD sizes | P30 | P40 | P50 | P60 | P70 | P80 | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Disk size in GiB | 1,024 | 2,048 | 4,096 | 8,192 | 16,384 | 32,767 | |||||
| IOPS per disk | Up to 5,000 | Up to 7,500 | Up to 7,500 | Up to 16,000 | Up to 18,000 | Up to 20,000 | |||||
| Throughput per disk | Up to 200 MiB/sec | Up to 250 MiB/sec | Up to 250 MiB/sec | Up to 500 MiB/sec | Up to 750 MiB/sec | Up to 900 MiB/sec |
When you provision a premium storage disk, unlike standard storage, you are guaranteed the capacity, IOPS, and throughput of that
When you provision a premium storage disk, unlike standard storage, you are guaranteed the capacity, IOPS, and throughput of that
that is interesting In my D2 machine and with a P30 I got only 3400 IOPS, so this is wrong ? Well according to the disk but the VM can only deliver 3200 IOPS with the 3400 IOPS delivered its perfectly normal then.
The same test again with a better Azure VM and the same disks.
| VM Type | Disk Type | MiB/s | I/O per s |
| Standard DS3 v3 (4 vcpus, 14 GiB memory) | HDD-S30 | 2.01 | 514.01 |
| SSD-E30 | 2.21 | 566.63 | |
| SSD-P30 | 21.58 | 5523.51 | |
| SSD-P60 | 51.00 | 13056.39 |
The requirements are there 5500 Iops for a disk that need to deliver 5000 IOPS that’s good. but what about the P60 disk , again a had cap to the VM max of 12800 IOPS
The latency is not that different for this you need a different kind of VM
25th | 100.256 | N/A | 100.256 HDD-S30
25th | 100.008 | N/A | 100.008 SSD-E30
25th | 4.416 | N/A | 4.416 SSD-P30
25th | 2.135 | N/A | 2.135 SSD-P60
Comparing the Azure VM’s selected on IOPS and select the right machine
selecting the F4 VM that can deliver 16000 lops according the sheet .
| VM Type | Disk Type | MiB/s | I/O per s |
| Standard F4s (4 vcpus, 8 GiB memory) | HDD-S30 | 2.01 | 514.01 |
| SSD-E30 | 2.21 | 566.63 | |
| SSD-P30 | 21.58 | 5523.51 | |
| SSD-P60 | 50.85 | 13018.46 |
Did not get the 16.000 lops in fact it produce almost the same results ad the DS3 only double the costs.
SSD-P60 latency measurement 4k blocks vs 64K blocks
25th | 2.171 | N/A | 2.171
25th | 3.088 | N/A | 3.088 <> 64kblocs
So this strange big machine still not hitting the limits CPU and memory is low. Seems good but not the performance
Checking the Microsoft site : https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-compute
You can see a different specs result. this means the machine can’t deliver the IOPS and the Size table thinks he can. Results are bad.
| Standard_F4s_v2 | 4 | 8 | 32 | 8 | 8000 / 63 (64) | 6400 / 95 | 2 / 1750 |
Then lets pick a Azure VM than can deliver the iops. a F16 big VM costly but can it deliver I compare both tables In the Azure portal and the Docs
But on the other side on the Docs https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-compute
| Standard_F16s_v2 | 16 | 32 | 128 | 32 | 32000 / 255 (256) | 25600 / 380 | 4 / 7000 |
| VM Type | Disk Type | MiB/s | I/O per s |
| Standard F16s v2 (16 vcpus, 32 GiB memory) | HDD-S30 | 2.01 | 514.09 |
| SSD-E30 | 2.21 | 566.63 | |
| SSD-P30 | 21.60 | 5529.96 | |
| SSD-P60 | 63.76 | 16321.29 |
This looks OK now 16000 IOPS.
But what If I build a stripe set from the SSD-P30 and SSD-P60 and HDD-S30 and SSD-E30 what would be the iops ? (it’s a bad idea to mix different disk types this is just a sample)
What if we create a stripe set ?
Worse performance than if I user the SSD-P60 alone. Bad config to do this.
HDD and SSD
Both Disks have around 500 IOPS each and now they can produce a 1000 IOPS that’s not bad
But what happens if I combine all the disks into a Storage space direct ? combining all the disk you have and build a new disk JBOD.
Also a Bad Idea and a waste of resources and Money an P60 disk combined with a S30
That’s all about the little side step, but it keeps me thinking…. -What if
Below is a list with similar iops performance And Instead of using 1 SSD-P60 I’ll use 3 disks on paper I should have 3x 16000 IOPS = 48000 IOPS and 3x 500MB/s =1500 MB/s that is massive right. stripe set or Storage space or storage space direct ? all valid options but what machine do I need to handle the performance.
I selected 3 types a E32,DS5 and a DS14 all with big price difference but similar specs .
| Standard_E32s_v3 2 | 32 | 256 | 512 | 32 | 64000 / 512 (800) | 51200 / 768 | 8 / 16000 |
| Standard_DS5_v2 | 16 | 56 | 112 | 64 | 64000 / 512 (688) | 51200 / 768 | 8 / 12000 |
| Standard_DS14_v2 3 | 16 | 112 | 224 | 64 | 64000 / 512 (576) | 51200 / 768 | 8 / 12000 |
First I build a Storage Pool on the DS5_V2
Nice Capacity good latency and decent performance a round 29000 IOPS of 3 disks, in a Mirror set I’ll loose a disk so the performance is good better than I expected. To hit the limits I should add 2 more disks to this config and see if they can handle the performance.
25th | 2.025 | N/A | 2.025
I’ll run the same test on a E32-8s_v3
Bigger VM much more performance, higher price.
So overall the cheaper VM can produce the same disk performance. but the machine is $1000 cheaper per month. Again it depends what you are doing with the VM
Now the same configuration with Storagespaces Direct just to see if the performance is better, keep in mind that every run the machine performance can be a bit different so in the same range I see this as the same performance.
The S2D results on a E32 VM
And even a step higher an expensive VM with 432 GB memory. With an S2D Cluster.
So same performance when Running a StorageSpace or S2D cluster and no change on the machine type. in fact the DS5 machine is slightly better. it saves $2000 per month. If you don’t need the CPU and memory from the VM.
So size does matter but it depends on what size you are looking right. Azure is like Lego but different. Combining the pieces makes a great solution.
Below I created a table Cost vs performance, I also compared the datasheet in the azure portal to the DOC pages and I think you should keep this page as a reference. https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-memory
This shows you that in complex configurations there is no one size fits all and it comes to testing and adjusting, Tools may help you but picking the right VM size and choose the right storage can take some time. As in this I only compared disks but what if I choose Netapp files or some other disks like ultra SSD’s
And Now I did this config with 3 P60 disk that cost $1000 each = $3,121.92 (in azure Calculator) it gets me ~30.000 IOPS
Now On the DS5 machine a 2 way mirror Auto created.
It nags me that I can’t get the max from the VM, the must be something wrong in my configuration. lets do some quick testing change VM and Disk types
With 6 times a SSD-P30 disk I’ll get 27.000 IOPS on the DS5 Machine
When using a Stripeset this hits the VM limit of 768 true put. Less IOPS but more speed. So Configuration is also KEY in the used hardware.
Lets tweak the config a bit and see if we can pass the 50.000 Iops and hit the machine limit.
With read cache enabled and 8 P30 disks. that’s not bad right.
The P40 disks have 7500 IOPS each will this break the record ? (6x P40 disk storage space)
First test same result a bit lower, but there is more to get. Testing now With 8 P40 disks
(8x P40 disk storage space)
(8x P40 disk storage space) Manual configuration.
(8x P40 disk storage space) Manual configuration. with 6 columns
That’s not bad the DS5 hits the limit.
On Microsoft Ignite 2015 Mark Russinovich did a demo, where he showed a virtual machine with Premium Storage that hit over 64,000 IOPS. Well This beats the record but the Azure hardware is much better now right.
Lets Switch to some big Azure VM
64 Cores lets see If I can use some of these cores in the S2D config.
Oh ok it seems I need more cores or less workload on this. But easily hit the IOPS limit on this machine.
Overall in this is what do you need and test this also with a different configuration. Not only on price but also on performance. In the first section I used 3x a P60 disk cost $3.000 a even better result I get with 8x P30 disk cost $1.000
Picking the right configuration can only be don based on testing and create some references for you. Azure machines and storage is changing all the time its getting better all the time. It all depends on your workload but there is no one size fits all !
Follow Me on Twitter @ClusterMVP
Follow My blog https://robertsmit.wordpress.com
Linkedin Profile Robert Smit MVP Linkedin profile
Google : Robert Smit MVP profile
