Category Archives: Microsoft Azure

VM Inception | Nested Virtualization in Azure

I bet that most of you have watched the movie “Inception”, where a group of people are building a dream within a dream within a dream. Before Windows Server 2016 you couldn’t deploy a VM within a VM in Hyper-V. Lot of people are/were encouraged to use VMware as it supported this capability called “Nested Virtualization”. But with the release of Windows Server 2016 & Hyper-V server 2016 this functionality has been introduced. This is specially useful when you don’t have lot of hardware to run your lab environments or want to deploy a PoC system without burning thousands of dollars.

Microsoft announced the support for nested virtualization Azure IaaS VMs using the newly announced  Dv3 and Ev3 VM sizes. This capability allows you to create nested VMs in an Azure VM and also run Hyper-V containers in Azure by using nested VM hosts. Now let’s have look on how this is implemented in Azure Azure Compute fabric.

Image Courtesy Build 2017

As you can see in the above diagram, on top of the Azure hardware layer, Microsoft has deployed the Windows Server 2016 Hyper-V hypervisor. Microsoft then adds vCPU on top of that to expose the Azure IaaS VMs that you would normally get. With nested virtualization, you can enable Hyper-V inside those Azure IaaS VMs running Windows Server 2016. You can then run any number of  Hyper-V 2016 supported guest operating systems inside these nested VM hosts.

Following references from MSFT provides more information on how you can get started with nested virtualization in Azure.

 

 

 

 

 

502 Bad Gateway error | Azure Application Gateway Troubleshooting

I was setting up an Azure Application Gateway for a project couple days back. The intended workload was setting up git on nginix.  But when I tried to reach the git URL I noticed that it was failing with 502 Bad Gateway error.

Initial Troubleshooting Steps

  • We tried to access backend servers from Application Gateway IP 10.62.124.6, backend server IPs are 10.62.66.4 and 10.62.66.4. Application Gateway configured for SSL Offload.
  • We were able to access the backend servers directly on port 80, but when accessed via Application Gateway this issue occurs.
  • Rebooted the Application Gateway and the backend servers. and configured custom probe as well. But the issue was with the Request time out value which is by default configured for 30 seconds.
  • This means that when user request is received on Application Gateway, it forwards it to the backend pool and waits for 30 seconds and if it fails to get a response back then within that period users will receive a 502 error.
  • The issue has been temporarily  resolved after the time out period on the Backend HTTP settings has been changed to 120 seconds.

Real Deal

Increasing the timeout values were only a temporary fix as we were unable to find out a permanent fix. I have reached out to Microsoft Support and they wanted us to run below diagnostics.

  • Execute the below cmdlet and share the results.

$getgw = Get-AzureRmApplicationGateway -Name <application gateway name> -ResourceGroupName <resource group name>

  • Collect simultaneous Network traces:
    1. Start network captures on On-premises machine(Source client machine) and Azure VMs (Backend servers)
      • Windows: Netsh
        1. Command to start the trace:  Netsh trace start capture=yes report=yes maxsize=4096 tracefile=C:\Nettrace.etl
        2. Command to stop the trace:  Netsh trace stop 
      • Linux: TCPdump
        1. TCP DUMP command: sudo tcpdump -i eth0 -s 0 -X -w vmtrace.cap 
    2. Reproduce the behavior.
    3. Stop network captures.

Analysis

The network traces collected on Client machine and Destination servers while the issue was reproduced indicates that,  during the time period the trace was collected, for every HTTP Get Request (default probing) from the Application Gateway instances on the backend servers, the servers responded “Status: Forbidden” HTTP Response.

This has resulted in Application Gateway marking the backend servers as unhealthy as the expected response is HTTP 200 OK.
 
The Application Gateway “gitapp” configured for 2 instances (Internal instance IPs: 10.62.124.4, 10.62.124.5)
 
Trace collected on backend server 10.62.66.4
 
12:50:18 PM 1/3/2017    10.62.124.4         10.62.66.4            HTTP      HTTP:Request, GET /
12:50:18 PM 1/3/2017    10.62.66.4            10.62.124.4         HTTP      HTTP:Response, HTTP/1.1, Status: Forbidden, URL: /
12:50:45 PM 1/3/2017    10.62.124.5         10.62.66.4            HTTP      HTTP:Request, GET /
12:50:45 PM 1/3/2017    10.62.66.4            10.62.124.5         HTTP      HTTP:Response, HTTP/1.1, Status: Forbidden, URL: /
 
Trace collected on backend server 10.62.66.5
 
12:50:48 PM 1/3/2017    10.62.124.4         10.62.66.5            HTTP      HTTP:Request, GET /
12:50:48 PM 1/3/2017    10.62.66.5            10.62.124.4         HTTP      HTTP:Response, HTTP/1.1, Status: Forbidden, URL: /
12:50:45 PM 1/3/2017    10.62.124.5         10.62.66.5            HTTP      HTTP:Request, GET /
12:50:45 PM 1/3/2017    10.62.66.5            10.62.124.5         HTTP      HTTP:Response, HTTP/1.1, Status: Forbidden, URL: /

Rootcause

Due to the security feature ‘rack_attack’ enabled on the backend servers, it has blacklisted the application gateway instance IP’s and therefore the servers were not responding to the Application Gateway, causing it to mark the backend servers as Unhealthy.

Fix

Once this feature was disabled on the backend web servers (niginx) , the issue was resolved and we could successfully access the web application using Application Gateway.

 

Backup ARM VMs in Azure | Tips & tricks

As you already know Microsoft Azure Fabric is now in version 2 which is sometimes referred to as Azure Resource Manager (ARM) deployment model. Most of the services from old Azure Service Management model are now available in the new model (the new portal) and today we are going to see how we can backup VMs deployed using ARM deployment model using a Azure Recovery Services Vault.

Note that you may notice another two services in your Azure subscription called Backup vaults & Site Recovery vaults which are redundant and has no use. (They are just placeholders which will be removed soon I assume)

Backup ARM VMs (1)

Essentially following scenarios are supported in a new Recovery Services vault. If you are using premium storage accounts for your VMs  keep in mind that it is only supported in a public preview and not generally available as of yet.

  • Azure Resource Manager VMs
  • Classic VMs

The process can be done in few easy steps.

Creating a Recovery Services Vault

A Recovery Services vault holds all the backups and recovery points of the VMs that are being protected along with the backup policy applied to that vault.  One important thing to keep in mind is that Recovery Services Vaults are geo specific, meaning if you need to backup a VM in one region the target vault should reside in the same region as well.

In the Hub menu, click Browse and then search for Recovery Services. I’ve already added it as a favorite by clicking the star right next. Then select Recovery Services vault and click Add.

Backup-ARM-VMs-2.png

Provide a name, select the target Azure subscription, create a new resource group or select an existing one and finally select the region for your Recovery Services vault.

Backup-ARM-VMs-3.png

Next you can select the storage replication option. The default is Geo-redundant storage and if you want a cheaper (but not durable as Geo-redundant) option you can opt out for locally-redundant storage.  Click the All Settings option in your vault dashboard to get started.

Backup-ARM-VMs-4.png

Select a Backup Target

You need to discover your Azure ARM VMs first before they are added to a recovery services vault. This will identify the VMs that can be protected by your recovery services vault.

Backup-ARM-VMs-5.png

Define a Backup Policy

A backup policy defines how frequent the VMs are protected and when the recovery points are created along with the retention range for those recovery points. You can edit the default policy to fit to your needs or create new policy here. You can choose between a daily or weekly schedule to backup your VMs.

Backup-ARM-VMs-6.png

Next select the desired VMs that you wish to backup and finally click Enable Backup.

Backup-ARM-VMs-7.png

Backup-ARM-VMs-8.png

Start the Initial Backup

By default the first scheduled backup is the initial backup. If you want to manually force the first backup it is also possible. In the vault dashboard click Azure Virtual Machines and right click on the desired VM and select Backup Now.

Backup-ARM-VMs-9.png

You can see the backup job progress by clicking All Settings > Jobs > Backup Jobs as below from the vault dashboard.

Backup-ARM-VMs-10.png

When you further expand the backup job you can see the status of each task running underneath.

Backup-ARM-VMs-11.png

Azure Cool Blob Storage | What, Why & How?

What is Azure Cool Blob Storage?

Few days back Microsoft Azure storage team added a new variant of  a storage offering called Cool Blobs. Like Amazon S3, Azure blob storage is a low cost object storage offering for Azure which enables you store your backup, media content such as images and videos, scientific data, compliance and archival data.

Why Cool Blob Storage?

Cool Blob Storage is ideal of infrequent accessed object data, that is data accessed less than once a month. Based  on the frequency of access, you can select between Hot or Cool access tiers for a storage account now. Cool Blob Storage provides following benefits for you as an end user.

  • Cost effective: Data stored at cool access tier comes at a lower price point as low as $0.01 per GB in some regions, where data you store in a hot storage tier start at $0.024 in some regions.
  • Compatibility: This is  100% API compatible with exiting Azure Blob storage and you can use this new type of storage accounts right away in your exiting applications.
  • Performance: Both Hot and Cool tiers have the same performance in terms of latency and throughput.
  • Availability:The data write SLA for Hot access tier is 99.99% where it is 99% for Cool tier. Also the read SLA is 99.99% for Hot tier where it is 99.9 for the Cold tier by leveraging the Read Access-Geo Redundant Storage, storage replica option in Azure.
  • Durability: Unlike Amazon S3 which guarantees you have Nine 11s (99.999999999%) of durability, Microsoft guarantees that your data will never be lost.  The AWS S3 SLA really interprets as “If you store 10,000 objects with us, on average we may lose one of them every 10 million years or so. This storage is designed in such a way that we can sustain the concurrent loss of data in two separate storage facilities.” Both Hot and Cool storage tiers in Azure provide the same high durability that Azure is currently offering which is 0% data loss.
  • Scalability and Security: The same scalability and security options in Azure Storage is provided in the new Blob storage accounts tiers as well.

How to deploy?

Let’s explore how you can create a new blob storage account with hot or cold access tiers in Azure GUI. Notice that this is only possible with ARM storage accounts not with classic storage. Also as of now this feature is only supported in storage accounts with standard performance.Blob Storage 1Changing the access tier is easy and takes only a click of a button.

Blob Storage 2

FAQs

Can I store my VM’s in cool/hot storage? No. Azure IaaS VM disks require page blobs and this is offered only in block blobs.
Can I convert my existing storage account to a Blob storage account? No. You need to create a new storage account or migrate data from an existing storage account to a new account.
Is this available in the classic model? No. This only supports ARM based deployments.
Can I have both hot/cool tiers in a single storage account? Not at this time. The access tier attribute is set at an account level and applies to all objects in that account.
Will I be charged for changing the access tier of my blob storage account? Changing the access tier at an account level will apply to all objects stored in the account. If you are changing from from hot to cool there won’t be any charge but changing from cool to hot will incur a per GB cost for reading all the data in the storage account.

 

 

Exporting your Azure Resource Groups to ARM Templates | Part 2

In my previous post I showed you how we can export Azure resource groups into ARM templates using the Azure Portal. For those of us who are not GUI fans (including myself) Azure PowerShell and Azure CLI provide cmdlets/commands to leverage the export feature for cloning, redeploying and automating Azure resource group deployments.

Azure PowerShell

With the latest Azure PowerShell you can execute below cmdlet to export a running resource group to an ARM template.

Export-AzureRmResourceGroup -ResourceGroupName <RG name> -Path <template path>

To export resource groups from a previous deployment you may use the below cmdlet syntax.

Save-AzureRmResourceGroupDeploymentTemplate -DeploymentName <Deployment Name> -ResourceGroupName <RG Name>-Path <template path>

Azure CLI

You can use the following syntax to export a running resource group to an ARM template.

azure group export <name> [template path]

Use below command syntax to export to an ARM template from a previously deployed Resource Group

group deployment template download [options] <resource-group> <name> [directory]

 

Exporting your Azure Resource Groups to ARM Templates | Part 1

Have you ever wanted to clone your resource group  deployment in Azure to another subscription or perhaps redeploy again without manual interaction with GUI? Now you can export your resource groups as ARM templates and redeploy wherever you want without having further barriers. Let’s explore how to use this feature in Azure.

Export from an existing Resource Group Deployment

When you select a resource group you can see the Export Template option in Settings.

Export RG to ARM (1)

Export RG to ARM (1)

Export from a previous deployment

In your resource group select the particular deployment slot and you will have the option to export that particular slot with parameters submitted for that specific instance of deployment.

Export RG to ARM (5)

Saving and Redeploying to a new resource group

Alternatively you have the option to Save the template and it will be saved under Browse > Templates in the Azure Portal.

Export RG to ARM (6)

Export RG to ARM (4)

Selecting the Deploy button will allow you to start a new deployment.

Export RG to ARM (3)

Keep in mind that currently not all the resource types are supported in with export feature. For an example you may encounter failure s when you try to export resources such as WebApps, Service Bus, Stream Analytics etc… Following is such an error which happened when we tried to export a resource group with Service Bus resources.

The schema of resource type ‘Microsoft.ServiceBus/namespaces’ is not available. Resources of this type will not be exported to the template. (Code: ResourceTypeSchemaNotFound)

This has been reported to Microsoft and this post will be updated once Microsoft provide a list of supported resource types/add more and more supported resource types to this feature. Right now I can confirm that IaaS resources are fully supported in this feature.

In this next post let’s see how we can leverage Azure PowerShell or Azure CLI to export resource groups into ARM templates.

SOS for Azure VMs with Set-AzureRmVM

“Save Our Souls” is the International distress call for help in maritime operations. Over the years SOS has become more common term to imply a call for help in a disastrous situation. In Microsoft Azure sometime you may have faced such situations especially with IaaS VMs. For an example RDP not working in a Windows VM or SSH ceased to function in a Linux VM. When all hope is lost you may contact Azure Support or try to restart the VM (from Azure Portal) or resize the VM as a last resort.

Now going into all of the above troubles is no longer required to rescue your Azure IaaS VMs. The latest Microsoft Azure PowerShell cmdlet improvements allows you to redeploy your virtual machine when you invoke a redeploy operation through Azure PowerShell.

Important

  • Below cmdlet works only with Azure Resource Manager based VMs.
  • Latest version of Azure PowerShell needs to be installed in the management PC from which you are invoking the redeploy operation.
  • Dynamic IP addresses will be changed after completing the redeploy operation.
  • Data on local disks (ephemeral storage) will be lost.

Following is the syntax for the updated Set-AzureRmVM cmdlet. Note that the -Redeploy switch is used to invoke a redeploy operation.

SetAzureRmVM Redeploy ResourceGroupName $rgname Name $vmname

The VM status changes from Running > Updating > Starting > Running during the operation. The final Running status means that VM has been successfully redeployed.

For a complete reference of the Set-AzureRmVM cmdlet please refer here.

Creating a SQL Database V12 Server in Azure

Few days ago my friend Business Solutions (Dynamics NAV) MVP Tharanga Chandrasekara came up with an interesting question. Creating a logical server for SQL Azure DB (PaaS) in the old Azure Service Management Portal is possible but why can’t we do that in the new Azure Resource Manager Portal. To find out what is happening I tried exploring the SQL PaaS option in the ARM portal.

When we create a new SQL database in the ARM portal we can create a logical server along with it as below.

SQL V12 1

But somehow when we checked two days back there was no Add button under SQL servers. We have tried the same thing in several Azure Subscriptions but there was no luck.

SQL V12 3

But today I checked again the same thing in one of my subscriptions and could see the Add button and could create a server without any problem.

SQL V12 2

Nothing out of the ordinary was mentioned in any forum as well and Tharanga has posted a question in User Voice. We were hoping the PG can shed some light into this. Whether it was a glitch on certain subscriptions or actually missing feature until now.

This led me to explore how to do this in ARM using PowerShell.

  • First you need to install the new Azure PowerShell module to start with. You can refer this to understand how to do so.
  • Then you can execute below cmdlets in Azure Powershell to login to your Azure Subscription and choose the exact subscription (if you have many Azure subscriptions under one account)

Add-AzureRmAccount
Select-AzureRmSubscription -SubscriptionId <Subscription ID>

  • Not all resources in ARM are available in all regions so it is always better to check whether the V12 database servers are available in the region you were planning to deploy.

(Get-AzureRmLocation | where-object {$_.Name -eq “Microsoft.Sql/servers” }).Locations

  • Next step is to create  a resource group in your desired region. I chose East US.

New-AzureRmResourceGroup -Name “jcbv12sql-RG” -Location “East US”

  • Then you can create the SQL V12 server and add firewall rules to allow any connections from outside Azure.

New-AzureRmSqlServer -ResourceGroupName “jcbv12sql-RG” -ServerName “jcbv12svr01” -Location “East US” -ServerVersion “12.0”

New-AzureRmSqlServerFirewallRule -ResourceGroupName “jcbv12sql-RG” -ServerName “jcbv12svr01” -FirewallRuleName “exrule1” -StartIpAddress “<First IP Address>” -EndIpAddress “<Last IP Address>”

Azure Automation PowerShell ISE add-on is now GA

Azure Automation team has announced the general availability of PowerShell ISE add-on for Azure Automation last week. With this add-on it is easier to author your Azure Automation runbooks using the familiar PowerShell ISE. Below are some of the notable features of this add-on.

  • Use Automation activities (Get-AutomationVariable, Get-AutomationPSCredential, etc) in local PowerShell Workflows and scripts
  • Create and edit Automation assets locally
  • Easily track local changes to runbooks and assets vs the state of these items in an Azure Automation account
  • Sync runbook / asset changes between a local runbook authoring environment and an Azure Automation account
  • Test PowerShell workflows and scripts locally in the ISE and in the automation service

Installing Azure Automation add-on for PowerShell ISE is pretty much straight forward. Although you can install the add on from the GitHub source, Microsoft recommends that you install the add-on from the PowerShell Gallery.

  • In an elevated PowerShell window execute below cmdlet. This will install the add-on only for the current user.

Install-Module AzureAutomationAuthoringToolkit -Scope CurrentUser

  • To automatically load the Azure Automation ISE add-on every time you open the PowerShell ISE execute below cmdlet.

Install-AzureAutomationIseAddOn

  • Also to load the add-on adhoc only when you want, execute  below cmdlet in the PowerShell ISE.

Import-Module AzureAutomationAuthoringToolkit

Managing Cloud Storage with Microsoft Azure Storage Explorer

Today you might be using different third party tools to perform management operations in your Azure storage accounts. CloudXplorer & CloudBerry are some good candidates but they are not free (as in beer). For those Developers who are using Visual Studio 2013/2015 the in-built cloud explorer is a perfect tool but what about the IT Professionals like us? Do we have a good and free alternative?

Microsoft has introduced a standalone version of Microsoft Azure Storage Explorer (Preview) with Azure SDK 2.8 release.  This tool is let’s you to quickly create blob containers, upload file content into blob containers, download files, set properties and metadata, and even create and get SAS keys to control access. Also you can quickly search for containers and individual blobs, and inspect a number of things like metadata and properties on the blobs.

Features in Storage Explorer

  • Mac OS X, Windows, and Linux versions (New in v0.7.20160107)
  • Sign in to view your Storage Accounts – use your Org Account, Microsoft Account, 2FA, etc
  • Add Storage Accounts by account name and key, as well as custom endpoints (New in v0.7.20160107)
  • Add Storage Accounts for Azure China (New in v0.7.20160107)
  • Add blob containers with SAS key (New in v0.7.20160107)
  • Local development storage (Windows-only)
  • ARM and Classic resource support
  • Create and delete blobs, queues, or tables
  • Search for specific blobs, queues, or tables
  • Explore the contents of blob containers
  • View and navigate through directories
  • Upload, download, and delete blobs and folders
  • Open and view the contents text and picture blobs (New in v0.7.20160107)
  • View and edit blob properties and metadata
  • Generate SAS keys
  • Manage and create Stored Access Policies
  • Search for blobs by prefix
  • Drag ‘n drop files to upload or download

This tool currently supports blob operations only and according to Microsoft support for Tables & Queues is coming soon.

Let’s take a look at this tool and see how we can manage Azure Storage using that. First you need to log into your Azure subscription.

Storage-Explorer-1.png

Once you are signed into your Azure subscription you can immediately start navigating through all of your storage accounts.

Storage-Explorer-3.png

You can perform following blob operations by right-clicking on a storage blob.

Storage-Explorer-4.png

Attaching Storage

If you want to connect to storage accounts in a different Azure Subscription or Azure China Storage Accounts or any publicly available storage service that you are not an administrator, you can  right-click on the Storage node and select Attach External Storage. Here you can provide the Account Name & the Access Key to connect to those external storage accounts.

Storage-Explorer-6.png

Also it is possible to connect to a blob container using a Shared Access Signature key and in order to do so the SAS key should provide List permissions for that particular blob.

Storage-Explorer-7.png

You can download this tool from storageexplorer.com